Skip to main content
← Back to news

GridCore: a scheduler that lets several local LLMs share one GPU

Tools & releases
Source:Hacker News
An open-source tool that schedules several local LLMs on the same GPU, keeping them from fighting over VRAM in turns. Useful for people running multiple models (an assistant, an embedder, a classifier) on one card who want predictable latencies.
Read original article

Related articles

The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti

The 16GB VRAM tier is the center of gravity of local AI in 2026. It's enough memory to run the best open dense models in Q4 formats — the 27B class, the new balance point of the open catalog — without giving up too much context, and a list price that (depending on the month) stays within a personal budget. The Blackwell generation offers three cards exactly in the middle of this tier: the RTX 5060 Ti, 5070 and 5070 Ti. Same generation, same GDDR7, three very different experiences. Here's how they compare: what each one really brings you, and where your budget should go.

Quantization: Get More From the Same Hardware

What quantization is, the Q4/Q5/Q8 levels, and how to pick the right one for your VRAM, with a concrete example on Qwen2.5 14B.

ML Drift: Google's open-source on-device GPU engine, and when it beats the generalists

The engine that runs models inside Chrome and YouTube is now open source: architecture, benchmark numbers, and an honest comparison with Ollama and llama.cpp.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🌱

Bonsai-2 27B

Models

27B reasoning model with ternary weights at 1.72 bits per weight, derived from Qwen3.8-27B with hybrid attention (~75% linear). In GGUF PTQ1_0 it takes 5.95 GB, retains 98.2% of the FP16 intelligence and reaches ~47 tok/s on an Apple M5 Max laptop, with a 262K-token context. Apache 2.0 license. Requires Prism ML's llama.cpp fork (ternary kernels are not in stock llama.cpp).

🤖

LFM3-1.7B

Models

Model for local agents and specialized applications, 1.73B parameters, tri-hybrid architecture (13 gated short convolutions, 5 sliding-window attention, 6 global attention). Explicit agentic post-training (harnesses like Hermes Agent, Pi, OpenClaw), 256K-token context, 16 languages including Italian, ~1GB of RAM at 4-bit. LFM Open License v1.0.

🤖

OpenMobile-2-9B

Models

9B smartphone agent (image-text-to-text) derived from Qwen3.5-9B and fine-tuned on OpenMobile-2 data: Android screenshots and native app tools. AndroidWorld Pass@1 79.9, MobileWorld 50.4; SPA-Bench rises from 31.9 to 59.6 after RL training. vLLM and SGLang support; the training environments (MobileGym++) and recipes are public. Apache 2.0 license.

⚡

SGLang

Inference

Open-source serving engine for LLMs with RadixAttention, continuous batching and first-class speculative decoding (MTP, EAGLE, draft models, ngram). Apache-2.0 license.

Browse all resources