vLLM v0.31.0rc3: randomized dummy inputs support in Model Runner V2
Hardware & acceleratorsRelated articles
vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the concepts (throughput, quantization, KV cache) for choosing when to use it instead of Ollama.
Nemotron-Parse: NVIDIA's OCR family for reading documents locally
NVIDIA re-released Nemotron Parse (v1.1, v1.2, 2.0) in the same week, on October 20: under 1 billion parameters, page structure included. Local setup with vLLM and the honest limits.
Local Images: ComfyUI 0.40 and comfy_attention — new for all models
ComfyUI 0.40 extends the comfy_attention attention engine to all models, turns on the local asset API, and standardizes 3D output. Plus the VRAM table that tells you what runs on your hardware.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.