Skip to main content
← Back to news

llama.cpp v0.9.0: OCP FP8 quantization, unified KV for multi-sequence and Chatterbox TTS

Tools & releases
Source:GitHub
The local inference engine adds OCP FP8 E4M3 quantization (saturating mode) and the flash_attn_ext_rows op that makes the unified KV cache efficient for multi-sequence inference. New architectures (Clover-35B-A3B-Preview, Limite 1B, Metis-1.6 and Mistral Large 4), Chatterbox TTS, Nemotron 3.5 Transcribe in 40 languages and faster prefill for gemma4 E2B/E4B; ggml moves to v0.29.0 with NCCL multi-node tensor parallelism.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.