Skip to main content
← Back to news

llama.cpp adds decision models via the /v1/systemone endpoint

Decision models
Source:Hugging Face
The llama.cpp server now supports decision models: send a state (text, JSON or a screenshot) plus typed questions, and the model returns a probability for each option in a single forward pass, without generating text. Three question types (choice, score, noul) and six models from 144M (Julia-1) to 27B (OpenJev, Clef), answering in 3-43 ms on an RTX PRO 6000, Apache 2.0 except OpenJev (CC BY-NC).
Read original article

Related articles

Qwen: the open model with the best multilingual support

Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

From the 3.1 8B to the 4 Scout MoE with a 10M context: the Llama family in a table, what runs on 16, 48 and 128 GB of RAM, the Community license translated for the Italian business and the choice by budget.

Where local models learn: open weights explained without the fluff

The model doesn't hook into the internet for its abilities: it absorbed them once at pre-training and executes them on your hardware. Open weights isn't open data, and the models come from many labs, not one.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🦙

Llama 3.1 405B

Models

Meta's largest open model: closed-frontier quality but datacenter requirements. In practice it needs high-VRAM multi-GPU or the cloud.

🦙

Llama 3.1 70B

Models

Frontier-class open model with 131K-token context. Needs 48-80GB VRAM, a Mac with unified memory, or a multi-GPU setup.

🦙

Llama 3.1 8B

Models

The de-facto local workhorse: solid quality, 131K-token context and a huge GGUF quantization ecosystem. The standard choice for chat and assistants.

🦙

Llama 3.2 1B

Models

Small, fast model with up to 131K-token context. Ideal entry point for chat and summarization on modest hardware, great on phones and mini-PCs.

Browse all resources