Skip to main content

Blog

Articles, guides, and insights on local artificial intelligence

14 articles found per "MoE"

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active
ModelsAdvanced

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active

The first Qwen-Max-class open-weight model (2.4T parameters, 95B active) outgrows any consumer node: the math on weights, multi-node hardware, 36-month amortization, and power costs, and when running it locally makes sense.

iaLocale Team
November 22, 2026
Mistral Large 4 in open weights: a 1T MoE (52B active) for on-premise inference
ModelsIntermediate

Mistral Large 4 in open weights: a 1T MoE (52B active) for on-premise inference

Mistral released Large 4 in open weights (1.03T total, 52B active, 1M context, open-weight license): an article teaching how to read MoE specs and do the RAM, quantization and speed math for on-premise inference.

iaLocale Team
November 14, 2026
Nemotron 3.5 Bolt: NVIDIA's 74B MoE with ~9B active — what changes on your desktop
ModelsIntermediate

Nemotron 3.5 Bolt: NVIDIA's 74B MoE with ~9B active — what changes on your desktop

74B parameters, 9B active: NVIDIA's MoE for local agents fits 24GB of VRAM via IQ2 quantization. Real VRAM, declared benchmarks and honest limits.

iaLocale Team
November 2, 2026
MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM
ModelsIntermediate

MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM

MiniCPM-V 4.7 (35B total, ~3B active, released October 12, 2026 by OpenBMB and ModelBest) is the first open MoE in the series, vision-language: multiple images, video, OCR. 13+ tok/s with 3 GB of RAM on an iPhone 17 Pro Max, 17+ on Android. Official quantizations BF16, FP8, INT4, AWQ, GPTQ, GGUF; engines Ollama, llama.cpp, vLLM, SGLang. The three techniques: Lookahead Expert Scheduling, fine per-level quantization, DSpark speculative sampling.

iaLocale Team
October 21, 2026
Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup
GuidesAdvanced

Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup

What actually fits in the 512GB Mac Studio M5 Ultra: the 4-bit rule for MoE (active vs total params), three GGUF setup paths (Ollama, LM Studio, llama.cpp), why the KV cache is the real bottleneck, and when the $9,999 pays back against APIs.

iaLocale Team
October 20, 2026
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
ModelsIntermediate

Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM

inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.

iaLocale Team
October 19, 2026
Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
ModelsIntermediate

Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally

Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.

iaLocale Team
October 18, 2026
Decision models in local: JEV-35B and the 3B-active System 1
Use casesIntermediate

Decision models in local: JEV-35B and the 3B-active System 1

You don't need a big LLM to decide. JEV-35B (35B, 3B active) in NVFP4 on a 32GB GPU and ej (11.4MB) on CPU show how to do routing, classification and quality control without generating a single token.

iaLocale Team
October 17, 2026
Mac Studio M5 Ultra: 512GB of unified memory on the desktop
HardwareIntermediate

Mac Studio M5 Ultra: 512GB of unified memory on the desktop

Four dies, 512GB of unified memory at 1.2 TB/s: the Mac Studio M5 Ultra runs 100-560B MoE models locally. The speed math, the cost comparison with the DGX Station, and when the Mac is the right choice.

iaLocale Team
October 16, 2026
gpt-oss: OpenAI's open reasoning model
ModelsIntermediate

gpt-oss: OpenAI's open reasoning model

21B total parameters but only 3.6B active, native MXFP4 weights and 128K of context: OpenAI's first open reasoning model fits in 16GB. What that means in practice.

iaLocale Team
October 11, 2026
Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB
ComparisonsIntermediate

Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB

A 27.8B dense or a MoE with 3.8B active, a 262K context against 256K and almost the same weight at Q4_K_M: two opposite ways of putting big-model quality on a 24GB consumer card. Which is faster, which goes deeper, and when to pick one over the other.

iaLocale Team
October 9, 2026
How to Read a Model's Spec Sheet: Family, Size, Context
ModelsIntermediate

How to Read a Model's Spec Sheet: Family, Size, Context

A guide to reading a model's specs: family, total and active size, context window, quantization, benchmarks and license, with a worked example on DeepSeek V4.1 Flash.

iaLocale Team
September 26, 2026
Qwen: the open model with the best multilingual support
ModelsBeginner

Qwen: the open model with the best multilingual support

Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.

iaLocale Team
September 19, 2026
Llama 3 and Llama 4: Meta's open family — which model runs on your hardware
ModelsBeginner

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

Llama 3 and 4 in a table: 3B/8B/70B and Scout MoE 109B, RAM required per model, the Community license and the choice by budget.

iaLocale Team
June 10, 2026