Blog
Articles, guides, and insights on local artificial intelligence
6 articles found per "qwen"

Speculative decoding and MTP: speed for free
Speculative decoding makes local AI faster without changing output quality: a small model drafts the tokens, the big model verifies them. Here's how the draft-verify loop works, MTP heads in Qwen models, EAGLE on Mistral Medium 3.5, and how to enable it all in three commands.

Qwen3.8-27B vs Gemma 4 26B A4B: the new frontier at 24GB
A 27.8B dense or a MoE with 3.8B active, a 262K context against 256K and almost the same weight at Q4_K_M: two opposite ways of putting big-model quality on a 24GB consumer card. Which is faster, which goes deeper, and when to pick one over the other.

Local Image Models: from SDXL to Qwen-Image
From SDXL's 2.6B parameters to Qwen-Image 2.1's 7B: how diffusion models work, how much VRAM you actually need, and which model to pick for your hardware.

KV cache: why your RAM needs more than just the model
What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.

Qwen-Image 2.1: Generate and Edit Images Locally
Qwen-Image 2.1 unifies text-to-image, editing, and RGBA transparency in a 7B open model (32-layer DiT). Local setup with diffusers, native ComfyUI support, VRAM needs (~14GB in bfloat16), and use cases (stickers, product photos, design workflows).

Quantization: Get More From the Same Hardware
Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).

