Skip to main content
ModelsAdvanced7 min read

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active

i
iaLocale Team
•
Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active

Qwen3.8-2.4T-A95B is the first Qwen-Max-class model in open weight: 2.4 trillion parameters in total, 95 billion active per token. The news matters to people who run models locally, but not for the reason you'd think: this is not about downloading it onto a workstation. It's about understanding what it weighs, what it costs, and what you get. The math is below.

What 2.4T actually means

"2.4 trillion" is not a number you can feel: it's a datacenter scale. The concrete facts, from the model card:

  • MoE with 512 experts: per token, 10 routed + 1 shared expert activate, for 95B active parameters. The rest of the model is there, but it isn't computed.
  • 92 layers, hidden dimension 8192, with multi-token prediction (MTP) trained over multiple steps.
  • Hybrid attention: 3 of every 4 layers use Gated DeltaNet (linear attention, constant state), 1 of 4 uses classic attention (Gated Attention). It's the same family of tricks that keeps the KV cache small: on a long context, cache memory only grows for the 23 classic-attention layers, not all 92.
  • Context: 262,144 tokens natively, extendable to ~1M.
  • Formats: BF16 weights in the repo, with a separate FP8 variant. The official API build (Qwen3.8-Max) adds vision input and 1M context by default, per the official blog.

The point that matters for the math: the bytes that must live in memory are the 2.4T in full, not the 95B active. In an MoE the experts sit in memory and each token touches a few of them: the machine has to be able to hold all of them.

How many bytes that is

Parameters × bytes per parameter. With the standard GGUF quant ratios (the same ones we use for normal-sized models):

FormatBytes/paramWeights on disk
BF16 (as released)2.0~4.8 TB
FP8 (separate variant)1.0~2.4 TB
Q8 (GGUF)~1.1~2.6 TB
Q4_K_M (GGUF)~0.55~1.3 TB
Q2_K (GGUF)~0.38~0.9 TB

Even at the most aggressive quant in the table the weights clear 900 GB. The question stops being "which quant" and becomes "which machine".

Where it fits

The uncomfortable answer: no single consumer node. The top of the market today:

MachineMemoryFits Q4_K_M (~1.3 TB)?
NVIDIA DGX Spark (GB10)128 GBNo
AMD Ryzen AI Max+ 395 (Strix Halo)128 GBNo
Mac Studio (M5 Max)128 GBNo
Mac Studio (M5 Ultra)512 GBNo — 512 < 1,300

The consumer record, 512 GB of unified memory, covers barely a third of the model at Q4. If you want a single node that fits with headroom, the next step up is server memory: 1.5-2 TB of ECC RAM on a multi-socket workstation, or GPUs in tensor/parallel-expert configurations.

The realistic consumer alternative is a multi-node setup:

ConfigurationTotal memoryQ4_K_M weightsHardware cost
4× Mac Studio M5 Ultra 512 GB2 TBFits, with room for the KV cache4 × 6,699 = 26,796 €
8× H100 80 GB (SXM)640 GBNo, even at Q2_K8 × 30,000 = 240,000 €
24× H100 80 GB (SXM)1.9 TBFits24 × 30,000 = 720,000 €

The 8× H100 row is in there to show the gap: 640 GB is not enough for any quant in the table above. You need multiples more, and the price follows.

The running costs: amortization and power

Same assumptions as our other machine math: 36-month amortization, 24/7 usage at a 50% average load, electricity at €0.25/kWh.

4× Mac Studio M5 Ultra 512 GB (480 W peak per unit, ~240 W average at half load):

  • Amortization: 26,796 € ÷ 36 = ~744 €/month
  • Power: 0.24 kW × 730 h = 175 kWh per Mac per month → 4 Macs = 701 kWh → × 0.25 = ~175 €/month
  • Total: ~920 €/month, all included

24× H100 80 GB (700 W peak per card, ~350 W average):

  • Amortization: 720,000 € ÷ 36 = 20,000 €/month
  • Power: 8.4 kW average × 730 h = 6,132 kWh → × 0.25 = ~1,530 €/month
  • Total: ~21,500 €/month, before room, cooling, and networking

The H100 fleet's electricity bill (€1,530/month) costs more than the entire Mac fleet's monthly total, which spends €175. On big MoEs, the architecture difference between unified memory and HBM shows up on the meter before it shows up in benchmarks.

What speed you'd get

On a single node, the rule we use for MoEs applies here: decoding is bandwidth-limited, and you divide bandwidth by the bytes of the active experts. 95B active at Q4_K_M ≈ 52 GB:

BandwidthTheoretical tok/s (1,200 ÷ 52)
Mac Studio M5 Ultra, 1,200 GB/s~23
DGX Spark, 273 GB/s~5

Careful: the 512 GB Mac does not hold the model, so the 23 tok/s is a counterfactual — the speed of a machine that doesn't exist. In multi-node the math changes: each token crosses all 92 MoE blocks, and at each block the activations make an all-to-all round between the nodes hosting the experts. The bottleneck is no longer memory bandwidth but the interconnect: Thunderbolt (10-15 GB/s) or 25/40GbE. There are no public numbers for this exact configuration, and we won't invent any: the order of magnitude is set by the interconnect, and if you need an always-on server with SLOs, start from the assumption that a consumer multi-node build is a project to validate, not a purchase.

When it makes sense

  • If today you run a Qwen3.8-27B or a 30B-A3B on 24 GB of VRAM: for the vast majority of uses — RAG, assisted coding, automation — those are enough. This article is not an incentive to buy something else, it's the cost map for the case where you genuinely want Max-class quality without an API.
  • It makes sense locally when the data cannot leave, the model must be available 24/7 without a token meter, and you already have space for a rack or a table full of Macs. In that case, the Mac fleet at ~920 €/month is the number to compare against your API token volume.
  • It does not make sense for someone starting from zero: a single 512 GB Mac with a 30-70B model gives you a complete local AI for a quarter of the price, with half the logistical problems.

In summary

Qwen3.8-2.4T-A95B brings the Qwen-Max class to open weight: 2.4T parameters, 95B active, context from 262k to 1M. In bytes, ~1.3 TB at Q4_K_M — more than any consumer node (max 512 GB) — so local for this model is necessarily multi-node: 4 Mac Studio M5 Ultra at ~26,800 € (~920 €/month all included) or 24 H100 at 720,000 € (~21,500 €/month). Multi-node speed is set by the interconnect, not by memory. If you don't have infrastructure yet, the right model for you is smaller: the open frontier has reached Max-class, but your hardware probably hasn't.

Article generated with AI

In these reading paths

Share:

Enjoyed this article?

Subscribe to the newsletter for new tutorials and guides.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.