Gated DeltaNet
ArchitecturesA linear-attention variant used in Qwen 3.5/3.8 models: it keeps a constant-size state updated by a delta-style rule (it removes stale information before adding the new one) instead of accumulating all key-value pairs. With few layers of this kind, KV cache memory grows little as the context grows, and classic attention layers alternate with them — for example 1 in 4 in Qwen3.8-2.4T-A95B.

