Tideway: VRAM and RAM managed as one pool for local MoE inference
ResearchRelated articles
GPU or CPU: when the CPU is enough
Do you really need a GPU for local AI? For many use cases, no: how prefill and decode work, what matters (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure on your own hardware.
Mac Studio M5 Ultra: 512GB of unified memory on the desktop
Four dies, 512GB of unified memory at 1.2 TB/s: the Mac Studio M5 Ultra runs 100-560B MoE models locally. The speed math, the cost comparison with the DGX Station, and when the Mac is the right choice.
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
Ling 3.1 Flash ships 560B open-weight parameters under an MIT license with 1M context. It won't fit in 24GB of VRAM, but the quantizations, 25B active MoE and the real numbers change the calculation.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

