Open-weight mixture-of-experts is eating the mid-range

The interesting thing about this year’s crop of open-weight releases is not that they are large, but that the large ones are sparse. Qwen3.5, GLM-5.2, Kimi K3 and MiniMax M3 all lean on mixture-of-experts layouts, which means the parameter count in the headline and the parameter count doing work on any given token are very different numbers.

That gap is what makes them practical. A 480B-class model that activates a fraction of itself per token has memory requirements closer to its active size than its total, and the total is mostly a storage problem. For anyone renting hardware by the hour, storage is the cheap part.

The practical consequence is that the mid-range has been hollowed out. A year ago the sensible self-hosted choice was a dense model in the 30-70B range. Now the choice is either something small enough to sit comfortably on one card, or something sparse enough that its active path does. The middle is awkward: dense enough to be expensive, not capable enough to justify it.

None of which says much about quality, only about economics. Benchmarks move around. The shape of what is deployable moves more slowly, and it has moved.