TL;DR

Kimi K3: building the best model is easier than keeping it online.

A few days ago I argued that Kimi K3’s architecture is a platform problem — every design choice was really an answer to “how do we move less data between GPUs?” Then reality delivered the sequel: within 48 hours of launch, Moonshot had to pause new subscriptions because demand outran their GPUs. The most efficient open model in the world couldn’t be served fast enough. Building the model turned out to be the easy part. This post is about the hard part — serving it — and about the one lever that changes the game for the rest of us: open weights, and the build-vs-buy decision they hand you.

Originally published at portfolio.hagzag.com. This is a follow-up to Kimi K3: When Model Architecture Becomes a Platform Problem.

The Platform Problem Came for Its Authors

In Part 1 I made a claim: K3 reads less like an ML paper and more like a distributed-systems design doc, because every headline feature — latent MoE compression, hybrid linear attention, block-grouped residuals — exists to reduce data movement between GPUs. Efficiency isn’t bolted on; it’s the whole design.

Here’s the twist I didn’t see coming when I wrote that. Within 48 hours of launch, demand pushed Moonshot’s GPU fleet to the edge of what it could serve. They paused new subscriptions, protected existing users, and said they’d add capacity and reopen in batches. A lab that engineered its model around compute scarcity got stopped cold by compute scarcity — not in the model, in the fleet.

That’s the thing about platform problems. You can solve them in the architecture and they simply relocate to the datacenter. Designing the most compute-efficient frontier model on earth does not exempt you from capacity planning. If anything, it guarantees the bill arrives sooner, because “cheap and excellent” is exactly the combination that triggers a demand spike you can’t provision for overnight.

The Tell: They Reached for QoS

The detail that made me sit up wasn’t the pause itself — every hot launch hits a wall. It was how they responded. Alongside adding capacity, Moonshot split their subscription into two separate plans: one for general chat/app/agent use, and one dedicated to coding workflows.

Read that as a platform engineer and it’s unmistakable: that’s quality-of-service by workload class. An agentic coding session can hold context and burn tokens for hours; a chat turn is short and bursty. Treating both as the same unit of load is how you let a noisy neighbor starve everyone else on the cluster. Splitting the plans lets them partition and schedule compute against the actual shape of each workload instead of a lowest-common-denominator average.

This is capacity management 101 — workload isolation, admission control, protecting your committed users first — applied to LLM inference. It’s the same instinct that made the model efficient, now showing up one layer out in how they run it. The architecture and the operations rhyme.

Efficient Isn’t the Same as Fast

There’s a paradox worth naming. K3 is arguably the most compute-efficient frontier model available, and yet users consistently reported it running noticeably slower than the top US alternatives. How does the efficient model end up the slow one?

Because per-token efficiency and end-user throughput are different quantities, and the gap between them is filled by your hardware. Efficiency lowers the work per token. Throughput — tokens per second in someone’s terminal — is set by how much capable silicon you can throw at that work, and how fast your interconnect moves activations between the GPUs holding your 896 experts. Cap the fleet, and even a lean model queues. Efficiency buys you more headroom per GPU; it doesn’t manufacture GPUs you don’t have.

Which is the honest footnote to Part 1’s “designed for scarcity” thesis: designing for scarcity helps you survive it, but it doesn’t make the scarcity disappear.

Why This Was Structural, Not a Stumble

It would be easy to file this as a young company underestimating launch traffic. It’s deeper than that. The labs pushing China’s open frontier generally have two ways to get compute: rent it from public cloud, or build their own clusters. Public-cloud capacity is finite and contended; building your own is capital-heavy and slow to stand up. And export controls cap access to the most advanced chips, so the binding constraint isn’t training a good model — it’s securing enough high-quality inference compute to serve it at global scale.

That reframes the whole “who’s winning” conversation. The model-quality gap has narrowed to weeks. The serving gap — the ability to actually deliver frontier intelligence to millions of users at low latency — is where hardware access still decides outcomes. Restricting chips doesn’t stop the models from being built; it constrains the ability to serve them directly. The bottleneck moved from the whiteboard to the loading dock.

The Release Valve: What Open Weights Actually Buy You

Here’s the turn, and it’s the part that matters most for anyone who runs infrastructure. A closed model that hits a capacity wall just… stays walled — you wait for the one vendor to provision more. An open-weight model has a release valve the closed ones structurally cannot: the moment the weights are public, anyone can serve them.

So the interesting question for a platform team stops being “is K3 good?” and becomes “given open weights, what do I actually do with them?” That’s a build-vs-buy decision, and it’s not binary — there are really three postures:

  • Buy — hit a managed inference API. Zero ops, someone else owns the capacity problem, you pay their per-token margin and live with their rate limits (and their pauses). Fine for spiky, low-volume, or experimental workloads.
  • Build — self-host the weights on GPUs you own or rent. You inherit the capacity problem and the control: the weights sit inside your VPC (data residency and sovereignty for regulated workloads), you can fine-tune and quantize for your exact workload, you set your own cost-per-task, and no vendor can rug-pull, deprecate, or throttle you.
  • Blend — route by workload. Send the sensitive, high-volume, or steady-state traffic to your own fleet; burst the rest to a managed API. This is the same QoS logic Moonshot reached for, just drawn at your trust and cost boundaries instead of theirs.

The honest counterweight: 2.8 trillion parameters is not a homelab project. Serving it usefully takes real HBM and a real interconnect, so “self-host everything” is idealistic for most teams. The pragmatic move isn’t all-or-nothing — it’s shifting a percentage of token spend from closed-cloud toward open-self-hosted, workload by workload, wherever sovereignty, cost, or independence justifies owning the stack. You don’t have to win the whole thing to stop handing every token to a single closed vendor.

And this isn’t hypothetical at the top of the market either. Reportedly, even a hyperscaler has weighed adopting K3 to cut inference costs on a usage-priced product — build-vs-buy is live all the way up the stack. When the weights are open, the leverage moves to whoever can serve them best: fastest fabric, lowest cost per task, closest to the data.

Conclusion

Part 1’s line was “the interconnect is the invoice.” Part 2’s is simpler: shipping the model was the easy part. Moonshot proved that the moment demand met their GPUs — and their fix, splitting workloads by class, was a platform-engineering move, not an ML one.

But the same event exposes the asymmetry that makes open weights matter. A closed model’s capacity ceiling is its users’ ceiling. An open model’s ceiling is a temporary inconvenience, because reach decouples from any one lab’s fleet the day the weights drop. That hands the rest of us a decision we didn’t have before: buy the intelligence, build the capability, or blend the two on our own terms. The platform problem never gets solved. It just gets relocated — and open weights are what let you choose where.