Kimi K3 architecture: KDA, MLA and MoE
Kimi K3 is easy to summarize badly. Calling it a 2.8-trillion-parameter model is accurate, but it hides the engineering choices that make the number possible.
The useful story is about information flow. K3 changes how information moves across tokens, across layers, and across experts. It also forces the serving stack to manage two different kinds of cache.
This guide follows the official technical report, not the launch-day benchmark race.
In this article
- The architecture in one pass
- Sequence: three KDA layers, then one Gated MLA layer
- KDA is a writable state, not a smaller KV cache
- Gated MLA is the exact-attention checkpoint
- Depth: Attention Residuals choose what to reuse
- Width: Stable LatentMoE routes through 16 of 896 experts
- Serving K3 means managing two cache lifecycles
- One million tokens is a systems claim too
- Read the benchmarks with their harnesses attached
- What to carry forward
Continue reading
This field note is part of Fanout Pro, with the complete archive, courses, labs, and every Daily paper guide.
See plans