Kimi K2: Open Agentic Intelligence
by Moonshot AI (Kimi Team)
The tech report behind a 1T-parameter open-weights MoE that beats most closed models on agentic benchmarks — including how they trained it without a single loss spike.
Overview
Kimi K2 is Moonshot AI's technical report on a Mixture-of-Experts language model with one trillion total parameters and 32 billion activated per token, posted to arXiv on 28 July 2025 and revised in February 2026, credited to the roughly 200-strong Kimi Team. Its central pre-training contribution is MuonClip, an optimizer extending Muon with a QK-clip technique applied to attention query and key projections; the authors report this let them pre-train on 15.5 trillion tokens with zero loss spikes, a specific and checkable stability claim at a scale where training runs routinely diverge and get silently restarted. The post-training half is the part most relevant to agent builders: a large-scale agentic data synthesis pipeline that manufactures tool-use trajectories, followed by a joint reinforcement learning stage in which the model improves through interaction with both real and synthetic environments. Reported results for the non-thinking model include 65.8 on SWE-Bench Verified, 47.3 on SWE-Bench Multilingual, 66.1 on Tau2-Bench, 76.5 on ACEBench (English), 53.7 on LiveCodeBench v6, 49.5 on AIME 2025, 75.1 on GPQA-Diamond and 27.1 on OJBench — all without extended chain-of-thought decoding, which is the comparison that makes the coding and tool-use numbers notable. Base and instruct checkpoints ship with a 128K context window under a Modified MIT License through the MoonshotAI/Kimi-K2 GitHub repository (about 11.1k stars), with documented serving paths for vLLM, SGLang, KTransformers and TensorRT-LLM. The arXiv posting itself is licensed CC BY-NC-ND 4.0, which is separate from the weights licence.
At a Glance
- Topic
- Models
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~1-2 hours to read; arXiv tech report, v1 28 Jul 2025, revised Feb 2026
- Provider
- Moonshot AI (Kimi Team)
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How MuonClip's QK-clip technique stabilises Muon-based training at trillion-parameter scale
- ✓Why a sparse MoE with 32B activated parameters can serve at far below 1T-parameter cost
- ✓How large-scale agentic trajectory synthesis produces tool-use training data without human demonstrations
- ✓What a joint reinforcement learning stage over real and synthetic environments actually optimises
- ✓How to read agentic benchmark results — Tau2-Bench, ACEBench, SWE-Bench Verified — and what each measures
- ✓Why non-thinking-mode scores are the honest comparison point against extended-reasoning models
- ✓What serving a 1T-parameter open-weights MoE requires in practice across vLLM, SGLang and TensorRT-LLM
Highlights
- •A frontier-scale training report with real numbers, not a marketing model card — optimizer, token count, loss behaviour and full benchmark tables
- •The zero-loss-spike claim over 15.5T tokens is unusually specific and is the paper's most reproducible contribution
- •Weights are actually released: base and instruct checkpoints under a Modified MIT License, not a research-only gate
- •Documents agentic capability as a training objective rather than a prompting technique, which is the shift most agent teams have not internalised
- •Companion GitHub repo (~11.1k stars) carries deployment recipes for four serving engines, so the paper is directly actionable
Who It’s For
Best For
- ✓ML engineers evaluating open-weights models for agentic or coding workloads
- ✓Researchers working on optimizers, training stability, or MoE architectures
- ✓Infrastructure teams planning to self-host a large MoE and sizing the serving stack
- ✓Anyone benchmarking open versus closed models on tool use and software engineering tasks
Prerequisites
- •Solid understanding of transformer architectures, attention and Mixture-of-Experts routing
- •Familiarity with LLM pre-training and post-training terminology (SFT, RLHF, RL environments)
- •Ability to read benchmark tables critically — the paper assumes you know what SWE-Bench Verified measures
FAQ
What is Kimi K2: Open Agentic Intelligence?
Moonshot AI's technical report for Kimi K2, a Mixture-of-Experts model with 1 trillion total and 32 billion activated parameters. It documents the MuonClip optimizer, a 15.5-trillion-token pre-training run with zero loss spikes, and a post-training pipeline built around synthesised agentic trajectories and joint reinforcement learning. Read it to understand how open-weights labs are now training for tool use rather than bolting it on.
Is Kimi K2: Open Agentic Intelligence free?
Kimi K2: Open Agentic Intelligence is free to access.
What level is Kimi K2: Open Agentic Intelligence for?
Kimi K2: Open Agentic Intelligence is aimed at a advanced audience. Recommended background: Solid understanding of transformer architectures, attention and Mixture-of-Experts routing, Familiarity with LLM pre-training and post-training terminology (SFT, RLHF, RL environments), Ability to read benchmark tables critically — the paper assumes you know what SWE-Bench Verified measures.
How long does Kimi K2: Open Agentic Intelligence take?
Expect roughly ~1-2 hours to read; arXiv tech report, v1 28 Jul 2025, revised Feb 2026. Most learners work through it at their own pace.
What will I learn from Kimi K2: Open Agentic Intelligence?
You'll learn: How MuonClip's QK-clip technique stabilises Muon-based training at trillion-parameter scale; Why a sparse MoE with 32B activated parameters can serve at far below 1T-parameter cost; How large-scale agentic trajectory synthesis produces tool-use training data without human demonstrations; What a joint reinforcement learning stage over real and synthetic environments actually optimises; How to read agentic benchmark results — Tau2-Bench, ACEBench, SWE-Bench Verified — and what each measures; Why non-thinking-mode scores are the honest comparison point against extended-reasoning models; What serving a 1T-parameter open-weights MoE requires in practice across vLLM, SGLang and TensorRT-LLM.
Topics
Sources
This page was written from 2 sources, 1 on domains other than arxiv.org.