GLM-5: from Vibe Coding to Agentic Engineering

Submitted 17 February 2026 (v2 on 24 February) by a 270-author team at Zhipu AI and Tsinghua University, the report runs from architecture to deployment. Pre-training documents a 744B-parameter mixture-of-experts model with 40B active parameters — double GLM-4.5 — trained on 27T tokens extended to 28.5T, using DeepSeek Sparse Attention to cut long-sequence attention cost 1.5-2x, Multi-Latent Attention with a 'Muon Split' optimisation, and multi-token prediction with parameter sharing to raise speculative-decoding acceptance. Mid-training extends context progressively — 32K over 1T tokens, then 128K over 500B, then 200K over 50B — and adds roughly 160B tokens of software-engineering data drawn from 10M issue-PR pairs. Post-training covers SFT across general chat, reasoning and coding/agent data with three thinking modes; reasoning RL on a GRPO backbone with an IcePop technique for training-inference mismatch; agentic RL on a fully asynchronous architecture with a token-in-token-out gateway, double-sided importance sampling and off-policy sample dropping; general RL with rule-based, outcome and generative reward models; and on-policy cross-stage distillation to prevent forgetting. A chapter on environment scaling describes 10,000+ verifiable executable SWE environments across nine languages via RepoLaunch, Harbor-format terminal environments, and web knowledge graphs for search agents, plus context-management strategies that lift BrowseComp from 55.3 to 62.0. Further chapters cover the slime RL framework, memory and parallelism infrastructure, adaptation to seven Chinese accelerator platforms with W4A8 quantisation, and evaluation across SWE-bench Verified, Terminal-Bench 2.0, BrowseComp, tau-squared-Bench and Vending-Bench 2.

Topic

Models

Level

Advanced

Format

Paper

Cost

Free

Duration

~3-4 hours for a full read; ~45 minutes for the asynchronous-RL sections alone

Provider

Zhipu AI / Z.ai (GLM-5 Team)

Hands-on

No

Certificate

None

• ✓How DeepSeek Sparse Attention cuts long-context attention cost by 1.5-2x in practice
• ✓How a 744B-parameter MoE with 40B active parameters is trained on 28.5T tokens
• ✓Why context extension is staged at 32K, then 128K, then 200K rather than done at once
• ✓How asynchronous RL decouples rollout generation from training, and what breaks when it does
• ✓Why a token-in-token-out gateway is needed to avoid re-tokenisation mismatch in agent RL
• ✓How 10,000+ verifiable executable software-engineering environments were constructed across nine languages
• ✓Which context-management strategy raised BrowseComp from 55.3 to 62.0 for search agents
• ✓How on-policy cross-stage distillation prevents forgetting across sequential RL stages

• •One of the most detailed public accounts of asynchronous agentic RL infrastructure, not just a benchmark card
• •Environment scaling is treated as a first-class engineering problem — RepoLaunch, Harbor-format terminal environments, bidirectionally verified web knowledge graphs
• •Weights are Apache-2.0 and the companion repo zai-org/GLM-5 (7.0k stars) ships GLM-5, 5.1 and 5.2 in both BF16 and FP8
• •Reports practical negative results, such as deterministic top-k operations being required to keep sparse attention stable during RL
• •A full chapter on adapting a frontier model to seven Chinese accelerator platforms with W4A8 mixed-precision quantisation — rare in any model report
• •Evaluated on agent-shaped benchmarks (Terminal-Bench 2.0, Vending-Bench 2, tau-squared-Bench, MCP-Atlas) rather than only static QA

This page was written from 3 sources, 1 on domains other than arxiv.org.

1. 1.arxiv.org — 2602.15763vendor
2. 2.arxiv.org — 2602.15763v1vendor
3. 3.github.com — GLM 5