slime Documentation — LLM Post-Training Framework for RL Scaling

by THUDM (Tsinghua University)

AdvancedDocumentationFree~8-12 hours to get one dense-model RL run working end to end; a multi-GPU cluster is required

The post-training stack that actually trained GLM — Megatron for training, SGLang for rollout, documented example by example.

Start LearningAdded Sep 10, 2026 · Updated Sep 10, 2026

Overview

slime is an Apache-2.0 LLM post-training framework from THUDM, at roughly 8.4k GitHub stars, whose documentation was last updated 28 August 2026 alongside a v0.1.0 release. Its architecture is three modules: Training on Megatron consumes data from the buffer and synchronises parameters back to rollout; Rollout on SGLang plus a router generates new samples and rewards; and a Data Buffer mediates between them, holding prompts, custom data and generation methods. The docs open with Quick Start, a Usage Guide, a Customization Guide, an Agentic RL Training Roadmap and an FAQ, then move to worked examples that are the real curriculum: dense models (Qwen3-4B, GLM4-9B), MoE models (GLM-4.7-Flash, Qwen3-30B-A3B, GLM-5.2 at 744B, DeepSeek R1), and task-shaped runs covering SFT training, Search-R1, multi-agent RL and coding-agent RL. An advanced section documents on-policy distillation, speculative decoding, low-precision training, reproducibility, fault tolerance, delta weight sync, and the SGLang and Megatron configuration surfaces separately. A developer track adds CI/CD, debugging, a trace viewer and profiling, and there is a dedicated AMD platform-support page. Supported model families span Qwen 3.6, 3.5, 3Next, 3MoE, 3 and 2.5, DeepSeek V3 variants and Llama 3. Credibility is unusually concrete: slime is the framework behind the GLM series from GLM-4.5 through GLM-5.3, and other RL systems are built on top of it — Dressage at Alibaba Accio, Miles at RadixArk, and vime, the vLLM project's own post-training framework announced in June 2026, which reuses slime's training stack and data-generation design.

At a Glance

Topic: Fine-Tuning
Level: Advanced
Format: Documentation
Cost: Free
Duration: ~8-12 hours to get one dense-model RL run working end to end; a multi-GPU cluster is required
Provider: THUDM (Tsinghua University)
Hands-on: Yes — code/exercises
Certificate: None

What You’ll Learn

✓How to split an RL post-training job across Megatron training, SGLang rollout and a data buffer
✓How to run GRPO-style RL end to end on a dense model such as Qwen3-4B or GLM4-9B
✓How MoE post-training differs in practice, using GLM-4.7-Flash, Qwen3-30B-A3B and 744B GLM-5.2 examples
✓How weight synchronisation and delta weight sync keep rollout policies aligned with the trainer
✓How to configure fault tolerance and reproducibility so a multi-day cluster run survives failures
✓How to build agentic RL rollouts for search, multi-agent and coding-agent tasks
✓How to apply on-policy distillation, speculative decoding and low-precision training to cut cost
✓How to profile, trace and debug a distributed RL run instead of guessing at throughput

Highlights

•Not a toy: this is the framework that post-trained the GLM series from GLM-4.5 through GLM-5.3
•Independent validation from downstream adopters — Dressage (Alibaba Accio), Miles (RadixArk) and vime (vLLM project) all build on slime's training stack
•Examples are organised by real model shapes and sizes rather than by API surface, so you can pick the one matching your cluster
•Documents the unglamorous parts long RL runs fail on: fault tolerance, reproducibility, weight sync and trace-level profiling
•AMD platform support is documented explicitly rather than assumed away, and the docs carry an August 2026 update stamp

Who It’s For

Best For

✓ML engineers running reinforcement learning post-training on multi-GPU or multi-node clusters
✓Teams building agentic RL environments for coding, search or multi-agent tasks
✓Researchers reproducing or extending the GLM post-training recipe
✓Infrastructure engineers evaluating RL frameworks against verl, NeMo RL and OpenRLHF

Prerequisites

•Working knowledge of RL for LLMs — PPO, GRPO, reward modelling and rollout mechanics
•Prior experience with Megatron-LM or another distributed training stack, and with SGLang or vLLM serving
•Access to a multi-GPU cluster; the examples assume real hardware, not a single card

FAQ

What is slime Documentation — LLM Post-Training Framework for RL Scaling?

Official documentation for slime, the LLM post-training framework built for reinforcement-learning scaling by THUDM. It targets engineers running real RL jobs on multi-GPU clusters rather than notebook demos, and it walks through the Megatron training / SGLang rollout / data buffer split with runnable examples for dense and Mixture-of-Experts models up to 744B parameters, plus the fault tolerance and weight-sync machinery long runs actually need.

Is slime Documentation — LLM Post-Training Framework for RL Scaling free?

slime Documentation — LLM Post-Training Framework for RL Scaling is free to access.

What level is slime Documentation — LLM Post-Training Framework for RL Scaling for?

slime Documentation — LLM Post-Training Framework for RL Scaling is aimed at a advanced audience. Recommended background: Working knowledge of RL for LLMs — PPO, GRPO, reward modelling and rollout mechanics, Prior experience with Megatron-LM or another distributed training stack, and with SGLang or vLLM serving, Access to a multi-GPU cluster; the examples assume real hardware, not a single card.

How long does slime Documentation — LLM Post-Training Framework for RL Scaling take?

Expect roughly ~8-12 hours to get one dense-model RL run working end to end; a multi-GPU cluster is required. Most learners work through it at their own pace.

What will I learn from slime Documentation — LLM Post-Training Framework for RL Scaling?

You'll learn: How to split an RL post-training job across Megatron training, SGLang rollout and a data buffer; How to run GRPO-style RL end to end on a dense model such as Qwen3-4B or GLM4-9B; How MoE post-training differs in practice, using GLM-4.7-Flash, Qwen3-30B-A3B and 744B GLM-5.2 examples; How weight synchronisation and delta weight sync keep rollout policies aligned with the trainer; How to configure fault tolerance and reproducibility so a multi-day cluster run survives failures; How to build agentic RL rollouts for search, multi-agent and coding-agent tasks; How to apply on-policy distillation, speculative decoding and low-precision training to cut cost; How to profile, trace and debug a distributed RL run instead of guessing at throughput.

Topics

reinforcement-learningpost-trainingmegatronsglanggrpollm-training

Sources

This page was written from 3 sources, 2 on domains other than thudm.github.io.