DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
by DeepSeek-AI
The paper that showed reasoning can be trained by reinforcement learning alone, with the weights to prove it.
Overview
DeepSeek-AI's R1 paper, submitted to arXiv on 22 January 2025 and last revised 4 January 2026, was subsequently published in Nature volume 645, pages 633-638. Its central claim is that reasoning ability in large language models can be incentivised through pure reinforcement learning, obviating the need for human-labelled reasoning trajectories to imitate. It releases two 671B-parameter mixture-of-experts models with 37B active parameters and a 128K context window: R1-Zero, trained by RL directly on the base model, and R1, which adds cold-start supervised data and runs a multi-stage pipeline of two SFT stages seeding two RL stages aimed at discovering improved reasoning patterns. The paper reports emergent behaviours - self-reflection, verification and adaptive problem-solving strategies - arising from the RL process rather than being demonstrated in training data, and shows those patterns can be systematically harnessed to guide smaller models. Six distilled checkpoints ship alongside the flagship, built on Qwen2.5-Math-1.5B, Qwen2.5-Math-7B, Llama-3.1-8B, Qwen2.5-14B, Qwen2.5-32B and Llama-3.3-70B-Instruct. Headline scores for R1 are 79.8% on AIME 2024, 97.3% on MATH-500, 71.5% on GPQA Diamond, 90.8% on MMLU, 49.2% on SWE-bench Verified and a 2029 Codeforces rating; the 32B distill reaches 72.6%, 94.3%, 62.1% and 1691 respectively. Everything is released under the MIT licence.
At a Glance
- Topic
- ML
- Level
- Advanced
- Format
- Paper
- Cost
- Free
- Duration
- ~1.5 hours read; also published as a Nature article
- Provider
- DeepSeek-AI
- Hands-on
- No
- Certificate
- None
What You’ll Learn
- ✓How pure reinforcement learning can incentivise reasoning without labelled chains
- ✓Why R1-Zero ships alongside R1 as an ablation of cold-start data
- ✓The multi-stage pipeline of two SFT stages seeding two RL stages
- ✓Which behaviours emerge: self-reflection, verification and adaptive strategy switching
- ✓How reasoning patterns transfer from a 671B MoE into smaller dense models
- ✓What six distilled Qwen and Llama checkpoints score against their base models
- ✓Where a 671B open-weight model lands on AIME, MATH-500, GPQA and SWE-bench
Highlights
- •Peer-reviewed in Nature, still unusual for a frontier reasoning-model paper
- •Weights plus six distilled checkpoints released under a permissive MIT licence
- •Reference repository carries 92k GitHub stars with reproducible evaluation configs
- •Reports the 128K-context 671B MoE architecture in enough detail to reproduce
- •The arXiv posting drew 1,351 points and over 1,000 comments on Hacker News
Who It’s For
Best For
- ✓Engineers planning a reinforcement-learning post-training run of their own
- ✓Researchers studying reasoning emergence and reward design
- ✓Teams evaluating open-weight reasoning models against closed APIs
Prerequisites
- •Solid grounding in transformer training and RLHF concepts
- •Familiarity with policy-gradient reinforcement learning terminology
FAQ
What is DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning?
DeepSeek-AI's paper on R1-Zero and R1, arguing that reasoning ability can be incentivised through pure reinforcement learning without human-labelled reasoning traces, and that the resulting patterns can be distilled into much smaller dense models. Written for engineers and researchers planning their own post-training runs, or evaluating open-weight reasoning models against closed APIs.
Is DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning free?
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning is free to access.
What level is DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning for?
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning is aimed at a advanced audience. Recommended background: Solid grounding in transformer training and RLHF concepts, Familiarity with policy-gradient reinforcement learning terminology.
How long does DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning take?
Expect roughly ~1.5 hours read; also published as a Nature article. Most learners work through it at their own pace.
What will I learn from DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning?
You'll learn: How pure reinforcement learning can incentivise reasoning without labelled chains; Why R1-Zero ships alongside R1 as an ablation of cold-start data; The multi-stage pipeline of two SFT stages seeding two RL stages; Which behaviours emerge: self-reflection, verification and adaptive strategy switching; How reasoning patterns transfer from a 671B MoE into smaller dense models; What six distilled Qwen and Llama checkpoints score against their base models; Where a 671B open-weight model lands on AIME, MATH-500, GPQA and SWE-bench.
Topics
Sources
This page was written from 3 sources, 2 on domains other than arxiv.org.