Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps)
by Maxime Labonne
The free, 83k-star roadmap from LLM math to fine-tuning, quantization, RAG, agents and deployment, with runnable Colab notebooks.
Overview
The Large Language Model Course is a free GitHub repository maintained by Maxime Labonne, co-author of the LLM Engineer's Handbook, who states that the course 'will always stay free'. It has more than 83,000 stars and 9,700 forks (checked September 2026), and was last updated in December 2025 and February 2026. The material has three parts. LLM Fundamentals is optional and covers the math for machine learning, Python for machine learning, neural networks and classic NLP. The LLM Scientist track goes through LLM architecture (tokenization, attention, sampling), pre-training (data preparation, distributed training), post-training datasets and synthetic data generation, supervised fine-tuning with LoRA and QLoRA, preference alignment (rejection sampling, DPO, reward models, GRPO and PPO), evaluation, quantization (GGUF and llama.cpp, GPTQ, AWQ, SmoothQuant) and newer trends such as model merging and test-time compute. The LLM Engineer track covers running LLMs, vector storage, retrieval-augmented generation, advanced RAG, agents and agent protocols, inference optimization (Flash Attention, KV cache, speculative decoding), deployment and securing LLMs against prompt hacking and backdoors. Each section links to curated references. The practical side is a set of Colab notebooks and tools: fine-tuning Llama 3.1 with Unsloth, ORPO and DPO fine-tunes, QLoRA and Axolotl runs, GPTQ, GGUF and ExLlamaV2 quantization, LazyMergekit for model merging, AutoQuant and LLM AutoEval.
At a Glance
- Topic
- Fine-Tuning
- Level
- All Levels
- Format
- Course
- Cost
- Free
- Duration
- Self-paced roadmap; weeks to months depending on track, individual notebooks ~1-2 hours each
- Provider
- Maxime Labonne
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Understand transformer architecture, tokenization, attention and decoding strategies behind modern LLMs
- ✓Fine-tune open models with LoRA and QLoRA using Unsloth and Axolotl notebooks
- ✓Align models with preference methods including DPO, ORPO, reward models, GRPO and PPO
- ✓Quantize models to GGUF, GPTQ, AWQ and ExLlamaV2 formats for cheaper inference
- ✓Merge models and build mixture-of-experts variants using mergekit and LazyMergekit
- ✓Build retrieval-augmented generation pipelines with embeddings, vector databases and advanced retrieval
- ✓Apply inference optimizations such as Flash Attention, KV caching and speculative decoding
- ✓Recognise and defend against prompt hacking, data poisoning and backdoor attacks on LLMs
Highlights
- •Among the most-starred LLM learning repositories on GitHub (83.1k stars, 9.7k forks as of September 2026)
- •Two role-based tracks (Scientist vs Engineer) so readers can skip to the half that matches their job
- •Runnable Colab notebooks for fine-tuning, quantization and merging, not just reading lists
- •Written by the author of widely used open tools such as LazyMergekit and AutoQuant; free with no paywall
- •Well received on Hacker News when it launched (178 points in December 2023)
Who It’s For
Best For
- ✓Software engineers moving into LLM engineering who want a structured syllabus
- ✓ML practitioners who want hands-on fine-tuning, alignment and quantization practice on open models
- ✓Self-learners who want a curated reading list instead of scattered blog posts
Prerequisites
- •Working Python; the optional fundamentals section covers the math and ML basics
- •A Google Colab account or GPU access for the fine-tuning and quantization notebooks
FAQ
What is Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps)?
The LLM Course is a free, open-source roadmap on GitHub by Maxime Labonne for developers and ML practitioners who want a structured path into large language models. It splits into an optional fundamentals track, an LLM Scientist track on building and post-training models, and an LLM Engineer track on shipping LLM applications, with Colab notebooks for hands-on fine-tuning and quantization.
Is Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps) free?
Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps) is free to access.
What level is Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps) for?
Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps) is aimed at a all levels audience. Recommended background: Working Python; the optional fundamentals section covers the math and ML basics, A Google Colab account or GPU access for the fine-tuning and quantization notebooks.
How long does Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps) take?
Expect roughly Self-paced roadmap; weeks to months depending on track, individual notebooks ~1-2 hours each. Most learners work through it at their own pace.
What will I learn from Large Language Model Course (LLM Scientist and LLM Engineer Roadmaps)?
You'll learn: Understand transformer architecture, tokenization, attention and decoding strategies behind modern LLMs; Fine-tune open models with LoRA and QLoRA using Unsloth and Axolotl notebooks; Align models with preference methods including DPO, ORPO, reward models, GRPO and PPO; Quantize models to GGUF, GPTQ, AWQ and ExLlamaV2 formats for cheaper inference; Merge models and build mixture-of-experts variants using mergekit and LazyMergekit; Build retrieval-augmented generation pipelines with embeddings, vector databases and advanced retrieval; Apply inference optimizations such as Flash Attention, KV caching and speculative decoding; Recognise and defend against prompt hacking, data poisoning and backdoor attacks on LLMs.
Topics
Sources
This page was written from 4 sources, 2 on domains other than github.com.