Liger Kernel Documentation: Efficient Triton Kernels for LLM Training
by LinkedIn
Patch a Hugging Face model with one call and cut fine-tuning memory with fused Triton kernels for RMSNorm, RoPE, SwiGLU, cross-entropy and DPO-style losses.
Overview
Liger Kernel is an open-source collection of Triton kernels for LLM training, built at LinkedIn and described in the tech report arXiv 2410.10989 and an ICML 2025 workshop paper. The documentation is split into Getting Started, Examples, High Level APIs, Low Level APIs and Contributing. Getting Started shows three ways to use it: load a model with AutoLigerKernelForCausalLM, call a model-specific patch such as apply_liger_kernel_to_llama before instantiating the model, or compose your own model from individual kernels like LigerFusedLinearCrossEntropyLoss. The Low Level APIs page documents the model kernels (RMSNorm, LayerNorm, RoPE, SwiGLU, GeGLU, cross-entropy, fused linear cross-entropy, softmax, sparsemax, multi-token attention and an mHC hyper-connections module), chunked alignment losses for CPO, DPO, ORPO and SimPO, and distillation losses for KL divergence and JSD. The Examples page runs the Hugging Face Trainer, Lightning with DeepSpeed ZeRO-3, Medusa multi-head retraining, Qwen2-VL vision SFT and an ORPO trainer. The project reports about 20% higher multi-GPU throughput and about 60% lower memory on LLaMA 3-8B with FSDP on 8 A100s, and up to 80% memory savings for post-training losses. It depends only on PyTorch and Triton, runs on CUDA and ROCm, and its supported-model list now includes Qwen3.5, Gemma 4, DeepSeek-V4 and GPT-OSS.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~2-3 hours to read the docs and run one example; longer for multi-GPU examples
- Provider
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Load a supported model with AutoLigerKernelForCausalLM so kernels are patched automatically
- ✓Patch a specific architecture with apply_liger_kernel_to_llama and toggle individual kernels by flag
- ✓Use fused linear cross-entropy to avoid materializing full logits and fit longer sequences
- ✓Run DPO, ORPO, CPO and SimPO alignment with chunked losses that cut post-training memory
- ✓Apply KL divergence and JSD distillation losses when training a smaller student model
- ✓Combine Liger with FSDP, DeepSpeed ZeRO-3 or DDP across multiple GPUs
- ✓Patch Megatron-Core with apply_liger_kernel_to_megatron for RMSNorm, cross-entropy and SwiGLU
Highlights
- •One-line patching for Hugging Face models, with forward and backward passes covered by exactness and convergence tests
- •Published benchmarks with stated hardware: LLaMA 3-8B on 4 A100s with FSDP trained about 20% faster with 40% less memory on Alpaca
- •Already integrated into Axolotl, LLaMA-Factory, TRL's SFTTrainer, the Hugging Face Trainer and ms-swift
- •The GitHub repo has about 6.7k stars and 980 commits, and kernels such as mHC from a December 2025 paper show it tracks new architectures
- •Runs on both NVIDIA CUDA and AMD ROCm with only PyTorch and Triton as dependencies
Who It’s For
Best For
- ✓Engineers fine-tuning open-weight LLMs who keep running out of GPU memory at longer context
- ✓ML teams running DPO, ORPO or distillation jobs who want to cut post-training memory
- ✓Platform engineers maintaining Axolotl, TRL or Megatron training stacks
- ✓Developers learning how Triton kernel fusion speeds up transformer training
Prerequisites
- •Experience fine-tuning models with Hugging Face Transformers or a similar trainer
- •Access to NVIDIA or AMD GPUs with torch >= 2.1.2 and triton >= 2.3.0 (ROCm needs torch >= 2.5.0)
- •Familiarity with FSDP or DeepSpeed for the multi-GPU examples
FAQ
What is Liger Kernel Documentation: Efficient Triton Kernels for LLM Training?
The Liger Kernel documentation is LinkedIn's guide to its open-source library of Triton kernels for LLM training and post-training. It is for engineers fine-tuning or aligning open-weight models with Hugging Face, Axolotl, LLaMA-Factory, TRL or Megatron who want longer context and higher throughput on the same GPUs.
Is Liger Kernel Documentation: Efficient Triton Kernels for LLM Training free?
Liger Kernel Documentation: Efficient Triton Kernels for LLM Training is free to access.
What level is Liger Kernel Documentation: Efficient Triton Kernels for LLM Training for?
Liger Kernel Documentation: Efficient Triton Kernels for LLM Training is aimed at a intermediate audience. Recommended background: Experience fine-tuning models with Hugging Face Transformers or a similar trainer, Access to NVIDIA or AMD GPUs with torch >= 2.1.2 and triton >= 2.3.0 (ROCm needs torch >= 2.5.0), Familiarity with FSDP or DeepSpeed for the multi-GPU examples.
How long does Liger Kernel Documentation: Efficient Triton Kernels for LLM Training take?
Expect roughly ~2-3 hours to read the docs and run one example; longer for multi-GPU examples. Most learners work through it at their own pace.
What will I learn from Liger Kernel Documentation: Efficient Triton Kernels for LLM Training?
You'll learn: Load a supported model with AutoLigerKernelForCausalLM so kernels are patched automatically; Patch a specific architecture with apply_liger_kernel_to_llama and toggle individual kernels by flag; Use fused linear cross-entropy to avoid materializing full logits and fit longer sequences; Run DPO, ORPO, CPO and SimPO alignment with chunked losses that cut post-training memory; Apply KL divergence and JSD distillation losses when training a smaller student model; Combine Liger with FSDP, DeepSpeed ZeRO-3 or DDP across multiple GPUs; Patch Megatron-Core with apply_liger_kernel_to_megatron for RMSNorm, cross-entropy and SwiGLU.
Topics
Sources
This page was written from 7 sources, 2 on domains other than linkedin.github.io.