LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs
by LLaMA Factory (hiyouga)
The reference manual for the fine-tuning framework most people actually reach for.
Overview
LLaMA Factory is an Apache-2.0 fine-tuning framework (74k-plus GitHub stars, 9k-plus forks) that unifies the post-training stack behind a single configuration format, one CLI entry point (llamafactory-cli train, chat and export) and a Gradio-based web UI called LLaMA Board. Its documentation splits into three parts. Getting Started covers installation, data preparation and the dataset registry, the WebUI, supervised fine-tuning, model saving with LoRA merging and quantization, inference, evaluation via llamafactory-cli eval, and an FAQ. Advanced covers acceleration methods, tuning algorithms beyond plain LoRA including GaLore and BAdam, distributed training with NativeDDP, DeepSpeed, FSDP and Ray, a Megatron Bridge integration, quantization strategies, trainer types, experiment monitoring through Weights and Biases and SwanLab, a full argument reference, instructions for registering an unsupported model, extras and best practices. A third section covers Ascend NPU backends, with several NPU installation paths, distributed NPU training and NPU performance tuning. The framework supports full-parameter 16-bit tuning, freeze-tuning, LoRA, QLoRA and OFT across LLaMA, Qwen, DeepSeek, Gemma, Mistral, Phi, GLM and InternLM plus vision models such as LLaVA and Qwen-VL, with 2 to 8-bit quantization via AQLM, AWQ, GPTQ, LLM.int8, HQQ and EETQ. The method is published as "LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models" at ACL 2024. Two honest caveats: the docs describe themselves as a work in progress, and independent reviewers note that taking a tuned model to high-traffic production still needs MLOps tooling the framework does not provide.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~4-8 hours from install to a merged LoRA adapter; reference docs are ongoing
- Provider
- LLaMA Factory (hiyouga)
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Format a custom dataset into LLaMA Factory's alpaca or sharegpt schema
- ✓Launch LoRA and QLoRA runs from YAML configs or the command line
- ✓Merge a trained adapter back into base weights and export it quantized
- ✓Choose between DeepSpeed ZeRO, FSDP and Ray for multi-GPU training
- ✓Apply preference optimization methods such as DPO, KTO, ORPO and SimPO
- ✓Evaluate a fine-tuned checkpoint with llamafactory-cli eval before deploying
- ✓Serve a fine-tuned model behind an OpenAI-style API using vLLM
- ✓Register a model architecture the framework does not yet support
Highlights
- •One config format covers pre-training, SFT, reward modelling, PPO, DPO, KTO and ORPO
- •LLaMA Board gives a dropdown UI over exactly the arguments the CLI takes
- •Documents newer LoRA variants and memory-efficient optimizers, not just vanilla LoRA
- •The Ascend NPU section makes it one of the few frameworks with non-CUDA install paths documented
- •Backed by an ACL 2024 paper and a 74k-star repository, so the configs are widely road-tested
Who It’s For
Best For
- ✓Engineers fine-tuning open-weight models on a single GPU or small cluster
- ✓Teams comparing several post-training algorithms without rewriting training code
- ✓Practitioners who want QLoRA on consumer hardware with documented settings
- ✓Researchers needing reproducible YAML configs to share with collaborators
Prerequisites
- •Comfortable with Python, CUDA drivers and installing a PyTorch environment
- •Basic understanding of LoRA, quantization and instruction-tuning dataset formats
- •Access to at least one GPU; QLoRA lowers but does not remove that requirement
FAQ
What is LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs?
Read the Docs documentation for LLaMA Factory, the Apache-2.0 framework that puts pre-training, SFT, reward modelling, PPO, DPO, KTO, ORPO and SimPO behind one CLI and a Gradio web UI for 100-plus language and vision-language models. It walks through installation, dataset formatting, LoRA and QLoRA training, adapter merging, quantized export, vLLM-backed inference, evaluation, and distributed setups using DeepSpeed, FSDP and Ray.
Is LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs free?
LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs is free to access.
What level is LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs for?
LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs is aimed at a intermediate audience. Recommended background: Comfortable with Python, CUDA drivers and installing a PyTorch environment, Basic understanding of LoRA, quantization and instruction-tuning dataset formats, Access to at least one GPU; QLoRA lowers but does not remove that requirement.
How long does LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs take?
Expect roughly ~4-8 hours from install to a merged LoRA adapter; reference docs are ongoing. Most learners work through it at their own pace.
What will I learn from LLaMA Factory Documentation — Unified Efficient Fine-Tuning of 100+ LLMs?
You'll learn: Format a custom dataset into LLaMA Factory's alpaca or sharegpt schema; Launch LoRA and QLoRA runs from YAML configs or the command line; Merge a trained adapter back into base weights and export it quantized; Choose between DeepSpeed ZeRO, FSDP and Ray for multi-GPU training; Apply preference optimization methods such as DPO, KTO, ORPO and SimPO; Evaluate a fine-tuned checkpoint with llamafactory-cli eval before deploying; Serve a fine-tuned model behind an OpenAI-style API using vLLM; Register a model architecture the framework does not yet support.
Topics
Sources
This page was written from 3 sources, 2 on domains other than llamafactory.readthedocs.io.