Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models
by Oumi
One YAML-driven CLI for the whole model lifecycle: SFT, LoRA, DPO and GRPO training, evaluation, LLM judges, data synthesis and deployment.
Overview
The Oumi documentation covers Oumi, an Apache-2.0 platform with roughly 9.4k GitHub stars for training, evaluating and deploying models from 10M to 405B parameters. It is organised into Getting Started (quickstart, installation, core concepts, tutorials), User Guides (training, inference, deploying models, evaluation, dataset analysis, LLM judge, running jobs on clusters and Kubernetes, data synthesis, hyperparameter tuning, quantization, customization and an MCP server), Resources (supported and custom models, dataset formats for SFT, pre-training, preference tuning and vision-language, and 200+ recipes) and full API and CLI references. The training guide explains a single YAML config with model, data, training, PEFT and FSDP sections and a trainer_type switch: TRL_SFT for text and vision-language SFT, OUMI for pretraining with streaming and packing, TRL_DPO for preference optimization without a separate reward model, and TRL_GRPO or VERL_GRPO for reasoning with verifiable rewards. The same config runs locally with oumi train, across GPUs with oumi distributed torchrun using DDP or FSDP, or on cloud providers such as AWS, Azure, GCP and Lambda through oumi launch, which builds on SkyPilot. Inference runs on vLLM, SGLang or Transformers, and six Colab notebooks cover a feature tour, LoRA fine-tuning, model distillation, evaluation, remote training and LLM-as-a-judge data curation. The project is current: v0.8 (May 2026) added oumi deploy for shipping models to inference endpoints, an MCP server and Transformers v5 support, and v0.9 (September 2026) added tool-call training, executable tool environments and RL over those environments.
At a Glance
- Topic
- Fine-Tuning
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~2-3 hours for the quickstart and Colab tour notebooks; ongoing reference while running training jobs
- Provider
- Oumi
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Install Oumi and fine-tune SmolLM-135M end to end from the quickstart YAML recipe
- ✓Configure the model, data, training, PEFT/LoRA and FSDP sections of an Oumi training config
- ✓Choose between TRL_SFT, TRL_DPO, TRL_GRPO, VERL_GRPO and OUMI pretraining trainers and prepare each dataset format
- ✓Scale from one GPU to multi-GPU DDP or FSDP with oumi distributed torchrun, and launch cloud jobs with oumi launch
- ✓Run interactive and batch inference through vLLM, SGLang or Transformers engines with the oumi infer command
- ✓Evaluate models on standard benchmarks and custom evals, and curate training data with built-in LLM-as-a-judge
- ✓Generate synthetic training data with oumi synth and search hyperparameters with oumi tune
- ✓Quantize a fine-tuned model with AWQ and ship it to a dedicated inference endpoint with oumi deploy
Highlights
- •A single CLI covers train, evaluate, judge, infer and deploy, instead of stitching TRL, evaluation harnesses and vLLM scripts together by hand
- •200+ ready-made recipes across 100+ models, including Qwen, Llama 3 and 4, Gemma 3 and 4, DeepSeek-R1, Phi, OLMo and vision-language models
- •Shipping fast in 2026: v0.8 (May) added oumi deploy, an MCP server and Transformers v5 plus TRL 0.30+; v0.9 (September) added tool-call training and RL over executable tool environments
- •Six Colab notebooks (tour, LoRA fine-tuning, distillation, evaluation, remote training, LLM-as-a-judge) let you try it before committing GPU hardware
- •Honest caveat: Hacker News launch posts drew little discussion, so community troubleshooting is thinner than for Unsloth or Axolotl; expect to rely on the docs and GitHub issues
Who It’s For
Best For
- ✓ML engineers who want one reproducible, config-driven workflow for SFT, preference tuning and GRPO
- ✓Teams fine-tuning small open models for a domain who need built-in evaluation and LLM judges
- ✓Practitioners moving between a laptop, a multi-GPU node and cloud clusters who want the same config to run on each
Prerequisites
- •Python 3.10+ and comfort with the command line and YAML configuration files
- •Basic understanding of fine-tuning concepts such as SFT and LoRA, and of Hugging Face models and datasets
- •A CUDA GPU or cloud credits for anything beyond the 135M-parameter quickstart
FAQ
What is Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models?
Oumi's documentation is the official guide to an Apache-2.0 open-source platform for the whole foundation-model lifecycle, written for ML engineers who fine-tune open models. It teaches one YAML-and-CLI workflow for SFT, LoRA, DPO and GRPO training, evaluation, LLM-as-a-judge, data synthesis, inference and deployment, so you can go from a 135M-parameter quickstart to multi-GPU or cloud training runs.
Is Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models free?
Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models is free to access.
What level is Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models for?
Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models is aimed at a intermediate audience. Recommended background: Python 3.10+ and comfort with the command line and YAML configuration files, Basic understanding of fine-tuning concepts such as SFT and LoRA, and of Hugging Face models and datasets, A CUDA GPU or cloud credits for anything beyond the 135M-parameter quickstart.
How long does Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models take?
Expect roughly ~2-3 hours for the quickstart and Colab tour notebooks; ongoing reference while running training jobs. Most learners work through it at their own pace.
What will I learn from Oumi Documentation — Open-Source Platform to Train, Evaluate and Deploy Foundation Models?
You'll learn: Install Oumi and fine-tune SmolLM-135M end to end from the quickstart YAML recipe; Configure the model, data, training, PEFT/LoRA and FSDP sections of an Oumi training config; Choose between TRL_SFT, TRL_DPO, TRL_GRPO, VERL_GRPO and OUMI pretraining trainers and prepare each dataset format; Scale from one GPU to multi-GPU DDP or FSDP with oumi distributed torchrun, and launch cloud jobs with oumi launch; Run interactive and batch inference through vLLM, SGLang or Transformers engines with the oumi infer command; Evaluate models on standard benchmarks and custom evals, and curate training data with built-in LLM-as-a-judge; Generate synthetic training data with oumi synth and search hyperparameters with oumi tune; Quantize a fine-tuned model with AWQ and ship it to a dedicated inference endpoint with oumi deploy.
Topics
Sources
This page was written from 6 sources, 3 on domains other than oumi.ai.
- 1.oumi.ai — indexvendor
- 2.oumi.ai — trainvendor
- 3.oumi.ai — training methodsvendor
- 4.github.com — oumi
- 5.github.com — releases
- 6.hn.algolia.com — search