Decision Models in Hugging Face
by Hugging Face
Get calibrated probabilities over fixed answers from an LLM, served through Transformers or llama.cpp, instead of parsing generated text.
Overview
Decision Models in Hugging Face was published on November 10, 2026 by Aritra Roy Gosthipaty, Lysandre, Pedro Cuenca and Tom Aarsen. It defines a decision model as one that reads a state plus a set of questions and returns a probability distribution over the allowed answers in one pass. The post explains why that beats asking a generative LLM for a label, then contrasts decision models with fixed-label classifiers, which need a retrained head for new labels, and with NLI-style zero-shot classifiers such as bart-large-mnli. The interface section covers the three question primitives: choice (a distribution over options), score (the expected value of an ordered scale) and noul (a yes/no Bernoulli probability), plus a confidence value normalised by option count. The ecosystem half is practical. It shows pipeline('decision') and AutoModelForDecision in Transformers, serving through transformers serve and through llama.cpp for GGUF models behind a shared /v1/systemone API, finding models via the Hub's decision-model tag, and training with Trainer, with TRL support still in an open pull request at publication. It links Unsloth's recipe, which converts Qwen3.5, Gemma 4 E4B or Llama 3.2 3B into decision models using 4-bit LoRA and reports test accuracy and VRAM from 2.5 GB to 14.4 GB per model.
At a Glance
- Topic
- Models
- Level
- Intermediate
- Format
- Guide
- Cost
- Free
- Duration
- ~30 min read; ~1 hour more to fine-tune a small model with the linked Unsloth notebook
- Provider
- Hugging Face
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓What a decision model is and how it differs from text-generating LLMs for labeling tasks
- ✓Why decision models avoid the retrained heads of classifiers and the label hypotheses of zero-shot NLI
- ✓How to write a state and typed questions using the choice, score and noul primitives
- ✓How to read the confidence value and per-option probabilities a decision model returns
- ✓How to run decision models with the Transformers decision pipeline and AutoModelForDecision
- ✓How to serve GGUF decision models with llama.cpp behind the shared /v1/systemone API
- ✓How to fine-tune a small open LLM into a decision model with Trainer or the Unsloth LoRA recipe
Highlights
- •Covers a new model type end to end: concept, interface, serving in two runtimes, Hub discovery and training
- •Written by Hugging Face Transformers and Sentence Transformers maintainers, with runnable pipeline and curl examples
- •Companion Unsloth recipe reports per-model test accuracy (77-81%), VRAM (2.5-14.4 GB) and training times of 10-49 minutes
- •Honest about maturity: TRL support was an open pull request at publication, so expect APIs to move
Who It’s For
Best For
- ✓Developers using LLMs for ticket triage, intent routing or content moderation
- ✓Teams that need calibrated probabilities from an LLM for thresholds and fallbacks
- ✓Engineers serving small local models with llama.cpp who want structured outputs without constrained decoding
Prerequisites
- •Comfort with the Hugging Face Transformers pipeline API and the Hub
- •Basic understanding of classification, softmax probabilities and calibration
- •For fine-tuning: LoRA basics and a GPU with at least 4 GB of VRAM, or a free Colab notebook
FAQ
What is Decision Models in Hugging Face?
A Hugging Face guide to decision models, LLM-based models that take a state and typed questions and return probabilities over predefined answers without generating text. It is for developers who use an LLM for routing, triage or classification and want calibrated, structured outputs. Afterwards you can serve, discover and fine-tune decision models.
Is Decision Models in Hugging Face free?
Decision Models in Hugging Face is free to access.
What level is Decision Models in Hugging Face for?
Decision Models in Hugging Face is aimed at a intermediate audience. Recommended background: Comfort with the Hugging Face Transformers pipeline API and the Hub, Basic understanding of classification, softmax probabilities and calibration, For fine-tuning: LoRA basics and a GPU with at least 4 GB of VRAM, or a free Colab notebook.
How long does Decision Models in Hugging Face take?
Expect roughly ~30 min read; ~1 hour more to fine-tune a small model with the linked Unsloth notebook. Most learners work through it at their own pace.
What will I learn from Decision Models in Hugging Face?
You'll learn: What a decision model is and how it differs from text-generating LLMs for labeling tasks; Why decision models avoid the retrained heads of classifiers and the label hypotheses of zero-shot NLI; How to write a state and typed questions using the choice, score and noul primitives; How to read the confidence value and per-option probabilities a decision model returns; How to run decision models with the Transformers decision pipeline and AutoModelForDecision; How to serve GGUF decision models with llama.cpp behind the shared /v1/systemone API; How to fine-tune a small open LLM into a decision model with Trainer or the Unsloth LoRA recipe.
Topics
Sources
This page was written from 2 sources, 1 on domains other than huggingface.co.