MLX LM — Run and Fine-Tune LLMs on Apple Silicon

mlx-lm (6.5k GitHub stars, 851 commits) is the LLM package built on Apple's MLX array framework from its machine-learning research team, exposing both a CLI and a Python API. Generation covers mlx_lm.generate and an interactive chat, streaming output, prompt caching for repeated long prefixes, model quantization with upload back to the Hugging Face Hub, and distributed inference and fine-tuning through mx.distributed; thousands of ready-converted models are available from the mlx-community organization. The fine-tuning guide (LORA.md) is the substantive part: mlx_lm.lora trains LoRA by default, with QLoRA against a pre-quantized base, DoRA, or full-weight fine-tuning, and mlx_lm.fuse merges adapters back into the base model, with GGUF export for select architectures. Datasets are JSONL in four documented shapes — chat, completions, raw text, and tools for function-calling data — loaded locally or from the Hub, with prompt masking to exclude instruction tokens from the loss. Memory guidance is concrete: drop batch size from the default 4 to 1-2, fine-tune 4-8 layers instead of the default 16, enable --grad-checkpoint, use gradient accumulation, or shorten sequences; the docs report roughly 250 tokens per second on an M1 Max with 32GB at batch size 1 with minimal layers. Validation loss is reported during training, with Weights & Biases and SwanLab integration. Documented model families include Mistral, Llama, Phi2, Mixtral, Qwen2, Gemma, OLMo, MiniCPM and InternLM2.

Topic

Fine-Tuning

Level

Intermediate

Format

Documentation

Cost

Free

Duration

~3-4 hours to run your first LoRA fine-tune end to end and fuse the adapter

Provider

Apple (ml-explore)

Hands-on

Yes — code/exercises

Certificate

None

• ✓Run a LoRA, QLoRA, DoRA or full fine-tune from one mlx_lm.lora command
• ✓Format training data as JSONL in chat, completions, text or tools shape
• ✓Quantize a Hugging Face model to 4-bit and push it back to the Hub
• ✓Trim memory with batch size, layer count, gradient checkpointing and accumulation
• ✓Fuse a trained adapter into base weights and export GGUF where supported
• ✓Serve generations from the CLI or Python API with streaming and prompt caching
• ✓Track validation loss and log runs to Weights & Biases or SwanLab
• ✓Use prompt masking so instruction tokens are excluded from the training loss
• ✓Scale across machines with distributed inference and fine-tuning via mx.distributed

• •Fine-tuning on a 16GB Mac is realistic with a 4-bit 8B model — no cloud GPU, no data egress
• •Independent benchmarking on an M2 Ultra found MLX had the highest sustained generation throughput of the Apple-silicon engines tested
• •Apple's unified memory means one shared pool, which is what makes local training feasible at all
• •Written and maintained by Apple's own machine-learning research team, not a community wrapper
• •Covers the whole loop — quantize, train, evaluate, fuse, serve — in one package
• •Memory-reduction advice is specific numbers and flags, not general hand-waving

This page was written from 4 sources, 2 on domains other than github.com.

1. 1.github.com — mlx lmvendor
2. 2.github.com — LORA.mdvendor
3. 3.kdnuggets.com — fine tuning language models on apple silicon with mlx
4. 4.arxiv.org — 2511.05502