Ray Documentation — Distributed Compute for AI and Python
by Ray Project
One API for distributed data, training, tuning and serving — the compute layer under most in-house AI platforms.
Overview
Ray (Apache-2.0, 43.5k GitHub stars, maintained by the ray-project organisation with Anyscale as commercial backer) is documented as a layered stack, and the docs are organised the same way. Ray Core is the foundation and defines three primitives: tasks (stateless functions scheduled anywhere in the cluster), actors (stateful worker processes) and objects (immutable values in a shared distributed object store) — enough to parallelise arbitrary Python. Above it sit five AI libraries, each with its own guide, API reference and example gallery: Ray Data for streaming datasets and large-scale batch inference; Ray Train for distributed training, wrapping PyTorch DDP and FSDP, Lightning and Hugging Face behind a ScalingConfig; Ray Tune for hyperparameter search with schedulers, checkpointing and early stopping across thousands of trials; Ray Serve for model serving with autoscaling, multi-model composition and FastAPI ingress, including a Ray Serve LLM module that exposes OpenAI-compatible endpoints; and RLlib for distributed reinforcement learning. A separate Ray Clusters section covers production deployment on AWS, GCP, Azure and Kubernetes via KubeRay, plus autoscaling and observability. A use-cases index reorganises the same material by job to be done — LLMs and generative AI, batch inference, model serving, hyperparameter tuning, distributed training, reinforcement learning, ML platform, end-to-end ML workflows and large-scale workload orchestration. The docs track the current release closely: Ray 2.56, released 30 June 2026, reworked Ray Data's memory handling, taking the team's reference pipeline from 300+ out-of-memory failures to zero and cutting end-to-end runtime from 1,055s to 447s.
At a Glance
- Topic
- ML
- Level
- Intermediate
- Format
- Documentation
- Cost
- Free
- Duration
- ~4-6 hours for the core library tutorials, then ongoing reference
- Provider
- Ray Project
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Parallelise Python with Ray Core tasks, actors and the distributed object store
- ✓Run large-scale batch inference over streaming datasets with Ray Data
- ✓Scale PyTorch and Hugging Face training across GPUs using Ray Train and ScalingConfig
- ✓Search hyperparameters at scale with Ray Tune schedulers, checkpointing and early stopping
- ✓Deploy composed multi-model inference graphs with autoscaling using Ray Serve
- ✓Serve LLMs behind OpenAI-compatible endpoints with the Ray Serve LLM module
- ✓Run production Ray clusters on Kubernetes or a cloud provider with autoscaling
Highlights
- •One runtime spans data prep, training, tuning and serving — no glue code between stages
- •43.5k GitHub stars and the compute layer underneath many in-house ML platforms
- •Ray Serve LLM exposes OpenAI-compatible endpoints, so it drops into existing clients
- •Ray 2.56 (June 2026) fixed the OOM and object-spilling failures Ray Data users reported
- •Every library ships a runnable example gallery, not just an API reference
Who It’s For
Best For
- ✓ML platform engineers building shared training and serving infrastructure
- ✓AI engineers whose job has outgrown a single GPU or a single machine
- ✓Teams running scheduled batch inference over millions of rows
Prerequisites
- •Confident Python, including decorators and asynchronous execution
- •Experience training or serving a model on one machine first
- •Basic cloud or Kubernetes familiarity for the Ray Clusters section
FAQ
What is Ray Documentation — Distributed Compute for AI and Python?
Ray is the open-source distributed runtime AI platform teams use to move a workload from one laptop to a GPU cluster without rewriting it, and these docs cover the whole stack. Ray Core provides tasks, actors and objects; Ray Data, Train, Tune and Serve build batch inference, distributed training, hyperparameter search and model serving on top. Work through them and you can run multi-node inference and training jobs yourself.
Is Ray Documentation — Distributed Compute for AI and Python free?
Ray Documentation — Distributed Compute for AI and Python is free to access.
What level is Ray Documentation — Distributed Compute for AI and Python for?
Ray Documentation — Distributed Compute for AI and Python is aimed at a intermediate audience. Recommended background: Confident Python, including decorators and asynchronous execution, Experience training or serving a model on one machine first, Basic cloud or Kubernetes familiarity for the Ray Clusters section.
How long does Ray Documentation — Distributed Compute for AI and Python take?
Expect roughly ~4-6 hours for the core library tutorials, then ongoing reference. Most learners work through it at their own pace.
What will I learn from Ray Documentation — Distributed Compute for AI and Python?
You'll learn: Parallelise Python with Ray Core tasks, actors and the distributed object store; Run large-scale batch inference over streaming datasets with Ray Data; Scale PyTorch and Hugging Face training across GPUs using Ray Train and ScalingConfig; Search hyperparameters at scale with Ray Tune schedulers, checkpointing and early stopping; Deploy composed multi-model inference graphs with autoscaling using Ray Serve; Serve LLMs behind OpenAI-compatible endpoints with the Ray Serve LLM module; Run production Ray clusters on Kubernetes or a cloud provider with autoscaling.
Topics
Sources
This page was written from 3 sources, 2 on domains other than docs.ray.io.