OpenEnv: Agentic Execution Environments

OpenEnv is an end-to-end framework for creating, deploying and interacting with isolated execution environments for agentic RL training, exposing the familiar Gymnasium-style reset(), step() and state() API over HTTP so environments can run as containerised backend services during distributed training. The documentation opens with a five-part Getting Started series that needs no GPU — Introduction & Quick Start, Using Environments (connecting, writing policies, running evaluations), Building Environments from scratch, Packaging & Deploying with Docker to the Hugging Face Hub, and Contributing environments back to the Hub — each with a runnable Colab notebook. Beyond that sit topic tutorials that are the real value: an end-to-end walkthrough that connects reasoning_gym, wires it into TRL through environment_factory, fine-tunes with GRPO and pushes the checkpoint to the Hub; MCP-backed environments built with FastMCP, covering when to use step() versus call_tool() and how step_async() fits the lifecycle; composable reward functions built from Gate, WeightedSum, LLMJudge and TrajectoryRubric; Wordle and 2048 GRPO training; evaluation by wrapping an environment in an Inspect AI Task via InspectAIHarness; BrowserGym harness rollouts; and training the real OpenCode and Pi coding agents with TRL's AsyncGRPOTrainer through a transparent proxy that captures per-turn token ids and logprobs while the agent runs its own tool loop. Built-in environments ship for Echo, sandboxed Python coding via smolagents, Chess, Atari and FinRL. The project is BSD-3-licensed, sits at roughly 2.5k GitHub stars, and is governed by a technical committee spanning Meta-PyTorch, Hugging Face, Nvidia, Microsoft, Unsloth, Modal, Prime Intellect, Reflection, Mercor, Fleet AI and RadixArk. The docs explicitly flag the project as experimental with APIs that may still change.

Topic

Agentic

Level

Advanced

Format

Documentation

Cost

Free

Duration

~4-6 hours for the 5-part Getting Started series (no GPU); the GRPO training walkthroughs add a day or more on a GPU

Provider

Hugging Face & Meta PyTorch

Hands-on

Yes — code/exercises

Certificate

None

• ✓Drive an agent environment through the Gymnasium-style reset(), step() and state() HTTP API
• ✓Build a custom environment from scratch and package it as a Docker container
• ✓Deploy an environment to the Hugging Face Hub and share or fork it publicly
• ✓Wire an OpenEnv environment into TRL via environment_factory and fine-tune with GRPO
• ✓Compose reward functions from Gate, WeightedSum, LLMJudge and TrajectoryRubric primitives
• ✓Expose and consume MCP tools inside the step loop, including step_async() and call_tool()
• ✓Evaluate an agent by wrapping the environment in an Inspect AI Task via InspectAIHarness
• ✓Collect reward-labelled teacher rollouts and SFT a student as a GRPO warm start
• ✓Train a loop-owning black-box coding agent using AsyncGRPOTrainer and a transparent token proxy

• •One of the few resources that carries you from 'agent runs' to 'agent trains' with a full GRPO loop
• •Nearly every tutorial ships as a runnable Colab notebook, and the first five need no GPU at all
• •Governed by a cross-vendor technical committee (Meta-PyTorch, Hugging Face, Nvidia, Microsoft, Unsloth, Modal, Prime Intellect) rather than a single company
• •Trains real black-box agents like OpenCode and Pi, not just toy games — the proxy trick for capturing logprobs from an agent that owns its own loop is documented in full
• •MCP environments are first-class, so tools you already expose to Claude or an IDE become training surfaces
• •The docs are candid that the project is experimental and APIs may change, which is more useful than a stability claim that isn't true

This page was written from 3 sources, 1 on domains other than huggingface.co.

1. 1.huggingface.co — indexvendor
2. 2.huggingface.co — indexvendor
3. 3.github.com — OpenEnv