AgenticFine-TuningModels

Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

by Hugging Face

AdvancedGuideFree~25 min read; reproducing a run needs multi-GPU access (the authors used 2x B200 per adapter)

Hugging Face's full recipe for RL-training a small LoRA per coding-agent harness, with the tasks, trainer settings and benchmark results.

Start LearningAdded Nov 21, 2026 · Updated Nov 21, 2026

Overview

This Hugging Face blog post, written by a team that includes Younes Belkada, Lewis Tunstall, Lysandre Debut and Ben Burtenshaw and published November 20, 2026, documents how the harness-hub project RL-trained LoRA adapters for Qwen3.8-27B inside three coding-agent harnesses: pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6. The harness runs unmodified while a proxy between it and vLLM records the exact tokens the trainer learns from. The post then covers task generation. Candidate tasks come from SWE-smith, SWE-Gym via TaskTrove, and Tasksmith tasks built from merged PRs. They are checked with `openenv validate` (858 of 1,127 Hub candidates passed) and filtered by difficulty against the base model, which leaves 171 qualified tasks out of 442 scored. Next come four short ablation runs on Terminal-Bench 2.1, training with TRL's `AsyncGRPOTrainer` (LoRA on all linear layers, learning rate 2e-5 after a sweep, verifier reward plus a 0.1 tool-efficiency bonus), and the safeagents proxy, which checked 1.3M tool calls during training and blocked 647. A section on merging the three adapters into a rank-96 super-adapter compares scaling weights and picks 1/2. In the authors' own evaluation, the pi adapter lifts the hard 51-task SWE-bench Pro subset from 40.0% to 64.0% and Terminal-Bench 2.1 from 40.0% to 56.5% with thinking off. The post closes on an open question about "reasoning leakage" in non-thinking mode. Models, datasets, the trajectories bucket and the Repo2RLEnv and safeagents code are all released.

At a Glance

Topic
Agentic
Level
Advanced
Format
Guide
Cost
Free
Duration
~25 min read; reproducing a run needs multi-GPU access (the authors used 2x B200 per adapter)
Provider
Hugging Face
Hands-on
Yes — code/exercises
Certificate
None

What You’ll Learn

  • ✓How to RL-train inside an unmodified agent harness by proxying the tokens between the harness and vLLM
  • ✓How to build verifiable coding tasks from SWE-smith, SWE-Gym and merged PRs using Repo2RLEnv and Tasksmith
  • ✓How to validate RL tasks with openenv validate and drop tasks the base model always or never solves
  • ✓How to configure TRL's AsyncGRPOTrainer with LoRA, a token budget and a tool-efficiency reward bonus
  • ✓How the safeagents proxy blocks reward-hacking tool calls during training and what it caught in practice
  • ✓How to merge several harness LoRAs into one super-adapter with PEFT concatenation and choose a scaling weight
  • ✓How to evaluate harness adapters on Terminal-Bench 2.1, SWE-bench Pro and out-of-distribution AutomationBench tasks

Highlights

  • •Gives exact numbers at every stage, including the learning-rate sweep, task-filter counts and failed settings, so the recipe can be checked and rerun
  • •Everything is released: the adapter collection, a merged super-adapter, an OpenCode GGUF, the RL task dataset, trajectories, and the Repo2RLEnv (about 800 GitHub stars) and safeagents code
  • •Reports where the method does not help: the Hermes adapter drops to 17.7% on out-of-distribution AutomationBench against a 23.0% base
  • •Raises its own open question about reasoning leakage in non-thinking mode instead of presenting the gains as settled

Who It’s For

Best For

  • ✓ML engineers post-training open-weight models for a specific coding agent or internal harness
  • ✓Teams weighing per-harness LoRA adapters against fine-tuning one general agent model
  • ✓Researchers building verifiable RL environments and reward-hacking guards for agentic training

Prerequisites

  • •Working knowledge of GRPO-style reinforcement learning and LoRA fine-tuning with TRL and PEFT
  • •Familiarity with Docker, vLLM serving and coding-agent benchmarks such as SWE-bench and Terminal-Bench
  • •Access to multi-GPU hardware (H200 or B200 class) to reproduce training rather than just read it

FAQ

What is Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost?

A long technical write-up from Hugging Face, published November 20, 2026, on training LoRA "harness adapters" with GRPO inside unmodified coding-agent harnesses (pi, OpenCode and Hermes Agent). It is written for engineers who post-train open models for agent use and want a reproducible recipe covering task generation, validation, async GRPO, adapter merging and evaluation.

Is Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost free?

Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost is free to access.

What level is Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost for?

Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost is aimed at a advanced audience. Recommended background: Working knowledge of GRPO-style reinforcement learning and LoRA fine-tuning with TRL and PEFT, Familiarity with Docker, vLLM serving and coding-agent benchmarks such as SWE-bench and Terminal-Bench, Access to multi-GPU hardware (H200 or B200 class) to reproduce training rather than just read it.

How long does Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost take?

Expect roughly ~25 min read; reproducing a run needs multi-GPU access (the authors used 2x B200 per adapter). Most learners work through it at their own pace.

What will I learn from Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost?

You'll learn: How to RL-train inside an unmodified agent harness by proxying the tokens between the harness and vLLM; How to build verifiable coding tasks from SWE-smith, SWE-Gym and merged PRs using Repo2RLEnv and Tasksmith; How to validate RL tasks with openenv validate and drop tasks the base model always or never solves; How to configure TRL's AsyncGRPOTrainer with LoRA, a token budget and a tool-efficiency reward bonus; How the safeagents proxy blocks reward-hacking tool calls during training and what it caught in practice; How to merge several harness LoRAs into one super-adapter with PEFT concatenation and choose a scaling weight; How to evaluate harness adapters on Terminal-Bench 2.1, SWE-bench Pro and out-of-distribution AutomationBench tasks.

Topics

harness adaptersGRPOLoRAcoding agentsTRLreinforcement learning

Sources

This page was written from 5 sources, 3 on domains other than huggingface.co.

  1. 1.huggingface.co — harness adapters at scalevendor
  2. 2.huggingface.co — harness adaptersvendor
  3. 3.github.com — harness adapters at scale.md
  4. 4.github.com — Repo2RLEnv
  5. 5.github.com — safeagents