Google Research RRSI Lets Agents Rewrite Their Harness Without Overfitting the Evolve Set

Google Cloud AI Research, with collaborators at UNC-Chapel Hill, Stanford, and Washington University in St. Louis, released RRSI—Regularized Recursive Self-Improvement of Agent Harnesses—as Apache-2.0 research code that lets an LLM agent rewrite prompts, tools, memory, control flow, and sub-agents while the backbone weights stay frozen. The paper is on arXiv as 2609.24972, the project site is regularized-rsi.com, and the implementation lives at github.com/google-research/rrsi. The repository README and paper are the grounding primaries. The authors are explicit that this is not an officially supported Google product.

The problem RRSI targets is familiar to anyone who has watched an agent “improve” on a fixed evolve set and then stall elsewhere. Modern agents are systems: a frozen policy wrapped in a harness of prompts, control flow, tool interfaces, memory, and context management. Recent methods automate harness edits by proposing candidates, scoring them on an evolve split, and keeping winners. That loop can memorize the suite—benchmark-specific fitting, noise chasing, and complexity accumulation that inflate evolve-set scores while out-of-distribution gains shrink or vanish. RRSI keeps the edit space open and regularizes how the search moves through it.

On the proposal side, an annealed edit budget follows a cosine schedule: early rounds may bundle several coordinated edits; late rounds allow a single attributable change. Every candidate is logged with component, hypothesis, diff, score change, and cost change so the proposer reads a ledger instead of re-testing falsified ideas. When progress stalls inside a noise band, budget shifts toward components the run has never exercised. On the selection side, a leakage critic rejects task names, entities, answers, or evolve-suite-specific logic before scoring. A noise-adjusted floor, estimated from repeated evaluations of the unchanged base harness, blocks gains that sit inside evaluation variance. A cost rule requires extra policy tokens to be paid for by measured gain. Components that stop helping become pruning targets. The team frames these as analogies to classical regularizers: edit budget as L0-style sparsity, pruning as Lasso/L1-style deletion, and the cost rule as Ridge/L2-style shrinkage of aggregate resource footprint.

Results in the paper and README use Claude Opus 4.8 as the frozen policy across three domain instances. On coding, Terminal-Bench 2.1 (evolve) rises from 74.2% to 80.2%, while SWE-bench Verified—never used for selection—rises from 82.0% to 83.8%. On the agentic workspace instance, Harvey LAB moves from 89.4 to 90.5 on the evolve split and from 86.9 to 89.2 on an in-distribution held-out split; JobBench, GDPval, and APEX-Agents gain +4.7, +3.5, and +3.7 points out of distribution. Engineering design shows EngDesign (evolve) +4.9 and Frontier-Eng +4.3 Medal points. All six held-out splits improve. With Gemini 3.5 Flash as the frozen policy on coding, Terminal-Bench 2.1 climbs from 64.6 to 78.7 and SWE-bench Verified from 76.8 to 79.0. Against Meta-Harness, AHE, TTHE, and HarnessX on the same starting harness and candidate budget, RRSI posts a smaller Harvey LAB evolve gain than several rivals but the only out-of-distribution average clearly above the unevolved harness H0 (43.6 versus 39.7). The project site also reports about 36% fewer policy tokens per trial versus unregularized evolution on the agentic workspace instance (README tables cite 2.42M versus 3.80M tokens per trial).

Operationally, the code needs Python 3.10+, accepts any LiteLLM model string, and defaults to Claude Opus 4.8 on Vertex AI. Each round drafts two candidates in separate git worktrees, screens them, evaluates both, and fast-forwards an evolve/ branch to the winner. Domains plug in through an adapter module; shipped instances cover coding (Docker + harbor for Terminal-Bench), Harvey LAB workspace tasks, and EngDesign. Smoke, baseline, run, and status commands share one CLI shape: python3 rrsi.py --domain <coding|workspace|eng> ….

RRSI is rare among harness self-improvement drops in treating overfitting as the product risk—and shipping the regularizers, benchmarks, and Apache-2.0 code to inspect the claim.