Recursive Self-Improvement for Agent Harnesses

Summary & Methodology Analysis

This paper addresses how to improve user-constructed harnesses in a task-specific manner to generate higher-quality execution traces for foundation models while remaining computationally lightweight and requiring only a few update iterations. The baseline coding agent, such as Claude Sonnet 4.6, Opus 4.7, or Opus 4.8, interacts with a task prompt and a harness using standard test-time reasoning and execution loops. Pairwise evaluation protocols use an LLM evaluator to compare two generated outputs based on specified criteria. The novel contribution of this work is representing the harness as a prompt-level specification of the agent loop, including agent roles, instructions, communication contracts, and workflow hops.

Building on this representation, the authors introduce Recursive Harness Self-Improvement, which iteratively refines the harness using pairwise feedback over its own revision history via trajectory-local self-comparison. This technique optimizes the implicit trajectory-local objective by replacing the broad competitor distribution with a local competitor distribution concentrated on the previous harness. Through this targeted optimization, the framework achieves significant computational savings. Specifically, for opus-4.8, Recursive Harness Self-Improvement reduces inference cost by up to 60 percent relative to the ultracode baseline.

The paper does not specify any limitations for this method. The evaluation spans a wide range of models and datasets, including Claude Sonnet 4.6, Claude Opus 4.7, Claude Opus 4.8, Meta-Harness, AutoHarness, TTHE, ADAS, GPTSwarm, AFlow, AlphaEvolve, ShinkaEvolve, OPRO, TextGrad, DSPy, GEPA, EPIC-KITCHENS, Something-Something V2, Dow 30, SPY, FRED, UniProt, Swiss-Prot, PDB, SIFTS, text-embedding-3-large, and all-mpnet-base-v2. The paper does not report additional performance or memory figures beyond the stated cost reduction.