Evolving Better Agent Skills Safely

Summary & Methodology Analysis

The paper addresses limitations in existing LLM agent skill evolution methods, which traditionally suffer from unverified forward updates that accept diagnostic edits without measuring real effects, alongside unregulated skill growth that accumulates redundant, conflicting, or task-specific heuristics over time. To solve this, SkillProx runs a closed-loop forward diagnostic evolution where candidate skill updates are re-executed on the same task batch, committing them only if they satisfy an outcome-grounded performance gate. Regressive updates are retained as rolled-back states, and both measured outcomes and rejected edit directions feed back into subsequent diagnoses. Furthermore, the accumulated skill is decomposed into auditable knowledge units, and their contributions are estimated using a frozen leave-one-out utility audit. Validation-gated proximal refinement is then applied to perform structured consolidation, demotion, or removal edits while preserving structural validity and validation task performance.

In terms of empirical performance, SkillProx improves average accuracy by 3.0 percentage points over the strongest gradient-based baseline across in-distribution and out-of-distribution benchmarks. Additionally, utility-aware consolidation reduces skill size by 3.12% while improving OJ hard accuracy from 46% to 54%. The evaluation leverages multiple models and benchmarks, including SpreadsheetBench Verified, WikiTableQuestions, HiTab, Qwen3.5-4B, Qwen3.5-27B, Qwen3.6-27B, SkillsBench, Trace2Skill, EvoSkill, CoEvoSkills, SkillComposer, SkillOpt, and SkillGrad.

The paper highlights notable limitations in current approaches. One-shot skill synthesis provides uneven or even negative gains across heterogeneous tasks because a skill constructed once cannot anticipate diverse future situations. Additionally, the forward gate constrains only the current training batch and does not imply monotonic performance across iterations, on the validation split, or on the test set. The paper does not specify precise hardware requirements, dollar costs, or exact latency figures beyond the performance improvements and size reductions noted.