Papers
arxiv:2608.07545

DarwinX: Evolving Agent Harnesses Through Natural Selection

Published on Jul 31
· Submitted by
Yifan Zhang
on Aug 14
#2 Paper of the day
Authors:
,
,
,
,
,
,
,
,
,
,
,

Abstract

DarwinX evolves agent harnesses via population selection with frozen models, improving verified performance across benchmarks without benchmark-specific patches.

An LLM agent's capability depends not only on model weights but on its harness: prompts, tools, skills, and control flow. Self-improvement loops already edit harnesses, yet single-lineage search is path-dependent and local wins often regress other tasks. We introduce DarwinX, which treats self-evolution as selection over a population of harnesses with the model frozen: a preserve-and-extend contract admits only variants that extend coverage without regressing, an archive keeps alternative lineages for recombination, and failure-, teacher-, and self-derived evidence share one edit interface. Fitness comes from each benchmark's own verifier: no gold solutions, no hand-picked winners. Across four benchmarks that progressively separate the evolution signal from the test, one loop adds about 17 points on average: Terminal-Bench 2.1 rises +7.7 to 83.2% on a matched base and to the verified frontier at 84.7% on a stronger one; TerminalWorld's held-out split reaches 68.3%, ahead of every off-the-shelf agent; WebArena-Infinity real-task pass@1 rises from 43.5% to 93.0% audit-clean; and a Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified. What evolves is general agent competence, not benchmark-specific patches, so it survives changes of task, verifier, and base model. A frozen model need not be a fixed agent: harness selection turns evaluation compute into durable capability.

Community

Paper submitter

We freeze the base model and evolve only the harness — prompts, tools, skills, control flow — by label-free natural selection: variants are scored on measured fitness (avg@k, no gold solutions), survivors are kept, and complementary ones are merged.

Matched-model, harness alone:

  • Terminal-Bench 2.1 75.5 → 83.2 avg@5 on GPT-5.5, past Codex at 83.1 (84.7 on GPT-5.6 Sol)
  • TerminalWorld 61.0 → 68.3 pass@1 on Opus 4.8, 41 held-out tasks, past Claude Code at 65.9
  • WebArena-Infinity 43.5 → 93.0 audit-clean pass@1 on 1,260 real tasks
  • SWE-bench Verified 80.8 → 84.2 by zero-shot transfer of the terminal harness

Two results we did not expect. The in-loop proxy saturates (0.505 → 1.000) while held-out pass@1 is 68.3 — a 31.7-point gap — and the variant that best fits the proxy is not the best generalizer, so keeping a population rather than following the single best lineage is what converts an overfit proxy into held-out gain. And we audited every WebArena-Infinity trajectory for validity: the base agent's raw 53.0 falls to 43.5 while ours goes 94.4 → 93.0, so the gap widens under scrutiny instead of narrowing.

Project page with interactive figures, every number regenerated from the run artifacts: https://huggingface.co/spaces/CoderDoge/darwinx

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2608.07545
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2608.07545 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2608.07545 in a dataset README.md to link it from this page.

Spaces citing this paper 1

Collections including this paper 1