cs.AIAug 27, 2026classified

Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification

Jinghan Xu, Yikai Zhang, Aili Chen, Weiyuan Li, Jiaqing Liang, Deqing Yang

cs.AI

Paper Guide Brief

Reading Brief

The paper introduces HarnessLens, a budget-aware framework for automated evolution of language-model agent harnesses. It addresses the inefficiency of existing propose-and-verify methods that evaluate all candidate modifications on a fixed task set, wasting rollouts and obscuring regressions. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Experiments across three agent harnesses and four benchmarks show average held-out performance improvements of 7.6-13.6% with substantially less evaluation budget than baselines.

Central Claim

Introduces HarnessLens, a budget-aware framework for automated harness evolution that uses behavior-aware verification with an attributable-evidence gate to selectively verify candidate modifications on relevant tasks, improving sample efficiency and reliability.

Contribution

Why It Matters

This contribution matters because it demonstrates that directing verification toward behavior-relevant tasks and requiring attributable evidence, rather than relying on aggregate scores over fixed task sets, enables more reliable and sampl...

Prerequisites

harness evolution, behavior-aware verification, attributable-evidence gate, task-space exploration, harness-space exploration

Atlas Placement

Artificial Intelligence (subfield)

Read If

You care about harness evolution, behavior-aware verification, attributable-evidence gate.

Skip If

You only care about tau2-bench, tau3-bench.

Methods

harness evolutionbehavior-aware verificationattributable-evidence gatetask-space explorationharness-space explorationtrajectory diagnosisbudget-aware frameworkpropose-and-verify

Tasks

agent harness optimizationlanguage-model agentself-improving agentsharness adaptationverification task selection

Datasets

tau2-bench Retailtau3-bench Banking KnowledgeTerminal-Bench 2.0BIRD Mini-Dev (Challenging)

Benchmarks

tau2-benchtau3-benchTerminal-Bench 2.0BIRD Mini-Dev

Noosaga Placements

Artificial Intelligencesubfield90%
The paper focuses on improving AI agents through harness evolution, a core topic in artificial intelligence involving reasoning, planning, and agent design.
Agent harnesses shape how language-model agents use instructions, tools, and runtime componentsWe introduce HarnessLens, a budget-aware framework for automated harness evolution
Evolutionary Computationframework70%
The paper's propose-and-verify paradigm for harness evolution is related to evolutionary computation, as it iteratively proposes and selects modifications, though it does not use traditional evolutionary operators.
Existing propose-and-verify methods typically score every candidate on a fixed task setHarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories
Multiagent Systemssubfield60%
The work involves autonomous agents and their evolution, which is related to agentic AI and autonomous agents, though it does not focus on multi-agent coordination.
Agent harnesses shape how language-model agents use instructions, tools, and runtime componentsself-improving agents use interaction feedback to revise the scaffolds that shape their behavior
Evolutionary Computationframework60%
Similar to the AI-level evolutionary computation framework, the method iteratively proposes and verifies candidate harnesses, which aligns with evolutionary computation concepts in machine learning.
Existing propose-and-verify methods typically score every candidate on a fixed task setHarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories
Natural Language Processingsubfield50%
The paper deals with language-model agents and their harnesses, which involve prompting, tool use, and natural language understanding, though the focus is on harness optimization rather than NLP itself.
Agent harnesses shape how language-model agents use instructions, tools, and runtime componentsAll LLM agent and evolution roles use deepseek-v4-flash-preview

Abstract

Agent harnesses shape how language-model agents use instructions, tools, and runtime components, but adapting these harnesses requires costly verification. Existing propose-and-verify methods typically score every candidate on a fixed task set, wasting rollouts on unrelated behaviors and allowing aggregate scores to obscure specific regressions. We introduce HarnessLens, a budget-aware framework for automated harness evolution. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Across three agent harnesses and four benchmarks, HarnessLens improves average held-out performance by 7.6-13.6% while consuming substantially less evaluation budget than competing baselines. These results demonstrate that behavior-aware verification with explicit attribution enables more reliable and sample-efficient harness evolution under constrained interaction budgets. Our code is available at https://github.com/jhxu5214/HarnessLens.

Paper Context

Source ContextWhole paper

Budget100,000 tokens

Coverage64,245 chars

Classified from the full extracted paper text (64,245 characters). The Paper Guide brief above is the user-facing synthesis; raw context is kept out of the page.

Full-paper context sent 64,245 of 64,245 extracted characters to classification.