When a coding agent stalls on a task, the reflex is to blame the model, or to wait for the next one. Sometimes that's right. More often the model is capable and the scaffolding around it is the problem: the wrong repository context loaded before the first call, a tool subset that omits the one command it needed, a history compaction that discarded the failure clue it was about to use. Every one of those is a harness decision, a choice about what to store, retrieve, and put in front of a fixed model, and at Aient we make hundreds of them. Most of the time we can't say which ones help.
We change a prompt overlay, watch the next handful of remediations, and form an opinion. It's an informed opinion, and it is still an opinion.
The paper
Meta-Harness (Lee et al., out of Stanford, KRAFTON and MIT) names this problem and then does something unfashionably plain about it. Treat the harness as an executable object. Keep a population of candidate harnesses. Evaluate them on a search set. Store each candidate's source, scores, and raw execution traces in a filesystem. Then let a coding agent, the proposer, read that filesystem the way an engineer reads a post-mortem, and propose the next candidate. The motivation is blunt: swapping the harness around a fixed model can move a benchmark by a factor of six, and a single harness evaluation can produce ten million tokens of diagnostic output, three orders of magnitude more than earlier text optimisers were built to digest.
The load-bearing choice is full-history filesystem access, and the paper earns that claim with an ablation rather than an assertion. Give the proposer scores only and its best classification harness reaches 41.3%. Add LLM-written summaries of the traces and the best result gets worse, 38.7%. Give it the raw traces and it reaches 56.7%, with the median trace-fed candidate beating the best candidate from either ablation. Summaries did not recover the missing signal; they compressed away exactly the details a diagnosis needs, which anyone who has been paged with a summary of an incident already knew.
The proposer behaves like the engineer, too. It reads a median of 82 files per iteration, split roughly evenly between prior harness code and execution traces, and in the TerminalBench-2 run it explicitly hypothesised that two regressing candidates were confounded by a shared prompt rewrite, separated the changes, and pivoted to the safer additive fix that became the run's best candidate: roughly 80 lines that gather an environment snapshot before the agent's first turn. Engineering memory with grep, nothing more mystical than that. The discovered harnesses hold up: 7.7 points over the best hand-designed context system while using a quarter of the context tokens, 4.7 points on 200 IMO-level problems across five held-out models, and first place among Haiku 4.5 agents on TerminalBench-2 (second on Opus 4.6, behind a score the authors could not reproduce; search and evaluation also share that benchmark, a discovery-problem framing the authors audit for leakage). The proposer, incidentally, is Claude Code. What matters for us, though, sits underneath the numbers: the demonstration that harness design responds to ordinary empirical method.
Where we actually are
Aient's harness is a production system, not a research artefact: durable execution boundaries, typed task lifecycles, remediation workflows with real operational semantics, end-to-end PR evidence with traces and recordings where configured, telemetry in ClickHouse. After an incident, an engineer can reconstruct exactly what happened. That evidence is genuinely good, and it is spread across eight systems, and none of them answers the one question an optimiser would ask: which exact version of the harness produced this result, on these tasks? So improving the harness stays a manual craft. Someone spots a pattern and writes a better prompt, and whether it helped remains, again, a matter of opinion.
The paper's toy harnesses are single Python files; ours has more surfaces. The harness is where our multi-agent structure lives: custom prompting, and the machinery that spawns swarms of subagents and lets them collaborate on a remediation. How many agents, which tools each one gets, what they share: every one of those is a harness parameter nobody is measuring yet, which is rather the point.
It also has harder constraints. We can't treat raw production traces as optimisation snacks. A generated prompt must never override platform safety instructions. And an optimiser that can ship its own code isn't a feature, it's an incident with good intentions.
What we're building
The adaptation we're building is deliberately unglamorous, and deliberately offline. A proposer agent gets a filesystem of candidate harness variants, their scores on a search set, and the full traces of how each one ran, with a holdout set it never sees and secrets stripped before it reads a line. It forms one theory about why a variant got worse, and proposes the next one. Nothing reaches production without an ordinary reviewed pull request.
The first target is small on purpose: our agent's context compaction, where an evaluator over historical replay cases already exists and the blast radius is bounded. Remediation prompts come after that, and only as prompt, context, and tool-policy variants; the durable workflow code stays frozen until replay-safety tests exist for it.
What this is not
None of this is running in production today. There is no self-optimising loop shipping fixes to its own scaffolding, and production-derived training of any kind stays blocked until a redaction and retention contract exists, not as a nicety but as a precondition. What exists now is the evidence, the evaluator for one surface, and the design. The computer is not self-optimising yet; it is at least looking embarrassed about that.
The reason to do this anyway is that our product's promise has four verbs: detect, fix, verify, learn. The first three are concrete in the codebase. "Learn" is still mostly curated memory and better prompts after a human notices a pattern; an operations agent that fixes production should learn from its own failures with the seriousness a good SRE team brings to an outage. Today the lesson lives in one engineer's head until they change teams. That is the alternative to what we are building: the transcripts accumulate, and the lessons get relearned by hand, one engineer at a time.