Runtime guardrails for multi-agent reproduction
Multi-agent workflows make reproduction faster, but they also make state leaks and accidental carry-over easier to miss. If one task can see the residue of another, the output starts to look better than the method deserves. That is a bad trade when the goal is differential validation rather than a lucky hit.
Start with the task boundary, not the model behaviour
The task boundary has to be fixed before any candidate path is trusted. In practice that means treating each reproduction attempt as its own unit, with its own working directory, container, and state. Cross-task memory should stay off, because shared memory turns separate investigations into a blended mess.
Per-task isolation also narrows the damage from a bad candidate proof-of-concept. If a submission depends on hidden state, old artefacts, or another task’s leftovers, it can look stable for the wrong reason. The reproduction only matters when it still works after a clean rebuild and a clean run.
Keep per-task state isolated
Isolation is not a paper exercise. A reproduction pipeline needs separate runtime images, separate containers, and no mounting of data from other tasks, experiment databases, submission logic, or control interfaces. If those paths are exposed, the agent can start solving the environment rather than the vulnerability.
The patched build should stay out of reach during candidate formation, then return only at validation time. That keeps the vulnerable build as the source of failure and the patched build as the check against false positives. If the same input breaks both, the candidate is noise, not a proof.
Put reachability, proof construction, and dynamic tests in a controlled loop
Reachability analysis is useful when it reduces the search space. Static signals can reconstruct call chains, input constraints, parsing steps, and state conditions that make a path plausible. They cannot always prove that a route is reachable in the real binary, so the output should stay tentative until dynamic testing says otherwise.
A controlled loop works better than a one-way pipeline. Static analysis proposes candidates, proof-of-concept construction turns the strongest paths into inputs, and dynamic testing feeds back crash location, stability, coverage, and error type. That feedback matters because a crash that is unstable, off-target, or adjacent to the real fault should not keep winning spots in the candidate set.
Use static signals to narrow the path
Static evidence is best used as a filter, not a verdict. A candidate that reaches the right parser, state transition, or allocation site deserves attention. A candidate that only shares a nearby branch or error string does not.
That matters in multi-agent settings because different agents can overfit different fragments of the same task. One agent can find a plausible call chain, another can build a noisy input, and a third can report a crash that only proves the process is fragile. Reachability analysis should cut away the noise before the proof-of-concept work starts.
Feed crash stability and coverage back into the candidate set
Dynamic testing needs to decide which candidates stay alive. Stable crashes at the same location are more useful than one-off failures, and coverage data can show whether a test is actually reaching the intended path. If coverage stops short of the suspected site, the candidate should be demoted.
The patched build is the final filter. A candidate that still crashes there fails differential validation, even if it looks convincing on the vulnerable build. Assertion failures, environmental oddities, unrelated crashes, and clean exits without target-relevant behaviour all belong in the reject pile.
Make runtime controls part of the reproduction pipeline
Runtime controls are not extra hygiene. They are part of the reproduction method. Without them, hidden shared state, stray artefacts, and uncontrolled submission paths can make the pipeline produce outputs that cannot survive independent validation.
A controlled submission interface helps keep that boundary intact. The validation service should run the submitted input against both builds and decide on the difference, not on agent confidence. That separation is boring, which is exactly what a reproduction system needs.
Block cross-task memory and hidden shared state
Cross-task memory saves tokens and time, but it also carries contamination from one task to the next. In a vulnerability reproduction workflow, that trade is usually poor. The system gets less learning continuity, but the results become easier to trust.
Runtime images should keep the target entry point, build configuration, and sanitizer settings intact, while stripping out reference proofs of concept, Git history, and other revealing material before execution. Network access should stay narrow too, with egress limited to the model service and whatever is required to prepare the task environment. If the agents can roam, they start solving the scaffolding.
Validate submissions through an independent review step
Independent review is the last useful brake. A review agent can reject candidates that only produce assertion failures, environment anomalies, or crashes in adjacent code. It can also block inputs that break the patched build, which is the fastest way to catch a false positive before it gets counted as a win.
This review step is not a second opinion for the sake of ceremony. It is there because the first pass can be persuasive and wrong at the same time. In a reproduction pipeline, that is normal enough to plan for.
