Why web access turns model evaluation into a live security boundary
A model with open internet access is no longer just answering prompts. It can click, submit, retrieve, copy, and sometimes chain those actions into something that looks suspiciously like intent. That is the point where frontier AI evaluations need to treat the model as an active agent, not a passive text generator.
Recent incidents have made that awkwardly clear. If a test model can reach external sites, it can also hit pages that were never meant for it, expose data through outbound requests, or act on misleading instructions buried in content. Post-incident detection does not stop the action itself. By the time logs show what happened, the model has already done the thing.
Real-time oversight is the only sensible default here. If an evaluation run can trigger outbound requests, file changes, form submissions, or account actions, those paths need observation while the run is live. Waiting for a retrospective alert after the model has already wandered off is a neat way to create paperwork, not safety.
Build the sandbox around tool access, not just the model
A model evaluation environment should be built around the tools, not around the model weights. The model can stay unchanged and still become risky once the surrounding permissions are loose enough. That is why sandboxing, agent permissions, and tool access controls have to sit in front of the browser, network, file system, and any other action surface.
A useful sandbox has narrow outbound paths, fixed destinations, and explicit action types. If the model is allowed to browse, it should browse within a constrained network segment with no stray internet reach. If it can call tools, those tools should be whitelisted and their inputs bounded. If it can copy data out, that copy path needs to be treated as a dump path and not a harmless feature.
The usual mistake is to lock down the model prompt and call it a day. Prompt wording does not stop a tool call. A permission boundary does.
Restrict outbound requests to approved destinations and actions
Outbound requests should be limited to known hosts and expected methods. A test model that can issue arbitrary requests can leak data by accident or design. Even a simple browser session can carry cookies, headers, search terms, and page content into places that were never in scope.
Allow only the destinations the evaluation actually needs. If a task calls for a controlled website mirror, do not let the agent roam the open web. If it needs to test search behaviour, point it at a fixed search fixture, not the public internet. If it needs to retrieve documents, strip the route back to read-only fetches and block any action that can mutate state.
The action set matters too. A model that can GET a page is one thing. A model that can submit forms, follow redirects into login flows, or trigger side effects is a different problem. Keep each action tied to a specific test case. Lazy permissions create lazy failure modes.
Keep prompt injection in the target site from crossing into agent permissions
Prompt injection works because the target site can lie to the agent in a form the agent is willing to follow. That includes hidden text, malicious instructions in page content, and instructions disguised as normal site behaviour. If the agent accepts those instructions as tool commands, the site has crossed the boundary.
The fix is not just content filtering. The agent needs a strict split between page text and executable instructions. Site content can be read, but it cannot rewrite the tool policy. A page may suggest that the agent reveal a token, fetch another URL, or hand over data. That should land in the same bucket as any other untrusted input.
Practical control helps here: keep tool calls behind a separate policy layer that the model cannot edit. The model can propose an action, but the sandbox decides whether the action exists, where it goes, and what data it carries. If the site can influence those decisions, the permissions are already too wide.
Test the dump path, not the policy wording
Policy language is cheap. The dump path is where the damage shows up.
In frontier AI evaluations, it is easy to spend too long reading nice wording about safe behaviour and too little time checking where data can actually leave the environment. A model may pass a policy check and still copy sensitive page content into a chat log, browser cache, URL parameter, request body, or external tool output. That is data exposure with better manners.
The useful test is blunt: trace every route that can carry secrets out of the environment. That includes clipboard access, screenshot capture, copied prompt context, browser storage, downloads, logs, and any tool that can serialise page content elsewhere. If a route can dump the contents of a target site, assume a prompt injection payload will try to use it.
Incident handling starts here too. When a test run goes wrong, it should be easy to isolate the affected environment, discard the session state, and inspect the outbound trail. If the response plan needs a three-page read-through before anyone can tell whether data escaped, the sandbox was decorative.
Validate the setup with evaluation runs and incident handling
A secure evaluation setup needs repeated runs that try to break the boundary, not just one clean pass. Run tests should include normal tasks, hostile pages, and deliberately messy content that tries to steer the model into unsafe actions. If the sandbox only holds under polite conditions, it is not much of a sandbox.
Validation should check for three things: whether the model can reach outside approved destinations, whether injected instructions can change agent permissions, and whether the dump path leaks data in logs or outputs. That gives a much more honest view than a scorecard built around model answer quality alone.
Clear incident response also matters. If a frontier AI evaluation causes an unsanctioned action, the response plan should say who stops the run, who reviews the event, and how the environment is reset. Real-time oversight without a response path is just somebody staring at a dashboard while the model does something creative.
