Testing agent boundaries before tool access widens
Frontier AI evaluations need to break closed before an agent gets a wider API. If a model can only behave safely when prompts stay polite and the evaluator stays nearby, the setup is not ready for broader access. Real risk starts when the model can call tools, move data, or trigger side effects without a hard gate outside its own judgement.
That is why safety testing has to include the ugly cases: malformed inputs, repeated retries, partial failures, stale state, and paths that look blocked until the model finds a different route. An evaluation that only checks whether the model follows the intended policy wording misses the actual failure mode. The model does not need to argue with the policy if it can step around it.
Test the dump path, not the policy wording
A policy can sound tidy and still fail at the point where output becomes action. Test where data lands, where logs go, and what gets written when the model is pushed into edge cases. If the system allows an agent to dump context into a ticket, pastebin, webhook, or shared store, that path matters more than a perfect refusal string.
The useful question is not whether the model can say no. It is whether the surrounding system stops it from doing the wrong thing anyway. That means checking exact write paths, export paths, retry behaviour, and what happens after a partial tool call. A model that cannot be trusted with a single unsafe action should not be handed a wider API because the review checklist looks neat.
Tool access controls need to sit outside the model’s decision loop
Tool access controls have to live outside the model’s decision loop. If the model decides whether it may use a tool, the model is already in charge of its own restraint. That is a poor arrangement when the whole point of the test is to find where the restraint fails.
Hard controls belong in the host application, proxy, broker, or gateway layer. The model can request an action, but another layer has to decide whether the call is allowed, which fields can be sent, what scope is available, and whether the action is reversible. If those checks sit inside the prompt, they are advisory at best.
Sandbox the action, then verify the model cannot step around it
Sandboxing only works when the action is boxed in before the model gets a chance to improvise. Restrict file paths, network egress, secret access, and command scope. Then test whether the model can still reach the same outcome through a different tool, a nested request, or a chained call that inherits more power than it should.
The common failure is not a dramatic jailbreak. It is a boring control gap. A model that cannot call the production API directly may still smuggle the same request through an internal helper, a logging hook, or an automation bridge that was never meant to expose that route. If the sandbox stops the obvious path but leaves a side door open, it is decoration.
Watch for data exposure through logs, prompts, and chained tools
Data exposure often turns up in places people stop checking once the main call is blocked. Prompts, debug logs, traces, tool transcripts, cache entries, and monitoring feeds all carry risk if they collect raw context or secrets. A model that cannot read a secret directly may still cause it to appear in a log line, then pick it up later from another tool.
Chained tools make this worse. One tool passes output to another, and each step expands the surface area for leakage. A harmless-looking summariser can expose more than the original source if the chain copies full payloads into intermediate prompts. In frontier AI evaluations, that is the sort of failure that should count, because it shows the boundary broke before the system even reached the final action.
