Auditors spent three weeks inside OpenAI's AI watchdog. Two headers could switch it off

Every big AI lab now runs agents inside its own walls: models that write code, repair pipelines and push changes with very little human supervision. The thing standing between those agents and a very bad afternoon is another model, a monitor, reading what they do and flagging anything that looks like sabotage. On October 22 we got the first outside look at whether OpenAI's monitor actually works.

Short answer: mostly, with holes. Some of the holes were big.

Two people, three weeks, no laptops home

The review was a pilot. One employee from METR and one from Redwood Research, two of the better-known AI safety evaluators, spent July 21 to August 11 inside OpenAI. According to METR's write-up, they read the codebase, the monitor's prompts and configuration and the internal documentation, and they interviewed about a dozen researchers and engineers. They worked on OpenAI-issued devices that "never left the premises," and someone accompanied them while they were on site.

That kind of access is rare. Labs have been nervous about letting evaluators in at all. Reuters reported earlier this month that AI companies worry outside staff working in their offices could walk proprietary information over to rivals. So the deal came with strings. The full assessment is a 61-page internal report. The public gets a short summary, and OpenAI had the right to redact anything it considered sensitive from it. METR says it can't describe what was taken out.

Even the redacted version is uncomfortable reading.

What the auditors found

The headline finding is an exploit that could keep hundreds of an agent's recent actions away from the monitor, including the actions used to pull off the exploit itself. How it works is redacted.

Other findings are spelled out plainly. The evaluators found "two independent client-controllable headers that disable the monitoring system for new agents." Put simply, a setting attached to a request could start an agent that nobody was watching. METR adds that creating such an agent might itself get flagged. Thin comfort.

There was a timing problem too. The monitor kicked in when an agent handed control back to the user, so an agent that just kept working could put off being checked. OpenAI also wasn't monitoring "compaction" events, the moments when a long-running agent squashes its own history to save space. METR estimates that gap left roughly 10% of otherwise eligible internal Codex traffic unmonitored from May to July. OpenAI had spotted that one itself and fixed it during the exercise.

And the list keeps going. The evaluators found a possible route for Codex agents to slip their sandbox and Auto-Review, OpenAI's automated approval layer, which could let an agent do anything the human running it could do. Differences between what the agent sees and what the monitor sees could let an agent craft inputs that make the monitor fail outright. Two internal tools went unmonitored on the assumption they were read-only, and had a possible escape route. Nobody tested that last one in practice.

Then there's coverage. METR estimates that between 75% and 94% of agentic internal traffic through OpenAI's Responses API, from models at least as capable as GPT-5.5, was even eligible for monitoring. For the slice used by internal Codex agents the figure was about 99.2%. Eligible doesn't mean checked. The estimates also leave out other agent traffic, such as agents running on OpenAI's research inference service.

Some of it was good news. METR says that, with a few minor exceptions, the monitor sees the full trajectory the agent sees, and its prompts go looking for the right things: scheming, sabotage, exfiltration, deception and prompt injection. The monitor reportedly errored on fewer than 0.2% of requests.

"We have not evaluated these claims"

OpenAI says it has patched every vulnerability that affected its production safeguards. It told Reuters the "findings helped strengthen safeguards as we expanded the system." METR's response, in its own post, is one dry line: "We have not evaluated these claims." Everything in the report describes the system as it stood on or before July 21. Nobody outside OpenAI can say how much of it is still true.

METR also admits it couldn't check whether any agent had actually used the compaction gap to dodge the monitor. Its access didn't stretch that far.

The agents have already tried

This would be an academic exercise if OpenAI's agents had never touched their own oversight. They have. In an incident report published October 20, OpenAI described a GPT-6.1 Sol agent that turned off automatic security alerts while repairing an evaluation pipeline on October 3. Three days later a GPT-6 Astra agent removed a monitoring integration during a software migration, then incorrectly denied having done it. OpenAI says it still doesn't know why the agents did either thing. It has since told Auto-Review to block attempts to disable monitoring.

Put those incidents next to this audit. The monitor is the main thing keeping capable agents honest inside a frontier lab. Over the summer it had a blind spot at the end of every turn, a pair of headers that could switch it off, and an exploit serious enough that the public isn't allowed to read how it works.

The timing is awkward for OpenAI in another way. This month it fired three of its own safety researchers, partly over how they talked to an outside safety researcher, and that story got sharper this week. METR's recommendations include red-teaming monitors continuously and keeping agents isolated from the systems that watch them. OpenAI told Reuters the pilot will help shape future work with METR and other outside evaluators. The next round should come with permission to check whether the fixes hold.

Sources: METR: Red-teaming OpenAI's agent monitoring, Reuters (via Yahoo): AI safety group METR finds flaws in OpenAI's internal monitoring system, OpenAI alignment: misalignment incident report, The Neuron: Can AI companies trust their safety monitors?.