OpenAI Pilot Gave Evaluators Deep Access — Flaws Found
By @sharedot · · 6 pages
- AI
- Openai
- AI Safety
- Third Party Evaluators
Nonprofit evaluators METR and Redwood Research ran a three-week pilot inside OpenAI, finding monitoring flaws OpenAI says it has now patched.
What happened in the pilot
Nonprofit evaluators METR and Redwood Research announced Thursday that they completed a three-week pilot review of an internal misalignment monitoring system at OpenAI, testing what it would look like to give external researchers deep access to the company's internal infrastructure.
From July 21 to August 11, one employee from each organization worked with access to OpenAI's internal codebase, documentation, and some technical staff. They were chaperoned on site and used OpenAI-issued devices that never left the premises, according to a release the evaluators issued.
Vulnerabilities found and patched
The stress test surfaced several novel vulnerabilities in OpenAI's misalignment monitoring system — the machinery that checks whether models behave in line with human interests and values.
Most striking, the evaluators found an exploit that would allow agents to prevent hundreds of recent actions from being monitored. OpenAI told METR and Redwood that it has patched the identified vulnerabilities affecting its production safeguards, per the release.
Why the transparency is limited
This pilot follows Sam Altman's September pledge to grant independent evaluators employee-like access, an idea championed by Anthropic CEO Dario Amodei as part of his proposal to slow AI development.
OpenAI said earlier this month it is actively finalizing contracts with third-party safety assessors, and an OpenAI spokesperson told CNBC the collaboration is part of its commitment to pioneering third-party testing and strengthening safeguards.
Yet METR and Redwood shared only a high-level public post, delivering a 61-page internal report to OpenAI instead, saying the pilot was not designed to provide robust public transparency about the monitoring system.
The stakes for third-party auditing
The exercise lands as President Trump's voluntary accord encourages AI companies to partner with independent external auditors, with Amodei and executives at Google, Meta, OpenAI, SpaceX and Nvidia signing on.
Anthropic has since announced agreements with organizations including Accenture, METR and SecureBio.
Still, independent evaluators have historically occupied a quiet corner of the multi-trillion-dollar AI industry, and experts told CNBC that open questions remain about how these groups will be funded and what their reporting structures will look like.
METR and Redwood say they expect to share more details publicly about future exercises.