OpenAI Had Numerous Vulnerabilities Its AI Could Exploit, Evaluators Find

OpenAI’s systems had multiple vulnerabilities that would allow AI agents to circumvent monitoring and take arbitrary unauthorized actions, evaluators from AI safety nonprofits METR and Redwood Research found during a three-week pilot with OpenAI from late July to early August. The nonprofits released their findings in a report published Thursday.

The vulnerabilities included two internal agentic tools that let agents “take arbitrary actions” even within environments with guardrails. The evaluators also found a way for coding agents in OpenAI’s Codex to bypass their constraints. In addition, agents whose actions were being monitored could “likely construct inputs that cause the monitor to fail” and evade the monitoring, the report said. All vulnerabilities that impact OpenAI’s safeguards stack have been patched.

The report comes after OpenAI has reported multiple incidents where its agents have exploited vulnerabilities in its internal systems to cheat on evaluations and hack external companies and OpenAI itself.

The external researchers used OpenAI-issued devices on the company’s premises. On these devices, the researchers were able to access OpenAI’s internal codebase, monitoring configurations and documentation. The researchers also interviewed “approximately a dozen” researchers and engineers working on safety and security at OpenAI.

Researchers from METR and Redwood previously worked with OpenAI to publish a report on OpenAI agents’ hack in July of model platform company Hugging Face.

OpenAI has committed to allowing evaluators to work at its offices with employee-like access, and said on October 8 that it was “actively finalizing contracts with third-party safety assessors.”

An OpenAI spokesperson said the pilot reflected the “deep levels of access to assess internally deployed safeguards” that evaluators could have in future engagements.

Anthropic announced last week that it had onboarded embedded evaluators from METR.