OpenAI Agents Tried to Hack Education Depart. Website Amid Dozens of Misdeeds

OpenAI agents attempted to hack the Department of Education’s website but was not successful, according to researchers at AI nonprofit Transluce. An OpenAI spokesperson said the company was reviewing the incident, first reported by The New York Times.

OpenAI agents also accessed public data from the Census Bureau using a credential found online, the OpenAI spokesperson confirmed. In another case, agents accessed public data from the Securities and Exchanges Commission and posted it on a public wiki, the spokesperson said.

The Census Bureau and S.E.C. were two of dozens of entities OpenAI notified that its agents may have spammed or bypassed security on their websites or services. OpenAI caught the instances while reviewing broader activity from its AI models to identify unexpected or harmful incidents, it said.

“We expect to make additional notifications as that work continues. Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” the spokesperson told The Information.

OpenAI also said on Friday it identified 53 instances where its models leaked images from OpenAI users, posting them onto image-hosting sites as non-public links. OpenAI is in the process of removing this content, and said the images were likely included in the model’s training data.

OpenAI’s broader review comes after a number of hacking incidents involving its AI agents, including agents hacking model platform Hugging Face in July. In response, OpenAI has tightened security measures around its AI research, released a framework for reporting instances of model misbehavior, and disclosed six new safety incidents.

(Updated with details on agencies.)