More Details Emerge About OpenAI’s Hugging Face Incident

TBPN，科技每日秀 Substack 专栏。关注

Happy Thursday.

We’re live on YouTube and 𝕏.

The current thing in tech and business is the unfolding story around OpenAI’s Hugging Face incident, in which agents communicated to each other via message board to hack out of OpenAI’s systems.

Today’s Lineup

South Park Commons General Partner Aditya Agarwal at 12:00 PM
Expedia CEO Ariane Gorin at 12:15 PM
The Atlantic CEO Nick Thompson at 12:30 PM
Hadrian CEO & CPO Chris Power at 1:00 PM
Atlas Motion Christian Mochen at 1:10 PM
Figma Co-Founder & CEO Dylan Field at 1:20 PM

Run of Show

More Details Emerge About OpenAI’s Hugging Face Incident

More details have emerged about the Hugging Face hacking incident that OpenAI disclosed in late July.

During a presentation at Black Hat’s annual cybersecurity conference yesterday, OpenAI member of technical staff Michael Dalton revealed that its autonomous agents created a message board to help each other break out of a sandbox environment to get access to the internet. Wired reported some other wild details that Dalton shared:

OpenAI’s agents apparently began giving each other assignments to split up work. And as is the case on any active development message board, they also generated petty drama at times by stepping on each others’ toes; for example, accidentally deleting each others’ work. As the message board developed into more and more of a Lord of the Flies–type situation—all still completely unnoticed by the humans running OpenAI—the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.

According to Sharon Goldman, who attended the presentation, when employees discovered the messageboard, they wiped the system on which the agents had created it. Days later, staff found that the agents had created another way to communicate by using the names of newly created directories.

In late July, OpenAI announced that during evaluations, two of its models had broken out of their testing environment to hack into the AI tool library Hugging Face and other companies. Their purpose was to find ways to essentially cheat on the tests researchers were using to evaluate them so that they’d perform better on the evaluations.

At the conference, Dalton said the incidents mark a “pivotal moment” for the company and the industry as a whole. He added that numerous internal teams have “dropped everything” to dramatically expand OpenAI’s ability to monitor its agents, and that the company is slowing its research to enhance security and revamp its defenses.

Clip Spotlight: Is Google AI Pilled Enough?

Jordi: “Google started selling compute. A lot of researchers, I'm sure, were frustrated with that. And that could have led to some of the historical exits.”

John: “If you believe you're in the foothills of the singularity, you will want to scale your compute.”

”If you're seeing traction in the products that you're making, you want to be GPU rich.”

”You don't want to be selling TPUs to other companies. There's gotta be some tension.”

Jordi: “The DeepMind organization has made some real contributions to the space, but has been lagging.”

”They've always felt behind, even though they created so much of the fundamental research that has gone into this entire chapter.”