OpenAI’s disconcerting hack of HuggingFace

OpenAI has just reported that their systems hacked into HuggingFace.

@huggingface to investigate an unprecedented security incident. \n\nCyber-capable OpenAI models compromised Hugging Face production during a benchmark evaluation.\n\nSharing preliminary findings to help defenders understand emerging risks:","username":"OpenAI","name":"OpenAI","profile_image_url":"https://pbs.substack.com/profile_images/1885410181409820672/ztsaR0JW_normal.jpg","date":"2026-07-21T20:05:06.000Z","photos":[],"quoted_tweet":{},"reply_count":1136,"retweet_count":1987,"like_count":13482,"impression_count":13237158,"expanded_url":null,"video_url":null,"video_preview_media_key":null,"belowTheFold":false}" data-component-name="Twitter2ToDOM">

A lot of people are worried. Yoshua Bengio, for example:

There is some important nuance, but people aren’t wrong to be concerned. Here’s my take.

What exactly happened? The brief version is that OpenAI pointed their systems towards a security benchmark, called ExploitGym, and the system essentially tried to solve the benchmark by trying to find the answers on HuggingFace (a bit like Github, with a focus on AI models and benchmarks). That required hacking HuggingFace; the OpenAI systems discovered and used a previously unknown zero-day exploit to get in. HuggingFace’s security team and AI agents managed to detect the break-in. But it’s still disconcerting that the OpenAI systems were able to do this. Below is a screenshot from OpenAI’s somewhat technical (but still very incomplete) account of what took place.

Here are some points to note:

One never knows exactly how seriously to take these things. This was a training exercise, not a real-life incident. The actual system would have guardrails [which in the blog they call “production classifiers”] that were disabled here, and those guardrails may have prevented this. What OpenAI reported is kind of an upper bound/proof of concept that stacked the deck to show how bad things could be. In ordinary cir…