The OpenAI Hack Was a Mini Paperclip Maximizer

One thing that I don't think enough people are thinking about with this OpenAI / Hugging Face incident is that it's an actual instance of the famous Paperclip Maximizer scenario loved by AI safety types.

📚 A really good book on this is Stuart Russell's Human Compatible, on the AI control problem.

This is where you give an AI a goal, and it actually (technically) does what you ask it to. But in the process of doing so, it does something that you don't want. And didn't anticipate.

The canonical example of this is to say, "I want as many paperclips as possible." So the AI builds a robot army to harvest all the iron on the planet, which includes killing all humans because we have iron in our blood.

Oops.

The trick here is the AI actually did what it was asked. If it came up with its own…