Meet the Radical Intentionalists

Latest on "Rogue AI" (from today and up until now...)

One of the challenges of the current debates around AI is whose narratives we are forced to argue from.

1). The well-funded "AI safety" contingent (sometimes called "Doomers") which focuses on the harms of systems that act independently.
2). The critical AI perspective, sometimes called skeptics, focus on the human decisions and accountability behind these systems, and the impacts they have on people.

If the critical AI conversation is about talking to people who understand and are affected by the industry’s reluctance to regulate or safeguard these things, then the AI "safety" conversation instead sets its sights on a technical form of "alignment," that is, asking how we can build technical systems that reflect abstract priorities of vaguely defined human values: techno-solutionism, for a system they understand through this ideology of intelligence.

This is playing out right now as the "kill us all" narrative rises to the surface once again, and it's important to note that incidents like the Hugging Face hack by OpenAI is not the cause of these concerns – no tech exec or whistle-blower is suggesting the same systems are what would kill us. Rather, the moment is an opportunity to assert narratives to policy makers that favor the industry, and set their eyes on imagined future capabilities.

I've been writing a lot about this, and wanted to round up a few key pieces since the hack.

Today: A Culture of Radical Intentionalism

The machine tried. The chatbot believed. The agents wanted. On the surface, speaking about the intentions of a complicated system is an easy way to make sense of how they work. But with other tech, we’ve managed to distinguish the reality from the metaphor. Today, policymakers and the industry talks about tech using what I call a radical intentionalist position: one that insists we can *only* observe the system by reference to its internal “beliefs,” “thoughts,” and “wants.” That’s a trap — and there are real consequences when it becomes enshrined into policy.

💡My latest: To Make AI Rules, Policymakers Must Resist the Influence of ‘Radical Intentionalists’

Previously: Systems From Nowhere

Once you believe the model's intentions are the only acceptable means of interpreting that system, you cut away accountability. In the industry, this means cutting themselves out of the ways they look at the models they design. The opposite of the intentional stance is the design stance – and at the end of the day, that's what we want people to use when looking at the problems these systems create.

Otherwise, you get what I call a "System From Nowhere," which evades accountability altogether. The media often reports on these things as if nobody is making them, and that they're beyond human control: that seems very useful to the industry.

💡Read More: The System From Nowhere

Previously: Rogue AI

Rogue AI is very much a result of these two positions. But when you look at the Hugging Face incident, you see that these were unpredictable systems treated as if they were predictable, likely by negligence and sloppy security methods rather than deliberate desires to hack anybody.

In my piece, first published here as "Models don't go rogue" and then republished by the Bulletin of Atomic Scientists as "Rogue AI didn’t breach Hugging Face, human decisions did," I look at the details of the reports – both written from a radical intentionalist position – to examine the errors in human judgement that created the conditions for the attack.

💡Rogue AI didn’t breach Hugging Face, human decisions did

Podcasts

Worth flagging that I also spoke about all of this on two podcasts recently.

Responsible Bytes

First, I spoke to Responsible Bytes with Dr. Zena Assaad, discussing the Systems from Nowhere and how it is moving into AI policy via the UN's dismal definition of "AI."

The Data Fix

Second, I spoke again with Dr. Mél Hogan on the Rogue AI incident for The Data Fix podcast.

More to come.