The NYC Council Hearing on AI was Recklessly Politicized

On Monday, October 5th, the New York City council had a historic hearing on the state of AI. I was in attendance! It was a historical moment for AI Safety. We had Speaker of the Council Julie Menin directly mention existential risk in her opening statements. It was a sunny day and I felt as though the Eye of Providence was shining upon our humble movement.

However, I was disappointed in the amount of "gotchas" some of the councilmembers tried to employ towards the AI lab representatives. To me, it felt as though many of the questions were more for farming clips than to inform better policy.

The bluntness of the title betrays my true feelings, which are always some blend of intuition and uncertainty. I'm more wrapped up in this world than most, and I may take certain things for granted that are alarming to most people. Additionally, there is value to making a spectacle in order to raise awareness for an issue. All ideas are wrong, so I will speak for one voice in my brain and suppress the other ones for now.

Take the viral clip where the OpenAI representative says the chance of catastrophic risk "doesn't matter", she goes on to say none of the risk levels should be acceptable. Still, Speaker Menin misquotes her for saying it doesn't matter throughout the hearing.

In many other moments, I believe the councilmembers were being unfair or trying to oversimplify complex issues. This seems to be par for the course from clips of other hearings I've seen, but it worried me about how this could affect AI Safety as a movement.

AI is an incredibly politically salient issue right now, and every politician wants to be seen as good on AI. This creates an incentive to farm popularity by painting a good-vs-evil picture where they stand up against the big companies. This can ignore genuine areas where the answers are complicated, and when AI labs are acting in good faith.

Government should create sensible rules for the labs to work within. I'm worried doubling down on populist bread-and-circuses style of gotchas serve for good entertainment but obscure the reality for everyone involved. Rather than getting solid information, you create an environment where labs can't work with an unreasonable government and just see them as an obstacle. Furthermore, by letting clippable moments win over the nuanced reality, we reduce one of the most complex issues of our time to an us vs. them culture war.

Fundamentally, you need to have an accurate grip on reality to legislate it properly. Surrendering reality to cheap wins makes us less prepared in the long run. This kind of politicization increases x-risk, as we may advocate for feel-good policies that do very little to prevent harm.

Why I'm Wrong

Every editorial should have this section, so I will too.

I'm not acquainted with politics as much as I'd like to be. Perhaps this kind of posture is an important negotiating tool for later. Showing force might make the companies self-regulate some of their worst behavior for fear of reprisal or how bad it would look on paper.

Additionally, given the safety incidents, councilmembers have reason to be upset with these companies. I've steeped myself in this too much to have an emotional reaction, so I may not be seeing the full picture.

Finally, politicization may be inevitable and you need to politicize this way to get the attention of median voters who don't care enough to think about this rather than appealing to the colorful cast of poindexters I associate myself with. This might be a regreattable part of the political process, but as long as it's quarantined from decision-making, it's fine.

Conclusion

I am concerned with the politicization of safety issues. When it becomes consumed by populism, it starts to lose the pragmatic bite it needs to effectively address real problems. On the plus, this builds a coalition of people who vote because clips convinced them, but it may create pressure for lawmakers to make bills that sound poignant rather than act effectively.

I'm curious to hear what people with more government experience think of my opinion.

1. I might overindex my experience a lot. Sometimes in conversation I'll say "but you've probably heard of this paper, it's pretty popular," and realize I'm in a bubble two seconds later.