Why do they even talk about x-risk?

Why do CEOs and other AI leaders talk about existential risk at all? What sociological mechanisms make it possible for them to do it? And how should we respond to the popular argument: “if Musk says something, the opposite must be true”?

I am a member of the Pause AI movement, specifically Pause AI Poland.

And I’m having trouble debating "AI skeptics", who have unfortunately become increasingly vocal in my country. I’m asking you to explain why exactly, the CEOs of these companies, both this most well-known ones, such as Elon Musk, Sam Altman, and Dario Amodei and many of other high-ranking employees, even talk about x-risk in the first place? After all, companies have always hidden the side effects of their products rather than being the ones to warn people about them.

Oil companies funded organizations that denied that climate change was anthropogenic, cigarette manufacturers fought against health warnings on packages, cell phone manufacturers and the telecom industry tend to shy away from acknowledging safety concerns about RF radiation, pesticide companies tend to exploit any scientific uncertainty in favor of a narrative of safety. The same goes for pharmaceutical companies — probably for pretty much any industry.

And honestly, under normal circumstances, the fact that major CEOs and other serious industry representative talk the way they do should speak in favor of the so-called “doomers.” It’s the same principle as in court: having someone who is hostile toward you testify in your favor is infinitely more powerful than having someone who likes you—or a family member—do the same. So, I wouldn’t have brought up this topic if it weren’t for this very strange argument from the “skeptics” namely that they are creating hype.

I think this belief is so stupid that it’s not even worth wasting time debating it. Nonetless, there is still legitimate question: Why exactly are CEOs and people from the industry talking about these risks at all? Okay, let’s exclude Anthropic from this for the sake of argument—you could say they have a genuine incentive to do so because they position themselves as the “more responsible”. From the standpoint of their own profitability, talking about the x-risks is counterproductive because it creates negative PR.

This could lead people to turn away from their products, start poisoning the data, or put pressure on politicians to pause AI development.

I know this forum is known for its rational approach, and people here love game theory and psychology. Maybe I can find an answer here to a question that comes up again and again in discussions in Poland.

Is it a matter of a unique intellectual culture shared by people in Silicon Valley, one that is largely foreign to Europeans? I mean, is it because the people working at these labs are aware of the risks, so the CEOs have to talk about them? Are tech leaders being sincere when they talk about these risks? And if so, to what extent? Apparently not entirely, given that they’re still pushing ahead? But on the other hand, maybe they genuinely can’t stop, because a) shareholders would oust them b) antitrust laws wouldn’t allow it c) prisoner’s dilemma. But on the other hand, it’s hard to believe that someone who genuinely fears AGI would devote so many years of their life to building it. And portraying these CEOs as “the good guys” is counterproductive from a PR perspective.

Just to be clear, I know it’s not just these CEOs who are talking about this. I’m aware of whistleblowers and many of the other arguments put forward by Yudkowsky as well. I’m wondering how to respond to an argument that has recently emerged in response to the leaders’ appeal, which goes roughly like this: "Why don’t I trust the good intentions of capital interests? Because they conflict with my own interests.". And I know you can argue that, alongside American capital, there is also European capital with an interest in what is often called “digital sovereignty", and just because a certain Austrian painter drank water doesn’t mean that drinking water is bad. But I do think there is something to the core of this argument—namely, that it really is unusual that these CEOs are talking about the risks, when such risks haven’t been talked about in other cases. Maybe it’s because the risk is so great. But was the risk from leaded gasoline that El wrote about in his book really negligible? I can understand why someone might find that suspicious.

1. other poeple who signed the CAIS statement but, in my view, have conflicts of interest that should logically have prevented them from doing so: Bill Gates (owner of Microsoft shares until May 2026), Lila Ibrahim (COO of DeepMind), Mira Murati (CTO of OpenAI), Kevin Scott (CTO of Microsoft), Mustafa Suleyman (CEO of AI company), Emad Mostaque (CEO of another AI company), Ray Kurzweil (researcher in Google), Adam D'Angelo (board member of OpenAI), CEO, Dustin Moskovitz (CEO of Asana, AI driven work platform) Miles Brundage (head of policy research, OpenAI), Allan Dafoe (AGI strategist, Deepmind), Helen King (strategic advisor, Deepmind), Jade Leung (Governance Lead, OpenAI), Marc Warner (CEO of Faculty, AI company), Vijay Bolina (Chief Information Security Officer, Google DeepMind)... okey. I think that's enough.
2. If Volvo, Mercedes, BMW, and Volkswagen agreed not to improve their engines, it would rightly be treated as an anticompetitive agreement designed to cut costs at the expense of their customers.