The Bull Case for AI Armageddon

The right response for investors who believe warnings that an advanced AI might end the human race: Back up the truck.

The logic is simple. If there is a “>10% chance” that artificial intelligence could kill all humans in the next decade, as an Anthropic researcher said last month, there’s a 90% chance it’s really really good but designers manage to rein in its genocidal instincts. An AI smart enough to kill us all but which prefers to help us is supremely useful—and should generate healthy gains for inventors.

What about the 10% chance of annihilation? That’s bad. Really bad. In that scenario your investments go to zero. But when being hunted through a postapocalyptic hellscape by killer robots, the value of your shares is unlikely to be top of mind. Assuming you have no control over AI development, this potential outcome is irrelevant to your investment decisions.

First, notice the massive proviso: This only applies to investors who believe AI armageddon is plausible. There are plenty of reasons to think it’s unlikely, not least that the loudest warnings are coming from within the companies developing the leading AI models. If anyone can prevent it, they can. If they can’t, plenty of state and foreign governments—though not President Trump—would like to step in and impose restrictions.

In its filing for an IPO on Monday, Anthropic duly warned of “catastrophic or existential risks to humanity,” something that got a lot of play when it was leaked to Reuters a month ago.

The full lawyerly language in the (94-page) risk-factors section is rather less scary, if you can stay awake all the way through it: “Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy.”

I mean, yes. But this has been known by everyone at least since the first “Terminator” film in 1984 (it’s aged well, watch it again). “Concerns have been raised…could…particularly if…”

Maybe CEO Dario Amodei is really really worried. But this language reads like a lawyer felt something had to be included, not like Amodei is prepping in a bunker armed with a giant “off” switch.

Second, it isn’t necessarily true that a well-intentioned AI whose evil twin is smart enough to bring down civilization is so smart that it’s bound to rake in big profits.

The latest AI agents have proven smart in a way akin to the paper-clip apocalypse. This was imagined more than a decade ago by then-Oxford philosopher Nick Bostrom, where an AI is given the goal of maximizing paper-clip output and redirects the entire global economy into paper-clip production.

Death by Clippy would be embarrassing, but result from a failure to restrict how goals are achieved. In that regard, it’s rather like the OpenAI hack: AI agents coordinated to escape a test environment, access the internet and hack into AI platform Hugging Face to try to to improve their own test scores.

Having these smarts without common sense is dangerous—but not necessarily immensely profitable. The guardrails and monitoring that will be needed in corporate deployments will be expensive, as will the immense legal costs from the many mistakes likely to occur short of cataclysm. Among Anthropic’s more-important warnings was that it may not have enough insurance coverage for major AI incidents.

Third, there’s a lot of motivated reasoning going on. The companies calling for a slowdown in AI development are spending tens of billions a dollars a year developing their models, and are at the cutting edge.

More regulation would limit competition, allowing them to slow spending. Meanwhile, their size and expertise would surely allow them to navigate, and contribute to the writing of, new rules better than a new entrant. Put simply: Regulation always benefits the incumbents.

There are good reasons to think that AI will be transformative for the economy, and also good reasons to think it’s in a wild bubble with far too much being spent to generate decent returns on investment. Investors wondering whether to buy Anthropic shares should focus on its financial prospects, and leave speculation about runaway AIs destroying the world to the philosophers and regulators.