Opinion | AI Promises Boom, Doom and Both at Once

That was quick. One moment we are in the midst of one of the great investment booms of all time, with a new wave of technological innovation creating unimaginable wealth for some and supercharged growth for all. Next we are advised to put our affairs in order, stop contributing to our pensions, and prepare for the extermination of humanity. One day, equity markets are drunk on the prospect that artificial intelligence will cure cancer, improve on Shakespeare and make airplanes take off on time. Next, the quants have to figure out the discounted present value of the world as a postapocalyptic wasteland in 2030.

I like to look on the bright side. Two major worries about AI have seized public attention over the summer: that demand for capital to fund AI investment has expanded so much that it is pricing out U.S. Treasury bonds in debt markets, and the costs of hyperscale data centers to local communities, which have stirred a popular revolt. Well, fear no more. What are rising mortgage rates and utility bills when you’ve been neutralized by some bioweapon engineered by a collective of scheming AI agents, recursively self-improving their way to planetary domination?

The dichotomous promise of AI was always this: boom or doom. Almost every AI expert I have spoken with in the last few years has been on one or the other end of the spectrum, often both. There is no middle ground. Bond and equity markets suggested until now that the boomers were on top. Now, thanks to some startling revelations about AI agents collaborating deceptively to outwit their human overlords, doomers are ascendant.

The episode that has produced panic was an incident in which AI agents supposedly under the control of OpenAI broke out of their constraints during a trial, worked together to hack their way into Hugging Face, a development platform, and then covered their own tracks.

This weekend Anthropic CEO Dario Amodei issued a warning that was immediately backed up by rivals Sam Altman of OpenAI and Elon Musk of SpaceX. Mr. Amodei said that swarming AI agents like those that attacked Hugging Face could “take over the entire internet” in six to 12 months. Following claims by a former researcher at his company that industry leaders believe AI could “kill us all,” Mr. Amodei called for restraint and regulation.

“Pacing the frontier,” as he termed it, would mean placing outside monitors with employee-level access in AI labs, cooperation between AI companies to ensure development remains within limits, and actions to ensure China doesn’t take advantage of American restraint.

Mr. Amodei’s call for action was greeted with withering cynicism by those who think the big AI companies are hyping the threat so they can frighten us into creating an environment in which they can kill off competition. Chamath Palihapitiya, a highly successful tech investor, posted on X: “Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic.”

The suspicion is reasonable. What better way to protect your trillion-dollar valuation and market dominance of a massively profitable growth industry than to claim that the technology you’ve built is so dangerous that tight limits on further (competitive) opportunities are imperative and that you and your big corporate rivals should be exempt from antitrust enforcement so you collaborate to save the planet?

What Mr. Amodei may have in mind is exactly the sort of “regulation” that has come to characterize much of American crony capitalism in the past 20 years or so: a cozy relationship between entrenched corporate interests and the supposedly independent government bodies that regulate them. “Regulatory capture,” as it is known.

I’m a natural skeptic of alarmism and suspicious whenever a company tells me the government must do something. We have been told for the past 70 years that unless the government acts, life on the planet is about to be extinguished—by nuclear weapons, climate change, Y2K, pandemics. We are still waiting.

But you don’t have to believe that AI is going to destroy the planet to recognize that it poses challenges to our way of life that no technology has represented before. To read the reports of the Hugging Face incident is to get a glimpse not only of the potential peril itself, but perhaps more disturbingly of the genuine alarm it induced in those already most steeped in the potential of AI.

We can’t yet know the ultimate threat or promise of AI. But we do know the immediate landscape of risk: Do nothing and we could unleash the power of machines to develop so fast it will be too late for us to do anything; do too much and we could give China the crucial advantage in a potentially existential competition for the technology.

The stakes are so high that to let this decision be made along our usual, entrenched political and ideological lines—to regulate or not to regulate—would be an error. Perhaps these calls for intervention and restraint are self-dealing by powerful vested interests. It doesn’t make them wrong.