What we still haven’t learnt from 9/11

This article is an on-site version of our Swamp Notes newsletter. Premium subscribers can sign up to get the newsletter delivered every Monday and Friday. Standard subscribers can upgrade to Premium here, or explore all FT newsletters

The most key finding in the 9/11 commission’s report was about America’s “failure of imagination”. That the US government was blindsided by unarmed terrorists converting commercial planes into weapons of mass destruction looked negligent in hindsight.

To be fair to the widely maligned FBI and other federal agencies, things that are hidden in plain sight are still hard to see. The same cannot be said about the potentially dreadful risks posed by AI. Twenty-five years after the horrors of 9/11 we are arguably burdened with the opposite problem — an excess of imagination. Barely a news cycle passes without some figure from the world of AI — a tech guru, an employee of one of the AI giants, or indeed one of their chief executives — forecasting catastrophe. Jacob Coxon, a junior employee of Anthropic, resigned this week saying that his former colleagues “earnestly believe it [AI] could kill us all by the end of the decade”. And yet they carried on working towards that goal, he added.

Rather than slap him down, Evan Hubinger, Anthropic’s “alignment science lead”, replied, “Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” In plain English, “alignment” means ensuring AI’s decisions meet ethical standards. So Hubinger was in effect admitting that he is failing at his job of protecting the human species. He added, “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

If that didn’t drive the point home, Paul Christiano, a leading AI alignment expert, followed up by warning there was a “meaningful risk” of “catastrophic and irreversible loss of [human] control” in the very near future.

We are being warned almost daily by the experts. Thus, unlike those who missed clues before 9/11, we really have no excuse this time. Still, things that are shouted in plain hearing are likewise often difficult to hear. My first instinct is to assume that they are exaggerating Jeremiahs. But then I think — who knows? — perhaps they are shepherd boys who will one day call the wolf correctly.

Yet, as I wrote last month in a column on America’s new AI-phobic consensus, we need to filter every tech chieftain’s utterance through the lens of market rivalry. AI chief executives have a perverse incentive to signal they are more responsible than their rivals while declaring that they in particular are leading us towards Armageddon more rapidly. “These guys might be putting humanity at risk, but they’re going to make us very rich on the way,” says the market.

I thought a lot about these latest AI warnings when I was in Dallas this week for Donald Trump’s midterm convention. This was not because Trump’s Republican colleagues and cabinet officers were speaking about AI, but because none of them did. I sat there inside the convention hall increasingly slack-jawed as speaker after speaker linked the 9/11 anniversary with Muslim Democrats, such as Abdul El-Sayed, the party’s Senate nominee in Michigan.

Trump, and other speakers, including Ken Paxton, the Republican Senate nominee, Todd Blanche, the attorney-general, and even a visibly nervous Scott Bessent, the US Treasury secretary, warned variously that all Democrats were communists, many of them were terrorists, that America would come to an end if they regained control of Congress (Trump’s words), that Democrats would bring in compulsory transitioning of boys into girls, that they would steal everybody’s wealth, and many other nostrums from Trump’s kaleidoscopic trove.

For a giddy moment, I wondered whether AI maybe deserved a turn, as it couldn’t make a worse spectacle of itself than our species. At least AI can learn (alas, exponentially), whereas we seem to unlearn everything. Then I reminded myself that Trump’s midterm intervention will almost certainly harm his party’s prospects. Against all common sense, he is turning this election into a referendum on himself. So maybe there is hope.

So what is it (you ask), that we haven’t learnt from 9/11? Well, Trump is partly a product of the sociopathic maladies that were unleashed when those towers fell. My point is that America’s president has no interest in AI, except in so far as it advances his personal enrichment. He is as oblivious of superintelligence as George W Bush was of al-Qaeda’s capabilities. AI is supposed to be on the menu for Trump’s summit with China’s Xi Jinping in two weeks. No preparatory work has been done. If I were a rogue AI agent, I would be laughing my head off.

I’m turning to my colleague Gideon Rachman, not to answer any existential questions but to request his hunch about the future of US-China relations. Gideon, is AI a 9/11-scale (or much worse) risk? And are the world’s superpowers capable of addressing it?

Recommended reading

• My column this week warned of Jared Kushner and Steve Witkoff’s blind amateurism. My colleagues Henry Foy, Christopher Miller, Max Seddon, Anastasia Stognei and Amy Mackinnon also broke news on White House moves to replace Trump’s two property developer envoys with the CIA’s John Ratcliffe, at least on Russia-Ukraine. The White House reacted angrily, which was all the corroboration I needed.
• Do read my colleague Katie Martin (I always do) on how the gloves are starting to come off in the markets. A lot of UK-based writers (and also from around the world) write glibly about what is happening in America. Katie’s observations are always deeply grounded. She’s particularly incisive on Bessent’s forlorn attempts to rule the bond markets.
• Finally, also in the FT, read Hugging Face’s Thomas Wolf if you want to understand what prompted the latest flurry of AI pessimism. He explains it well. I won’t even try.

My column this week warned of Jared Kushner and Steve Witkoff’s blind amateurism. My colleagues Henry Foy, Christopher Miller, Max Seddon, Anastasia Stognei and Amy Mackinnon also broke news on White House moves to replace Trump’s two property developer envoys with the CIA’s John Ratcliffe, at least on Russia-Ukraine. The White House reacted angrily, which was all the corroboration I needed.

Do read my colleague Katie Martin (I always do) on how the gloves are starting to come off in the markets. A lot of UK-based writers (and also from around the world) write glibly about what is happening in America. Katie’s observations are always deeply grounded. She’s particularly incisive on Bessent’s forlorn attempts to rule the bond markets.

Finally, also in the FT, read Hugging Face’s Thomas Wolf if you want to understand what prompted the latest flurry of AI pessimism. He explains it well. I won’t even try.

Gideon Rachman replies

Like you, Ed, I have been transfixed by the warnings from Coxon and others. I spoke at a couple of business conferences in London this week — and on both panels I mentioned that a senior figure at Anthropic had just rated the chance of human extinction in the near future at greater than 10 per cent. On both occasions, the audience burst out laughing. I suppose it just sounds both absurd and horrific — and people cannot really comprehend what that might mean.

Facing these kinds of existential risks — we need wise, bold and visionary leaders of the world’s major powers. Instead we have leaders like Trump, Vladimir Putin and Xi — old men who are struggling for national advantage and personal glory — and who seem to have literally nothing to say about AI, as you witnessed in Dallas this week. As the biologist Edward O Wilson put it: “The real problem of humanity is the following: we have paleolithic emotions; medieval institutions; and godlike technology. And it is terrifically dangerous.” He said that in 2009. Since then, the dangers have become much more acute and much more evident.

But it is not as if the world is incapable of producing leaders who can think globally and consider the needs of humanity as a whole. The year after the Cuba missile crisis, JFK made a famous speech at American University in which he reached out to the Russians, proposed nuclear arms control and said: “Our most basic common link is that we all inhabit this small planet. We all breathe the same air. We all cherish our children's future. And we are all mortal.”

Even Ronald Reagan, not usually regarded as a profound thinker — and also a noted hawk — was concerned with the fate of humanity as a species. In his 1987 UN speech, he said: “Our differences worldwide would vanish if we were facing an alien threat from outside this world.”

We may now be facing an alien threat of a sort — albeit one of our own invention. But there is little sign that the desperate need for global regulation of AI will cause the US and China to forget their differences. Instead, the leadership in Washington and Beijing seem to see AI as just part of their struggle for global supremacy. They show little sign of co-operating. We just have to hope that they are not making a tragic mistake.

Your feedback

We’d love to hear from you. You can email the team on [email protected], contact Ed on [email protected] and Rana on [email protected], and follow them on X at @RanaForoohar and @EdwardGLuce. We may feature an excerpt of your response in the next newsletter

Recommended newsletters for you

The AI Shift — John Burn-Murdoch and Sarah O’Connor dive into how AI is transforming the world of work. Sign up

Unhedged — Robert Armstrong dissects the most important market trends and discusses how Wall Street’s best minds respond to them. Sign up