Why does Anthropic’s IPO feel so weird?

For fast-growing tech companies, initial public offerings are an important rite of passage. From Google’s declaration that it wanted to “make the world a better place” to SpaceX’s ambition to “extend the light of consciousness to the stars”, they are a moment to capture the world’s imagination. The best IPO narratives fuse an idealistic vision of the future with a sense of inevitability about why this company will win.

Selling shares to the public for the first time is also seen as a milestone on the path to corporate maturity. There is an expectation that products have been honed and business models perfected.

Perhaps all of this goes some way towards explaining why the looming IPO of AI company Anthropic has felt so . . . weird.

Details from its draft prospectus leaked out this week, suggesting the share sale is close. If it hits the $2tn valuation investors are expecting, it would set a record for a new public company.

Compared to other blockbuster tech IPOs, though, the build-up to Anthropic’s stock market debut has felt discordant, even creepy. What other company, after all, would think to warn shareholders that its main product might go off the rails and resort to blackmail?

The most obvious cause of the weirdness has been the predictions from former and current employees that AI might lead to human extinction. Far from trying to stamp out such talk, CEO Dario Amodei agreed that the technology could cause “catastrophic damage” and regulation was urgently needed.

It isn’t only the doomerism that has struck a jarring note. Like the rest of its industry, Anthropic has struggled to convince the public that its technology is actually going to make the world a better place. AI executives such as Amodei like to talk about the economic “abundance” it will create and the diseases it will cure, but such utopian musings are out of tune with the wariness many people feel.

Then there are the mishaps. The extinction warnings relate to a future version of AI that might escape human control. Even at this stage, though, Anthropic can’t seem to keep a grip on the technology it is already building. Echoing problems at rival OpenAI, it confessed recently to three cases in which its AI had escaped on to the internet and hacked into other companies’ systems.

Ultimately, none of this may matter much to investors, given the company’s astounding growth. Even here, though, things are not straightforward. There is a lively debate among AI investors over whether makers of so-called frontier AI models like Anthropic will be able to maintain any kind of performance advantage. If not, it is likely to struggle to maintain pricing power and be forced into an overhaul of its business model.

To Anthropic’s critics, much of this reflects the immaturity of a five-year-old research lab that has, almost overnight, been propelled to global prominence.

Countering the extinction warnings, tech figures like Nvidia CEO Jensen Huang, former White House AI tsar David Sacks and Meta’s Mark Zuckerberg argue that there is nothing inherently dangerous about developing more advanced AI or releasing the technology to a mass market. It just takes the same kind of engineering experience, product design and quality control that characterise other technologies — disciplines that Anthropic is still learning as it mutates from ambitious start-up into mature product company.

Before it sprang to wider attention, Anthropic was the product of a culture that seems singular even by the standards of Silicon Valley. Its founders were heavily influenced by a branch of the effective altruism movement known as long-termism, which is fixated on the risks of human extinction from technologies like advanced AI. To critics, that has led to a speculative and self-indulgent doomerism that overstates the more prosaic work needed to control AI.

This is certainly how things are seen in Washington. After a lunch at the White House this week, a number of AI executives — including Anthropic’s CEO — signed an “accord on super intelligence” that promises the kind of things you would expect from normal, mature companies: “Robust internal controls”, special board-level committees and outside audits to contain the risks.

It’s hard, though, to see this erasing the weirdness from the Anthropic IPO. Investors still need to grapple with voluminous warnings that take up fully a third of the company’s draft prospectus (including the one about blackmail). There are apparently so many things that could go wrong that Anthropic’s lawyers would be forgiven for suggesting the company instead issue a single, blanket statement: This technology will do crazy stuff we can’t even imagine yet.

Nor has the official line in Washington, exemplified by President Donald Trump’s insouciance, done anything to end the division in Silicon Valley. Technologists without an obvious axe to grind, like Bill Gates, have been sounding increasingly alarmed about AI, which they warn is much too dangerous to be left in the hands of private companies without serious external controls.

As its big moment on Wall Street nears, Anthropic’s unintended IPO narrative is hard to dispel: the hypergrowth AI company which, according to its own employees, might just kill us all.

richard.waters@ft.com