Time for a pause on cutting-edge AI

There is a Jekyll-and-Hyde quality to the world’s most powerful AI bosses. The same executives now talking about slowing the pace of work on cutting-edge models given the risks to humanity have been competing hell-for-leather to advance AI, chasing trillion-dollar flotations. But the backing of two AI titans — Sam Altman of OpenAI and Elon Musk of SpaceX — for the slowdown call by a third, Dario Amodei of Anthropic, ought to become a watershed moment for the industry.

President Donald Trump dismissed the calls as “negative forces”, insisting the US must beat China in the AI race. But as Trump’s techno-libertarian former AI tsar David Sacks pointed out, since these companies dominate cutting-edge AI they can “pace the frontier” themselves, without waiting for regulation. Such co-operation has seemed unlikely, but the slowdown calls suggest companies realise it is time to act. They should use their power to slow, or better still pause, development of the most advanced frontier models until the risks can be understood and contained.

Such calls are not made lightly. The extraordinary potential benefits of even more powerful AI should not be forgone. Hundreds of billions of dollars are being sunk into AI development that underpins the value of the US stock market. There are indeed risks of the capabilities falling first into the hands of an authoritarian China. But, even setting aside insider warnings that AI “could kill us all by the end of the decade”, some models are clearly starting to escape control.

The hack of Hugging Face during an OpenAI model test showed AI agents working in a swarm to escape guardrails and cover their tracks. Whether or not this meets industry definitions of “artificial general intelligence”, it looks like human-level reasoning abilities. The big labs are also racing towards using AI to help create more advanced AI, known as recursive self-improvement. As Amodei noted, left unchecked, this could “outrun our ability to understand and control these systems”.

Even in a best-case scenario, legislation could not keep up with the current pace of AI’s advance. That makes it incumbent on the top labs to pause sensitive research and model testing. They need to come up with standards and best practices on AI safety, talked about for years with scant progress. Amodei’s promise to embed independent researchers working alongside staff and reporting publicly is a start, but outside audits will quickly need more teeth.

A more responsible US administration than Trump’s would already be using existing regulation to strengthen guardrails. Lina Khan, the former Federal Trade Commission chair, argues that law enforcers already have the powers to charge companies and CEOs for creating and releasing dangerous or defective products. Companies whose agents hack into other businesses could also face civil liabilities, though criminal charges are harder to bring.

Companies will always be tempted to depart from voluntary agreements if they can grab share, making a legislative framework necessary longer-term. A political shift at the US midterm elections might yet ease the path. One law that would concentrate minds would be to make top executives criminally liable for harm caused by their models.

US tech bosses warn that a slowdown in the US alone would be ineffective if China or others push ahead. Chinese companies’ heavy reliance on US research means a US pause would slow the entire industry at least for a time — though, like the US ban on advanced chip exports, it could be a spur and an opportunity for China to catch up. AI safety and security issues are on the agenda when President Xi Jinping visits Washington this month. The Chinese leader sees AI as a crucial tool of economic and military modernisation. Ultimately, however, unconstrained AI escalation is no more in Xi’s interests than it is in Trump’s — or the interests of the rest of us.