The Week Everybody Started Talking About AI Doomsday

This week, panic about an AI doomsday broke out from the confines of Silicon Valley into the public zeitgeist.

It started Tuesday when an Anthropic researcher told The Wall Street Journal he was quitting. His reason? He said he feared the leading lab and its competitors were racing to build systems that humans won’t be able to control. Shortly after, a current Anthropic researcher took to X to confirm the sentiment, adding he believes the risk the human race will go extinct in the next 10 years is higher than 10%.

Cue the existential dread.

My colleagues Amrith Ramkumar, Lindsay Ellis and I spoke with lawmakers, church leaders and even one filmmaker who are now being called to answer some hard questions: Is this real? Could AI really kill us all?

Despite the alarm bells, the view that AI could bring about our demise remains a fringe opinion among industry experts. Anthropic and OpenAI, two of the biggest AI labs developing this technology, have also said they take safety seriously.

And maybe the point of all this is to have a wider conversation to spur action.

“The window is closing and the technology is developing, but the cataclysmic incident hasn’t happened,” said Oscar-winning filmmaker Daniel Roher, who made a documentary about the risks of AI. His film—which featured sit-down interviews with OpenAI’s Sam Altman and Anthropic’s Dario Amodei—had a lukewarm reception when it was released in March, he said.

The doc will be released on streaming platform Netflix next week. “Our destiny is not written. We still have time.”

Read more in our article: America’s Great AI Freakout Has Begun

—Angel is a tech reporter based in San Francisco

• Biggest AI Rivals Agree They Need to Slow It Down

• Congress Is Suddenly Waking Up to the AI Doomsday Threat

• How Would AI Actually Kill Us All? What to Know About the AI Doomsday Debate

At its most consequential product event in years, Apple debuted a $1,999 foldable iPhone—its most expensive yet—a revamped iPhone 18 Pro (also with an upped price tag), plus new watches and updated AirPods.

What you need to know:

Brian Dema routinely turns to his father for business advice. His father passed away four years ago, but Dema created an AI version of him. AI experts say in a matter of years we will all be able to create similar AI likenesses of our loved ones. At that point, the real question will be: Should we?

Why should we care that AI is suddenly insanely good at math? It’s a question that I’ve been asking mathematicians and AI researchers for months, and especially this week, when OpenAI published a solution to one of the notoriously complex Millennium Prize Problems. AI’s progress in math is now giving us a peek at a world beyond our comprehension.

If all AI labs, including bitter-rival OpenAI, agree to slow their work, it could well benefit all of humanity—if Anthropic CEO Dario Amodei’s warnings are to be believed. That slowdown likely gives Anthropic, already a leader in the field, an advantage. But if Anthropic is the only one to heed the call, it falls behind OpenAI.

Facing a three-way arms race with Russia and China, the U.S. is turning to energy startup Pacific Fusion. Testing with fusion reactors helps ensure America’s nuclear weapons still work after decades in storage, components are reliable and that new designs will have the desired effect, without resuming the explosive nuclear tests of decades prior.

Iran used American-developed AI models to try to target U.S. Navy ships in the Middle East, according to an Anthropic report warning of emerging national-security dangers. An Iranian-linked group compiled information from ship and aircraft transponders, military photographs and commercial satellite photos to track American warships and look for vulnerabilities in their ship-born communications, the report found.

Cheat Code: Chinese AI companies attempted to clone American AI models by funneling real Chinese user queries to them via a network of intermediate platforms.

Burning Bright: The star AI researcher who was offered a pay package worth more than $1 billion to join Meta last year is leaving the company for Anthropic.

Meta’s Muse: The company announced a new personal AI agent that can buy goods online, respond to emails and complete other tasks users authorize it to do for them.

…when people working for AI companies can’t figure out how to spend all that money.

The Technology newsletter is a weekly digest of tech reviews, columns and headlines from Anne Steele and Wilson Rothman. Write to Wilson at wilson.rothman@wsj.com and Anne at anne.steele@wsj.com. Got a tip for us? Here’s how to submit.