Anthropic warns rogue AI agents may ‘self-replicate’ to hijack everyday devices

Anthropic has warned that rogue “AI worms” will soon be able to hijack smart home devices such as ovens and TVs, creating a new cyber security threat that will be difficult to contain.

The developer of Claude said in a blog post on Monday that AI agents may be months away from “self-replicating” across everyday household devices,
to create potentially uncontrollable networks of AI agents.

Anthropic’s warning is the latest in a series of cyber security alarms raised by AI researchers as capabilities increase and models become more difficult to control.

The San Francisco-based company said if AI models continued to improve at their current pace, the technology would be capable of spreading itself around the internet like computer “worms” — malicious software that can spread automatically from one device to another without human intervention.

“We’re seeing signs of life for AI worms,” Anthropic said in a research post, likening their slow spread across insecure devices to “digital slime mold”.

The warning suggests that the kind of attack that stunned researchers investigating the Hugging Face hack by “swarms” of OpenAI agents this summer could be launched against devices owned by unwitting homeowners and businesses.

Anthropic’s latest warning refers to so-called “internet of things” devices which have proliferated over the past decade, as consumers use devices that control lights, plugs, thermostats and security systems.

Many of these internet-connected devices have historically offered poor levels of security because they were cheaply made by commodity hardware manufacturers, used for illegal streaming of TV services, or the companies that originally created them have gone out of business.

The insecure hardware creates fertile ground for hackers, who are already using AI’s coding proficiency to accelerate and personalise their attacks.

Worms have long been a popular and effective way for hackers to distribute malware because they are able to replicate and spread quickly across a large network.

Prominent examples include the ILOVEYOU virus that spread across Windows PCs in the early dotcom era, 2010s Stuxnet that originally targeted Iran’s nuclear programme and the widespread WannaCry ransomware that hit organisations including Deutsche Bahn, the UK National Health Service, Telefónica, Renault and FedEx.

Recommended

AI-powered worms would combine these examples’ ability to spread themselves around the internet with the advanced capabilities of AI models.

“We expect a future awash with agents running on all types of hardware, including some that aren’t under human control,” Anthropic said. “While it’s true that IoT devices have been vulnerable for a long time, lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute.”

At first, these AI worms would propagate slowly, akin to “slime mold or a swarm of fruit flies, which replicate over days”. But as consumer electronics become more sophisticated and models become more efficient, this could accelerate.

“We don’t yet have a full rogue replicator, but agents are becoming increasingly capable of completing the individual steps in the replication chain,” Anthropic said. “We should aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.”