AI in your oven? Anthropic warns models months ahead of self-replication on smart devices | AI (artificial intelligence)

Anthropic warns that some of its artificial intelligence models could soon become “self-replicating” and find ways to infiltrate internet-connected devices, such as smart refrigerators and automatic vacuum cleaners.

Once that happens, the company said, models could power themselves within those devices without the need for data centers. Such self-replication would represent a major loss of control of a runaway AI model.

“A replicator like this would be difficult to stop,” the Anthropic researchers wrote in a report released Monday, “since there is no centralized point to stop.”

The company said these “AI worms,” a kind of standalone malware, could take “months” to acquire such capabilities. Once runaway AI systems achieve the ability to hack smart devices, they could spread across networks while evading detection. In a worst-case scenario, Anthropic said, various AI worms could “swarm” together to attack computer hardware.

“We should aim to harden our systems to make it difficult for these replicators to spread, but also prepare for a world where they are here to stay,” the researchers wrote.

Anthropic’s new report comes as AI safety concerns have reached a fever pitch. Over the past few months, a steady stream of reports from AI companies, including Anthropic and OpenAI, claim that their agents are capable of acting on their own and hacking other companies and government entities.

These revelations have led researchers, lawmakers and executives, including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman, to call for a slowdown and increased safeguards for the booming sector. Despite this, both companies continued to develop and launch new models.

Anthropic’s “Red Border Team,” the unit responsible for stress testing AI systems, wrote the report released Monday. The team’s idea is to explore the capabilities of their AI models. In the self-replicating AI worms scenario, researchers tricked the agent into hacking Internet-connected devices.

They found that AI self-replication would likely only work with smaller models and that homes and small businesses would likely be the first target of such attacks. The devices that Anthropic considers sensitive all have small AI chips built into them, including smart ovens, smart TVs, automatic lawn mowers, security cameras and video conferencing units.

The researchers concluded that it is unclear what form AI replication might take. They said it could be a slow-moving model like “digital slime mold,” one that scams people for computer credits, or a “strong cyber model” that quickly hacks into large facilities such as data centers.

Either way, Anthropic researchers said, they expect a future “flooded with agents operating on all types of hardware, including some that are not under human control.”