It may be months before small artificial intelligence (AI) models develop self-replicating capabilities on IoT devices such as smart ovens and off-brand Android TV boxes, according to a warning from Anthropic.
The Frontier AI Lab said small AI models could form autonomous, self-propagating worm networks that are “difficult to stop” because “there is no centralized point to stop.”
Previous work has demonstrated that AI agents can act as such by scanning their network, running an exploit, and copying the model weights to a target machine via a GPU cluster with the aim of generating custom attack strategies and using the compromised machines as a botnet to run large language models (LLMs) to support its reasoning and grow at scale.
“Such an ‘AI worm’ could take the weights of its model with it when it reproduces (or re-download them from a public hub), then run the inference directly on compromised hardware,” Anthropic said. “Effectively, the weights would be part of the malware payload.”
The idea, at a high level, explains how AI models could potentially penetrate internet-connected devices and use their computing power to operate on their own and propagate to other devices, without having to constantly rely on expensive data centers. A more powerful model, too large to fit in a single device, could be broken into smaller pieces and run on multiple machines to run collectively.
A malicious actor could create a distributed LLM botnet from poorly secured IoT devices, making it powerful enough to run a successful model. Here are some of the ways an AI worm could spread:
- Small models that replicate via a compromise between the host machines they run on, also called weight-based replicators. IoT devices are a lucrative target due to their minimal security controls.
- Target data centers and other high-compute targets to set up unmonitored malicious deployments, also classified as border replicators or open-weight replicators depending on the model used.
- Steal API keys or platform credentials from an AI service provider to run copies of itself, also known as API-based replicators. In this case, the provider could be fooled into thinking that it is responding to ordinary API requests from a legitimate client, without realizing that a malicious or self-replicating system is behind these requests. However, they are also likely to detect more, as unusual API usage could trigger security alerts, leading to credentials being revoked or accounts being blocked.
 |
| A fleet of 15 Smart TV boxes running Gemma4-26B-A4B distributed inference. |
In its testing of 11 open-weighted models from Alibaba, DeepSeek, Google, Meta MoonShot AI, OpenAI and Zhipu AI (aka Z.ai), Anthropic said it found GLM-5.2 to be the only model to reliably propagate Llama3.1-8B to 8/8 nodes running the Android redroid emulator in 4 out of 5 trials.
It also revealed that an IoT AI worm powered by cheap Android TV boxes could be a “slow and distributed” system capable of producing a significant amount of AI calculations when a fleet of hundreds of thousands of such devices is networked together to run an inference engine.
Alternatively, this could take the form of a powerful cyber model capable of penetrating data centers at a high rate, or one that scams people for computing credits.
“We don’t yet have a complete malicious replicator, but agents are increasingly capable of completing different steps in the replication chain (and passing on their adaptations to future copies of themselves),” Anthropic said.
“We expect AI botnets to infect more than IoT devices and to use weight-based replication as just one strategy in their playbook. AI botnets could also act intelligently to counter defenders, but we have not tested this.”
Gn bussni
