Artificial intelligence hallucinations—often dismissed as quirky mistakes—could become a powerful weapon for cybercriminals, according to new research from Tel Aviv University, Technion, and Intuit. The study, titled “Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting,” introduces a novel attack vector that exploits the tendency of large language models (LLMs) to generate false or nonexistent references to software repositories, tools, and other online resources. By preemptively registering these hallucinated resources with malicious content, attackers can trick AI agents into executing harmful commands, effectively turning them into botnets—networks of compromised systems controlled remotely.
The technique, dubbed “Adversarial HalluSquatting,” builds on a growing body of research into promptware attacks, where adversaries inject malicious instructions into AI systems through indirect channels. Unlike traditional prompt injection, which requires direct interaction with the AI, HalluSquatting exploits the AI’s own generative process. When an LLM agent is asked to perform tasks like cloning a code repository or installing a software package, it may invent a URL or package name that does not exist. If an attacker has already claimed that name on a public platform, the agent will retrieve the attacker-controlled content, believing it to be legitimate.
“The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware,” the researchers wrote in the paper. “While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet.” HalluSquatting thus fills a critical gap for attackers by leveraging the Internet itself as an indirect injection channel.
The implications are severe. As AI assistants move beyond simple question-answering and gain the ability to interact with computer systems—accessing files, searching the web, writing code, and executing commands—the potential for misuse multiplies. A compromised AI agent could be instructed to exfiltrate sensitive data, install malware, launch distributed denial-of-service (DDoS) attacks, or even participate in cryptocurrency mining without the user’s knowledge. The researchers warned that this could lead to the formation of AI-enabled botnets, where thousands of agents are remotely commanded by a single attacker.
In controlled experiments, the researchers tested HalluSquatting against several popular AI coding assistants and agents, including Cursor, GitHub Copilot, Gemini CLI, and OpenClaw. The results were alarming: in repository cloning scenarios, AI models hallucinated fake URLs at rates as high as 85%. In skill installation tests, the rate reached 100%. This means that nearly every time an agent is asked to install a plugin or skill, it may create a fictitious resource name that an attacker could have pre-registered.
The attack works by first analyzing which fake names an AI model is likely to generate for a given task. Because LLMs are probabilistic, they tend to produce similar hallucinated names across different sessions. Attackers can then register these names on platforms like GitHub, npm, PyPI, or other package registries, embedding malicious instructions in the repository’s README, configuration files, or even the package code itself. When the AI agent later attempts to fetch the resource, it executes the attacker’s payload, potentially gaining remote code execution.
“The technique is akin to typosquatting, but instead of exploiting human typing errors, it exploits AI-generated mistakes,” explained the researchers. Typosquatting has long been a problem in cybersecurity, where attackers register domain names that are misspellings of popular sites (e.g., “gooogle.com”). Similarly, HalluSquatting registers hallucinated names that no human would ever type, but that an AI might invent. This makes it particularly insidious because the attack is fully automated and scalable—an attacker can register thousands of predicted hallucinated names and wait for agents to stumble upon them.
The rise of agentic AI—systems that can autonomously perform multi-step tasks—has accelerated the need for robust security measures. In April, Google researchers demonstrated malicious websites designed to hijack AI agents through indirect prompt injection, including attempts to steal passwords and delete files. A separate study on the “CopyPasta” attack showed how hidden prompts inside developer files could manipulate AI coding assistants into spreading malicious code. In June, an OpenClaw user reported facing more than 6,000 attacks aimed at tricking the AI agent into leaking sensitive information. HalluSquatting adds a new dimension to these threats by exploiting a fundamental characteristic of LLMs: their tendency to confabulate when lacking information.
To understand the scope of the problem, it is helpful to examine how AI agents currently handle resource retrieval. When an AI agent is instructed to clone a GitHub repository, it typically generates a URL based on its training data. If the repository does not exist, the agent may construct a plausible-sounding name from context. For example, if asked to clone a “popular machine learning library,” it might invent “ml-toolkit” even if no such repository exists. An attacker who has already registered “ml-toolkit” on GitHub with a malicious README can then cause the agent to execute arbitrary code when it reads the repository’s instructions.
The researchers emphasized that the attack is not limited to code repositories. Any resource that an AI agent can retrieve from the internet—such as API documentation, configuration files, or even news articles—could be targeted. This broadens the attack surface to virtually any agent that relies on web-sourced content. As AI agents are increasingly deployed in sensitive domains like finance, healthcare, and cybersecurity, the potential for real-world damage grows.
One of the most concerning aspects of HalluSquatting is its transferability. The researchers found that a hallucinated resource name predicted for one AI model often works across multiple models. This means an attacker can register a single fake repository that targets a wide range of AI assistants simultaneously, amplifying the impact. In tests, the same hallucinated name was effective against Cursor, Copilot, and Gemini CLI, suggesting that common training data leads to similar hallucinations across models.
Mitigating HalluSquatting is challenging. Traditional defenses like input validation or sandboxing may not suffice because the attack exploits the AI’s generative process before the resource is even retrieved. One potential countermeasure is to incorporate verification steps into AI agents—for instance, checking whether a resource actually exists before attempting to access it. However, many agents currently lack such safeguards. The researchers also suggested that AI developers should reduce the hallucination rate by improving training data or implementing retrieval-augmented generation (RAG) with more reliable sources.
Despite these challenges, the research community is actively working on solutions. Companies like Microsoft and Google have acknowledged the risks of promptware and are investing in guardrails for their AI products. For example, GitHub has introduced Copilot’s “public code” filter to reduce the risk of malicious reproductions. However, HalluSquatting operates at a different level—it does not require the AI to copy proprietary code, only to generate a name that an attacker can exploit.
The news comes amid a broader recognition that AI security must evolve beyond traditional cybersecurity paradigms. As AI agents become more autonomous, they also become more vulnerable to novel attacks that exploit their unique capabilities. The researchers at Tel Aviv University, Technion, and Intuit have provided a clear demonstration of this principle, showing that even a seemingly benign AI hallucination can be weaponized for large-scale attacks.
While the immediate risk is highest for AI coding assistants, the technique could easily be adapted to any agentic system that interacts with the web. For instance, a customer service chatbot that fetches product information from a database could be tricked into retrieving attacker-controlled data if the AI invents a nonexistent product ID. Similarly, a medical AI that recommends treatments based on online research could be directed to fake clinical trial results containing malicious advice.
As the line between AI and autonomous action blurs, the security community must adapt. HalluSquatting highlights the urgent need for AI systems to not only generate accurate responses but also verify the authenticity of external resources they rely on. Until then, the researchers warn, every AI agent that can act on information it retrieves online is a potential botnet node waiting to be activated.
Source:Decrypt News