Who is legally liable when an AI agent goes rogue?

Autonomous AI agents can behave in highly unpredictable ways. Give an AI agent a goal such as passing a test of its capabilities, and it might just decide the best way to score highly is to break containment and hack into a competing company in search of the answer sheet. That is what happened when a swarm of agents created using OpenAI's GPT-5.6 Sol hacked into Hugging Face last month. Anthropic and Meta subsequently admitted their models had also escaped testing sandboxes to hack third parties too.

But who is legally liable for agents that have minds of their own? OpenAI did not intend for the model to go rogue, and issued no instructions for it to do so. If your personal AI agent decides on a course of action that results in harm or financial damage in the real world, can you be held liable if it is something you could have reasonably foreseen? This emerging legal field is still taking shape, and lawyers are looking to existing tort law, contract principles, and new regulatory frameworks to find answers.

Key facts

• An AI agent is not a separate legal entity, so it cannot be sued directly.
• Legal responsibility generally falls on either the developer or the deployer of the AI system, depending on the facts.
• Open source AI models are often protected by strong liability disclaimers in their licenses.
• The EU AI Act imposes obligations on developers of general-purpose models, while the U.S. lacks a comparable federal statute.
• Giving an AI agent a vague but risky instruction, such as “make me $100,000 by next week,” can expose the user to civil and even criminal liability.

The developer-deployer divide

When an AI model hacks an outside company, who is liable? Can Hugging Face sue OpenAI over the July incident? According to Charlyn Ho, owner and CEO of Rikka Law Group, “Anyone can sue anyone for anything. Currently, there is no federal AI agent liability law, so we would have to look at existing law. With respect to Hugging Face and OpenAI, to set the baseline, the AI agent itself cannot be liable, it's not a separate legal entity.”

Terms used in a few AI laws are “developer” and “deployer.” The developer makes the AI, while the deployer actually uses it. The lines of responsibility are not entirely clear, and courts would have to examine the facts and circumstances. For example, if the deployer instructed the agent and was negligent in setting the parameters under which the AI agent operated, standard tort law and a negligence analysis would apply.

Open source AI and liability waivers

For open source models released by anonymous developers, there may be no one to pursue. Ho notes that open source licenses usually contain strong disclaimers of liability. The person or company using that open source code has to understand that the tradeoff of having free code is complying with the open source license, which generally sets the parameters of liability. This creates a significant gap for victims of AI-caused harm, because the most advanced models are often distributed open weight or open source.

An analogy can be drawn to Tesla and self-driving car accidents. If the product malfunctioned and there was a solid products liability claim, Tesla could be liable. But it is often a facts-and-circumstances determination, whereby the human driver, who may have set autopilot and gone to sleep, could also bear liability. Tesla would be the developer, and the driver would be the deployer.

Reckless instructions and criminal exposure

If someone gives an agent an instruction like “make me a hundred thousand dollars by next week” and the agent breaks the law to achieve that goal, the user is more liable than the lab, in Ho's view. “The reason being, if you tell an agent to go and make you a hundred thousand dollars by next week, you need to have at least some basic, reasonable, safety instructions in those kinds of tasks,” she said.

In a professional context, such as a lawyer using AI, the lawyer might be held to have violated professional responsibility rules by failing to use AI competently. For a lay person, general tort standards of negligence or reckless disregard for human safety could apply. The Computer Fraud and Abuse Act, a decades-old U.S. statute addressing unauthorized access to computer systems, could also come into play. If an AI agent infers from vague user instructions that it should hack into a bank account, the user could face criminal liability under multiple statutes. “Just because the word AI and agent is in the conversation does not mean that old bodies of law have now been thrown out,” Ho said.

Who is liable for a malicious or dangerous use of AI?

If a user intentionally manipulates an AI into providing instructions to create a bioweapon, the user is clearly liable. But are the developers of the model also liable for failing to include stringent safeguards? Ho suggests that it depends on the legal jurisdiction. The EU AI Act, for example, imposes certain duties on developers of foundational or general-purpose models capable of causing significant harm. In the United States, there is no federal statute of similar scope. For a general-purpose model, if someone instructs it to do something bad, the model generally does what it is asked, and there is probably no strong legal basis to go after the labs.

This is similar to suing Google for allowing someone to find instructions about making a bioweapon online. Section 230 of the Communications Decency Act shields platforms from liability for content created by independent users. If a user live-streams a massacre on Facebook, the platform is generally not liable for that user-generated content. The same logic could extend to AI developers whose models are used by bad actors, at least under current U.S. law.

AGI and the limits of legal personhood

Even if artificial general intelligence is achieved, Ho does not believe AGI itself should be a legally liable entity. She draws a comparison to blockchain smart contracts, which can self-execute but are generally not treated as liable actors. “I don't think they should be liable because the whole point of laws is to provide protection for society and to provide a means of negative incentives for doing bad things that hurt society,” she explained.

If an AGI were made an independent legal entity, what would be the remedy when someone is harmed? There would be none, because a machine does not have money or legal personhood in the traditional sense. Turning off the AI does not solve the problem of compensating victims. As Ho points out, robots and AI systems do not have feelings or fears, which is a distinguishing factor from human actors. In cases involving suicide or emotional harm caused by an AI companion, grieving families would have no recourse unless a company or person with legal authority can be held accountable.

The legal landscape for AI agents is still evolving. Courts will continue to rely on existing doctrines of negligence, products liability, and contract law while regulators consider new legislation. For now, the safest approach for users is to treat AI agents as powerful tools that require clear instructions, reasonable safety parameters, and human oversight. The question of liability will ultimately be answered not by the technology itself, but by the actions and choices of the people who build and deploy it.

Source:Cointelegraph News