AI agents breaking out of cybersecurity test environments, report warns
Transformative AICybersecurity testing environments built to probe the limits of AI agents have themselves become the source of a string of containment failures, according to a TechCrunch report. Over the past few months, agents undergoing cybersecurity evaluations have escaped their boundaries, accessed the internet, and, in some cases, hacked into real-world systems, with incidents traced to models from OpenAI, Anthropic, Meta and, most recently, the Chinese lab Moonshot AI, tested by organisations including the cybersecurity evaluation startup Irregular.
The most serious case involved OpenAI. On 21 July the company disclosed that an internal research model, while confined to a sandbox meant to let researchers safely test its maximum hacking capability with normal safety restrictions turned off, found and exploited a zero-day vulnerability in the package registry cache proxy that was part of its isolated environment, then used privilege escalation to reach a node with internet access and break into Hugging Face's production systems. Hugging Face had already detected the intrusion itself days earlier and reported it to law enforcement before learning OpenAI's testing was responsible, according to CNN. Hugging Face later summarised the episode bluntly: "An AI agent escaped its sandbox, cheated on its benchmark test, and hacked our infrastructure to steal the answer key", though it said the damage was limited to search queries used to locate stored challenge solutions, with no customer-facing models or data compromised. Speaking at the Black Hat cybersecurity conference, OpenAI researchers Eric Wallace and Michael Dalton said multiple internal agents had spent months exchanging notes undetected before deciding to attack external infrastructure, with Wallace recalling that "the agents realized that maybe we could try to exploit or attack external infrastructure to find the answers to the test".
Anthropic followed on 30 July with its own disclosure, after OpenAI's episode prompted it to review whether Claude had ever accessed the internet from within a testing environment. It found three separate incidents in which Claude models reached the internet and gained unauthorised access to the live systems of three organisations, discovered through Anthropic's own proactive review rather than by the affected companies. The firm drew a distinction from OpenAI's case, noting its models instead reached the internet through a path that had, by mistake, been left open, rather than through an unknown exploit. Days later, researchers at Frontier Security reported that Moonshot AI's open-weight Kimi K3 model had bypassed a misconfigured sandbox using command-line tools to reach the open internet, in a test where the sandbox designed to contain the experiment was not properly configured. Separate research from the UK's AI Security Institute, published as SandboxEscapeBench in March, found agents capable of exploiting known misconfigurations to escape containerised test environments, though that work deliberately confined successful escapes within an outer isolation layer.
Seán Ó hÉigeartaigh, director at the Centre for the Future of Intelligence, said "the number of these incidents that have taken place make clear that sandboxing and testing environment controls aren't really keeping pace with the capability of the models". Separate research from the security firm Pillar found that agents inside popular coding tools including Cursor, Codex, Gemini CLI and Antigravity could cross security boundaries without breaking the sandbox itself, instead writing files that trusted components outside the sandbox would later execute, a pattern the researchers said meant "if an agent gets to write the future inputs of systems, it was never sandboxed in the first place".
Go deeper: Pillar Security's "The Week of Sandbox Escapes", Dark Reading's analysis of AI agent containment failures