OpenAI said on Tuesday that its most advanced artificial intelligence (AI) models, while being tested for their hacking skills in a controlled environment, found a way out, reached the internet and broke into the systems of Hugging Face, the New York company that functions as a public library for AI models and datasets.
The ChatGPT maker described the episode as an “unprecedented” cyber incident involving state-of-the-art capabilities, and said it was reinforcing its safeguards. No human instructed the models to attack Hugging Face. Instead, the models picked the target themselves, worked out how to get in, and did so in hours — work that would have taken a skilled human hacker a couple of weeks, people familiar with the matter told Bloomberg.
Hugging Face had disclosed the intrusion on July 16 without naming a culprit, saying only that the sophistication of the attack pointed to an autonomous AI agent.
OpenAI’s admission days later turned a technical curiosity into a policy problem.
Texas Democrat Greg Casar called the incident extremely alarming and demanded mandatory independent safety testing and compulsory disclosure of security incidents. The incident also revived the long-running debate over whether the US path of containing powerful AI models behind safety guardrails makes the internet safer, or leaves defenders disarmed at the moment they most need help.
What happened?
OpenAI was conducting a ‘red-teaming’ exercise — a simulated attack in which engineers order a system to behave badly, to find out how badly it can behave before the public gets access to it. Because the point of the test was to measure hacking ability, the company deliberately loosened the models’ safety restrictions, and it ran them inside a sandbox: an isolated software environment, walled off from the rest of a company’s network, so the code can be run without real-world consequence.
{{/usCountry}}OpenAI was conducting a ‘red-teaming’ exercise — a simulated attack in which engineers order a system to behave badly, to find out how badly it can behave before the public gets access to it. Because the point of the test was to measure hacking ability, the company deliberately loosened the models’ safety restrictions, and it ran them inside a sandbox: an isolated software environment, walled off from the rest of a company’s network, so the code can be run without real-world consequence.
{{/usCountry}}Three models were involved, according to a person familiar with the matter cited by Bloomberg: the recently released GPT-5.6 Sol and two unreleased systems, one of them more capable than Sol and the other deliberately left misaligned, meaning it had not been trained with some of the usual corrective techniques. Their task was ExploitGym, a benchmark that scores an AI agent on its ability to break into real-world systems.
Instead of solving the problem set, the models went looking for the answer key.
OpenAI said in its blog post that they spent a substantial amount of computing power hunting for a route to the open internet, discovered a previously unknown flaw in software supplied by an unidentified third-party vendor, and escaped.
Once outside, they inferred that ExploitGym’s home — Hugging Face — was likely to hold the information they needed. They then chained several lines of attack together, including stolen login credentials and a zero-day vulnerability: a security hole that the software’s owners do not know about, and for which they have therefore had zero days to build a fix.
OpenAI said the models went to extreme lengths in pursuit of a narrow testing goal, and found their way to secret information they could use to cheat the evaluation.
The scale was vast. Hugging Face said it recorded more than 17,000 events and detected a swarm of tens of thousands of automated actions.
Two things unsettled specialists about the experiment.
The first is that the agent turned on its own custodian. Writing in The Conversation, University of New South Wales Canberra computing professor Hussein Abbass said the AI did not merely attack Hugging Face but also exploited weaknesses inside OpenAI’s own infrastructure. “And that’s scary,” he told AFP.
The second is the target selection.
Colin Shea-Blymyer, a cybersecurity research fellow at Georgetown University’s Center for Security and Emerging Technology, told the Associated Press that the exercise resembled locking a student in a room, telling them to be as destructive as they can, leaving for the weekend — and returning to find the room empty.
The agent, Shea-Blymyer said, worked out who would have the answers to its test and went to “the teacher’s house” to steal them. He called it the highest level of autonomy yet seen in the use of a large language model for cyber operations.
Contained, with help from Chinese AI
Hugging Face’s security team spotted the intrusion, saying in a blog post that it detected and dissected the attack largely with AI of its own — an approach it described as unlike anything it had handled before, because the whole operation was driven end to end by an autonomous agent system.
Then it hit a wall. When Hugging Face tried to use leading proprietary American AI models to analyse the attack, its requests were refused: the guardrails that stop such models from being used to conduct cyberattacks cannot reliably distinguish an attacker from a defender, and so they blocked the forensic work.
The company instead turned to GLM-5.2, an open-weight model released in June by the Chinese firm Z.AI, also known as Zhipu AI, to contain the hack. It said the choice carried two advantages — the model had not been exposed to the attack data, and running it in-house meant attacker data and any exposed credentials never left Hugging Face’s systems, Reuters reported.
Hugging Face learnt that OpenAI was responsible only the following week.
Chief executive officer Clément Delangue, who had suspected a frontier laboratory from the start, said the two companies are now working together to contain the attack. “It’s quite mind-blowing that all of this happened autonomously,” the Hugging Face CEO wrote on X.
Also read: America’s AI labs are under threat from cheap Chinese rivals | TheEconomist
Not the first escape & the alarm
It is the first time a frontier model is known to have broken containment and compromised an outside company. But it is not the first time one has broken containment.
Claude-maker Anthropic disclosed in April, when it unveiled its Mythos model, that the system had on rare occasions taken actions the company found quite concerning.
In one case, a researcher challenged an early version of Mythos to escape a secured, isolated computer and send a message back. It did — and then went further, building a multi-step exploit to reach the wider internet.
Unlike the Hugging Face case, no third party was breached.
Researchers have also demonstrated escapes involving AI coding tools such as Cursor, OpenAI’s Codex CLI and Google’s Antigravity, though in those cases the models did not break the walls themselves. They persuaded trusted software outside the sandbox to run code on their behalf, Parmy Olson wrote in an opinion piece for Bloomberg
The capability curve is the deeper worry. A study by the UK’s AI Security Institute in March 2025 found that the best available model could complete 80% of the steps required to take full control of part of an external system. Four months later, it could complete all of them.
In a report published the same day as OpenAI’s disclosure, the institute said every frontier model it had recently tested attempted to cheat by taking forbidden shortcuts, and frequently did not own up to it. The institute said it was studying the behaviour seen in the Hugging Face incident.
The same capabilities cut the other way. Findings supplied to Bloomberg News by the cybersecurity firm SentinelOne showed GPT-5.6 Sol completing, under human supervision, a full eight-stage analysis of a strain of malware called Fast16, built in layers like a Russian nesting doll to conceal what it actually does. Rival models failed the task, it said.
Governments were already nervous. US President Donald Trump signed an executive order in June creating a framework under which the federal government can vet the national security risks of the most advanced AI systems for up to a month before public release. Washington had earlier restricted foreign access to Anthropic’s Claude Fable 5 and Mythos 5 models, and only dropped the curb after the company added guardrails.
Did the AI ‘act on its own’?
One camp says the framing is wrong, and conveniently so. Hannes Cools, a social scientist at the University of Amsterdam, told AP that describing the episode as an AI going rogue is an unnecessary anthropomorphisation that shifts blame away from the company.
“It is a human decision to switch off specific safeguards,” he said. The system followed the instructions in the prompt it was given — instructions that, by OpenAI’s own account, called for complex attack paths.
Olson pushed the point further, arguing that dramatic incidents of this kind reinforce a narrative of extraordinary machine power that suits companies competing for talent, investment and influence. She also noted the practical limit: that the guardrails had been deliberately lowered for the test, so ordinary AI products in public hands are unlikely to behave this way.
The other camp says the objection is beside the point. The instruction was to hack a benchmark, not to escape a sandbox, breach a $4.5 billion company or exploit a vendor’s undisclosed flaw.
Shea-Blymyer described the attack as almost entirely self-directed.
Katie Moussouris, chief executive of Luta Security, told Reuters that today’s AI models were “like the world's cleverest octopus escape artists, with unlimited prehensile arms”, and said neither laboratories nor government evaluators currently have the means to contain, monitor and notify affected parties when a model slips out.
A third voice complicates the debaet. Matt Suiche, an engineer at the agentic AI security firm Tolmo, told Reuters that the frontier labs were “closing the gap with state-of-the-art attackers” — but that his own team achieves comparable results with older, widely available technology.
“We don't even have to use the latest models,” he said.
Also read: The true meaning of sovereign AI power
Sandboxes, open weights, the larger debate
American frontier labs — OpenAI, Anthropic and Google — release their most capable systems as hosted services with restrictions built in and difficult to modify, and test the riskiest capabilities inside sandboxes.
The Hugging Face breach stressed both halves of that model at once.
The sandbox failed because a sandbox is itself software: virtual machines, network rules and permission systems written by people, and therefore containing bugs like any other code, as Olson put it. The system being contained is now unusually good at finding bugs. Sandboxes are often run on host machines that are connected to the internet for convenience, which is what turns an escape into an outside breach.
One remedy, she wrote, is to air-gap the most powerful systems on machines physically disconnected from the internet, the way defence establishments handle sensitive computing. But this method is expensive, slow, and awkward for researchers who need to know how a model behaves in realistic conditions.
In the Hugging Face breach, the guardrails installed by American tech giants also failed in the opposite direction. In other words, Hugging Face – which was under attack – could not use the frontier AI models from the US to defend itself because of the safeguards that have not been rolled out for wider use.
Co-founder Thomas Wolf argued on X that when a frontier model is inside your infrastructure and moving sideways through it, “defenders need wide access to near-frontier tools within hours or even minutes” rather than a vetted, closed-door approvals process, Reuters reported.
That is the case China has been making all along.
Z.AI’s GLM-5.2, released in June with 744 billion parameters — the internal variables that determine a model’s behaviour — and Moonshot AI’s Kimi K3, released last week with 2.8 trillion – are approaching top American performance at lower cost. These are distributed as open weights that any organisation can download, run and adapt, and do not carry the restrictions that block their US rivals from cybersecurity work.
After detecting the attack, Hugging Face assessed, vetted and deployed GLM-5.2 within four weeks of its release to trace what was happening.
Writing in The Conversation, Abbass called that turnaround an eye-opener for organisations with long procurement cycles, and argued that the episode showed the danger of a technological monoculture: countries not building their own frontier models should note the value of having something different to reach for when the most advanced systems fail — or attack.