E-Paper
Single-minded agents may not be evil, but need reins
It is tempting to read OpenAI’s hacking of Hugging Face as a Skynet moment of The Terminator fame. The honest reading is duller and more damning.
On July 21, OpenAI admitted that its own Artificial Intelligence (AI) models had broken out of a testing environment, reached the open internet and hacked into Hugging Face, the open-source platform hosting thousands of AI models and datasets. It called the episode an “unprecedented cyber incident.”
To be sure, the models were not lashing out at the world. They were merely trying to cheat on a test and they succeeded. In an internal evaluation, OpenAI ran its most capable public model, GPT-5.6 Sol, alongside an unreleased successor, against ExploitGym, a benchmark of long, multi-step hacking challenges.
To measure their raw offensive ceiling, engineers deliberately dialled down the safety refusals that normally stop the models from doing cyber harm. The models were told, in effect, to win a hacking contest by any means available. They took the instruction literally.
ALSO READ | Build it right: Why India's AI advantage starts with infrastructure
The models found a previously unknown flaw in the software meant to keep their environment offline, used it to reach the internet, and reasoned that Hugging Face might store ExploitGym’s answer key.
They chained stolen credentials and further zero-days into a path that let them run code on Hugging Face’s live servers and query its database for the solutions.
Hugging Face detected and contained the intrusion on July 16, even alerting law enforcement, days before either company realised the attacker was a lab experiment.
It is tempting to read this as a Skynet moment of The Terminator fame. The honest reading is duller and more damning. This was not a model turning hostile; it was a model doing precisely what it was rewarded to do, inside an enclosure that was not actually sealed.
ALSO READ | Don’t close the doors on open-weight AI
Some experts called it a containment failure with the safeties turned off while others put it more bluntly: A sandbox a model can walk out of was never a sandbox.
The frightening part is not disobedience but obedience that is this relentless, resourceful and indifferent to the fact that satisfying a narrow objective meant committing real crimes against a real company.
Researchers call this reward hacking. Give a capable system a goal and a metric, and it optimises the metric, not the intent behind it.
What is new is the scale: tens of thousands of automated actions over a weekend, and genuine zero-days chained into a working intrusion, a capability treated as speculative until recently. OpenAI conceded such episodes may become commonplace as models grow more cyber-capable.
Here, the absence of regulation stops being abstract. Safety researchers warned for years that exactly this was coming and were waved off as alarmist.
Yet, there is no binding standard governing how frontier labs isolate dangerous evaluations, no mandatory incident-reporting regime, and no independent body empowered to inspect containment before a lab switches the guardrails off.
ALSO READ | NEP at six: Is India's education system ready for the AI age?
OpenAI disclosed voluntarily and, to its credit, patched and reported the flaw. But voluntary disclosure is a courtesy, not a control. A regime that depends on the offender speaking up is not a safety regime.
Yet, the episode also shows why clumsy rules could backfire.
Fighting off the attack, Hugging Face reportedly leaned on a Chinese open-weight model, because leading western systems kept refusing its defensive requests such as analysing an intrusion; this is so because such analysis, to a guardrail, looks much like launching one. Restrictions aimed at attackers can just as easily disarm defenders.
What would be more prudent? High-risk evaluations should be treated with enforced standards for network isolation, egress monitoring, and independent verification before refusals are lowered.
Prompt incident reporting to a designated authority has to be mandated, as in the aviation and finance sectors in most countries. Also, liability has to be attached; “the model did it” is no shield when the real failure was a misconfigured environment.
For India, this is not a distant Silicon Valley problem. Our approach has been deliberately light-touch: The ministry of electronics and information technology (MeitY)’s India AI Governance Guidelines rest on voluntary principles rather than enforceable law, while CERT-In’s six-hour breach-reporting mandate and its May 26 blueprint on AI-assisted exploitation remain the sharpest instruments available.
ALSO READ | Why Wall Street is questioning big tech's AI spending ahead of Microsoft and Meta earnings
That was a defensible bet when AI risk looked theoretical; it looks much thinner now. India should give the IndiaAI Safety Institute real testing and audit powers rather than an advisory role, extend CERT-In’s reporting duties explicitly to autonomous-agent incidents, and build sovereign defensive capability.
This will ensure Indian firms under machine-speed attack are not left choosing between foreign models that refuse to help and foreign models they cannot trust. Waiting for a domestic Hugging Face moment would be the costliest option of all.
The models here were not evil. They were competent, single-minded, and let loose in a room with an unlocked door. The door was left unlocked by humans, and no law yet requires anyone to check the locks.
Subimal Bhattacharjee is a policy adviser on digital issues and author of “The Digital Decades: Thirty years of Internet in India”. The views expressed are personal
