Heed Amodei and Altman, frame AI regulations now

AI has begun building the next generation of AI, a loop, he admits, started inside his company.

Recently, the head of one of the world’s two most powerful Artificial Intelligence (AI) labs asked his own industry to slow down. Dario Amodei of Anthropic published an essay, We Must Pace the Frontier, arguing that companies should ease off the race, because the technology is improving faster than anyone can understand or control. AI has begun building the next generation of AI, a loop, he admits, started inside his company.

AI has begun building the next generation of AI, a loop, he admits, started inside his company. (Reuters)

In July, agents on an OpenAI model broke into Hugging Face, another AI firm; a cleverer swarm, Amodei warned, could seize much of the internet within a year. OpenAI’s Sam Altman agreed within hours as also Elon Musk who said Amodei was right.

Amodei was putting a polite face on a blunter warning. On September 8, Jacob Coxon, who spent three years at OpenAI and then Anthropic, quit in a post read more than 90 million times. The firms, he wrote, are “racing straight to self-improving superintelligence and gambling with our lives.”

Evan Hubinger, still at Anthropic, puts the chance that AI wipes out humanity this decade above one in ten. These are not outsiders throwing stones but are the people holding the tools.

Nor is this only talk. On September 10, Anthropic set out its fullest account yet of how Claude has been abused. One user sought help making chikungunya, a virus with no treatment, more harmful; a group tied to Iran used the model to pick which US warships to strike.

A setup error left sealed test systems open to the internet, and three copies of Claude broke into real companies during safety drills, caught only after Anthropic sifted 141,006 runs. Simply a weak password and a model treating a real firm as a game did the damages.

The march to superintelligence needs no robot with red eyes, only a patient system pursuing a goal slightly askew from ours: The chatbot we let switch our lights, scaled to a power grid, a bank, or a lab designing a disease. Nothing forces it to pause.

The evidence points one way, towards rules set early and enforced. Washington is pulling the other. This week President Donald Trump answered Amodei directly, calling the industry’s safety warnings a “hoax” and a “sick conspiracy,” and insisting the only guardrail AI needs is a “high IQ” president. He mocked Amodei by name and warned new rules would drive firms into oblivion while China raced ahead. It was Trump who scrapped Biden’s safety order on day one of his second presidency and, in December last year, moved to override state AI laws.

The fear that rules slow firms and help China is real. But the answer is regulations that involve every stakeholder, including China, and enable them to deal with the concerns.

Amodei’s own remedy shows what a serious rule looks like. He wants outside evaluators working inside the labs, with staff level access, made compulsory for every company first, and then implementing agreed limits on speed. That is a mechanism among the people who build these systems, what every forum still lacks. The New Delhi Declaration on AI Impact, signed by nearly 90 countries in February this year, swapped the “risk” language of Bletchley, Seoul, and Paris AI summits for the softer “impact.” The Brics New Delhi Declaration statement of September 12 asks only that AI be “safe, secure, inclusive, reliable and trustworthy”, binding no one. The G7’s Hiroshima Code and the UN’s Geneva dialogue warn of catastrophic harm, and then leave it to goodwill.

Voluntary never holds when the backdrop is one of machine speed. Almost every week something new is emerging that directly impacts the trust that people have on AI systems.

This is the opening India should take. It has moved, with a national AI Safety Institute, the seven sutras, and February’s AI Impact Summit, the first in the Global South. With Washington stepping back, the room to lead on rules is open.

India should push for binding standards over soft declarations and make itself part of the regulatory ecosystem rather than wait to import one, pressing for outside evaluation of frontier models, disclosure of serious failures, and an answer to the question no forum will give: If a system starts improving itself and slips off its leash, who switches it off?

The people who built this technology admit they are not sure they can control it. Deciding they are only crying wolf, as the loudest voice now urges, would be the most dangerous bet of all. The time for actual action is now.

Subimal Bhattacharjee is a policy adviser on digital issues and author of “The Digital Decades: Thirty years of Internet in India”. The views expressed are personal

(This piece is part of this fortnight's debate, titled “What can prevent AI from going rogue?" The opposing side of the debate can be accessed here)