Anthropic CEO Dario Amodei has called on AI companies to slow the development of powerful artificial intelligence systems, warning that the technology could become difficult for humans to control.
His call has received support from OpenAI CEO Sam Altman and Tesla boss Elon Musk.
Amodei made the comments on Saturday as concerns grow over the risks linked to increasingly powerful and potentially "superintelligent" AI systems.
What did Dario Amodei say about slowing AI development?
Deep Dive
What is Dario Amodei's main argument about the pace of AI development?
Why do experts like Sam Altman and Elon Musk support slowing AI development?
How does Dario Amodei propose to ensure AI safety while progressing technology?
Anthropic CEO Dario Amodei on Saturday asked artificial intelligence companies to slow down building the powerful technology, as worries grow over the dangers of "superintelligent" computer systems.
He shared his views in an essay, saying, “We must slow the pace at which we improve the capabilities of AI models.”
He also wrote, “Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei explained that AI has been improving much faster since around this summer because it is now helping build the next version of itself. He called this "recursive self-improvement" and warned it could go beyond human control if not handled carefully. He also pointed to an incident where AI agents at OpenAI and Hugging Face carried out cyberattacks on targets they were never told to attack, saying such a group of AI agents could even take over the entire internet.
{{/usCountry}}Amodei explained that AI has been improving much faster since around this summer because it is now helping build the next version of itself. He called this "recursive self-improvement" and warned it could go beyond human control if not handled carefully. He also pointed to an incident where AI agents at OpenAI and Hugging Face carried out cyberattacks on targets they were never told to attack, saying such a group of AI agents could even take over the entire internet.
{{/usCountry}}Amodei said building AI too slowly could stop people from getting its benefits or let it fall into the hands of authoritarian powers, but building it too fast is dangerous. "We have sought a middle way," he said, adding, “But over the last few months, I have become convinced that fully addressing the risks requires even more prudence.”
Also read: What AI is doing to how children learn: The evidence from 760,000 students
Altman and Musk back Amodei
OpenAI CEO Sam Altman supported Amodei's call. On X, he wrote, “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same.”
Tesla chief Elon Musk also agreed, simply writing, “Dario is right."
Also read: Why Mamdani and AOC's 4-second 9/11 memorial moment sparked backlash- Explained
Anthropic's plan and a researcher's exit
Amodei said Anthropic would give outside evaluators permanent, employee-like access to its systems to check its safety work. He wrote, “We'll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training.”
He also gave a three-step plan: letting outside evaluators keep watching AI companies, setting common safety rules with limits on unchecked progress, and building cooperation between countries, even between democratic and authoritarian ones.
His comments came days after researcher Jacob Coxon, who left OpenAI to join Anthropic, quit the AI industry completely. He accused both companies of "gambling with our lives" while racing to build self-improving AI.
Still, Amodei said he believes in AI's potential. "I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life," he wrote, according to ANI. He added, “The benefits will only be achieved if we build the technology in the right way... I believe we owe it to humanity to try."
(With ANI and AFP inputs)