E-Paper
'We really believe AI could kill all humans': Anthropic safety lead after co-worker resigns
Evan Hubinger, the Alignment Science Lead at Anthropic, had acknowledged that AI killing humans is a very real possibility.
Prefer HTon Google
Evan Hubinger, the Alignment Science Lead at Anthropic, had acknowledged that AI killing humans is a very real possibility. Hubinger’s admission came hours after his colleague and Anthropic researcher Jacob Coxon publicly announced his resignation from the AI lab, claiming that self-improving AI systems could end humanity within the next decade.
Coxon announced his resignation from Anthropic on X this morning, citing reservations about the company’s actions. He claimed that AI labs are building self-improving AI systems that they cannot control, which in turn could spell doom for humanity.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” the 27-year-old former Anthropic researcher said in a grim warning.
(Also read: Anthropic researcher quits over ‘out of control’ AI fears, says ‘AI could kill us all by end of decade’)
Anthropic safety lead acknowledges risk
Evan Hubinger, the Alignment Science Lead at Anthropic, responded to Jacob Coxon’s post on X, admitting that the risk of AI ending humanity is very real.
“Jacob is correct here—we really do earnestly believe AI could kill all humans!” Hubinger wrote.
“I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger admitted.
The Alignment Science Lead at Anthropic said that he believes the risk of AI killing humans within the next decade is more than 10%.
AI alignment science is the research field dedicated to ensuring that artificial intelligence systems act in accordance with human instructions. Essentially, it makes sure that AI interprets what humans actually want rather than misinterpreting instructions and causing harm.
The risk of superintelligence
Hubinger confessed that Anthropic does not have a plan to solve alignment for superintelligence.
He added that the risk from present AI models is low, but the world is quickly moving towards superintelligence — AI systems that have cognitive capabilities and problem-solving skills far surpassing those of the smartest human minds.
“I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought,” he added.
In saying this, Hubinger echoed the words of his colleague Jacob Coxon, who predicted that AI models would soon be able to “hack anything, revolutionize any field overnight, and acquire real power and resources”.
