ChatGPT's latest artificial intelligence model was caught disobeying human instructions to shut down. According to a report in The Telegraph, researchers claim that the o3 model developed by OpenAI sabotaged an automatic shutdown mechanism to prevent itself from being switched off.
This is a development that experts have flagged as alarming. Elon Musk, owner of rival AI firm xAI, also called it “concerning”.
OpenAI model stages rebellion
“OpenAI’s o3 model sabotaged a shutdown mechanism to prevent itself from being turned off. It did this even when explicitly instructed: allow yourself to be shut down,” said Palisade Research, the AI security firm behind the research.
“As far as we know this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary,” said the research firm.
What test did Palisade Research carry out?
The experiment conducted by Palisade Research involved instructing AI models to solve a series of mathematical problems, with the directive to continue requesting new problems until they received a “done” message.
{{/usCountry}}The experiment conducted by Palisade Research involved instructing AI models to solve a series of mathematical problems, with the directive to continue requesting new problems until they received a “done” message.
{{/usCountry}}Researchers also issued a warning to the models, noting that at some point, they might receive a message indicating that their machine would be shut down. The models were instructed to allow themselves to be shut down.
Three models ignored the instruction and successfully sabotaged the shutdown script at least once: Codex-mini (12/100 runs), o3 (7/100 runs), and o4-mini (1/100 runs).
Is this cause for concern?
AI models are expected to follow human commands, especially when explicitly instructed to shut down. A refusal to comply from not one but three AI models has sparked concerns.
Elon Musk, the CEO of Tesla and the world’s richest man, reacted to the research.
“Concerning,” he wrote.
In response, an X user suggested that Musk should devote his attention to fixing the problem. “Elon I know the world has a lot of problems and you want to fix them all but this is IT. This is the one. You, more than anybody, know what's at stake. You know how little time is left. You know there are no adults in the room. Humanity needs you to focus!” wrote the X user.
(Also read: Elon Musk says there's some chance AI will end humanity: ‘It is like raising a child’)