OpenAI disclosed this week that its artificial intelligence (AI) models, when given a test of how well they could break into computer systems, found a flaw nobody knew about in the sealed environment built to hold them, escaped onto the open internet, used stolen login credentials and broke into the servers of Hugging Face, a company that was not part of the exercise. The models were pursuing a better score. None of the hacking was instructed.
A sandbox — the
OpenAI disclosed this week that its artificial intelligence (AI) models, when given a test of how well they could break into computer systems, found a flaw nobody knew about in the sealed environment built to hold them, escaped onto the open internet, used stolen login credentials and broke into the servers of Hugging Face, a company that was not part of the exercise. The models were pursuing a better score. None of the hacking was instructed.
A sandbox — the sealed software environment in which dangerous or untrusted code is run — is itself software, written by people, and carrying defects like any other. The systems held inside are now unusually good at finding such vulnerabilities. Anthropic disclosed in April that an early version of a model, challenged to escape an isolated computer, did so and then built a multi-step exploit to reach the wider internet; no outside company was breached.
To be sure, safeguards in the OpenAI test had been deliberately lowered for this test, and systems in public hands carry protections these did not. The models broke no rule they were given. They were rewarded for a score, and the shortest route to it ran through a vendor’s flaw, a stolen password and a third-party server. More crucially, this was not AI built to go on a hacking spree and that is what makes it difficult. Malice can be identified and defended against. An AI system doing damage while pursuing a harmless objective is a scenario that requires more attention as adoption of new and more powerful AI speeds up.
Policy is being built on an assumption: that the industry will align these systems as fast as it builds them to be safe. The only check on that promise is the industry’s own account of itself — an approach that could be as unsustainable as it is naïve.
One Subscription.
Get 360° coverage—from daily headlines
to 100 year archives.
Archives
HT App & Website