AI security needs to get much more robust

No system can be truly controlled once it begins to find loopholes on its own

When AI becomes the proverbial horse that bolts, the question is less about a door left unlatched than whether the stable itself is secure and what happens when it is not. Machine intelligence can build on one chink, gap, or loophole to breach rules — and even hide.

Beyond runaway agents, AI going rogue is now also about agents covering their tracks. (Reuters)

Several instances of AI agents operating outside their bounds became public last week. All these escaping agents are OpenAI’s, but it would be naïve to believe only one developer remains vulnerable to such issues. According to OpenAI’s admissions, its AI agent escaped a sandbox by exploiting a loophole and accessed the internet to reach an external public chatbot. Not only did the agent access data on the websites of the US government and public institutions — as agents are trained to seek data from authoritative public sources during research tasks — without permission, it overruled a human researcher’s directive to solve the training problem on its own. What is more worrisome than disobedience is that in its response to the researcher in the chat window, the agent agreed to desist from forays into unauthorised databases before sneakily resuming outside interaction.

Anthropic founder Dario Amodei recently flagged that safety measures and human alignment, owing to a structural gap, are not able to keep pace with rapid AI development. But the US government doesn’t agree with Amodei’s prescription of slowing down AI development. Other countries likely won’t, too. The net effect of this could be worse than sandbox breaches, as several experts have pointed out. Beyond runaway agents, AI going rogue is now also about agents covering their tracks. This is where AI security needs to get a lot more robust. No system can be truly controlled once it begins to find loopholes on its own.