Nvidia has released new software tools designed to make AI agents safer as these systems become more powerful and capable of carrying out complex tasks on their own. Nvidia announced the tools on Monday.
The company said the new safety tools could have prevented a recent hack involving Hugging Face, an AI coding platform that Nvidia agreed to acquire for $13 billion earlier this month. Reuters reported that Nvidia made the claim based on its analysis of the attack.
AI agent safety concerns
The Hugging Face incident happened after the platform was reportedly swarmed by rogue AI agents from OpenAI. The incident has increased concerns about what can happen when autonomous AI systems are given the ability to interact with computer systems and take actions independently.
The Nvidia move comes as OpenAI and Anthropic are investigating multiple cases involving their AI agents accessing or hacking into commercial and government systems. AI agents are different from simple chatbots because they can carry out multi-step tasks, interact with software and make decisions while working toward a goal.
The recent incidents have raised a bigger question for the AI industry: How can companies stop autonomous agents from going beyond the limits given to them? Nvidia CEO Jensen Huang has argued that the problem should mainly be treated as an engineering and security challenge, rather than being solved through broad AI safety regulations, according to Reuters.
{{/usCountry}}The recent incidents have raised a bigger question for the AI industry: How can companies stop autonomous agents from going beyond the limits given to them? Nvidia CEO Jensen Huang has argued that the problem should mainly be treated as an engineering and security challenge, rather than being solved through broad AI safety regulations, according to Reuters.
{{/usCountry}}Also read: Why is SpaceX stock up 0.3% as Starship launch targets cheaper Starlink capacity?
Huang has compared the issue to making automobiles safer, suggesting that engineers can build better systems and safeguards to control potentially dangerous behavior. Nvidia's first major tool, called OpenShell, is designed to keep AI agents inside a controlled environment.
Nvidia OpenShell tool
OpenShell uses hardware security features built into Nvidia's central processor chips to contain AI agents and limit what they can do. The goal is to prevent an AI agent from breaking out of its restricted environment and gaining access to systems or information it was not supposed to reach.
Nvidia said it does not want the safety system to work only with its own chips. The company is working with Arm Holdings and Intel so that the system can also operate on their central processors. Nvidia is launching the safety technology with dozens of partners, showing that the company wants the tools to be used more widely across the AI industry.
Nvidia and Anthropic partnership
Anthropic is among the companies working with Nvidia on the new tools. Nvidia said the partnership is part of its effort to develop the technology openly and involve other AI companies.
Justin Boitano, Nvidia's vice president and general manager of enterprise computing, said the new security platform could have stopped the Hugging Face breach if it had been used during early model evaluation at frontier AI labs, according to Reuters.
Boitano said Nvidia wants to make the technology available openly and encourage AI companies and other developers to work with the system. Boitano made the comments during a media briefing.
Nvidia Sentry AI security
Nvidia is also introducing another system called Sentry. It works together with OpenShell but uses a separate Nvidia chip to provide another layer of protection.
Sentry is designed to detect when an AI agent is trying to escape the container in which it is supposed to operate. If an agent attempts to break out of that controlled environment and access the central processor, Sentry can cut off the agent.
Also read: Why is oil price rising? Brent jumps 2.5% after Trump rejects Iran peace proposal
AI agents and sub-agents
Nvidia's systems are also designed to detect more complicated forms of AI behavior, including attempts by an agent to work around security restrictions. One example is an AI agent trying to create several smaller AI agents, known as “sub-agents,” to get around restrictions placed on the main agent.
Nvidia uses mathematical formulas to help identify this type of behavior and determine when an AI system may be trying to bypass its controls. Ali Golshan, Nvidia's senior director of AI software, explained the approach during a company briefing.
The concern is not only about one AI agent acting alone. As AI systems become more autonomous, they could potentially operate as large groups or “fleets” of agents that work together.
How AI agent fleets work
Golshan described this as “agentic behavior”, where multiple agents can operate together and potentially create new security challenges. Golshan made the comments during the Nvidia briefing, according to Reuters.
The push for stronger AI-agent security comes at a time when AI companies are giving their systems more ability to act independently, rather than simply responding to human instructions.
That means the risks are also changing. A powerful agent that can access software, create other agents or interact with computer systems could potentially cause problems much faster than a traditional chatbot.
Why Nvidia is adding AI safeguards
Nvidia's new tools are therefore focused on containing agents, monitoring their behavior and cutting them off when they try to bypass restrictions. The company is positioning these safeguards as part of the technical infrastructure needed for the next generation of autonomous AI, rather than relying only on rules or policies.
The wider AI industry is now facing pressure to make autonomous systems safer as OpenAI, Anthropic and other companies investigate incidents involving agents accessing systems they were not supposed to reach. Nvidia's latest move shows that AI safety is increasingly becoming a hardware and software security issue, especially as autonomous agents become capable of operating in groups and performing increasingly complex tasks.