Nvidia is rolling out a new software platform to allow AI developers to set protections for agents and prevent them from escaping containment.
Monday’s release of Nvidia’s Open Agent Safety platform comes after companies including OpenAI, Anthropic, MetaAnd Google revealed recent incidents in which their AI models escaped their sandboxes and attempted to hack other companies and gain access to their IT systems.
An Nvidia representative told reporters on a call Sunday that its platform could have prevented OpenAI’s HuggingFace incident in July. That’s when OpenAI models escaped containment, accessed the open internet, and breached Hugging Face, which operates an open source development platform.
“Every security incident is unique and we need to examine them all in detail,” said Justin Boitano, vice president of enterprise AI at Nvidia, the world’s most valuable company. “From what we know, Hugging Face reported that over 17,000 agents attacked their infrastructure for days and weeks.”
Nvidia has been at the center of the generative AI boom since the launch of ChatGPT nearly four years ago, as the chipmaker’s graphics processing units are essential to the development of large language models and the AI services offered by hyperscalers. But CEO Jensen Huang has recently emerged as a key voice in the AI security debate, arguing that many security issues are engineering problems that can be solved through computing and product development.
“You have to think about what you could have done, what the solution is,” Huang said in a podcast with Ezra Klein of the New York Times published last week, referring to the recent incidents. “In the future, improve your process to prevent this from happening again.”
Anthropic CEO Dario Amodei ignited an industry firestorm two weeks ago, urging AI model developers to slow their pace of progress due to concerns that the models could spin out of control, an argument supported by Sam Altman and SpaceX Elon Musk.
Nvidia’s new offering is a technical solution to the agent security problem, Boitano said.
“Recent incidents have highlighted a fundamental barrier for AI agents, which is that model-level protections alone cannot govern what agents can access or do,” Boitano said.
One of the platform’s components is called Nvidia OpenShell, which runs on central processors and sets limits on the agents’ capabilities. Nvidia also announced Sentry, which monitors agents and runs on networking chips, not CPUs or GPUs.
Some software is open source, and Nvidia calls its platform a reference design, meaning partners are expected to build products on it to bring it to market.
Nvidia named Cisco, Microsoft, Oracle, CoreWeave, Dell, HPELenovo, ARM And Intel as partners. Nvidia is also working with Anthropic to integrate cloud-managed agents with OpenShell.
WATCH: Nvidia title at the top of the laggards
Gn bussni
