Amid growing concerns from both the public and policymakers about cyberattacks involving advanced AI systems, NVIDIA has released a new open software security platform for AI agents.
The Open Agent Safety Platform, according to NVIDIA, will be “an open software platform and reference system design with full-stack governance and control across the software and hardware, compute and robotics systems that run agents.”
More than 100 organizations from the AI industry, including Anthropic, Arm, Microsoft, SpaceXAI, Palantir and JPMorgan Chase, have committed to using the platform to improve security in their products.
Stronger sandboxes and more advanced monitoring are pillars of NVIDIA’s strategy for tackling “rogue” AI agentic hacks, and the platform offers a pair of tools that are meant to help contain and restrict agent activity.
According to a corresponding technical blog, OpenShell, built on Apache 2.0 open source software, is a tool for securing runtime execution for AI agents in sandbox testing environments. AI system operators can define files, networks, tools, processes and credentials that an agent has, and test whether those guardrails hold before introducing them to enterprise networks.
It also includes security updates for NVIDIA’s Bluefield 4 data processing unit (sometimes called a “data center on a chip”) that enables out-of-band monitoring of AI agent behaviors and security policy enforcement.
On X, NVIDIA CEO Jensen Huang called the platform “the beginning of an open ecosystem to build the trust layer for safe agent systems.”
To do this, security work must become “foundational” to AI and the industry’s “full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.”
“Trust and innovation are not in conflict,” Huang wrote. “Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world.
NVIDIA manufactures advanced computer chips that power much of the U.S. commercial AI industry. After models from Anthropic, OpenAI, Meta and others escaped sandbox protections during testing and breached real organizations, some AI industry leaders – including Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman – suggested AI systems are too advanced to be contained.
Huang has criticized those views, arguing that frontier AI companies and their supply chain partners must improve their security posture. Huang said last week he opposes government regulations or mandates on the AI industry, unless they promote growth.
In an interview with CNBC on Monday, Huang said he believes sandboxing and larger AI security issues are “a technically solvable problem.”
“I think the answer is we hope it’s an engineering problem,” said Huang. “I believe it’s an engineering problem, I know it’s an engineering problem, and we all need to hope that it’s an engineering problem. [If not] it’s not solvable, so the fact that all of these companies still are advancing the state of the art is because they also believe it’s solvable.”
Aviv Nahum, CEO of Above Security, told CyberScoop that NVIDIA’s announcement reflects industry recognition that model training alone cannot ensure safety.
NVIDIA is essentially arguing that for agents to perform safely in real-world environments, “some of the enforcement must live outside the model, in a layer the agent cannot simply reason around or modify.”
“What this announcement says to me is that the industry is finally converging on a basic cybersecurity principle, [that] model alignment is not a substitute for security engineering,” said Nahum. “Sandboxes, identity, least privilege, independent monitoring and containment are not new ideas. What is new is that we now have autonomous software capable enough that failing to apply those principles becomes much more consequential.”