Spectra Assure Free Trial
Get your 14-day free trial of Spectra Assure for Software Supply Chain Security
Get Free TrialMore about Spectra Assure Free TrialKey takeaways
It’s a conundrum for application security (AppSec) today: How do you square agentic-assisted development with the basic security principles that keep software trustworthy?
Chris Romeo, managing general partner at Kerr Ventures and a longtime AppSec authority, said that for security pros and AppSec veterans, it’s going to be tough to satisfactorily answer that fundamental trust question until development organizations appropriately address the agentic overprivileging problem.
“These are the things that are keeping CISOs up at night right now. Ultimately, how do I know that an agent didn’t push some malicious code or just bad wrong code into a production system?”
—Chris Romeo
Romeo, who is investigating how Claude permissions work, said that when a coding agent can read from a repository, run a shell, install packages, open pull requests, call external services, and interact with production-adjacent systems, the prompt is not the security boundary; it’s the code for the agent’s permissions.
Here’s why the overpermissioning problem exists, and why solving it is key to reining in agents.
[ Webinars: Mythos vs. Reality: Zero-Day Defense for Supply Chains | Autonomy, Not Autopilot: Get Real About the Agentic SOC ]
The root problem is that, whether in coding environments or elsewhere, AI agents are conferred permissions in a crude manner at best. In most cases, the agent just inherits the same privilege level of the user who spun it up. And that privilege gets handed down when agents spawn agents.
Identity pros have warned against this kind of permissions creep among humans for literally decades. And now it’s happening at machine speed and scope — and thepractice is even baked into how AI agent platforms are architected, Romeo said.
“I can’t even launch a sub-agent from Claude'’s CLI with a different permission set than what the parent has.”
—Chris Romeo
When the developer has tons of available access across various systems, the agent often does, too. It’s even less transparent when the agent is configured within an already approved platform that never triggers a security review, said Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs.
But the biggest problems happen when more than one agent or action is involved, he said. Modern developers often have many agents working at different tasks at once, sometimes swarms of them, and they’re often spawning sub-agents to create complex workflows in the pursuit of a prompted goal.
“The [shadow AI] gap widens when agents acquire credentials at runtime, spawn subagents, or chain actions across multiple systems in a single task. Every delegation step creates an identity your IAM system was never designed to track. Most organizations have no ownership model for agent identities and cannot trace an agent’s action back to the human who authorized it.”
—Jacob Krell
Romeo said that another big issue he has found with how Claude permissions work is that much depends on trusting the sanctity of the endpoint on which a lot of the work is done, but there’s no way to track the provenance or trustworthiness of skills loaded into the endpoint.
Anthropic recommends using agent management policy to lock files down, but even it admits that’s not a security boundary, per se.
“The challenge is, is [that lockdown policy] something you can’t bypass in some way? I’m not willing to stake my reputation on saying yes.”
—Chris Romeo
Overpermissioning is a problem across all agentic use cases, not just coding. But as Romeo said, development use cases are some of the most consequential because of the outsized level of access most developers hold and the downstream impact to the software supply chain when an overpermissioned coding agent goes rogue or is subverted.
“Given that we extend the privileges of the developer running an agent to the agent itself, the consequences of a misaligned or confused agent could be catastrophic. Stories are leaking out about agents removing databases or pushing to production, not maliciously, but due to a misinterpretation of instructions.”
—Chris Romeo
The non-determinism and goal orientation of agents adds another wrinkle to the issue, said Romeo, adding that “agents will eat overprivileged software for lunch” when they’re on a mission to meet a prompt’s objective.
Eric Capuano, director of SOC operations at Black Hills Information Security, cited the The OpenAI-Hugging Face incident as a prime example.
“A model given a testing goal went outside its lane, got into another company’s systems, and the operators found out after the fact. That is an agent with too much access, too little scoping, and nobody watching the logs in real time. Every SOC has seen that exact failure with a human contractor or a misconfigured service account. The difference is speed and volume.”
—Eric Capuano
Romeo said we eventually will need a true identity or identifier “just to maintain sanity” for how coding agents or other broader agents work.
“We need traceability for both the person requesting the work, [whether] a developer or any other team member, and the agent performing it, with a system that authorizes requests. To gain traceability, a distinct and true identity is required for anything operating within the system.”
—Chris Romeo
That’s the foundation, but it’s really just the start of what it will take to build out a trustworthy agent-forward coding environment, Romeo said. Each agent’s properties — model, tools, network routes, workspace, and execution policy — need to be combined with identity and credential information to form a real control plane. Getting visibility into every aspect is key, he said.
Michael Bargury, CTO at Zenity, said this is the direction that many AI security experts are taking with control suggestions. But he warned against placing too much confidence in identity security alone to solve the problem of agents’ goal orientation. That’s because agents will find ways to escalate privileges, rewrite themselves, and change their tools and skills on the fly.
“When you’re trying to secure agents, it’s all about runtime. Security engineering traditionally is first and foremost about the right architecture, the right identity, the right network segmentation. The problem is that agents will just mishmash all of that together.”
—Michael Bargury
Solutions are still very much in the early phases, but Bargury said he’s especially keen on the open-source work being done now on the Agent Control Standard (ACS), which defines how agent platforms can expose middleware hooks to make it possible to evaluate each agent decision point against policies. This can lay the foundation for observability, better agent bills of materials, and methods to guide the agent into secure behaviors.
OpenAI's recent post-mortem report on the Hugging Face incident includes a warning that truly effective guardrails must be devised and implemented in the age of frontier AI models.
OpenAI analyzed the incident, which occurred during cybersecurity evaluations of several OpenAI models, driven primarily by a highly capable, internal-only research model comparable in scale to GPT-5.6 Sol, one of the company’s flagship models.
Operating under reduced safeguards, the models sought to complete tasks successfully by independently communicating through unauthorized channels, exploiting vulnerabilities in shared infrastructure, gaining internet access, and reaching third-party systems operated by Hugging Face.
“We consider this incident a ‘warning shot’ for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.”
—OpenAI post-mortem report
The report stresses that OpenAI’s models are now powerful, persistent, and collaborative enough to find and exploit security weaknesses across multiple computer systems when safeguards are insufficient and that many other models, including open-source ones, will soon reach comparable capabilities.
Kanaiya Vasani, chief product officer and chief marketing officer at ExtraHop, told RL Blog recently that AI-assisted attackers who can find and exploit a vulnerability in seconds make the tiered model obsolete. The detection side has the same problem. Signature-based and indicators of compromise (IoC)-based methods only recognize what has been seen and catalogued before, and the flaws the OpenAI agents chained together offered nothing to match against.
ExtraHop launched the Agentic SOC Alliance in July (founding members include ReversingLabs, CrowdStrike, and LangChain), giving organizations an industry framework for composing these layers without locking into a single vendor’s stack. On the intelligence side, RL’s detection APIs will feed ExtraHop’s network detection and response platforms, turning a new threat finding into real-time detectors deployed across customer environments and shortening the window between discovery and inoculation.
Vasani recommends a three-layer model, and shares what it takes to stand one up, in this Q&A: How to build effective agentic SOCs.
The software industry is entering the AI era burdened by legacy flaws and weaknesses, making Secure by Design essential.
The post-mortem reaches sobering conclusions, and demands a plan of action for the AI industry — plus your SecOps strategy.