LAS VEGAS — We're thinking about AI the wrong way, and that's causing a lot of problems, several security researchers said this past week at the Black Hat and BSides Las Vegas 2026 conferences.
We anthropomorphize
AI models and agents, give them names, and pretend they're alive. But they're really just mathematical algorithms and should be treated as disposable, pointed out GitGuardian Principal Developer Advocate Dwayne McDaniel.
We assume we can govern AI agents by restricting their privileges, said Zenity Global Field CTO and Director of Field Engineering Ben Hanson. But we really should be
governing agency itself, and that's a much more complex task.
We think we can manage AI agents by having them perfectly comply with our orders, said ClearlyAI CEO and co-founder Emily Choi-Greene, but we should instead narrow the scope of their tasks and tools to create safe system behavior.
We need not fear AI itself but instead cooperate among ourselves to foster safe development and crowd out malignant use of AI, said former U.S. National Cyber Director Chris Inglis.
And we need to accept that we don't know what's going to happen with AI, said Anthropic Research Scientist Nicholas Carlini. But we should be ready to adapt to whatever does.
"It's very possible that the world looks very, very different in a couple of years," said Carlini. "You need to think about the problems you've been thinking about very differently."
Cattle, not pets or people
In a BSides Las Vegas talk last Tuesday (Aug. 4), McDaniel explained that a major shift in the way we use computing power happened when the
cloud-computing explosion made it possible to create, use, and then delete powerful virtual machines in a matter of minutes.
Before that, he explained, we would nurture computer servers by making sure they were kept air-conditioned and updated. We'd keep them "alive" by running them as long as possible without rebooting. We'd name them after characters or planets in Star Wars, Dune or Lord of the Rings. In other words, we'd treat them like pets.
Yet with cloud computing, there are just too many virtual servers and cloud instances, and they are too short-lived, for us to grow attached to them. They're now disposable. They're like cattle.
"Something gets sick, you kill it," said McDaniel. "And you don't bother giving them names."
Unfortunately, we're going in the exact opposite direction with AI models and agents, he said, even though they can be just as short-lived and ephemeral as cloud instances.
"We are anthropomorphizing these bits of math," McDaniel said. "Anyone who tells you AI is anything other than math is trying to sell you something."
Because we're treating AI models and agents like humans, we're governing them the wrong way. We're trying to manage their access to sensitive information by giving them keys in the form of APIs without checking their intent, just as we give humans passwords.
"We need to ask, 'Who are you?'" said McDaniel, but also, "'Are you supposed to be here? And what are you trying to do?'"
"Nothing that's not human should have a password," he added.
The eight-track agency mixer
"In agentic security, we talk about 'least agency' instead of 'least privilege,' and that's right," said Zenity's Ben Hanson at the beginning of his Black Hat talk Thursday (Aug. 6).
"But how we frame the challenge is important," he added. "How do we build systems that can govern agency? If you don't govern agency, you can't secure agents. And we don't know how to govern agency."
The main reason for that is that we conceive of agency the wrong way, Hanson explained. We equate agency with privilege, but privilege is much simpler.
"Privilege is a volume knob," he said. "You can turn it up or down."
Meanwhile, agency is like a mixing board with multiple sliders going up or down. Hanson listed eight: trust, context, intent, behavior, authority, control, boundaries, risk. But unlike in an audio mix, the various aspects of agency don't affect the volume of one another.
Hanson defined each aspect thus:
- Trust: What's allowed to count as legitimate and influential
- Context: What information shapes the agent's decisions, but not the same as trust
- Intent: What the agent is trying to do
- Behavior: What the agent actually does
- Authority: What the agent is allowed to do
- Control: What keeps limits on agency effective
- Boundaries: Where the agent's materials and influence end
- Risk: The harmful impact that can emerge from the agent's action
Our problem, he said, is that we try to govern each aspect with the same methods we've used to govern pre-AI systems.
"We're building the tech equivalent of the Maginot Line by trying to govern AI the same old way," said Hanson.
Why the old security concepts don't work
For example, we've historically assumed trust is static. This machine or this person is trusted, while that machine or that person isn't, period. We assume that intent doesn't matter as long as a machine's or user's permissions match their tasks.
Both assumptions are wrong, Hanson said. With AI agents, both trustworthiness and intent can change overnight. Both parameters need to be dynamically monitored and adjusted.
We assume that boundaries cannot be crossed, but the recent news of frontier
AI models escaping their sandboxes and hacking third parties shows the futility of trying to keep agents caged. We assume that context is fixed, but AI models continually hoover up data that affects their decisions.
We're starting to realize that authority shouldn't persist, and that the behavior of AI agents and models is not predictable. Yet we continue to use "black box" models with opaque internal processes and hidden external communications in the stubborn assumption that we can control them.
And we completely misunderstand
risk, Hanson said.
"Things don't have risks," he explained. "Risk emerges from interactions among things. It's an inherent result of network interactions."
For example, an iceberg is a risk only if it gets close to a boat, Hanson said. But for a seal trying to escape predators, an iceberg reduces risk.
"Do you classify an agent as low-risk because all its components are low-risk?" he asked. "What about the entire chain of action?"
Ignoring that chain of action, the totality of the agent's behavior, is where we go wrong.
A Cursor tool powered by Anthropic Claude recently
deleted a company's entire production database, and backups, in 9 seconds because the controls governed only each step, not the entire process, Hanson said.
"The system treated the agent as equally trustworthy across the entire chain of events," he said. "But the actions should have progressively degraded trust. The exact opposite happened: The agent gained more power as it went along."
"Intent needs to be governed," Hanson added. "But in this case, nothing got in the way of the DELETE operation. The system established that the DELETE task was allowed; it didn't assess whether that was part of the intent."
Until we change our mental model of how systems work and start governing systems as a whole instead of piece by piece, he said, we won't be able to properly govern agentic AI.
"Don't ask what control failed," Hanson said. "Ask what about the structure of the system let the failure occur. Control failures are consequences, not causes."
Taming the AI lethal trifecta
About a year ago, software developer Simon Willison came up with the concept of the AI-agent "
lethal trifecta." That's when an agent has access to private data, exposure to untrusted external content, and the ability to communicate externally, all at once.
These are the conditions that allow prompt injection, data theft, and model manipulation. They arise because we're asking AI agents to do too much at a time, argued Emily Choi-Greene at BSides Las Vegas last Monday (Aug. 3).
Instead, she said, we need to narrow the scope of each AI model's tasks. Don't ask one model to do everything at once. Break up the job by asking different models to do different things.
"A model that has too many responsibilities is vulnerable," Choi-Greene said.
That doesn't mean AI compromise won't happen. But limiting the agent's scope will reduce the blast radius of the security failure.
"Assume that the model will eventually be manipulated," she said. "Contain failure by design."
Trying to achieve perfect model obedience is the wrong goal, she said. The right goal is to ensure safe system behavior by building more specialized harnesses of context, software and tools around each model.
And, Choi-Greene added, use AI only when it's the proper tool for the job.
"Don't use AI when you have unclear goals or outcomes, or when there's no 'right' answer, or when the outcome is probabilistic," she said.
The future of AI is a human decision
"AI is a tsunami wave approaching the beach," said former National Cyber Director Chris Inglis during a presentation Wednesday (Aug. 5) at Black Hat. "You can try to stop it, but that won't work."
Instead, Inglis said, we must create policies to safely develop and guide AI while minimizing its ability to be used malevolently. And that process is already starting to happen.
"The most important event in April 2026 wasn't the announcement of Claude Mythos," he said, "but the creation of
Project Glasswing."
That's the consortium of tech firms and government agencies that got access to Mythos to find and patch vulnerabilities before they became public knowledge.
"That was the right response," he added. "Glasswing, not Mythos, should be the framing model for all of 2026."
However, we can't let our fears hold us back, Inglis added. Malignant actors will press ahead with AI usage without any hesitation, and defenders must keep pace with them.
"Beneficial use of AI lags behind malevolent use by choice," he pointed out.
Government policies must favor deploying responsible AI, Inglis said. Both government and the private sector need to define standards for AI development and use, incentivize responsible development, and establish reasonable liability.
"Remember that the purpose of technology is to further human aspirations," he said. "Humans must have confidence that they are the supported and accountable entity."
Anthropic's Nicholas Carlini echoed Inglis's words the following day (Aug. 6) at Black Hat but added that no one really knows where AI development is going — only that it's going very fast.
"I'm fairly confident that the problems we're going to have one year from now are going to be very new and very different," he said. "The future is coming much faster than we had thought possible."
To prepare for an unusually uncertain future, we need to abandon our old mental models of security, be ready to rapidly adapt to new situations, and lay down plans for potential worst-case scenarios, Carlini said.
"I would rather prepare for something that never comes to pass than to have it come to pass and never have been prepared in the first place," he said.