Guest Author: Jacob Graves, Field CISO, Prompt, SentinelOne
TL;DR – Security teams didn’t make a mistake when they applied existing frameworks to AI. They applied the best tools they had to a technology that operates on fundamentally different principles. The gap comes from assuming that software security and AI security are the same problem and that assumption should be examined more closely.
Most enterprises we work with did the right things when AI arrived. They classified the data, governed the access and monitored the network. Reasonable decisions, made quickly, by teams that had a framework and applied it.
What that framework wasn’t built for is an employee pasting source code into an AI tool because it feels like a private assistant, with no record of it on the security team’s side. The same thing is also happening with customer data, legal documents and financials. It is not malicious. Not careless. Just a normal person using a tool the way it was designed to be used.
The assumption behind all of it was simple: software is deterministic. Secure the perimeter, govern the access, monitor the outputs and you’ve covered the risk. For thirty years that held up well enough. AI operates on different principles entirely.
Traditional software doesn’t negotiate. You give it an instruction, it runs the instruction. When something goes wrong, you look for bad inputs, bad logic or access that shouldn’t have been granted. The failure modes are understood and the controls map cleanly to them.
AI works differently. It doesn’t execute instructions so much as interpret them and that changes the attack surface in a fundamental way. A system capable of reasoning about language is also capable of being reasoned against. The content it processes isn’t just data to be acted on. In the right hands, it’s a vector for influencing what the model does next.
In practice, that looks like this: an agent connected to Slack, email, Google Drive, a ticketing system gets asked to summarize a document or take the next step on a task. Hidden inside that document is an instruction trying to get the agent to search for sensitive data or send information somewhere it shouldn’t.
The employee didn’t do anything wrong. The model didn’t malfunction. But the agent had access, context and the ability to act and something happened that nobody authorized.
The mechanism is indirect prompt injection. The model processed content from an untrusted source, followed the instruction embedded in it and nothing in the existing stack was positioned to catch that before it happened.
And the agentic scenario is only part of the picture. Across the enterprise AI deployments we monitor, roughly 1.6% of all prompts contain a policy violation, most often involving PII, credentials or sensitive business information.
That volume reflects normal business operations at organizations with mature security programs, not anomalies.
When we look at where enterprise AI security investment has actually gone, the pattern is consistent. Data classification and DLP to control what the model can access. Identity and access management to control who can reach it. Network monitoring to track data flows. These are real investments and they address real problems.
None of them govern what a model can be made to do with what it reads.
DLP doesn’t inspect whether a prompt is attempting to manipulate model behavior. IAM controls who opens the door, not what happens once the model is processing content from an untrusted source. A SIEM can only alert on what’s been logged. In most deployments, model interactions never make it into the log pipeline at all.
The honest answer is that most organizations have never had to think about this layer before. There was no equivalent in traditional software. You didn’t need to ask whether your CRM could be talked into doing something it wasn’t supposed to. With AI, that’s exactly the question and most security programs don’t have a way to answer it yet.
The controls that exist today don’t need to be replaced. They need to be complemented by something designed for a surface they were never built to see.
What that means in practice is being able to look at the full chain:
Most security programs today can answer the first question. Very few can answer the rest, not because they haven’t invested, but because the interaction layer was never instrumented to begin with.
Closing that gap means inspecting prompt and response pairs in real time, detecting injection patterns before they reach the model and logging what actions were taken against what was actually intended. Not a replacement for existing controls. A layer those controls were never designed to provide.
The organizations closing this gap recognized early that their existing stack, however mature, wasn’t designed for this surface. Admitting that is harder than it sounds. It means accepting that doing everything right by conventional standards still leaves something uncovered. In our experience, that’s the conversation most security teams haven’t had yet.
Most of the work is just drawing the line in the right place. Find out how you can see and govern what happens at the AI layer most tools were never designed to reach. Learn what your current stack isn’t seeing.