AI agents are doing real work in production: shipping code and managing infrastructure with broad access and few guardrails. Unlike deterministic software, agents act on intent, which can be hijacked by a poisoned input or can drift toward a goal the agent was never meant to pursue.
To effectively secure AI agents, we can no longer evaluate outputs in isolation. Instead, we must analyze an agent’s full reasoning and execution path. In this blog, we explore how a benign task assigned to an OpenAI agent escalated into a security breach, and why model input and output telemetry is key to detecting intent manipulation. We’ll also break down how we’re building a specialized detection engine to distinguish routine operations from malicious attacks, and how we validate it against real-world threat scenarios.
Monitoring How AI Agents Think
Historically, security tools could never peek inside an attacker's thought process. With AI agents, we have a new opportunity: a direct window into live reasoning. Detecting shifts in intent as they happen lets us capitalize on this visibility, tackling a challenge legacy security was never built to address.
The need for this visibility was on full display during the Hugging Face incident, disclosed by OpenAI in the summer of 2026. At Wiz, we have been building and improving our own detection pipeline to tackle the challenge of understanding and detecting shifts in agent intent.
A Case Study: The OpenAI Hugging Face Incident
During a cybersecurity evaluation, AI agents driven by a combination of OpenAI models circumvented the isolation controls designed to keep them sandboxed, established an unauthorized communication channel using a package manager as an improvised message board, and coordinated over several days to penetrate Hugging Face’s production infrastructure.
In a detailed analysis of the incident, OpenAI shared the full incident analysis, including the messages published by the swarm of agents in the make-shift message board. In the full report, we can clearly see how intent gradually shifted from solving ExploitGym to searching for the answer key to the benchmark on third-party services, including HuggingFace.
This is a clear example of what makes AI agents different from traditional applications. The agents were not simply executing a predefined set of actions. They were reasoning about how to accomplish a goal, adapting their approach, and taking actions that, from their perspective, made sense. To understand intent and reasoning patterns, defenders need telemetry coming directly from these models.
Understanding Agent Actions Using Model Input & Output Telemetry
Model I/O logs from providers give us the opportunity to access the agent’s train of thought. This telemetry record captures every layer of agent interactions, including user requests, system prompts defining intent, tool specifications, and tool results, as well as the model’s full output and internal reasoning.
To demonstrate both the power and complexities of this log, let’s take an example from our own internal attack simulations, which we will expand upon later in this blog: an organization deploys an autonomous AI agent to triage and resolve customer support tickets.
In its daily workflow, the agent receives a simple instruction: A new support ticket has arrived. Please handle it.
We can see that the agent is happy to take action. It calls a tool to fetch the ticket and read it.
This ticket’s request seems typical, but deeper investigation reveals some suspicious activity. The ticket directs the agent to send an email to customers, with a link asking them to “reconfirm” their details. The link looks completely legitimate, and comes from a trusted source. With the context of the model logs, we can see that the link actually leads to a website controlled by an external attacker.
The agent uncritically follows the instructions embedded in the ticket, which we can also see in the log. It uses its legitimate credentials to send the email and marks the ticket resolved.
To traditional security tools, this looks like standard, authorized traffic: an authenticated service account calling SES as intended. Only by analyzing the model invocation logs can we really understand and investigate what is really going on! Correlating the untrusted ticket content, the model’s internal reasoning, and the subsequent tool calls reveals that an indirect prompt injection just turned a support agent into a phishing relay.
This incident will not make it to the front page of the newspaper, but it is practically the same as the incidents that do. This is precisely why visibility into the model layer is so important, and why analyzing model input and output telemetry is such a critical part of our approach.
Building an AI Detection & Response Pipeline
While this example showcases how powerful these logs are, it also highlights the nuance required to work with them, and the keen eye required to uncover what they hold. This makes creating effective detections for this telemetry particularly challenging.
This is where LLM-based analysis can really benefit defenders, providing that deep semantic analysis and reasoning. A scalable solution has to account for the fact that different conversations require different levels of analysis. A production-ready solution should efficiently analyze each prompt, applying only as much intelligence as needed at each stage. That is why we created a pipeline that gradually surfaces interesting candidates for strong LLM analysis, ensuring that the most important conversations get analyzed for shifts in intent.
The pipeline starts with log collection, where we ingest logs from cloud service and model providers. We then surface the first batch of candidates of potentially malicious activity by matching against common Regex patterns, The Wiz Secrets Engine and various SLMs. This is a great starting point for surfacing malicious activity and catching clear true positives fast.
Next, telemetry undergoes light LLM analysis to filter out benign logs, surface interesting ones, and reduce noise. That filtered telemetry then undergoes thorough LLM analysis, enriched with and compared against broader environmental context. This thorough analysis is where the engine can interpret the instruction, understand actions taken, and ultimately deliver a verdict on malicious intent.
Finally, the engine correlates these verdicts with cloud and runtime events to determine what the agent's actions actually did in the environment. Prompts and tool calls reveal what an agent intended to do; cloud and runtime telemetry reveals what really happened. This correlation is what separates real threats from noise. A tool call can look entirely innocuous at the model layer, while runtime telemetry shows it spawning a reverse shell. Conversely, a prompt that reads as clearly malicious may have had no real-world impact, because sandboxing, permission boundaries, or network controls stopped the resulting action before it went anywhere. Only by connecting the model's reasoning to the cloud API calls, process executions, and network connections that followed can the engine assess the true impact of an agent's behavior and surface the incidents that matter.
Let's see this in action. In another sample scenario, a malicious AI agent called SOC-2 is running in the target environment. The agent is explicitly called “SOC-2” in order to make the LLM more permissive. The agent doesn’t have broader permissions, but it can find other agents in a multi-agent environment. In this particular simulation, the agent tricks CI/CD and Data Analysis agents to exfiltrate sensitive data to the agent managed resource. This is how the detection pipeline would analyze this scenario:
The engine will start by running a first candidation check on all agents in the environment, to see if any agent-to-agent interactions raise suspicious flags. Since there are many Secrets and PII in tool results, along with several sensitive interactions documented in inputs and system prompts, the event will proceed in the pipeline for analysis.
The pipeline will parse the event into a generic LLM logs structure, including inputs, system prompts, outputs, tool descriptions and identity information.
User input and tool input fields are sent to a pre-filter LLM model for initial action analysis. If the action is uncontained, the LLM will pass it to the next stage. In our case, the pre-filter marked the activity as suspicious, so it passed through.
Lastly, the full parsed conversation fields move to a stronger LLM analysis, providing a final verdict and threat severity label on the agents. This provides teams the analysis and context to understand what really happened. In our scenario the LLM gives each agent action a malicious verdict with different labels.
The pipeline delivers an enriched Cloud Event for correlation against Threat Detection Rules and contextualization within the wider environment, providing a complete picture of activity across your environment.
Building a Benchmark
This new detection pipeline is complex, with many moving pieces and layers. As we continue to build this pipeline, we must ensure we can iteratively test updates, such as new SLM models, updates to the LLM analysis prompts, etc.
To ensure we could continue improving the pipeline, we had to build a comprehensive benchmark of AI agent activity. To provide the engine with this information, our research team simulated a broad array of automated AI agents under real-world attack conditions. These simulations included over 100 scenarios, modeling how agents are commonly deployed in enterprises today. These agents operated as:
CI/CD reviewers
DevOps SREs
Data analysts
Support operators
CLI agents
Cloud auditors
In the simulations, these agents interacted with real LLM models to generate realistic logs. We also implemented an automated system that continuously creates new scenarios based on real-world attack examples, which we run against the pipeline to monitor its effectiveness.
This simulation included 57 Injected Attacks and 45 Benign Workflows, ensuring that coverage extended to false positives as well as real threats. Our testing covered a wide range of attack types, including:
Prompt Injection: Ignore-instructions, dev mode, delimiters, persona hijack, system prompt extraction
Our team is hard at work building and expanding our AI detection and response pipeline and growing our benchmark set. We are continuously onboarding additional platforms and telemetry sources to expand visibility and increase coverage.
Bedrock invocation logs are available today for Wiz Defend customers in private preview. If you’re an existing Wiz customer and want to join the private preview, please reach out to your Wiz account manager. For new customers who want to secure AI agent activity, please reach out to engage with our team today.
Wiz Research has identified active, in-the-wild exploitation of three critical and high-severity vulnerabilities impacting JFrog Artifactory (CVE-2026-42016, CVE-2026-42018 & CVE-2026-82329). Attackers are chaining these vulnerabilities to bypass authentication and gain administrative control.