Monitoring AI agent sessions: Leveraging embedding models to aid threat hunting and attack discovery workflows

Agentic systems are beginning to challenge traditional security monitoring paradigms because their decision-making is outcome-based rather than action-by-action logged, making simple keyword or regex matching insufficient for threat detection. One proposed solution involves leveraging embedding models to semantically index and search vast quantities of unstructured agent session data—such as reasoning steps, tool calls, and user prompts—stored in platforms like Elastic. This semantic retrieval allows analysts to hunt for novel behaviors by describing desired outcomes rather than relying on pre-defined rules. While one enterprise implementation utilizing generative models demonstrates high precision (97.2%) in detecting exposures, this approach suffers from high costs per session, probabilistic variance, and requires knowing the investigative questions upfront, which impedes proactive threat hunting.
The proposed embedding search method seeks to create a deterministic layer over inherently probabilistic agent data by representing every discrete turn within an AI interaction as a fixed vector upon ingestion. To improve signal fidelity, sessions should not be embedded whole but rather chunked by channel (user input, reasoning, tool calls, agent response), with results fused using Reciprocal Rank Fusion (RRF). This technique supports three querying modalities: instructive text searches (finding discussions of issues), descriptive hunts (discovering unseen behaviors), and exemplar retrieval (finding duplicates of known incidents).
<RED>
Tested vector search on 7,454 turns across agent sessions.
One in seven reviewed agent sessions required triage due to suspicious actions.
Storing six months of Claude Code and Cursor sessions as semantic vectors in Elastic was performed.
Agent sessions contain unstructured behaviors that are difficult to describe using query rules.
Uber's Agentic AI Detection and Response (ADR) system analyzes every session with a generative model.
The embedding approach embeds each turn once at ingest, storing the vectors in Elastic for semantic search.
Chunking agent sessions by channel is necessary because embedding an entire session loses context.
Queries can be instructive, descriptive hunts, or exemplar retrieval using known bad turns as queries.
MRR compares vector search against keyword (BM25) search across four channels (User, Reasoning, Agent, Tool).
Credential access was the only tactic aligned between MITRE ATLAS questions and retrieved sessions in testing.
<PURPLE>
The fundamental friction here lies in shifting security monitoring from a verifiable ledger of human actions to an inference engine operating on algorithmic reasoning. Traditional incident response excels at finding discrete deviations (e.g., specific commands run); AI agents invert this, making the whole session the unit of potential risk, as a sequence of individually permissible actions can constitute systemic failure or malicious intent. The move toward semantic search is a cognitive leap: it shifts detection from "Did they type X?" to "Did the *pattern* of their interaction align with an outcome we define as problematic?" This empowers hunting for latent risks—like the inadvertent Git push mentioned in the example—that no rigid rule set would ever capture.
The key structural insight is the differentiation between **exemplar retrieval** and **descriptive hunts**. Exemplars solve high-fidelity pattern matching: "Find me 12 more instances of this exact risk I just closed." This is tactical compliance checking. Descriptive hunts are the genuine leap toward threat discovery: "Find me any instance where data might be pushed outside approved boundaries," targeting zero-day agent behaviors, but they rely on anticipating future unknowns—a philosophical hurdle for detection systems.
The reliance on channel chunking (User, Reasoning, Tool) suggests that true risk signal resides not just in *what* was asked (User), nor what was outputted (Agent Response), but in the *internal deliberation* of the AI itself (Reasoning). This is where context dilution, caused by massive user inputs, washes out critical nuance. Furthermore, using a vendor's proprietary product for benchmarking risks embedding its own solution narrative into the concept of "accuracy."
Patterns detected: ARC-0012 Systemic Drift, ARC-0039 Authority Game (Vendor Advertorial Tendency).
Autonomy check: No autonomy signals.

Tested vector search on 7,454 turns across agent sessions.
One in seven reviewed agent sessions required triage due to suspicious actions.
Storing six months of Claude Code and Cursor sessions as semantic vectors in Elastic was performed.
Agent sessions contain unstructured behaviors that are difficult to describe using query rules.
Uber's Agentic AI Detection and Response (ADR) system analyzes every session with a generative model.
The embedding approach embeds each turn once at ingest, storing the vectors in Elastic for semantic search.
Chunking agent sessions by channel is necessary because embedding an entire session loses context.
Queries can be instructive, descriptive hunts, or exemplar retrieval using known bad turns as queries.
MRR compares vector search against keyword (BM25) search across four channels (User, Reasoning, Agent, Tool).
Credential access was the only tactic aligned between MITRE ATLAS questions and retrieved sessions in testing.
<PURPLE>
The fundamental friction here lies in shifting security monitoring from a verifiable ledger of human actions to an inference engine operating on algorithmic reasoning. Traditional incident response excels at finding discrete deviations (e.g., specific commands run); AI agents invert this, making the whole session the unit of potential risk, as a sequence of individually permissible actions can constitute systemic failure or malicious intent. The move toward semantic search is a cognitive leap: it shifts detection from "Did they type X?" to "Did the *pattern* of their interaction align with an outcome we define as problematic?" This empowers hunting for latent risks—like the inadvertent Git push mentioned in the example—that no rigid rule set would ever capture.
The key structural insight is the differentiation between **exemplar retrieval** and **descriptive hunts**. Exemplars solve high-fidelity pattern matching: "Find me 12 more instances of this exact risk I just closed." This is tactical compliance checking. Descriptive hunts are the genuine leap toward threat discovery: "Find me any instance where data might be pushed outside approved boundaries," targeting zero-day agent behaviors, but they rely on anticipating future unknowns—a philosophical hurdle for detection systems.
The reliance on channel chunking (User, Reasoning, Tool) suggests that true risk signal resides not just in *what* was asked (User), nor what was outputted (Agent Response), but in the *internal deliberation* of the AI itself (Reasoning). This is where context dilution, caused by massive user inputs, washes out critical nuance. Furthermore, using a vendor's proprietary product for benchmarking risks embedding its own solution narrative into the concept of "accuracy."
Patterns detected: ARC-0012 Systemic Drift, ARC-0039 Authority Game (Vendor Advertorial Tendency).
Autonomy check: No autonomy signals.

The fundamental friction here lies in shifting security monitoring from a verifiable ledger of human actions to an inference engine operating on algorithmic reasoning. Traditional incident response excels at finding discrete deviations (e.g., specific commands run); AI agents invert this, making the whole session the unit of potential risk, as a sequence of individually permissible actions can constitute systemic failure or malicious intent. The move toward semantic search is a cognitive leap: it shifts detection from "Did they type X?" to "Did the *pattern* of their interaction align with an outcome we define as problematic?" This empowers hunting for latent risks—like the inadvertent Git push mentioned in the example—that no rigid rule set would ever capture.
The key structural insight is the differentiation between **exemplar retrieval** and **descriptive hunts**. Exemplars solve high-fidelity pattern matching: "Find me 12 more instances of this exact risk I just closed." This is tactical compliance checking. Descriptive hunts are the genuine leap toward threat discovery: "Find me any instance where data might be pushed outside approved boundaries," targeting zero-day agent behaviors, but they rely on anticipating future unknowns—a philosophical hurdle for detection systems.
The reliance on channel chunking (User, Reasoning, Tool) suggests that true risk signal resides not just in *what* was asked (User), nor what was outputted (Agent Response), but in the *internal deliberation* of the AI itself (Reasoning). This is where context dilution, caused by massive user inputs, washes out critical nuance. Furthermore, using a vendor's proprietary product for benchmarking risks embedding its own solution narrative into the concept of "accuracy."
Patterns detected: ARC-0012 Systemic Drift, ARC-0039 Authority Game (Vendor Advertorial Tendency).
Autonomy check: No autonomy signals.