Modernising threat hunting: Using Semantic Search to Find Risky AI Agent Behavior

The monitoring of autonomous AI agent behavior presents a significant challenge to traditional Security Operations Center (SOC) threat hunting models, which rely on discrete, rule-based logging—a model that fails when an outcome is dictated by an emergent agent action rather than explicit human instruction. Current solutions involving having generative models review every session are costly and non-deterministic, making them unsuitable for continuous threat hunting where novel questions about historical data must be posed.
To address this, semantic search via vector embeddings is proposed as a superior method. This approach involves pre-processing all agent sessions—specifically chunking them by channel (user input, reasoning, response, tool calls)—and storing the resulting numerical vectors in a database like Elastic. At query time, natural language descriptions of risky behaviors are converted into vectors and compared against stored data to rank similarity, allowing analysts to focus human review only on high-scoring anomalies.
While one successful enterprise implementation (Uber's ADR) uses expensive LLMs for deep triage, the embedding model approach provides a scalable, deterministic way to search probabilistic data across six months of recorded interactions. A key finding suggests that the agent reasoning channel often holds richer contextual signals regarding incidents than direct user prompts.

* Tested vector search on 7,454 turns from personal Claude Code and Cursor agent sessions.
* Found one in seven sessions required human review for risky behavior.
* Stored six months of sessions as semantic vectors in Elastic.
* Behavior detection shifts from explicit queries to describing desired outcomes that are then found via similarity ranking.
* Agentic AI breaks the traditional logging model where a human decides every action.
* Uber’s Agentic AI Detection and Response (ADR) uses generative models to analyze sessions across 7k+ hosts daily.
* Embedding models convert text into rows of numbers (vectors), allowing for similarity comparison instead of lexical matching.
* Sessions must be chunked by channel (User, Reasoning, Agent, Tool) before vectorization due to contextual loss in whole-session embeddings.
* Method testing showed that the reasoning channel yielded higher relevance for incident detection than the user channel.
* Exemplar retrieval—using a known bad turn as a query—was found to be stronger at finding similar workflows than descriptive hunts alone.

The shift from keyword matching to semantic search represents an attempt to architect persistence and resilience against emergent behavior; it moves detection away from prescriptive rules (what *should* happen) toward inductive inference (what *is* happening, even if novel). By modeling the entire session's context as a probabilistic vector cloud, one can begin to map the subtle shape of risky workflows rather than just matching specific keywords. This operationalizing of "describing the behavior" creates a vital feedback loop: an investigation leads not just to remediation, but to a new, better-formed search query, refining the collective institutional memory in the vector store.
The fundamental tension lies between depth and scale. Generative model triaging is deeply intelligent but probabilistically costly and non-deterministic; embedding search offers massive scale and determinism at the cost of relying on pre-baked descriptions. The evidence suggesting the reasoning channel is richer than the user channel implies that security focus must follow the *internal cognitive process* of the AI, not just its external articulation to a human supervisor. This suggests system oversight should pivot from monitoring task completion (the prompt) to monitoring the agent's internal deliberation over allowed functions and context sourcing.
The ultimate vulnerability exposed here is systemic: when agents multiply the blast radius of ordinary mistakes by gaining unchecked implementation capability, the standard defensive perimeter becomes insufficient. The reliance on descriptive hunting also creates a knowledge asymmetry; the system can only find behaviors for which a human has yet to think to describe.
Autonomy check: No autonomy signals.
Patterns detected: none