Google Threat Intelligence Group
Google Threat Intelligence
Visibility and context on the threats that matter most.
Contact Us & Get a DemoIntroduction
Adversarial misuse of AI has increased the risk of data theft and extortion events, because when proprietary source code is exposed, defenders must scramble to identify and patch vulnerabilities while attackers deploy machine-speed AI tools against them.
By structuring the analysis process, enforcing skeptical validation steps, and injecting domain-specific human expertise directly into the pipeline, we’ve achieved a leap in efficacy. Combining AI models with a deeply structured, human expert-driven orchestration layer to tip the scales so that defenders can beat adversaries to the punch.
Today, we use the Agentic Vulnerability Discovery Harness (AVDH) to rapidly analyze code and find exploit paths during proactive reviews, penetration tests, red team operations, and incident response engagements. By combining multi-agent orchestration with our frontline subject-matter expertise, this framework helps to augment the discovery and validation of routine vulnerabilities, enabling humans to focus their impact.
To help defenders implement similar approaches for their own environments, we are sharing the details of this internal, point-in-time architecture for the first time. AVDH can also be used alongside CodeMender’s ongoing scanning to create a two-layered defense strategy.
Real-World Results
In the 10 months that we’ve been using AVDH, we’ve seen it have a significant impact. During a recent incident response investigation involving stolen corporate repositories, the harness discovered over 100 true-positive critical vulnerabilities in just two days — achieving results in a fraction of the time required for manual review.
This has greatly accelerated how Mandiant discovers vulnerabilities at scale. We have used it to analyze environments spanning tens of millions of lines of code, and execute thousands of pipelines to generate tens of thousands of findings. This rapid analysis has uncovered dozens of assignable flaws in widely used web extensions and open-source projects, resulting in 12 assigned CVEs, including CVE-2026-13242, CVE-2026-55803, and an additional dozen currently in active disclosure.
While fast, broad, high-precision scanning has been one of the key benefits of AVDH, it has also acted as a force multiplier during our targeted adversary simulation engagements. We recently processed a client’s web application source code through the harness, and quickly found a remote code execution (RCE) vulnerability that enabled initial access.
AVDH has repeatedly proven invaluable for navigating mature defenses and accelerating complex exploit chains.
Architecting the Pipeline
Harnesses have become a vital tool for cybersecurity uses of large language models (LLMs). They help mitigate much of the model’s unpredictability, driven by inherent, non-deterministic behavior, and dramatically improve their effectiveness at code analysis.
The programmatic infrastructure of a harness orchestrates agents in a strictly deterministic manner toward objective completion. For AVDH, we used the Google Agent Development Kit (ADK), an LLM framework that implements the most common agent orchestration patterns, and provides flexibility for configuring custom and third-party integrations. This approach aligns with the agentic orchestration capabilities now available in Google Antigravity, which provides a centralized workspace for builders to steer and manage these agentic workflows.
Our decades of frontline experience discovering and remediating vulnerabilities across every software domain helped us structure AVDH around the proven methodologies our consultants execute daily. AVDH chains specialized agents together in a sequential pipeline, much like the waterfall approach to software development: each phase is completed before the next begins. This pipeline yields a prioritized, risk-rated list of findings, primed for a human expert to review.
Just as frontline security experts rely on organizational context, an agentic harness requires rich environmental inputs — such as asset inventories, software bills of materials (SBOMs), architecture documentation, and threat intelligence. When fed into a distilled human knowledge base, this contextual data allows agents to dynamically select relevant skills, language rules, and vulnerability patterns for deep analysis.
Figure 1: Sequential vulnerability discovery methodology
Threat Modeling
A critical first step when using AI for code security analysis is to establish a threat model for the target codebase. Software architectures can vary wildly, and without a threat model, we can lose valuable context, such as attack vectors, business logic, and reachability.
While traditional source code review engines rely on rigid pattern-matching rules, an LLM offers the distinct advantage of distinguishing code accessible to a standard user from code restricted to an administrator, or code that is never executed at all.
Our pipeline begins by dispatching an Explorer agent to identify the core purpose of the target codebase. This agent determines the software domain (such as web or desktop application), reviews discovered documentation, flags directories to exclude from scanning (such as those containing unit tests), and dispatches Specialist Explorer subagents.
These Specialist Explorers then delve into their respective focus areas, including authentication, authorization, routing, and other domain-specific categories. Their output is passed to a Threat Model Synthesis agent, which aggregates the findings into a cohesive threat model.
Figure 2: Codebase reconnaissance workflow diagram
Once this stage of analysis is complete, the consultant is presented with both textual and visual representations of the threat model for verification before analysis continues. This approval gate helps ensure that the rest of the pipeline has an accurate foundation to operate on.
Figure 3 shows an example layout of a visual threat model generated by the harness, indicating which application components are exposed and how they connect.
Figure 3: Visual representation of a threat model for a sample codebase
Entry Point Discovery
With the threat model established, we deploy parallelized Discovery agents to analyze every in-scope file. These agents use the lightweight Gemini Flash Lite model to process code at scale to extract critical application entry points, such as HTTP routes, inter-process communication (IPC) listeners, and other domain-specific attack vectors. Simultaneously, they isolate and extract all identifiable sources of user input nested in these identified entry points.
Figure 4: Entry point discovery workflow diagram
Context Enrichment
Once entry points are selected for analysis, the harness assigns each to a dedicated Enrichment agent. In enterprise applications, analyzing an entry point in isolation is rarely sufficient — critical components like sanitizers, permissions, and routing conditions are often highly distributed.
Furthermore, vulnerabilities frequently hide deep within nested function calls, multiple hops and files away from the initial source. To bridge this gap, the Enrichment agent navigates the codebase to aggregate contextually relevant code for its assigned entry point. It evaluates this aggregated data to determine whether the entry point requires further analysis by the Access Control agent, the Data Flow Analysis agent, or both.
Figure 5: Context enrichment workflow diagram
Hypothesis Generation
Effective code analysis hinges on observing two primary properties: control flow and data flow. While control flow dictates the execution order of tasks and instructions, data flow traces how information moves and transforms throughout the application.
Our AVDH delegates these critical tasks to the Access Control and Data Flow Analysis agents, respectively.
At this stage, these agents perform minimal self-validation. Their primary objective is expansive brainstorming. To manage the sheer volume of hypotheses produced, this creative process is kept in check by a Confidence Filter configured by the consultant.
Figure 6: Hypothesis generation gating diagram
The Access Control agent evaluates the protections surrounding the target entry point to determine its overall accessibility to application users. Its primary purpose is to validate security assumptions, and confirm whether privileged functionality is restricted or inadvertently exposed to unauthorized users. This analysis exposes flaws where a check was never made, or made against the wrong identity, including missing authorization, privilege escalation, and cross-site request forgery (CSRF).
Meanwhile, the Data Flow Analysis agent tracks the flow of user input from the initial entry point throughout the entire application. It traces data as it traverses nested function calls, sanitizer transformations, and storage boundaries like databases.
The agent's goal is to determine if this user-supplied data ever reaches a dangerous "sink," a function where malicious input could execute and cause harm. This deep tracing unearths vulnerability classes such as SQL injection, cross-site scripting (XSS), command injection, and path traversal.
Hypothesis Validation
Once hypotheses are generated for the target codebase, our harness dispatches a new set of agents to validate them. In LLMs, the temperature parameter dictates the variability and randomness of the output: lower temperatures yield predictable, stable responses, while higher values can produce radically different results each time.
Our harness uses this by dispatching multiple Validation agents configured with high temperature settings to assess each hypothesis, alongside a single Validation Synthesis agent tasked with processing their verdicts to make a final decision. Using a higher temperature enables our validation to cover a much broader spectrum of possibilities rather than more predictable, expected responses. Ultimately, this temperature configuration provides richer, more comprehensive context for the agent making the final determination.
The Synthesis agent evaluates the reasoning and verdicts from the Validation agents to determine if the hypothesis meets our rigorous quality criteria and aligns with the overall threat model. From here, there are three possible outcomes:
-
Confirmed finding: The hypothesis is robust, and the Validation agents have independently verified it.
-
Disproven hypothesis: The Validation agents surface significant conflicting evidence disputing the validity of the flaw.
-
Rejected hypothesis: The hypothesis does not align with the established threat model, or does not qualify as a vulnerability.
Figure 7: Hypothesis validation workflow diagram
Human Subject-Matter Expertise
Expert Validation
Once the harness deduplicates and risk-rates the confirmed findings, we continue the analysis with rigorous human expert review. We perform due diligence by dynamically replicating the exploitation and executing Proof-of-Concept (POC) code to verify that the AI assumptions are accurate and that no unseen compensating controls hinder the attack path.
Once validated, the consultant synthesizes the AI-generated finding with their own expert analysis and prepares it for formal disclosure. Conversely, any findings that fail to pass this dynamic testing phase are discarded.
We encourage network defenders considering implementing similar vulnerability discovery harnesses to manually validate findings.
Figure 8: Human-in-the-loop handover diagram
Distilled Knowledge
While human-in-the-loop validation of confirmed findings effectively minimizes false positives, we still need to address false negatives.
To determine if the AI agents had missed any vulnerabilities, we engineered a rules-based approach that directly injects Mandiant subject-matter expertise into the analysis pipeline. It uses highly-specialized prompts distilled from our consultants' collective knowledge, similar to the skills engineering concept.
Integrating this human intelligence directly into our AI-driven analysis significantly elevates the precision of the results. To ensure this knowledge system remains modular and scalable, we structured it as a hierarchy with the software domain at the top, followed by three primary rule categories: language, framework, and vulnerability.
Figure 9: Agentic rule system hierarchy
Framework and language rules apply across the entire pipeline, equipping the agents with consultant insights into the specific technologies employed within the target codebase. These rules encompass critical details, such as common entry point definition patterns and unique attack surfaces, with additional contextual information essential for threat modeling.
In contrast, vulnerability rules apply exclusively during the final stages of the pipeline, prescribing precisely how to discover, validate, and risk-rate specific types of vulnerabilities. This structured system ensures the entire analysis pipeline is infused with Mandiant’s human expertise in a maintainable, highly modular way.
Figure 10: Methodology rule application diagram
Measuring Success
Accurate benchmarking and evaluation are critical to maintaining and continuously improving an agentic code analysis pipeline. We developed a rigorous internal methodology for measuring the performance of our orchestration harness, ensuring that prompt adjustments and rule updates consistently drive positive, data-backed improvements without introducing quality regressions.
We recommend implementing an analogous benchmarking system to gauge progress and efficacy with your code analysis pipeline.
Benchmark Targets
While public code vulnerability datasets exist, training data contamination presents a significant challenge for evaluating LLMs. It is possible that modern frontier models have already ingested these public repositories, making it nearly impossible to determine if a model is genuinely reasoning through a vulnerability or simply recalling a memorized solution.
To ensure high-fidelity evaluation, we developed a suite of proprietary, synthetic codebases. These custom benchmarks span software domains, programming languages, vulnerability depths, and architectures, from traditional monoliths to modern microservices.
Crucially, our security consultants manually verify every injected vulnerability to ensure it is genuinely reachable and dynamically exploitable. As we tune the harness and its underlying prompts, we enforce strict review processes to actively prevent the AI from overfitting to these benchmark codebases.
Benchmark Grading
Our grading process pairs AI evaluation with expert human-in-the-loop review. When our harness analyzes a benchmark directory, the output is passed to a dedicated Grading agent. This grader evaluates the pipeline's findings against our ground-truth dataset, demanding precise vulnerability matches rather than relying on loose semantic similarity.
From there, the grading pipeline branches out to handle edge cases:
-
False positive triage: Harness findings that do not map to the ground truth are routed to a secondary agent to definitively classify them as either false positives or legitimate vulnerabilities.
-
Duplicate resolution: If the pipeline produces multiple findings that map to a single ground-truth issue, another agent analyzes the cluster to determine whether the findings are duplicates.
Finally, a human expert manually reviews the graded data to validate the accuracy of the AI judges. We perform this rigorous testing cycle across multiple domains and architectures for every major release of the harness, averaging out the results to account for the inherent non-determinism of LLMs.
Framework and language rules apply across the entire pipeline, equipping the agents with consultant insights into the specific technologies employed within the target codebase. These rules encompass critical details, such as common entry point definition patterns and unique attack surfaces, with additional contextual information essential for threat modeling.
In contrast, vulnerability rules apply exclusively during the final stages of the pipeline, prescribing precisely how to discover, validate, and risk-rate specific types of vulnerabilities. This structured system ensures the entire analysis pipeline is infused with Mandiant’s human expertise in a maintainable, highly modular way.
Figure 11: Benchmarking process diagram
Conclusion
Securing the software development pipeline has emerged as a defining challenge in modern enterprise defense. Our ongoing research has shown that defenders face extraordinary challenges in responding to the rapidly-growing capabilities of adversarial AI.
To match these emerging threats, securing the code pipeline must be a critical component of a modern defense strategy. Manual source code review can’t keep pace with AI, and traditional scanning engines consistently miss the broad spectrum of vulnerabilities hidden in modern software.
However, the success of our harness proves defenders can reclaim the advantage against adversarial AI. By embedding frontier models within an expert-defined harness, defenders can automate the discovery of routine vulnerabilities.
Handling these standard findings transforms source code visibility into a scalable defense, freeing our consultants and other defenders to focus entirely on complex flaws. We believe that the process of building and refining this harness has demonstrated that AI is most effective when deployed as a practical multiplier for human expertise.
While our tool was built for point-in-time assessments and deep, proactive vulnerability discovery, our recent blog post describes how CodeMender complements this by providing continuous, AI-enabled monitoring for software development and vulnerability management. For organizations looking to deploy these capabilities out-of-the-box, Google AI Threat Defense offers an always-on platform. It includes CodeMender’s code scanning and remediation to analyze systems, prioritize threats, patch vulnerabilities, and continuously monitor for new attacks. Combining AVDH for targeted, deep analysis with CodeMender’s ongoing scanning creates a two-layered defense strategy. This approach leverages point-in-time remediation for complex chains while maintaining continuous visibility over the development lifecycle.
Want a deeper look at how we built and deploy this pipeline in real-world environments? Join us at Cyber Defense Summit September 15-16, 2026 in Washington, D.C. where we will be presenting "How Mandiant Orchestrates Gemini to Find Zero-Days Before Adversaries." We will walk through live demonstrations, share lessons learned from deploying agentic workflows, and discuss the future of AI-driven offensive and defensive capabilities. Register for the Summit here.