Turning an Attacker's Toolkit on Our Own Detections

Automated detection engineering is achieved by applying the multi-agent approach used in vulnerability hunting to build defenses. The process involves using multiple narrow AI agents in a staged pipeline: planning, generation, deduplication, adversarial validation, red-teaming, reachability testing against real data, and reporting. This system is designed to mirror the attacker's job of finding vulnerabilities by focusing on building detections that catch those attacks. A key element is that human oversight remains critical; AI generates candidates which are then subjected to adversarial review and human approval before shipping, ensuring quality through a structured process.
The success of this approach relies heavily on having pre-prepared, high-quality customer telemetry within a normalized data lake, as this real data provides the necessary scoreboard for testing detection effectiveness against actual, realistic activity. The process evolved from slow, manual, human-driven work to an automated orchestration where skills and rules are explicitly codified, allowing agents to operate in parallel with an adversarial reviewer enforcing strict quality standards.

* A multi-agent approach is used for building detection systems, mirroring vulnerability hunting methods.
* The system involves a staged pipeline: plan, generate, dedupe, adversarial validation, red-team, reachability, and report.
* Candidates are tested against real customer telemetry in Mitiga's Cloud Security Data Lake.
* The process incorporates an adversarial reviewer to check candidates against written rules.
* Automated detection engineering does not deploy detections automatically; it opens a pull request for human review and merge.
* Quality gates include deterministic checks ensuring candidates load against real data before validation.
* The system relies on pre-existing, normalized customer telemetry for reachability scoring.
* A key step involves agents testing detectors against the attack they are designed to catch in a controlled environment.

The narrative describes a shift in the locus of human effort: moving from manually authoring every detection to overseeing and defining the quality of automated systems. The central tension lies between the speed afforded by multi-agent automation and the necessity of stringent, explicit guardrails, which requires translating tacit, expert knowledge into machine-readable rules. The system's durability is not in the agents themselves, but in the layered structure that forces systematic scrutiny—from defining skills (Act two) to imposing gates (Act three).
The pattern observed is a resistance to letting automation create opaque systems; instead, the process mandates transparency through adversarial testing and data-backed scoring. This suggests an underlying principle that efficacy is not merely about generating output but about provable causality against reality. The transition from functional success to hardening quality reveals a pattern of systemic improvement where failure modes (like poor formatting or invalid data loading) become critical determinants for system evolution rather than mere bugs.
What assumptions underpin the push for automation here? It assumes that complex adversarial reasoning can be decomposed into discrete, verifiable stages. The implication is that cognitive sovereignty in this domain shifts from knowing *how* to write a rule to understanding and designing the validation framework for the agents themselves. If the goal is to move detection engineering toward an outcome where the system proves its claim against live threats, the next evolution must focus on how to automate the generation of the "adversarial test," establishing dynamic proof that moves beyond historical correlation into real-time causal verification of security posture.