A detection rule can show up as deployed on a coverage dashboard and still never fire when an attacker uses the technique it was built to catch.
Conifers assessed 14,652 detections in its customer base, including rules written by customers and detections managed by vendors in SIEM, endpoint, cloud, identity, email and network tools. The research found that 47% of detections in the average organization need attention.
The failures fall into five groups
Logic bugs mean a detection runs but won’t fire when the targeted technique is used, because of wrong operators, broken parsing or conditions that can never be true. Missing telemetry leaves a detection running against data sources that stopped flowing, were never onboarded or changed format.
A detection can also query the wrong table or index, one that doesn’t contain the events it needs, and still look healthy on every dashboard. Duplicated detections cover the same behavior with slight variations, adding alert volume and maintenance work without adding coverage. Noisy detections fire so often and with so little precision that analysts learn to ignore them.
According to Conifers, all five share one trait. Each of these detections still shows up as deployed in a coverage dashboard.
“The underlying problem with threat detection isn’t that detections were poorly written. It’s that the telemetry beneath them changes, making detections stale and ineffective without anyone realizing it,” said Rutger de Boer, CTO at DTX.
He noted that the research “validates what we see with our customers every day.”
“By continuously testing detections at scale, organizations gain confidence in their security posture while significantly reducing the manual effort required from analysts,” de Boer added.
Detections security teams can’t edit
The SIEM accounts for a minority share of the detection surface, according to the findings. Endpoint platforms, cloud security tools, identity providers, email gateways and network sensors ship their own detections, and together they generate most of what lands in the analyst queue.
Security teams can’t see inside or change the logic of vendor-controlled detections. That leaves them two options when one of those detections is noisy, the researchers say. They can suppress it and accept a blind spot, or they can let it keep flooding the queue.
Known threats with no coverage
Conifers also measured how much of each organization’s threat landscape, as identified by its own threat intelligence, has matching detections, hunts or compensating visibility. On average, organizations had coverage for 63% of the threats their own intelligence flagged as relevant, leaving more than a third of those known threats without operational defenses.
The gap comes from the manual and slow process of turning intelligence reports, feeds and briefings into detection logic and hunting hypotheses, according to the researchers.
Researchers argue that closing these gaps requires AI that connects directly to threat intelligence and turns it into detections and hunts automatically. Under this approach, detection engineers and hunters direct the strategy and make the judgment calls, while the system handles the loop from intelligence to hunts, hunts to detections, and coverage gaps to telemetry fixes.
Measured against MITRE ATT&CK techniques relevant to each organization, based on its threat landscape, technology stack, tools and controls, average protection stood at 64%. That leaves one in three relevant techniques without a reliable detection.
“For years, the industry has measured detection strength by counting rules and tools. But deployed is not the same as protected,” said Tom Findling, CEO of Conifers.
In Findling’s view, security teams need to know “which detections work, which threats remain uncovered, and how quickly intelligence becomes protection.”