Claude Mythos is arguably the strongest cybersecurity model that Anthropic has built. But we know that model capability is only part of what determines how well an AI vulnerability product performs.
To test that, we put Anthropic’s Claude Security, which runs on Mythos, and Aikido Code Security Audit head-to-head on the exact same target to see which harness can deliver the best coverage and at what cost. Code Security Audit is part of Aikido’s AI Code Analysis suite. We also tested Codex Security.
We ran all three products against a private benchmark application with 89 known vulnerabilities identified across the set beforehand.
The results? Claude Security with Mythos found 60 of 89 vulnerabilities (67%) at a cost of $157, while Aikido’s Code Security Audit found 68 of 89 (76%) for $75. Codex Security had 58 of 89 for $125.
Aikido found 8 more vulnerabilities than Claude Security with Mythos, including 1 critical and 3 high severity issues, while costing less than half as much. Aikido found a full ten more than Codex Security. These results are yet another example that performance and efficiency depend on the system around the model, and even with the most capable model, performance can still lag behind.
The harness still makes the difference
As we have written before, vulnerability discovery is fundamentally a search problem. A tool must decide where to look, which hypotheses to chase, how deeply to investigate them and when a finding has satisfying evidence to report.
Claude Security uses a sophisticated process involving repository mapping, scanning agents, and multi-stage verification. Aikido has its own sophisticated process, but differs in how it allocates that work, using lower-cost, smaller models that are chosen specifically for the tasks that they perform best at. These smaller models can explore a wider range of independent hypotheses, while expensive frontier reasoning is reserved for the stages where it is most likely to make a difference in the outcome.
That allocation is what let this run spend its budget on breadth of investigation rather than pointing the most expensive model at everything, and this is what resulted in substantially higher recall.
This higher recall included several issues with real-world impact that Aikido’s swarm of agents caught but Mythos missed. One was a critical cross-tenant authorization flaw that allowed users to update their own account settings and change their organization assignment to add themselves to another organization, giving them access to that organization’s data. Aikido also found subtler and easy-to-miss issues, including a timestamp-format mismatch between the application and database that allowed expired API keys to remain valid.
Frontier intelligence gets expensive fast
Mythos starts at $10 per million input tokens and $50 per million output tokens. That adds up fast when a review expands into hundreds of agent trajectories across a large repository.
The difficulty is that agentic code audits don’t have a fixed amount of work. Depending on factors like repository size and application complexity, the number of leads to chase and the required reasoning tend to vary from one repository to another.
Claude Security leaves this variability with you. Users are flying blind because Claude Security doesn’t give you the cost of the scan until the run is already finished. So when the dust settles, you can end up with a cost much higher than you’d expect.
Aikido makes a different choice. We take on that uncertainty ourselves. Before the scan begins, we dynamically estimate its cost from the repository’s size, reviewable code, and complexity and show that estimate to the user.
As a user, you get to raise or lower the coverage level to fit that estimate before committing to it. Our run on the benchmark repo would’ve cost you $75 at recommended depth while Claude Security would have cost a flat $157.06.
And if you feel the repository deserves even more intensive coverage than what we have auto-estimated, you get the freedom to increase this.
In our Claude Security run, no cost estimate was provided beforehand, and the total became clear only after the scan had completed.
The repository we used for the scan is about ~15000 lines of code. Having no option to view an upfront cost estimate or tune the analysis effort level to your budget is manageable on a repository this size. But it becomes a much bigger problem as the vulnerability search costs grow with repository size, application type, and pipeline complexity.
Point an expensive model at a monorepo, or run it on every pull request, and a missing pre-scan estimate can leave you with costs substantially higher than what you’d expect.
What this means
We’ve argued for a while that harness design determines security outcomes more than the model which sits underneath it. This benchmark provides us with a first glimpse at a head-to-head number, by testing Mythos 5 and Codex Security against Code Security Audit on the same repositories, and the harness still made the difference.
Of course, one benchmark doesn’t close the argument, but it does provide evidence that the ‘harness over model’ no longer rests on assertion alone.