Image: assets-eu-01.kc-usercontent.com · rights & removal
From Opus 5 to Opus 5.5: better fixes at 58% less cost in the SonarQube Remediation Agent
Reporting by SonarSource Security ResearchRead the original at sonarsource.com
Executive Summary
The SonarQube Remediation Agent operates by processing backlog issues, either manually assigned or scheduled against a main branch, without requiring manual prompting. The fixes are generated by an LLM, utilizing user-provided API keys for providers like OpenAI or Anthropic, with billing running to the user's account. The agent attempts fixes in a sandbox environment and re-runs analysis; only fixes that clear the check result in a pull request for team review. An improvement of 11.9% was seen in the share of issues resolved end-to-end due to the agent closing unfinished tasks, with Opus 5.5 addressing residual issues from previous models. Furthermore, the measure of changes ready for review increased by 15.8%, reflecting that more generated fixes are considered immediately actionable by reviewers. Cost per fix decreased by approximately 58% because inference costs are billed directly to the user, reducing the cost associated with finding and fixing issues.
Facts Only
* Issues fully resolved resulted in an 11.9% increase.
* Changes judged ready to merge as-is increased by 15.8%.
* Cost per fix decreased by about 58%.
* Functional test pass rate increased by 0.3%, remaining unchanged.
* Wall-clock time per issue increased by 4%.
* The agent generates fixes using an LLM where the user supplies an OpenAI or Anthropic API key for execution.
* Unresolved issues from previous models were addressed, leading to fewer manual sweeps.
* The process involves generating a fix, applying it in a sandbox, re-running analysis, and retrying if checks are not cleared.
Full Take
The narrative centers on the emergent value of closed-loop, automated remediation rather than incremental quality gains observed in code. The primary tension arises between efficiency metrics (cost reduction and resolution rate) and process integrity (runtime). While performance indicators like functional test pass rates remain flat, suggesting no introduction of behavioral errors, improvements in merge readiness and resolution completion suggest a successful shift in the *quality* of output that aligns with human workflow expectations. The trade-off observed is temporal: wall-clock time increased by 4%, which acts as an easily measurable cost against efficiency gains. This suggests a system optimizing for speed over absolute latency, accepting slightly slower execution to achieve significant economic leverage. The implication for agency lies in shifting focus from monitoring the agent's internal process (the sandbox attempts) to validating the final output upon merge. If the mechanism successfully reduces human intervention required per fix while maintaining functional correctness, the cost reduction becomes transformative for addressing accumulated technical debt, provided the perceived cost increase is accepted as a function of efficiency. What does this trade-off imply about where organizational attention should focus: on optimizing the agent's internal loop, or on defining what constitutes an acceptable outcome for human review?
From the original · SonarSource Security Research
Issues fully resolved, no manual follow-up needed: up 11.9% Changes judged ready to merge as-is: up 15.8% Cost per fix: down about 58% Functional test pass rate: up 0.3%, so unchanged Wall-clock time per issue: up 4% A cost cut that size usually shows up somewhere else, as more rework or more fixes that quietly break something. It was not here.Read the full story at sonarsource.com
Sentinel — Human
Confidence
The text appears to be a highly informed analysis grounded in specific performance metrics, demonstrating strong contextual understanding rather than purely generative prose.
Signals Detected
low severity: Sentence length variance exhibits natural variation; complex, evaluative phrasing interspersed with direct data reporting.
low severity: The text successfully synthesizes complex technical results (metrics) into a narrative argument about the implications of those results, showing a coherent focus on economics and process.
low severity: The argumentation flows logically from specific metrics to broader implications (cost, readiness, functional safety) without relying on repetitive, template-like transitions.
low severity: Specific numerical claims and technical framing are highly detailed and context-specific, suggesting direct engagement with proprietary benchmarks rather than general LLM fabrication.
Human Indicators
Use of specific, domain-heavy metrics (11.9%, 15.8%, 0.3%) that require deep knowledge of the specific system being benchmarked.
The nuanced discussion distinguishing between different performance indicators (resolution rate vs. merge readiness vs. functional pass rate) shows analytical depth beyond typical LLM summary.
The framing is deeply rooted in an internal product evaluation context, indicating firsthand experience or very close access to the underlying data.