Spectra Assure Free Trial
Get your 14-day free trial of Spectra Assure for Software Supply Chain Security
Get Free TrialMore about Spectra Assure Free TrialKey takeaways
AI could soon amplify the consequences of the software industry’s failure to eliminate longstanding coding flaws — a failure that has defied years of pushing Secure by Design, a proactive application security (AppSec) approach that emphasizes incorporating security considerations throughout the software development lifecycle.
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) recently analyzed vulnerability data from 2024 and 2025 and found that software continues to be plagued by many of the same basic weaknesses that have persisted for years, and in some cases decades.
CISA identified memory safety and improper input validation as among the most prevalent weaknesses across both Common Vulnerabilities and Exposures (CVEs) and other vulnerabilities in its Known Exploited Vulnerabilities (KEV) Catalog. Injection flaws, including SQL injection, cross-site scripting, and command injection, also featured prominently, as did path traversal and missing authentication.
About 4o% of the vulnerabilities in CISA’s KEV Catalog in 2024 stemmed from a handful of what it called “stubborn weaknesses” and well-understood design errors that CISA has described as preventable through Secure by Design practices. The agency noted in its report:
“Threat actors continue to succeed, in part, because simple, preventable software weaknesses remain unaddressed. Resolving fundamental issues would eliminate a significant portion of today’s most common compromises.”
In the age of AI, Secure by Design is more important than ever. Here’s why.
[ Webinar: Secure by Design — Why Trust Matters for Risk Management ]
Improving the software industry’s track record on the elimination of longstanding software weaknesses is more urgent now that AI is rapidly accelerating vulnerability discovery and exploit development, said Ted Miracco, CEO of Approov.
AI is also making software attacks cheaper by speeding up tasks such as reverse engineering, vulnerability discovery, API mapping and scraping, and the creation of exploit and automation scripts. That means organizations can’t count on vulnerabilities remaining hidden from bad actors until a patch is ready, Miracco said.
“CISA’s findings reinforce the need to move from treating vulnerabilities as isolated defects to addressing the underlying architectural and engineering practices that allow them to recur.”
—Ted Miracco
For him, Secure by Design should involve the eradication of entire classes of vulnerabilities while reducing the attack surface so that no vulnerability can result in an operational compromise.
That will require a new incentive structure for the software industry, said Yasir Zahid, cybersecurity leader and founding member of Secure.com.
“Software ships fast, security reviews happen late, and finding a flaw after release is cheaper in the short term than preventing it at design. So teams stay stuck reacting to individual findings instead of eliminating whole flaw classes at the source.”
—Yasir Zahid
Zahid said input validation failures aren’t just a result of developers not knowing about them; rather, the systems they work in don’t enforce safe defaults or catch unsafe patterns before code merges.
Zahid said that reordering incentives will require two immediate changes:
“Right now, the cost of a vulnerability lands on the customer, not the producer.”
—Yasir Zahid
CISA’s analysis shows that, for the moment at least, the AI-sparked growth in vulnerability discovery has not quite translated into a corresponding expansion of the threat landscape. For instance, the CVE list more than doubled between FY2024 and FY2025 from 14, 234 CVEs to 30,517 CVEs. But the KVE Catalog, which lists exploited flaws, grew only by about 20% over the same period, indicating that attackers concentrated on a relatively small set of vulnerabilities.
CISA found that attackers generally aren’t relying on exotic techniques or cutting-edge zero days to compromise organizations. Instead, they are scanning for and exploiting known vulnerabilities and basic security weaknesses that organizations often fail to patch, properly configure, or retire in time.
That concentration of exploited vulnerabilities could become harder to manage if AI-enabled vulnerability discovery expands the pool of flaws that defenders must assess and prioritize.
Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs, takes it as a warning that CISA states in the report that it is establishing the baseline before AI-enabled vulnerability discovery scales up. “At AI speed, organizations still prioritizing by Common Vulnerability Scoring System (CVSS) scores instead of CISA’s newer Stakeholder-Specific Vulnerability Categorization (SSVC) framework won’t keep up,” he said.
The persistence of some vulnerability classes among the top 10 for nearly two decades suggests the industry has been able to absorb the cost of repeatedly discovering and patching flaws that could have been prevented in the first place, Krell said. But AI threatens to change that equation.
“When discovery and exploitation scale faster than remediation can respond, eliminating weakness classes during development stops being a security ideal and becomes an operational necessity.”
—Jacob Krell
The parties who are in the best position to drive change are the buyers of software, not developers or security teams, he said, adding that CISA’s Secure by Demand Guide already lays this out. Enterprise customers and federal agencies can ask vendors which vulnerability classes they’ve eliminated and what the road map looks like for the rest. If they made those questions a hard requirement in every software contract, vendors would treat memory safety road maps as competitive differentiators instead of slide deck promises, he said. “The lever exists, but buyers aren’t pulling it,” he said.
CISA’s report also raises questions about what Secure by Design should look like in the age of AI, when AI systems and AI agents can act autonomously, often with very permissive privileges.
Ryan McCurdy, vice president at Liquibase, said that Secure by Design also needs to be extended beyond considering vulnerabilities in code. “As AI agents gain access to databases and other production systems, they can make changes that create real risk without exploiting a vulnerability at all,” he said.
When agents can change permissions, remove a constraint, expose sensitive data, or make a destructive schema alteration using completely valid commands, the focus has to cover both what AI can do and what it can generate, he said.
“Organizations need to know what changed, whether it was authorized, and whether it meets policy before that change becomes part of a production system.”
—Ryan McCurdy
Data integrity is another important Secure by Design concern in the AI era, Aproov’s Miracco said. AI systems are only as trustworthy as the data and inputs they receive, and if attackers use spoofed, manipulated, or unauthorized clients to inject data into backend systems, they can potentially influence downstream analytics, business decisions, or AI models, he said.
“Secure by Design therefore needs to encompass not only confidentiality and availability, but also the integrity and provenance of data entering critical systems.”
—Ted Miracco
James McQuiggan, a security awareness advocate at KnowBe4, said some of the challenges to adopting Secure by Design and Secure by Default principles for AI may be addressed in turn with AI. That’s because AI significantly enhances Secure by Design by automating threat modeling, security testing, and predictive analysis, allowing systems to be built with a deeper understanding of potential risks, he said.
McQuiggan said AI-powered code analysis can help developers identify and fix vulnerabilities early, ensuring that security is embedded from the ground up, as well as enabling more resilient designs that potentially mitigate risks before they become exploits.
Meanwhile, AI can strengthen Secure by Default by adjusting security settings based on real-time threat intelligence, ensuring optimal protection without user intervention.
“AI-driven automation ensures security remains effective over time, preventing misconfigurations and enhancing overall system resilience.”
—James McQuiggan
Katie Norton, a DevSecOps and software supply chain security analyst with IDC, said baking security into products is a good idea that is reflected in CISA’s Secure by Design initiative, but those benefits do not mean that companies can stop investing in security tools.
In fact, assuming a product is safe because it has security baked in will be able to secure itself against threats “is a huge red flag,” Norton said. That's especially true in the AI coding era.
“That kind of complacency can be just as risky as ignoring security altogether. Security is adversarial by nature, and attackers are constantly looking for creative ways around even the most carefully designed controls. So, while I’m enthusiastic about products that bake in security, I’m skeptical we’ll ever reach a point where we trust the thing to secure itself without external checks.”
—Katie Norton
Software delivery can no longer depend on trust, said Ryan McCurdy, vice president of marketing at Liquibase, because attackers can compromise trusted infrastructure and use it to make malicious changes look legitimate.
“Knowing where a change came from is important, but it doesn’t tell you whether the change itself is safe. The source of the change isn’t what determines risk.”
—Ryan McCurdy
Attestation answers a question about origin. Deciding whether a release is safe to ship or consume takes analysis of the final, assembled artifact using binary-level inspection tools that will catch malware, tampering, and behavior changes that a clean provenance record will never surface.
Saša Zdjelar, chief security officer at Nvidia and former chief trust officer at ReversingLabs, said Secure by Design has helped mature industry conversations about software security. But there's still a lot of work needed considering lessons from recent high-profile supply chain attacks. “3CX and SolarWinds have more to do with malware implants and integrity issues than traditional vulnerabilities,” he said.
Those types of software supply chain weaknesses can't be found through traditional application security testing (AST) tools or software composition analysis (SCA), Zdjelar said, "because that's just not what they’re designed for." Secure by Design is still too wedded to traditional AST and SCA without encouraging better context of how software is compiled and deployed, he said.
For Secure by Design to deliver on it's promise, Zdjelar said, organizations need more holistic tools that work for producers and consumers of software — akin to how vehicle crash testing evolved.
“[You] wouldn't crash-test a radio volume knob and a windows up-down button and a seatbelt separately and a rear car seat separately and a visor separately. You crash-test the vehicle when it's been fully assembled so that you know how the system as a whole operates or will perform in that type of environment.”
—Saša Zdjelar
The post-mortem reaches sobering conclusions, and demands a plan of action for the AI industry — plus your SecOps strategy.
The TeamPCP actors, alleged to be behind one of the most active supply chain threats, were arrested — but this is not the end of Shai-Hulud.
UAT-10147 leveraged agentic AI to go beyond scripting to deliver a backdoor. The method highlights the need for agentic SOCs.