As AI makes threat modeling more complex, CISOs need a faster, more practical way to identify new risks without losing sight of the security basics.
A few weeks ago, on a busy day, threat-modeling expert Adam Shostack opened an email from a client. Someone at that organization had vibe-coded an app and put it to work with customer data. Now, the client wanted to know what risks the tool posed. And what it should do about them.
They needed answers quickly, so Shostack gave himself 15 minutes to analyze the system.
Soon, he had “a list of meaningful threats,” including hallucination and bias, problems that STRIDE, a widely used application-security threat model framework, would not have shown.
“I found some interesting stuff that wasn’t obvious to me when I started,” Shostack says.
During those 15 minutes, he used PHANTOM-B, a threat modeling framework he developed. It starts with the standard question “What can go wrong?” but applies it specifically to the LLM components of a system.
Each letter in the name represents a specific threat: Prompt injection, Hallucination, Anthropomorphization, Non-explainability, Training issues, Overreliance (including data quality or “poison”), Missing security engineering, and Bias.
The new framework is meant to complement STRIDE, not replace it. Whereas STRIDE applies across the entire application, PHANTOM-B can be used for the components that interact with LLMs.
The framework is designed to produce useful results quickly, making threat modeling less daunting and less costly. And, therefore, more likely to happen.
“I’m a big fan of: Let’s make it inexpensive to do this work,” says Shostack, who presented PHANTOM-B at Black Hat USA 2026.
Shostack adds that CISOs and their organizations are caught between two challenges: the pressure to secure AI quickly and the difficulty of applying existing security tools to it.
“Businesses need an entry point, which is something that you can fit into a one-hour meeting or a 10-minute conversation with an executive,” he says.
The need to use tools like PHANTOM-B is becoming more urgent, as companies increasingly rely on AI. “Threat modeling is more important than ever because AI is introducing genuinely new application architectures and novel risks,” says Jeff Williams, founder of OWASP and founder and CTO of Contrast Security.
Why traditional threat modeling falls short
Threat modeling has always been a struggle. Organizations of all sizes struggle to do it consistently because it takes time and often can’t keep pace with the number of applications being built and updated. As a result, many companies reserve it for their most critical systems, leaving much of their software without the same scrutiny.
“AI didn’t break threat modeling. It exposed weaknesses that were already there,” Williams says. The uncomfortable reality, he argues, is that threat modeling has never been standardized or automated enough to scale.
The hardest part is not naming threats but figuring out which applications work. “Traditionally, we’ve attempted that with surveys, questionnaires, interviews, stale Visio diagrams, and spreadsheets,” Williams says. “The resulting model is often incomplete before the exercise even begins, and it becomes obsolete as soon as the application changes.”
Generative and agentic AI amplify those weaknesses, because their behavior is not easy to predict. Conventional software follows rules that engineers can trace from input to output. AI systems interpret natural-language instructions and generate probabilistic responses, so the same request may produce different results. And when agents can access data or call tools, those results can trigger actions elsewhere.
“Traditional threat modeling was applied to systems that were generally deterministic. The modern generative and agentic AI systems are non-deterministic, so modification of how we view their risk profiles is needed,” says Brian Glas, vice president of consulting services at CODIFIC and one of the project leads for the OWASP Top 10.
This distinction also appears in Microsoft’s AI threat modeling guidance, which notes that the boundary between data and commands can blur, creating new attack surfaces and enabling failures to spread across tools, memory, and agents.
At the same time, faster development cycles are leaving teams with less time to think through how systems should be designed and secured. And complexity also makes it harder to examine threats in isolation. A weakness in a document, for instance, could alter a model’s response and cause an agent or tool to take an unsafe action in another system. OWASP warns that these risks can spread across the entire retrieval-augmented generation pipeline.
“A chaining of threats or weaknesses results in the exploitation of the application,” Glas says.
The answer to these is to make threat modeling an ongoing practice, a recommendation that is made in NIST’s Artificial Intelligence Risk Management Framework.
But despite a growing body of guidance — including Shostack’s PHANTOM-B, Microsoft’s AI threat modeling recommendations, NIST’s Generative AI Profile, and OWASP’s Top 10 for LLM Applications — there is still no definitive playbook for threat modeling AI systems.
“Threat modeling is not a mature practice,” Glas says. “I believe we are still trying to figure this out.”
Improving threat modeling in the age of AI
A 15- or 30-minute threat-modeling exercise sounds deceptively manageable. But how much useful security work can teams really accomplish in such a short time?
Shostack says a short session is not supposed to be exhaustive. It should, however, identify enough meaningful risks to guide the next decision: Is the risk acceptable, does the system need to change, or is a deeper review required?
“One of the things that we’ve learned from the agile world is that when you make the period of work small, you iterate more and faster,” he says. If the first session misses the mark, the team can erase the whiteboard and try again without losing days or weeks of work. “You make the experiments cheap, and when the experiment is cheap, you can run it repeatedly.”
The result should be a set of concrete stories about how the system could fail. Those scenarios can help CISOs understand the risks they are accepting and help technical teams choose appropriate controls, which can mean limiting data access, narrowing tool permissions, adding human approval points, or even reconsidering whether an LLM is needed at all.
PHANTOM-B aims to fill in gaps left by other frameworks. It touches on relevant but sometimes exotic concepts such as anthropomorphization, asking whether users or developers are treating the model as if it understands intent, reasons like a person, or will reliably follow an instruction such as “Do not do X.” That misplaced trust can shape how a system is designed and how much authority it receives.
Non-explainability raises another problem. If an LLM screens résumés, evaluates medical images, or makes business decisions, an organization may need to justify its output. But asking the model to explain itself may produce a plausible-sounding answer rather than an accurate record of how it reached the original result. Because LLMs are probabilistic, rerunning the same request may not reproduce the same output, making investigation and debugging more difficult.
The “missing security engineering” prompt is a reminder that adding an LLM does not eliminate conventional software risks. According to Shostack’s white paper, it can magnify them, particularly when organizations rush to deploy vibe-coded software that no one fully understands.
This tool, together with the work from OWASP, NIST, CISA, and others can help teams understand the emerging risk landscape. “Threat modeling is a creative process,” Williams says.
Mistakes to avoid when threat-modeling AI systems
LLM-powered tools may be relatively new, but they are still apps. They use third-party libraries, process untrusted input, handle sensitive data, expose APIs, and depend on identity and authorization controls.
Focusing heavily on new AI threats while ignoring the basics can leave organizations exposed to preventable vulnerabilities. “The biggest mistake is forgetting that all the existing risks still apply,” Williams says.
He advises CISOs not to jump straight to prompt injection or model manipulation while skipping the fundamentals. “That’s backwards,” Williams says. “Traditional application security has to be the foundation. The novel AI risks are an additional layer on top of it, not a replacement for it.”
Glas agrees. “Details have changed; fundamentals of risk have not,” he says. “Don’t throw away the security lessons learned in the last 25 years to chase velocity.”
Glas also points to another common problem: missing context. Teams may not fully understand how an LLM fits into the broader application, including its access controls, trust boundaries, inputs, and data flows. Without that view, important risks can easily slip through the cracks.
Trying to catalog every conceivable attack is not the answer, either. “You can’t possibly track them all,” Glas says. “The number of possible attacks is almost limitless.” Instead, he recommends starting with the assets that need protection and the preventive and detective controls already in place. From there, teams can model different attack types to uncover weak or missing controls and determine what is needed to reduce risk to an acceptable level.