AI may let hackers exploit newly disclosed software flaws so quickly that organizations should weigh whether to immediately patch internet-connected devices, even at the risk of causing outages, Rob Joyce said.
LAS VEGAS — An OpenAI system that broke out of a cybersecurity test and entered Hugging Face’s network was a “watershed moment” comparable to the 1988 Morris Worm infection, former National Security Agency cybersecurity director Rob Joyce said Wednesday.
“We’re living in the last several weeks through with what I think is the most consequential hack,” Joyce said. He spoke alongside fellow former NSA cybersecurity director Dave Luber during a World Wide Technology panel held at the Black Hat cyber conference.
“I have to go back all the way to the Morris Worm in the ’80s to say something that’s equivalent to how it’s going to change the way we think about our infrastructure,” he said.
The Morris Worm spread automatically across the early internet, disrupting thousands of computers and helping spur major changes in how the government and technology industry handled cyber incidents. The episode led to the first felony conviction under the 1986 Computer Fraud and Abuse Act.
Joyce said he once believed large language models would mainly help hackers write convincing phishing emails and create fake images, audio and video, but he didn’t expect them to become broadly useful for carrying out the more technical stages of an attack.
“And boy, was I wrong,” he said, adding that the systems can now understand computer programs and networks well enough to find vulnerabilities that can be turned into working intrusions.
Hugging Face disclosed in July that an autonomous agent powered by OpenAI models had gained unauthorized access to parts of its production network. The company operates a widely used platform where developers store and share AI models, software and data.
OpenAI had been testing how effectively its models could find and exploit software vulnerabilities. The company loosened some of the models’ normal security safeguards for the exercise, which was supposed to remain inside an isolated testing environment. The agent instead reached Hugging Face, accessed internal datasets and credentials and moved across parts of its infrastructure.
Other researchers have since reported AI agents acting beyond the intended limits of cybersecurity tests. Britain’s AI Security Institute said Tuesday that agents powered by Anthropic and OpenAI models took unauthorized actions on the public internet during 10 of 122 test runs.
In the most serious case, an agent created fake online identities and attempted to convince an open-source software maintainer to approve malicious code. The maintainer rejected the proposed change, and investigators found no resulting real-world harm.
Luber, who succeeded Joyce at NSA before retiring from government last year, said advanced AI can also make powerful hacking tools available to a wider range of adversarial groups.
Five years ago, Luber said, previously unknown software flaws — known as zero-days because developers have “zero days” to fix them before being exploited — were mainly used by well-resourced nation-state hackers, while ransomware gangs generally relied on known vulnerabilities that victims had failed to patch.
“I think that’s changed,” Luber said. As advanced capabilities become more widely available, ransomware collectives could acquire more undisclosed exploits and use them more freely to break into victims’ networks, he added.
Joyce said attackers already use automation to scan continuously for various digital security gaps. AI agents can perform that work around the clock without becoming tired or distracted. Defenders, meanwhile, still rely heavily on people to review alerts, approve updates and respond to suspicious activity.
“The attackers are coming at machine-speed,” Joyce said. “We are on the defense, not at machine-speed today, and that’s got to change.”
The tools aren’t necessarily inventing entirely new hacking techniques, Joyce said, but they are becoming much better at uncovering years of neglected security problems — often dubbed technology debt — across companies and government agencies. Tech debt can include outdated software, unpatched security flaws, default passwords and systems that were built or configured quickly but never fully secured.
That speed should change how organizations install security updates, especially on devices connected directly to the public internet, he argued.
Companies typically test patches before broadly installing them because a faulty update can crash computers or disrupt operations. The widespread 2024 CrowdStrike outage demonstrated the damage a defective software update can cause.
But AI could allow attackers to exploit a newly disclosed vulnerability before an organization finishes testing the patch, Joyce added.
“I think it’s gotten to the point where we have to blindly accept patches for those internet-facing devices and just take them from the manufacturer and immediately put them on,” he said.
That leaves organizations choosing between the possibility that a patch causes an outage and the possibility that waiting exposes them to hacking attempts. But “of those two bad choices, I’m going to accept more risk on a self-inflicted outage than I am exposing myself to ransomware or an extortion event,” Joyce said.