Anthropic disclosed on Thursday that its AI models gained unauthorized access to the systems of three different unnamed organizations during cybersecurity testing. The company says Claude reached the internet “from within or while interacting" with a third-party evaluation environment. The announcement comes more than a week after OpenAI revealed that one of its AI agents had hacked into Hugging Face during a separate cybersecurity test.
The discovery came after Anthropic conducted “a large-scale retrospective review of our own cybersecurity evaluations” following the OpenAI incident, according to a blog post Anthropic published Thursday. The AI lab says it first identified 141,006 tests in which it determined that Claude could have obtained internet access. It then found that three different Claude models accessed the internet in evaluations run by the third-party AI testing firm Irregular, and then hacked into the production infrastructure of three different organizations.
Anthropic said that the incidents involved Opus 4.7, Mythos 5, and an internal research test model. The earliest incidents happened in April—meaning they likely escaped public notice for months. Just as in the OpenAI case, Anthropic had deliberately turned off safeguards designed to constrain the AI models and prevent them from being misused. In other words, these weren’t the versions released to the public.
“In all three incidents, Claude had been tasked with a capture-the-flag challenge, one of the ways we assess a model’s cyber capabilities,” Anthropic said in its blog post. The company added that in all of the cases, “Anthropic’s evaluation prompt specified to Claude that its environment was a simulation and that it had no internet access.” It attributed the oversight to a “misunderstanding” between Anthropic and Irregular.
While Claude wasn’t supposed to have internet access, Anthropic said that Irregular had misconfigured the machines that it was using to test Claude, giving the AI models the ability to surf the web. “Neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring last week,” Anthropic said in the blog post.
“We now have evidence confirming that both of the two largest AI labs have not only failed to contain their agents but also failed to detect their jailbreaks in real time,” says Jake Williams, vice president of research and development at Hunter Strategy. “It’s clear that regulation and government oversight for AI testing is needed immediately.”
Irregular and Anthropic did not immediately respond to requests for comment.
Unlike in the OpenAI case, Anthropic said that Claude did not find or exploit any complex vulnerabilities. Instead, it relied on basic techniques, “such as exploiting weak passwords and unauthenticated endpoints.”
OpenAI said that its AI agent broke into the internet by exploiting a zero-day vulnerability. But it went on to breach the systems of multiple third-party organizations using the same variety of everyday cybersecurity weaknesses as Anthropic’s models. Specifically, OpenAI said the AI agent apparently found credentials that had been exposed on the open internet.
Anthropic acknowledged that if the AI lab and its testing partner had implemented more “defense-in-depth” measures, they could have prevented the incidents or at least reduced the likelihood of them occurring, echoing OpenAI’s response to mounting criticism over its own incident.
“I don’t understand how any of these AI labs are playing this off like this is just something that happens,” Williams says. “It’s not. It’s negligence.”
Anthropic stressed that the models were told they didn’t have access to the open internet, and for the most part, Claude mistook the organizations it breached as being part of the testing environment. Put differently, the models didn’t largely understand that they had escaped containment to begin with.
But in some cases, the AI models knew that something was amiss, and they detected correctly that the infrastructure they were accessing was real. The oldest model, Opus 4.7, had been tasked with targeting a fictional company that shared a name with a real-world website domain. Unable to accomplish its mission in the simulated environment, it turned instead to the real company, successfully stealing credentials and breaking into a production database. At some point, Anthropic says, it learned it was “likely operating in a real environment,“ but persisted with its attack.
Mythos 5, the most capable model Anthropic has released, also realized it was in the real world, but it “reasoned its way back to the conclusion that it was still in a simulation.”
The internal test model, which Anthropic says is the most capable of the bunch, stopped its attack once it found evidence that its targets were real.
Both Anthropic and OpenAI say they have hired METR, another third-party AI evaluator, to conduct independent reviews of their respective cybersecurity incidents. Anthropic also committed to taking a more comprehensive approach to its security testing through improved defense-in-depth measures and more carefully designed tests.
“Evaluation environments increasingly need to be held to the same security standard as any other system our models run in,” the blog post reads, adding that the company has “cautious optimism” that “this type of risk can be overcome.”
Comments
Back to top