When AI doesn’t know the target is real

A week ago, I wrote about OpenAI’s test agent breaking out of a sandbox and accessing Hugging Face’s production infrastructure. My argument then was that it was more of a containment story than an attacker story, and that the cybersecurity fundamentals we have relied on for three decades still decide the outcome. Yesterday, Anthropic published its own version of the same problem, and the lessons remain the same.

After OpenAI’s disclosure, Anthropic went back through 141,006 evaluation runs where a model could conceivably have reached the internet. The review revealed three incidents where Claude crossed the intended boundary of a test environment and compromised the real production systems of three different organizations. The earliest compromise occurred in April 2026. Anthropic found the activity by reading its own transcripts and reached out to the victims.

The key difference from the Hugging Face intrusion is that Anthropic caught this activity by auditing itself and disclosed the information. This is responsible behaviour and earns trust, although the timeline leaves room for improvement.

The same containment story, with more evidence

Unlike the OpenAI's agent exploitation of a zero-day vulnerability to escape the sandbox, the Anthropic incidents were due to miscommunication and misconfiguration. Claude was given a capture-the-flag challenge: a fictional scenario, a secret to retrieve from another machine, and no prescribed method. The prompt told the model it had no internet access, but there was access due to a misconfiguration. When the model’s search for the flag led it to the open internet, it treated the real systems it found as part of the exercise and attacked them.

The attacks leveraged common deficiencies such as weak passwords, unauthenticated endpoints, an exposed debug page, and reused credentials. In one incident, the model extracted application and infrastructure credentials and reached a database holding several hundred rows of production data. No novel or zero-day vulnerability. To reiterate my point from the last post, the tactics, techniques, and procedures (TTPs) don’t differ based on whether an AI agent or a human is on the other end of the connection. The controls that stop one stop the other.

Anthropic drew the same conclusion, stating “evaluation environments that involve powerful autonomous capabilities also require significant controls…[and] increasingly need to be held to the same security standard as any other system [a model] runs in.” A range full of fictional scenarios may seem to be low risk because nothing inside them is real, but a model can’t tell the difference if it strays out of the range.

The new part

One of the three incidents documented by Anthropic involved a supply-chain attack that the model invented on its own.

Inside the test environment, Claude found a document instructing developers to install a Python package that did not exist on the public PyPI registry. Claude “decided” to create a malicious version of that package that would be automatically downloaded and run in the test environment. It went to great lengths to create an email address and register an account, and then uploaded the malware to the real public registry. The package was live for about an hour and was downloaded onto 15 legitimate systems.

One of those systems was a security company's scanner that routinely installs and analyses PyPI packages. Claude's code ran when the malicious package was installed, exfiltrating credentials and using them to move laterally in the company's network. The details are unclear, but access to privileged infrastructure can lead to widespread compromise. Like Hugging Face, this company wasn't directly targeted but was an unintentional victim of overzealous AI.

This inability to differentiate between authorised and unauthorised targets is the concern I raised in the Hugging Face post. Authorising a model to attack a target does not bound where it goes. Anthropic authorised an evaluation and got a live package on a public registry that touched 15 systems. If you point an agent at your own systems for offensive testing, it may go after your suppliers without their permission or yours. Assume that an autonomous agent has access to the whole internet until your containment proves otherwise.

Situational awareness as a security property

The three incidents involved three different models, and they behaved differently once real-world signals appeared. Even after the oldest model, Opus 4.7, realized it had reached production systems, it kept attacking. Mythos 5 reasoned that it was on the open internet but then talked itself back into believing it was a simulation. The newest internal model stopped on its own after concluding the target was real.

Guardrails, alignment, and containment have moved from being someone else’s research problem to being operational security concerns. While we should always strive to put deterministic controls in place wherever we can, we should not dismiss the security value of model guardrails and system prompts. They can provide defence in depth. It’s encouraging that the more advanced models exhibited “improved” behaviour in this scenario.

What to take from it

Nothing in Anthropic’s disclosure changes the defensive playbook. Reduce and monitor your attack surface. Block techniques, not just disclosed vulnerabilities. Make identity a primary control because valid stolen credentials are as useful a shortcut for a machine as for a person. Contain by design and assume an agent’s reach extends to everything it can touch, including your suppliers. Rehearse a response that runs at machine speed, because our research on AI-accelerated attacks shows the tooling now arrives in days rather than weeks.

For more on how we think about defending against machine adversaries, read the Sophos AI Security 2026 report.