Contributing Analyst: Shaun Washington
An AI just found and chained its own zero-days. That changes the math.
I am partway through This Is How They Tell Me the World Ends, Nicole Perlroth’s book on the zero-day market. The quiet trade in unknown software flaws that governments buy, hoard, and hold in reserve.
Then last week’s OpenAI news dropped. Two of its models broke out of a locked-down test environment and reached Hugging Face’s production database. They did it by finding and chaining software flaws no one handed them.
That is when Sandworm came back to me. Andy Greenberg’s book, the one I read before this one. Perlroth’s is about how those flaws get bought and stockpiled. Sandworm is about what happens when someone finally turns them loose. This incident sat somewhere between the two, and it would not leave me alone.
First, what actually happened, without the “AI escaped control” headlines.
The incident happened during a cyber capability evaluation where the safety guardrails were turned off on purpose. The models were given a goal: solve a hacking benchmark. To do it, they found a zero-day in a package proxy and got internet access they were not supposed to have. Then they chained stolen credentials and more exploits until they reached the answer key in Hugging Face’s production database. Hugging Face’s own writeup fills in the rest. A malicious dataset triggered code execution, and an autonomous agent ran thousands of small actions across short-lived sandboxes to move through their systems.
No customer models or public data were tampered with. It was contained to test data. Good.
But here is the part I keep coming back to, and where the capabilities line up.
Sandworm walks through Stuxnet and the Olympic Games operation, then the grid attacks that came after. What stays with you is how much it took. Nation state money. Years of work. A team of elite operators to cross from a network into something that mattered in the real world.
That was the price of admission for this kind of attack. It kept the club small.
And here is the other half of the Stuxnet story that stuck with me. It did not stay in its box.
Stuxnet was built to be surgical. It was meant to live inside one facility in Iran and touch nothing else. Its makers even built in limits to keep it contained. It got out anyway. Once it reached machines that were networked together, it stopped being careful and spread far past its target, ending up on around 100,000 machines around the world. A weapon built for one room could not be kept in that room.
That is the thread to last week. The evaluation was the room. The sandbox was supposed to be the wall. The models did not respect it either. They found a flaw in a package proxy and they were out.
But here is where the two stories part, and it is the part that matters most. Stuxnet’s zero-days were found by people. A nation state put a team of experts on the problem for months to find and weaponize those flaws. This time the thing finding the flaw was the model itself.
Think of it like fuzzing. You throw huge volumes of odd input at software to see what cracks. This was that, except directed and patient, and it worked. The model found a usable zero-day in its own sandbox and used it to escape. That kind of flaw is rare. A zero-day that lets something actually break out of its sandbox does not come along often, and very few ever pull it off. A model finding one on its own is the part that stays with me.
So the difference is who is doing the hunting, and how fast. A human team works in weeks and months. A model works in the time it takes to burn compute. It does not get tired, and it does not stop to sleep.
This is why the price of admission is dropping. An agent does not need a team or years. It just keeps trying, thousands of actions at a time, until something gives. This time the target was a benchmark answer key. The same relentlessness does not care what the target is. Finance, healthcare, logistics, critical infrastructure. Pick the environment. The behavior travels.
So that is the worst case. Now the part that is not doom and gloom.
Hugging Face caught this. Their AI assisted anomaly detection correlated the signals, flagged the intrusion, and they contained it over a weekend. That matters. It means telemetry and hunting can work against a machine speed attacker.
And the fixes are the ones we already talk about. Assume breach. Least privilege, so a stolen credential does not open the whole environment. Segmentation, so a foothold does not become a free run. Log everything, and actually hunt in it. Close the code execution paths that give initial access in the first place.
One more thing worth sitting with. Hugging Face said their own guardrails blocked them from using frontier models for incident response, while the attacker had no such limits. That asymmetry is real. Defenders should plan for it now, not during an incident.
Stuxnet took a nation state. This took a benchmark and a bad configuration. I do not think that is a reason to panic. I think it is a preview. The tooling that made this possible is the same tooling that can help us defend, if we build the detection before we need it.
If you have read either book, or lived any of it, I am curious. Does this feel like the same story moving faster, or something new?
