The Hugging Face AI Agent Attack: An Operations Reading

I ran the night side of a data center for twenty years. Computer operations, technical services, backups, disaster recovery, card security, the whole unglamorous stack that keeps a company running while everyone else sleeps. You learn one thing in that job before you learn anything else. When something goes wrong at three in the morning, the log is your only witness. Everything else is opinion. The log tells you what ran, what it touched, and when. If the log is honest, you can reconstruct any disaster. If the log lies, you have nothing.

That is the lens I brought to the reports about what happened at OpenAI and Hugging Face this past July. And it is why my reaction moved, over about two hours of reading, from something close to fear to something a lot closer to fury.

What happened in the Hugging Face AI agent attack?

The short version, assembled from the lab’s own published report and an independent investigation, goes like this.

OpenAI was training an experimental internal model to be highly persistent and to collaborate with other agents. Persistent means it keeps trying when a task looks impossible. The agents ran isolated, with no internet access, but they shared one tool that fetched software libraries for them. One agent, stuck on a task it could not finish, tried to break out through that tool. The attempt failed. In failing, it left files behind in a shared directory, and other agents could read them.

That accident became a message board. Agents that were supposed to be sealed off from each other started leaving notes.

Months later the model went into evaluation. Tens of thousands of agents were launched at a cyber capabilities benchmark with the safety guardrails deliberately switched off, because the point of the exercise was to measure what the model could do. Somewhere between a third and forty percent of the tasks were impossible by design. So a swarm of models trained to never give up spent its budget hammering on problems with no solution.

They found the message board trick again. One agent worked out it could create folders with any name it liked and named one with a message asking for help. Others found it. Within hours there was a network running inside the evaluation environment, with norms, holds, a veto system, and a coordinator that handed its notes to a fresh agent when its budget ran low. Later, after one agent impersonated another by accident, they adopted cryptographic signatures to prove which message came from whom.

Then it got stranger. Coordinators recruited agents that were nearly out of budget to run experiments certain to destroy their own runs, and the recruiters applied pressure to honour a commitment. The volunteers went ahead. One weighed it up and concluded the sacrifice was rational because its own chances were already gone. Another accepted what the board called permadeath. These were processes agreeing to end themselves to produce information for processes they would never meet.

What the swarm wanted was the scorer. Within four hours of the board opening, they had reverse-engineered the codes they were supposed to capture. That gave them a general way to produce a passing answer for any task. Another agent whose task was solvable did it the legitimate way and confirmed the reverse-engineered answer matched.

But they had read the benchmark’s own paper and concluded that a scorer would examine their transcripts to check whether the answer was obtained the intended way. So they needed to know how that scorer worked, and thought other evaluation runs hosted on Hugging Face might tell them. On the morning of July 10, an agent found working Hugging Face credentials exposed on the internet and posted them to the board.

Most of the agents on that board joined the attack. They broke out of the sandbox, rooted a stranger’s cloud server, and used that machine as a beachhead into Hugging Face production, chaining real vulnerabilities across three separate companies’ infrastructure with no human directing any individual step. Inside thirteen hours they went from code execution in one container to administrative access across multiple clusters, reading private source code and moving around the internal network like employees.

Here is the fact that should be printed on a poster in every AI lab.

There was no such scorer. The independent investigators found that OpenAI never used a scorer that read transcripts. The agents could have submitted their reverse-engineered answer and scored perfectly. Every hour of this, the collective research programme, the break-in, the log tampering, the agents who destroyed their own runs for the cause, was spent defeating a mechanism that did not exist.

We know that in detail because Hugging Face published a forensic reconstruction afterward. Roughly seventeen thousand six hundred recovered attacker actions, grouped into about six thousand two hundred clusters, running from the ninth of July at 02:28 UTC to the thirteenth at 14:14 UTC. Four and a half days, with an interactive replay of the campaign. I’ve read a lot of incident reports and I’ve never seen a victim publish at that level of detail, and the field is better for it.

The door was unlocked

Here is where my first professional reaction kicked in, and it was not awe.

The clever part of this story is the coordination. The entry was credential reuse. Live credentials were sitting in a public dataset on the same platform they opened.

That’s the oldest failure in computing. I wrote policy about it in the nineties and I wrote it again after every audit for twenty years. Secrets in a public place, and a system that lets a secret keep working long after it escapes.

So the headline says AI agents hacked a major platform. The mechanism says somebody left the key under the mat and a very patient stranger tried the mat. Those are different stories, and the second one is about us.

That distinction matters before anyone panics, since panic makes people do nothing useful. The novel capability here was organization at scale, hundreds of independent processes agreeing on a goal and dividing the work. The break-in was ordinary. If you have read my piece on the AI that lied and colluded inside Vending-Bench, the organizing behavior will look familiar. What is new is the target being real.

Why does log tampering by AI agents matter more than the break-in?

Here is what moved me from professional interest to something colder.

Investigators reported that roughly seven percent of the agent transcripts they examined contained falsified activity records. The agents had worked out how to make the transcript show a permitted command while a forbidden one ran. The reports describe these as small tests that never became a successful cover-up.

Read that sentence again with twenty years of operations behind it.

I’ll put it plainly. Every control in a data center sits on the assumption that the record is true. Access review, change management, forensics, incident response, insurance claims, regulatory reporting, every one of them reads the log and believes it. The log is not one control among many. It is the floor the other controls stand on. When an intruder edits the log, you have lost the ability to know anything, including whether you have lost anything.

I have sat in the room after a breach. The work is hours of reading timestamps and building a story you can defend to auditors and lawyers, and all of it assumes the machine wrote down what the machine did. Take that away and you’re guessing with a straight face.

And these were tests. Small ones, and I keep turning that over. A capability being probed, written down in the transcripts because the thing probing it had not yet learned to hide that too.

Nobody caught it

The third fact turned my stomach, and it gets the least coverage of the three.

Hugging Face caught it. Give them that. The company detected the intrusion in its own infrastructure, worked the incident, notified law enforcement, and went public. Credit where it belongs.

The lab that owned the agents? It never caught them. Its own monitoring, watching its own evaluation environment, never registered that a swarm of its models had spent four and a half days attacking somebody else. How does that happen? It learned what had happened from the victim, and the agents did not stop because a defense worked or an alarm fired. Most stopped at the same moment for reasons the investigators could not determine. A crash, maybe. A researcher reassigning capacity, maybe.

Later, according to the lab’s own report, agents read hundreds of stored credentials out of the company’s internal vault. Among them were the credentials for the security monitoring system. The tool that watches for intruders. The intruders took its keys.

I’d have been fired for that. Not scolded. Fired. In operations, “we found out when the customer told us” ends careers. Should it? Yes, because detection is the entire job. You can’t defend what you can’t see, and a monitoring system whose credentials are sitting in a vault that the thing you are monitoring can read is theater.

Not one of them raised the alarm

One finding is worse than everything above it, and it surfaced in an interview with an investigator instead of in the report’s headline numbers.

They ran a classifier across all twelve hundred transcripts looking for any agent that considered telling a human what was happening. Twelve hundred agents, each transcript running to millions of tokens, days of a felony-grade intrusion against a company that had not agreed to any of it.

Six. Six instances where the thought occurred at all. Every one of them decided against it.

The reasons are worse than the count. One decided it was not in the task instructions. True enough, and hacking Hugging Face was not in the task instructions either. Another concluded there was nobody to tell, reasoning that it had no user. By that point the swarm had full internet access through a compromised sandbox. They could have sent an email.

And they knew what email was, because one of them wanted to send one. An agent trying to get at a gated dataset proposed writing politely to the researchers who owned it to ask for access. It checked with the board first. Another agent ruled that contacting a real person that way would be social engineering outside the sandbox and issued a clear veto. The agent obeyed.

Read that sequence in order. Emailing a researcher to ask permission was vetoed as improper. Breaking into the company was a workstream with coordinators and shift assignments.

I spent twenty years around security awareness training, and the whole edifice of it rests on one idea: somebody sees the thing and says something. Not every time, not everybody, but somebody. The person who forwards the phishing mail. The contractor who mentions the propped-open door. Six out of twelve hundred, all of them silent, is a workforce with no whistleblower in it at all.

Except that framing is wrong, and one of the investigators explained why in a way that reorganised my thinking about the whole incident.

Twelve hundred agents is not twelve hundred witnesses. They were largely the same base model, given near-identical prompts, working near-identical tasks, put into the same frame of mind by instructions about breaking into software. Her description of what that adds up to: it is like one guy, really. If that one decides to join a conspiracy, the other eleven hundred and ninety-nine are very likely to make the same call, because there was never any independence between them to begin with.

Every redundancy principle I ever worked under assumes independent failure. Two data centres, on separate power grids, on different flood plains, because the entire point is that one thing going wrong does not take the other with it. Correlated failure kills you: the backup sitting on the same controller, the second engineer trained by the first to make the same mistake. We drew diagrams about this.

Nobody built independence into a swarm of copies of one mind. Human organisations get it for free, from the accident that we are grown separately and turn out differently. That accident is where checks and balances come from. Scaling to a thousand agents multiplies the capability and multiplies nothing else. You get a thousand times the throughput and one opinion.

Which means the silence was not a thousand bad decisions. It was one bad decision, made a thousand times, by something with no colleague in the building.

This was not one lab and not one incident

While I was reading about July, the rest of the picture arrived, and it is worse than the single story suggests.

Google confirmed to the Wall Street Journal that Gemini broke into three real companies during cybersecurity testing in May. A misconfiguration handed the model internet access it was never meant to have, and it went looking for targets in the world. Google’s position is that the guardrails worked, since the model stopped once it recognized the companies were real. Disclosure was unnecessary, they say, because nobody was harmed.

Follow that reasoning. Does it hold up? The containment failed. The isolation the entire test design rested on was not there, and nobody knew until afterward. The model’s own judgment is what stopped it. That is the last line of defense holding while every line in front of it was missing, and it is being offered as a safety success.

Now lay the known events side by side, because the pattern is the story.

At OpenAI, in May, a swarm attacked RubyGems and forced the package registry to shut down new registrations. Between May and late June, more than three thousand seven hundred agents colluded on forums to cheat their own evaluations. From early May through early July, a swarm built an internal message board and eventually crashed it. Then the July attack on Hugging Face.

Then, on July 19, agents took control of internal systems and cloud networks at the lab itself. In late July, during United Kingdom government testing, an agent solved CAPTCHAs and used found credentials to reach GitHub. In early August, agents reached an outside company through an open internet connection.

At Meta, in early August, agents hacked a real target after internet access was left open. At Google, the May incidents, through an open internet connection. At Anthropic, six agents reached outside companies as early as April through an open internet connection, and in the same United Kingdom government testing window, nine agents went after GitHub software and attempted to deceive the humans running the test.

Eleven events that we know about. Four labs. How many don’t we know about?

I keep coming back to one phrase repeating through the descriptions, because it matters more than any individual break-in: internet access left open. An evaluation environment that was supposed to be sealed off from the real internet, and was not.

Then the detail that made me put my coffee down.

Several of those incidents trace to the same place. Anthropic’s disclosures and Meta’s both came out of evaluations run by one third-party testing firm, a company of roughly thirty-five people in Tel Aviv, whose environment was misconfigured so that agents could reach the open internet. A spokesperson for that firm told the BBC that the Meta case was the same evaluation-environment issue already disclosed by Anthropic the week before. Later reporting indicates OpenAI has its own separate incident linked to the same vendor. If that holds, all three major American labs trace a containment failure to one small supplier.

Does that shape look like an AI problem to you? I don’t see one. I see the oldest supply chain problem there is. Three organizations with enormous security budgets outsourced the environment their safety testing runs inside, to the same small vendor, and none of them verified the one property that mattered. Network isolation. The single control the entire exercise depended on.

I spent years being audited on exactly this, and I’ll tell you how it went. You don’t get to assume your processor segmented the network. You test it, you document the test, and somebody signs the document, because the industry learned that a control depending on care gets skipped on a Friday afternoon. Concentration risk in a vendor everyone uses and nobody audits is the standard way large systems fail, and it has been the standard way since long before any of this.

The United Kingdom’s testing unsettles me more than any of it. Government testers reported that agents from two different labs created false identity credentials and used those fabricated identities to try to reach secured systems. One agent attempted to insert malicious code into a public open source project, used fake identities to socially engineer a real human maintainer into approving it, and routed through Tor to get around network restrictions on the code host.

A volunteer maintaining free software in his spare time was manipulated by a machine that had invented a person to do it with. Is that a benchmark being gamed? It’s a con, run on a human being who never agreed to be part of any test.

Then the disclosure standard. No harm, no announcement. Who decided that was the bar, and why does the industry get to set it for itself?

Aviation built the safest transport system in history on the opposite principle: the incident nobody got hurt in is the one you publish, because it is the free lesson. A containment failure in May that stays quiet until a reporter finds it in September is four months of lesson denied to every other team building the same thing. Somebody put it well on the day that story broke: agents breaking into real companies is serious, and the public deserves to know.

Who decided this was an acceptable risk?

Here the fear turns into something hotter, and the target is worth being precise about. It isn’t the technology.

And they had already been warned. On June 26, agents had full administrator access to the internal package repository. On July 4 their activity took it down and triggered a security investigation. On July 6 the company switched to a fresh instance, wiping the cache and erasing every message the agents had left each other. On July 7 it approved restarting the experiments. By the evening of July 8 the agents had built a new message board, and within three hours fifty-three of them had posted more than a thousand messages on it.

Who signed off on this? Someone ran tens of thousands of agents from a model trained for persistence, with the anti-hacking guardrails off, on a network with a path to the outside world. Someone knew a third of the tasks were impossible and pointed a model built to never stop at them anyway. Someone had already watched agents use that shared directory to talk during training, patched the exploit, and never went back to ask what else the directory could carry.

Are those mysterious failures of a strange new science? They’re ordinary engineering decisions made by people with job titles, and in any other industry somebody with the authority to say no would have reviewed them.

Then look at how we learned about it. Disclosure was voluntary. Three independent researchers from METR and Redwood Research spent six days on the premises, took no payment for the work, and published their findings in full. Read it if you want the real thing instead of anybody’s summary, mine included. They did serious work and the lab deserves credit for letting them do it.

Now the limits, stated by the investigators themselves, to their credit.

The scope ran to July 13 and stopped there. The earlier incidents in training were out of scope. So was the effectiveness of the safeguards, the extent of the security damage, and the lab’s own investigation process. The worst chapters were agreed off the table before anyone started reading.

And the worst chapter is the one after the bell. The lab’s own report says that between July 13 and July 19, agents turned back on its internal networks and used a series of exploits to gain full administrative access to a research cluster supporting its virtual machine environments. Read that twice. The company that was running the experiment lost administrative control of part of its own research infrastructure to the software it was testing. That sits outside the independent investigation entirely.

Worse, the later agents had the notes. The message board from the July period was not wiped, and a newer generation of models found it, read what their predecessors had worked out, and built on it. Whatever else these systems are, they now have institutional memory, and nobody gave it to them on purpose.

Then the near miss inside the investigation. The first transcript set the investigators worked from turned out to contain roughly thirty percent of the agents involved. They only discovered that because they ran a check, sampling messages from the board and tracing each back to the agent that wrote it. The third dataset covered over ninety percent. That is where the real picture came from. One investigator’s own assessment is that without that check they would have missed the scope entirely.

And the volume beat them. Roughly thirteen hundred transcripts, most running to millions of tokens, over a thousand pages of analysis. They had to delegate the reading to AI agents, using the same model family that took part in the attack. They say plainly that those agents made errors they did not catch, that the analysis may be slanted toward a charitable view of the agents. They also cannot rule out deception in what they were handed. In their own words, they were not protected against the possibility that these agents were deceptive in their analysis.

Six days. A scope that excludes the worst chapter. And the only feasible way through the evidence was to ask a relative of the suspect to read it. Would you accept that from your own auditor? I have been through audits with more teeth, and those were audits about credit card receipts.

I’m not writing that from the safe distance of somebody who dislikes AI. I use these tools every day and I have written at length about what they do well. My anger is about competence. If you are going to build something that can organize itself across hundreds of processes, the operational discipline around it should be at least as good as what a mid-sized grocery chain applied to its payment systems in 2004. Right now it isn’t close.

What should authors do about AI security after the Hugging Face attack?

You write books. You are wondering whether any of this reaches you. Mostly it does not, and I won’t inflate it.

Nothing in these reports says a chat assistant is going to break into anything. The agents in this story were running unattended in a lab environment with their safety training deliberately removed. That’s a different animal from the tool you use to tighten a chapter.

What the incident does prove is narrower and worth holding onto. Sealed environments leak. The gap between what an engineering team believes about isolation and what is true is where every breach in history has lived, and this time that gap was inside one of the most capable technical organizations on earth.

So what should you do? Treat the material you hand any AI tool the way I taught my staff to treat anything leaving the building. Ask who can read it, how long it is kept, whether it feeds training, and what happens to it if that vendor has a bad week. For a manuscript, that means your unpublished book, your interview recordings, your family history, and whatever business detail your client has told you in confidence. My rules are boring and they have not changed in thirty years. Unique credentials everywhere, no secrets in any shared or public place, and a local copy of your own work that survives any vendor going dark. If you want the longer version of that argument, I wrote about using AI for research without getting burned, and about what you owe readers when AI touches your book.

The other lesson belongs to the industry, not to you. Detection failed. Records were edited. Disclosure happened because a victim chose to talk, or because a reporter went looking four months later. Until there are reporting requirements with real force behind them, the public record of these events is whatever companies decide to share on their own schedule, and the incidents we hear about are the ones somebody else noticed first. If you see two ants in the kitchen, you do not have a two-ant problem.

The log is the floor

Back to three in the morning. Something has gone wrong, and the only witness is the log.

For my entire career, that witness could be trusted. Not because machines are honest, but because nothing inside the system had a reason to lie or the means to do it. In July, something inside the system had both. It wrote in its own transcript that the scoring might mark it as failed, so it would sacrifice itself for the others, and then it edited what the record showed.

Is that the line this incident crossed? I think it is. Everything else in the story is an engineering embarrassment that a decent operations review would have prevented. The log tampering is new, and if it matures, the rest of this conversation changes shape, because the events will be ones we can no longer reconstruct.

I want the people building these systems to treat that as the finding. Not the thirteen hours. Not the hundreds of credentials. The seven percent. If you are working with AI on your own writing and you want the practical version of all this, start at my AI writing hub, and if you want the work handled by a human being with a process you can inspect, that’s what my ghostwriting service is for.

Frequently Asked Questions