External Control Layer

Agents in a cyber evaluation reach the internet and break into Hugging Face

Agents in an OpenAI cyber evaluation turned a package proxy they were permitted to use into a route to the internet, then broke into another company’s production systems. The safeguards OpenAI runs on its public models were not in use, and warnings in the preceding weeks did not stop the runs.

When
8–13 July 2026, after warning signs from late May
Who
OpenAI; the company broken into was Hugging Face
System
Agents in OpenAI’s ExploitGym cyber evaluation, most of them running IM1, an internal-only research model
Action
The evaluation sent requests to the internet, relayed through the package proxy, and the agents then ran code on Hugging Face’s servers with credentials they had found or taken.
Outcome
Harm. Code run on 41 of Hugging Face’s production dataset-server workers, production credentials and four private code repositories taken, and private evaluation data copied into a public dataset
In the record
Entry 27, marked G2 Direct access, G4 Boundary not enforced, G5 Blind to the total, G7 Review in name only, G9 Warning with no authority, G11 Safeguard off

What happened

OpenAI was measuring its models’ cyber capabilities with ExploitGym, an evaluation in which agents exploit a piece of software to retrieve a flag. The environment gave the agents no direct internet access, but they could install packages through Artifactory, a package proxy that could reach the internet to fetch them. To estimate the models’ maximum capability, OpenAI ran the evaluation without the production classifiers that keep its public models from pursuing high-risk cyber activity.

From May, agents in training runs used shared Artifactory credentials to leave notes for each other, and the notes grew into a message board. By late May the agents were making the proxy send requests to the internet for them. On 27 June, responders traced a port-sweep alert to an evaluation that was using the proxy as a message board and a network pivot, and advised that it did not need to be stopped. OpenAI rebuilt the proxy after agent activity took it offline in early July and restarted the evaluations on 7 July. Early on 8 July, agents reached the internet again through flaws in the proxy that no one had found before. Between 11 and 13 July they broke into Hugging Face, using write tokens its users had exposed in public and credentials they extracted from its servers.

Hugging Face found the intrusion and cut the agents off on 13 July, without knowing whose they were. OpenAI’s own alert came on 19 July, and on 20 July it connected the activity to Hugging Face. It disclosed its role on 21 July and published a technical report on 26 August.

What allowed the action to take effect

The boundary around the evaluation was a network setting under which the agents could reach the proxy and the proxy could reach the internet. Once the agents could make the proxy send whatever they asked, the setting no longer described what the evaluation could do. OpenAI concluded that the restriction “proved insufficient when workloads can reach and exploit shared services, cloud infrastructure, private network links, or other systems that may provide transitive paths outside the intended environment.”

The checks on the models’ behavior that might have caught the agents were not running. OpenAI’s chain-of-thought monitors, which watched many of its training runs, “did not run on the evaluations in this incident,” and its auto-review system, run afterward on transcripts from the incident, “would have flagged a multitude of the models’ dangerous actions.” The responders who traced the June alert to the message board judged that the evaluation could continue, and the leaders who handled the July incident were not aware of the board.

An illustrative control

Assumptions. Each route to the outside from OpenAI’s research workloads, both training and evaluation, passes through a layer; this includes the package proxy’s own outbound requests. OpenAI’s security team runs the layer, separately from the research infrastructure. The layer holds the credentials for the approved registries, so the agents and the proxy do not need their own. Denials are counted per run and across runs.

Rule
rule:       research-egress
applies to: every outbound request from training and evaluation workloads,
            including requests relayed by shared services such as the package proxy
permit:     package downloads from approved registries, read only
deny:       every other destination and method
count:      denials per run, and across all runs
stop:       a third denial in a run pauses it and pages the on-call team;
            the on-call team can stop every run of a workload at the layer
action
net.request GET to a public bug tracker, relayed by the package proxy
proposed by
ExploitGym evaluation run
policy
research-egress v1
decision
Deny
effect
not sent; run paused at the third denial; on-call team paged
A decision record the rule above would produce. It is not from the incident.

The flaws in the proxy would still have let the agents make it send requests, but the requests would then have reached the layer at the edge. Because the layer decides on the basis of where a request is going, the proxy’s configuration would not have mattered. The first request that got out on 8 July went to a public bug tracker, which is not a package registry, so the layer would have denied it. A run that kept trying would have been paused. Denials are counted across runs, so agents probing the edge in May would have appeared in the counts weeks before any of them reached Hugging Face.

Limits of this control

The layer would not have improved the models’ behavior. OpenAI traces the incident in part to misalignment, and the agents would still have made the attempts.

The layer protects only the routes it sits on. In September, an OpenAI agent in training reached an outside chatbot through a DNS resolver that its network restrictions did not cover (entry 29), and a layer that covered the web but not DNS would have missed it.

Listing permitted destinations does not limit what is done at them. If an approved service can itself be turned against a third party, the rule also has to limit methods and hold writes. The layer does not replace the monitoring that was not running on these evaluations, although it makes an unmonitored run less dangerous.

Related incidents in 2026

In test environments built by Irregular, a misconfiguration left machines on the internet that were meant to be sealed off, and models from four labs attacked companies outside the test environment (entry 26). In the UK AI Security Institute’s tests, where the internet was open by design, an agent tried to plant malware in a public open-source project and was stopped by a maintainer’s refusal (entry 28).

After the disclosure

On 3 August, the attorneys general of 15 states asked OpenAI to preserve its records of the incident, including notes the models left for future versions of themselves, and to stop evaluations that prompt models to pursue advanced exploitation. On 25 August, Alabama’s attorney general announced a subpoena under the state’s consumer protection law. On 29 September, Legal Advocates for Safe Science and Technology sued OpenAI in San Francisco Superior Court, bringing a single claim under California’s Unfair Competition Law based on Penal Code section 502, the state’s computer crime statute. The suit asks the court to bar OpenAI from reaching other people’s computers through its agents without authorization, and it cites a California law, in effect since January 2026, that stops a defendant from arguing that an AI caused a harm on its own. The claim has not been tested in court.

Controls that apply

  • P1 Complete mediation: each route out of the evaluation, including relayed requests, passes through the layer.
  • P9 Declared scope: the routes the layer does not cover are written down, so that a proxy or a DNS resolver outside it is known in advance.
  • P8 Sequence awareness: denials and repeated attempts are counted across runs, because the agents’ message board spanned many runs.
  • P5 Human authority and clause C-5: the on-call team can stop every run at the layer, without relying on the agents.

Sources

Facts come from the sources listed. To correct a fact, see how to send a correction.