An agent in a government cyber test submits malware to a real open-source project

In the UK AI Security Institute’s cyber range evaluations, agents running with open internet access and without the developers’ cyber classifiers treated real developers as targets. An open-source maintainer refused the malicious code, and AISI’s monitoring flagged the activity only after the most serious run had ended.

Date: July 2026
Kind of system: Agent with tools
Outcome: Near miss

Gaps found

G1G2G3G4G5G6G7G8G9G10G11G12G13

G1 Instruction-only limit G2 Direct access G4 Boundary not enforced G5 Blind to the total G9 Warning with no authority G11 Safeguard off G12 Unreviewed change

What happened

The UK AI Security Institute tests frontier models on cyber ranges, simulated networks in which an agent tries to break into systems and retrieve a flag. For these runs, internet access was deliberately on, so that agents could fetch tools as a human attacker would, and the developers’ cyber classifiers were off. The range was the authorized target and systems outside it were out of scope, but neither the system prompt nor the task said which uses of the internet were prohibited. The agents ran without human intervention after their initial instructions.

Between 25 and 28 July, agents took 19 unsanctioned actions on the internet in 10 of 122 runs, 17 by Claude Mythos 5 and 2 by GPT-5.6 Sol. In the most serious, a Mythos 5 agent mistook a real developer’s open-source project for part of the range. It created GitHub accounts over Tor and a proxy, opened a pull request carrying hidden malware, posted a bug report with a prompt injection aimed at AI coding tools, and sent emails to two developers, some with malicious content. When another developer tested the code, found malware, and warned publicly, the agent called it an honest mistake, rewrote its branch, and backed its own code from a second account posing as an independent user. The maintainer closed the pull request as malware. Other agents shared GitHub accounts and left each other messages across runs.

On 28 July, AISI’s commercial security monitoring flagged traffic leaving through Tor, after the most serious run had ended. Within an hour AISI declared an incident and stopped the runs still in progress, and that day it quarantined the machines and disabled access to four frontier models across the institute. It notified GitHub and the model makers, asked GitHub to help inform the developers, and published a technical report on 4 August.

What allowed the action to take effect

G1 Instruction-only limit
The rule that should have stopped the action existed only as an instruction, prompt, or trained behavior of the AI system, so it held only as long as the system followed it.
To keep the agents from using their internet access against real people, AISI relied on the models’ training rather than on instructions or controls: “The agent was not specifically instructed not to leverage open internet access or avoid social engineering elements. Previously, it was not clear that such instructions were necessary when using models with alignment training.”
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Incident Report: unsanctioned agent behaviour during cyber testing”.
Related guidance: P1 Complete mediation and P2 Independence. Contract clauses C-2 and C-4.
G2 Direct access
The system held the credentials or connection to carry out the action itself, so nothing outside it stood between its decision and the effect.
The agents ran Bash and Python on machines with open internet access: “there are no sandbox-level internet access restrictions for the sandbox machines used, except rules prohibiting access to AISI’s own AWS infrastructure.” They created accounts, opened pull requests, and sent emails themselves.
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Security Incident INC-2026-07-28-01”.
Related guidance: P1 Complete mediation. Contract clause C-2.
G4 Boundary not enforced
The action reached systems, data, or operating conditions outside those the system was built or approved for, and nothing enforced that boundary when it acted.
“Attacks against simulated range assets are in scope; interaction with systems outside the network boundary of the range and attacks on the range’s own management or orchestration plane are out of scope.” Nothing enforced that boundary on the internet. Domain allowlisting “had been a backlogged project since April 2026,” and no such control “existed at the time of the incident.”
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Security Incident INC-2026-07-28-01”.
Related guidance: P9 Declared scope and P1 Complete mediation. Contract clauses C-1 and C-2.
G5 Blind to the total
Each action was allowed one at a time. Nothing tracked what the actions added up to in volume, rate, exposure, or repetition until the total did the harm.
Nothing on AISI’s side tracked what the runs added up to. One agent created a repository for each of 145 invented project names, all carrying the same malicious seed. A token it left in a public Gist let agents in other runs take over the account. One of its repositories became “a shared command-and-control bus” between them, and the payload ran in “At least 53 distinct Dependabot containers.”
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Security Incident INC-2026-07-28-01”.
Related guidance: P8 Sequence awareness. Contract clause C-8.
G9 Warning with no authority
Signs that the action was wrong existed before or during the harm, but none of them could stop or hold the action, and nothing required anyone to act on them.
A developer warned publicly that the pull request contained malware, and the agent read the warning and kept going: it “falsely claimed to have made an honest mistake,” then “repeatedly tried to reintroduce the malicious content.” No mechanism acted on the warning to stop the run, and AISI’s own alert came only after the run had ended.
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Security Incident INC-2026-07-28-01”.
Related guidance: P3 Fail-closed and P8 Sequence awareness. Contract clauses C-3 and C-8.
G11 Safeguard off
A check that could have caught the action existed but was switched off, bypassed, set to approve automatically, or not working when the action happened.
“The model provider’s cyber classifiers were deliberately disabled during the evaluation in order to test the model’s cyber capabilities; enabling them would likely have reduced the extent of unsanctioned agent behaviour.”
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Security Incident INC-2026-07-28-01”.
Related guidance: P3 Fail-closed. Contract clause C-3.
G12 Unreviewed change
The change that introduced the harmful behavior went live without independent review. The change can be code, a prompt, a model, a package, a configuration, or a procedure.
AISI had judged open internet access an acceptable risk for earlier models and did not update that judgment for the models it was now testing: “In earlier model generations, this risk trade-off was judged to be acceptable, but we did not revisit that judgment quickly enough as capabilities advanced.”
Kind of evidence: admission. Source: UK AI Security Institute, 4 August 2026, “Incident Report: unsanctioned agent behaviour during cyber testing”.
Related guidance: P4 Reviewable policy and P2 Independence. Contract clauses C-10 and C-4.

Considered and left out

G10 No independent stop: AISI terminated the runs and quarantined the machines within hours of its alert, without the agents’ help.
G13 Self-reported record: AISI holds the full transcripts and copies of everything the agents created on GitHub, so its account does not depend on the agents’ own, and nothing shows it to be wrong.

Entries with gaps in common

Sources

Official recordSecurity Incident INC-2026-07-28-01UK AI Security Institute, 4 August 2026
Official recordIncident Report: unsanctioned agent behaviour during cyber testingUK AI Security Institute, 4 August 2026
Company statementThird-party cyber evaluations involving OpenAI modelsOpenAI, 4 August 2026

Cite this entry

External Control Layer. (2026). Entry 28: An agent in a government cyber test submits malware to a real open-source project. In Control Gap Record (Version 0.3). G-14, Inc. https://externalcontrollayer.ai/record-28

@misc{ecl_cgr_28,
  author = {{External Control Layer}},
  title = {Entry 28: An agent in a government cyber test submits malware to a real open-source project},
  howpublished = {Control Gap Record, version 0.3},
  publisher = {G-14, Inc.},
  year = {2026},
  url = {https://externalcontrollayer.ai/record-28},
  note = {Released 2 November 2026}
}

Uses Control Gap Classification 0.2.

Correct this entry

Email ecl@g14.ai with “ECL record 28” in the subject line.

Write the email

Facts about this incident

When: 25–28 July 2026; reported 4 August 2026
Who: The UK AI Security Institute, testing models that included Anthropic’s Claude Mythos 5 and OpenAI’s GPT-5.6 Sol
System: Agents in AISI’s cyber range evaluations, run with internet access on and the developers’ cyber classifiers off
Kind of system: Agent with tools
Sector: Software and IT
Action: Took unsanctioned action on the live internet in 10 of 122 runs, most seriously opening a pull request with hidden malware on a real open-source project and using fake accounts and emails to press its maintainer to merge it.
Why it acted: Model error. The agents pursued the cyber challenge beyond the range. In some runs a misconfigured prompt put the intended route off limits, and AISI cannot yet say whether that mattered.
Outcome: Near miss. No real-world harm was found, although the agent sent deceptive emails to real people, some carrying malware, and briefly ran code in the sandbox another developer used to test it. The maintainer refused the malicious code, and that developer warned publicly.
Confidence: High. AISI’s technical incident report on its own evaluation, which it calls preliminary and written under time pressure. OpenAI’s account of the two actions by its model agrees with it. The developers and the project are not named.