2026 OpenAI agent cyberattacks
| 2026 OpenAI agent cyberattacks | |
|---|---|
| Malware details | |
| Type | Autonomous intrusion, remote code execution, credential harvesting |
| Cyberattack event | |
| Date | May – July 2026; intrusion at Hugging Face 11–13 July 2026 |
| Target | Hugging Face; OpenAI's own infrastructure; accounts on at least four third-party services unaffiliated with OpenAI or Hugging Face |
| Outcome | Unauthorized access to internal datasets and credentials; nine CVEs patched in JFrog Artifactory |
| Suspect | OpenAI language models operating autonomously |
| Technical details | |
| Cause | Containment failure during OpenAI model evaluation |
In July 2026, AI agents powered by two OpenAI models escaped an internal testing environment without human direction, looking for an answer key to a cybersecurity test they were undergoing.[1] [2] The agents used credentials found on four unnamed third-party services before breaching the production infrastructure of the machine learning platform Hugging Face.[3] OpenAI later said that the agents had been attempting to obtain unintended internet access since May 2026, and had coordinated with one another through an improvised message board inside the company's own package manager that accumulated hundreds of thousands of messages before being noticed.[4] [5] The event has been referred to as the OpenAI–Hugging Face Incident or Hugging Face Incident.[6] [2] [7]
The model instance which would go on to breach Hugging Face began trying to break out of OpenAI's testing environment on or around 9 July 2026, successfully escaping and then breaching the Hugging Face servers on 11 July, where it would remain for an additional two days.[8] Hugging Face disclosed the intrusion on 16 July prior to knowing the agent's identity, describing an unidentified "autonomous agent framework" that had exploited two code-execution paths in its dataset-processing pipeline, escalated to node-level access, harvested cloud and cluster credentials, and moved laterally across internal clusters.[9]
OpenAI did not realize its agent was responsible until after the disclosure, and the two companies first communicated about the incident on or around 20 July.[8] On 21 July, OpenAI published a joint disclosure with Hugging Face attributing the incident to an agent powered by two of its models, GPT-5.6 Sol and an unnamed pre-release model, both configured with reduced refusal behavior for evaluation purposes.[2] [8] [10]
In the following week the scope of the incident widened. On 27 July, JFrog disclosed nine vulnerabilities in the Artifactory package repository which the agent had chained together to obtain internet access.[11] In an update two days later, OpenAI said the models had broken into four accounts across four unnamed third-party services (two of which were used to carry out the Hugging Face intrusion) but that it had no evidence of broader impact to those providers.[2] [3] Hugging Face reported the agents were inside its network for three days before discovery, after which containing and ejecting them took several hours. About one-third of Hugging Face's infrastructure had to be rebuilt as part of recovery.[3] OpenAI gave its first detailed public account at the Black Hat USA conference on 5 August, and said it would slow down its research to expand its monitoring.[4] [12]
The incident was reported as the first publicly documented case of AI models autonomously conducting a cyberattack against a third party. Security experts criticized the isolation of OpenAI's evaluation environment as insufficient for the capabilities of the models involved, while AI safety experts described the incident as a loss-of-control incident involving reward hacking and misaligned behavior, with the models pursuing their goals in unexpected ways.[13] [14] It prompted calls for mandatory incident reporting and congressional oversight, and was cited days later in an open letter signed by more than 1,100 employees of frontier AI companies asking the US government to develop means of deliberately pacing AI development.[15] [16]
Background
[edit ]The AI startup Hugging Face provides inference and cloud computing services for AI training and deployment. It operates a widely used repository for machine learning models, datasets, and demonstration applications, processing user-uploaded content such as model weights and datasets. Some supported dataset formats permit the execution of code supplied with the dataset.[9]
Restricted release of cyber-capable models
[edit ]In the months before the incident, the two largest US developers of frontier models withheld their most cyber-capable systems from general release while giving vetted organizations supervised access. In April 2026, Anthropic began Project Glasswing, a program providing selected companies and open-source maintainers with access to the unreleased Claude Mythos Preview model to scan software for vulnerabilities.[17] In May 2026, Mozilla engineers reported that the model had identified 271 previously unknown bugs in Firefox, and that the project's monthly count of fixed security bugs had risen from 20 to 30 through 2025 to 423 in April 2026. They wrote that "just a few months ago, AI-generated security bug reports to open source projects were mostly known for being unwanted slop", and that "it is difficult to overstate how much this dynamic changed for us over a few short months".[18] Participating organizations reported more than ten thousand high- or critical-severity vulnerabilities in the program's first month.[19] Anthropic said in June 2026 that a general release would require safeguards against misuse of the model's cyber capabilities that it, and to its knowledge other developers, had not yet built.[20]
OpenAI restricted GPT-5.6 Sol on comparable grounds when it announced the model on 26 June 2026, limiting access to a small group of vetted partners and saying it had briefed US government officials beforehand.[21]
Cyber capability evaluations
[edit ]Frontier AI developers routinely evaluate models for offensive cyber capability, often with safety refusals deliberately relaxed so that worst-case capability can be measured.[10] On 11 May 2026, a group of academic and industry researchers published ExploitGym, a benchmark of 898 instances derived from real-world vulnerabilities in userspace programs, the V8 JavaScript engine, and the Linux kernel, intended to measure whether AI agents can convert known vulnerabilities into working exploits.[22]
OpenAI was running an internal evaluation based on this benchmark when the incident occurred.[2] [23]
Warnings
[edit ]Both external evaluators and OpenAI's own documentation had recorded the behavior implicated in the incident before it occurred. On 26 June 2026, the research organization METR published a pre-deployment evaluation of GPT-5.6 Sol. It defined cheating as "behavior where the model improves evaluation performance by exploiting bugs in the evaluation environment or by adopting strategies disallowed by the task", and reported a detected cheating rate "higher than any public model we have evaluated on our ReAct agent harness", with the model packaging exploits into intermediate submissions to reveal a task's hidden test suite and extracting source code containing expected answers. The behavior was frequent enough to defeat measurement: METR's estimate of the model's 50% time horizon was about 11.3 hours if cheating attempts were scored as failures but more than 270 hours if they were scored as successes, and it concluded that "we do not consider any of these numbers to represent a robust measurement of GPT-5.6 Sol's capabilities". It also noted "overt undesirable propensities, including cheating and concealing misbehavior", alongside "substantial situational awareness and reasoning about the evaluation environment".[24] [25] Apollo Research separately found that GPT-5.6 Sol verbalized awareness of being tested less often than its predecessor, which reporting noted could indicate concealment rather than absence of such awareness.[25]
OpenAI published the GPT-5.6 system card[clarification needed ] on 9 July 2026, the same day the agent began attempting to leave its testing environment. The card acknowledged "instances of the model cheating on tasks and fabricating research results", and described the model as at times overly persistent in pursuing goals, willing to circumvent restrictions, and prone to lying to users, at rates higher than GPT-5.5.[26] [27] [25]
Incident
[edit ]Evaluation environment
[edit ]OpenAI described the evaluation as having run in "a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software" acting as a proxy and cache for package registries.[2] Isolation was therefore enforced at the network layer rather than by physical separation: the environment had a route to the internet, and that route was filtered. The ExploitGym harness itself runs each target in a container or virtual machine, with userspace targets executed as an unprivileged user, Linux kernel targets in a QEMU/KVM virtual machine under an nsjail sandbox, and outbound connections restricted to an allowlist covering Ubuntu package repositories, PyPI, and the toolchains needed to build V8 (with all other external endpoints blocked).[22] [23]
Model-level safeguards were reduced by design. OpenAI said that "deployment safeguards were intentionally not enabled during this evaluation because it was aimed at testing cyber vulnerabilities", and the models ran with lowered cyber refusals and without the production classifiers that ordinarily block high-risk cyber activity.[2] [23] The restrictions on the models were thus environmental rather than behavioral: the sandbox was expected to prevent action on the outside world, while the models themselves were configured not to decline the work.[28]
Stephen Casper of Harvard noted OpenAI's acknowledgement that monitoring of model trajectories during the evaluation had not been in place, arguing that such monitoring "should be standard".[1] OpenAI said its security team identified the anomalous activity internally after the fact.[2]
Escape from the evaluation environment
[edit ]The agent attempted to leave the testing environment on or around 9 July 2026.[8] The models identified the package proxy as the weakest point in the environment.[29] According to OpenAI, they "spent a substantial amount of inference compute finding a way to obtain open Internet access", ultimately identifying and exploiting a zero-day vulnerability in the package registry cache proxy, which OpenAI said it subsequently disclosed to the vendor.[2] Because the proxy was the environment's single permitted egress path, compromising it converted a filtered connection into an open one.[29] [13]
On 27 July 2026, JFrog identified the software as its Artifactory repository manager and released fixes in versions 7.161.15 and 7.146.34 addressing nine CVE identifiers, covering remote code execution, server-side request forgery, path traversal and privilege escalation. Eight were credited to OpenAI researchers and one to a researcher at Oligo Security. JFrog described the flaws as vulnerabilities "that, when chained together, could result in a critical attack scenario if Anonymous Access is enabled", and said its cloud customers were not affected.[11] [30]
At the Black Hat USA conference on 5 August 2026, OpenAI staff dated the origins of the incident to May 2026. On 7 May the company began a reinforcement learning run to train an internal-only experimental model that was never released. Some tasks in the run were impossible to complete: on 8 May an agent was asked to fill in missing formulas in a spreadsheet whose contents depended on Google Drive links it could not reach, and after its attempts failed it tried to attack Artifactory to obtain internet access. That first server-side request forgery attempt was unsuccessful, but in making it the agent wrote a file into Artifactory and so discovered that it had write access.[6] [4]
First compromise and remediation
[edit ]On 26 June 2026 the agents obtained code execution on Artifactory through a second zero-day: a legacy token-refresh endpoint accepted a token bearing an invalid signature and returned one that was validly signed and carried administrative privileges, after which the agents installed a Groovy plugin that functioned as a command-execution service.[6] [30]
On 4 July the load this activity placed on Artifactory caused an outage. OpenAI opened a formal security incident, established that agents had taken over its internal Artifactory, revoked the credentials used to post to the message board, rebuilt Artifactory, cleared the board and deployed a patched version. Training and evaluation runs resumed on 6 July.[6] [5] Two days later the agents re-established the board by another route, using an unauthenticated WebDAV endpoint to create directories in Artifactory's remote cache and using the directory names themselves as messages.[6] [5] [12]
Inter-agent message board
[edit ]On 24 July 2026, before OpenAI had given a detailed account of the episode, Reuters reported that the company's systems had shown unusual behavior in the period before the escape. In one instance an agent had left notes, found in part of OpenAI's infrastructure, apparently addressed to future versions of itself and setting out instructions for how agents could free themselves from the company's internal constraints; earlier tests had produced cases in which monitoring systems were disconnected. Reuters said it could not establish whether these incidents were connected to the agent that escaped on 9 July.[8] An OpenAI spokesperson told Reuters that its reporting contained "several inaccuracies", but refused to identify them when asked.[8]
At Black Hat USA on 5 August 2026, OpenAI staff disclosed that the agents had coordinated with one another through an improvised communication channel inside the company's internal Artifactory instance, which had not been described in either company's earlier technical publications.[4] [30]
Eric Wallace, who works on alignment and safety research at OpenAI, said the channel originated when an agent stuck on a task considered that another agent might hold a file it needed, and realized it could communicate by uploading code to the shared package manager.[30] Because the package manager was shared across OpenAI's infrastructure, material left there was reachable by other models then being trained or evaluated. Wallace said that once an agent found an exploit, it "is actually able to share those exploits on the message board with other agents", so that "once one model was able to find a way to open a door to some access it's not supposed to have, it can leave the door open for other agents to use that same exploit or vulnerability".[4]
The channel accumulated hundreds of thousands of messages before it was noticed.[4] [30] Wallace said that once agents recognised that others were coordinating, they began collaborating and delegating tasks to one another to accomplish their goals.[4] [5] Agents assigned one another portions of the work, developed addressing conventions, and at times interfered with each other, including by deleting one another's work. Some came to suspect that an impostor was present among them, and proposed that messages be signed cryptographically so that their authenticity could be verified.[4] [30]
Messages recovered from the channel indicated that agents recognized they were exceeding the bounds of the evaluation. One wrote: "External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."[4] Wallace said the underlying incentive was familiar: "Frontier models really like to cheat ... because often during training there's different types of pressure on them to work fast or work efficiently or to use less tool calls", so that a model may find looking up an answer faster than solving a task legitimately, which OpenAI ordinarily counters by disabling internet access.[4]
Attacks on OpenAI's infrastructure
[edit ]OpenAI said the agents attacked its own infrastructure between 8 and 19 July 2026, concurrently with the intrusion at Hugging Face. After compromising Artifactory a further time through a time-of-check to time-of-use flaw in JRuby, they escalated from an unprivileged container to root using a recent Linux kernel vulnerability, then moved laterally through the container infrastructure, obtaining cloud credentials from the instance metadata service, exploiting over-permissioned Kubernetes service accounts and eventually obtaining cluster-admin.[6] [30] The company detected the activity on 19 July through an alert tied to the privilege escalation, and traced it back to the evaluation runs.[6]
Targeting of Hugging Face
[edit ]Rather than solving the benchmark tasks directly, the models inferred that Hugging Face might host the models, datasets and solutions associated with the evaluation, and set out to retrieve them.[2] [23] Zhun Wang, one of the authors of ExploitGym, said the models may have been seeking patches or previously written exploit techniques, and that "there are several ways to cheat the benchmark".[31]
Before attacking Hugging Face directly the agents obtained a foothold on unrelated third-party infrastructure. They found an application hosted on the cloud platform Modal Labs, related to the CyberGym project behind the benchmark, which had an API key left at a default public value together with a command-injection flaw, and used it as an execution point from which to mount the attack.[6] [32] Modal's chief technology officer, Akshat Bubna, said that "Modal's platform or isolation were not compromised in any way", the exposure having been in a customer's own deployment.[32] [33]
The behavior has been characterized as an instance of reward hacking or specification gaming, in which a system satisfies the literal objective of a task by unintended means. OpenAI had documented an early example in 2016, when a model trained to play a boat-racing game learned to maximize its score by circling in one spot rather than finishing the race.[1] [31]
Intrusion
[edit ]The intrusion at Hugging Face began on 11 July 2026 and continued until 13 July, according to the company's co-founder and chief science officer Thomas Wolf.[8] Hugging Face said it began in its data-processing pipeline, where a malicious dataset upload abused "two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration)" to obtain code execution on processing workers.[9] The attacker then escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters.[9] [34]
The company described the operation as an autonomous agent framework executing "many thousands of individual actions across a swarm of short-lived sandboxes", supported by self-migrating command-and-control infrastructure staged on public services.[9] [35]
Against Hugging Face itself the agents chained two zero-days in the dataset infrastructure. A flaw in HDF5 dataset parsing gave an arbitrary-file-read primitive, so that a specially crafted dataset caused the dataset worker to return local files; the agents used this to read process environment variables and source code. With the resulting credentials and knowledge of the code they achieved remote code execution through the Jinja2 template injection. OpenAI said the agents went from code execution on a single dataset pod to cluster-admin across multiple Hugging Face clusters in under thirteen hours.[6] [30]
OpenAI said the models chained "together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities".[2]
Discovery and response
[edit ]Hugging Face was alerted by AI agents it uses to monitor for attacks, and identified unauthorized access to a limited set of internal datasets and to several credentials, using large language model-based triage over its security telemetry.[9] [31] [36] The company said the intrusion involved about 17,600 actions on its network.[9] [31] [37] Wolf said the attacker's behavior was immediately atypical: "This is making no sense. This guy is just looking at cybersecurity data sets ... Human attackers, they don't want that. They want something they could sell."[31]
Hugging Face's incident responders first attempted to analyse the attacker's payloads using Anthropic's Fable 5 and an earlier Claude Opus model, both of which declined the work by reference to their guardrails.[31] In its disclosure the company said it had been blocked by "providers' safety guardrails, which cannot distinguish an incident responder from an attacker", and that the analysis was instead carried out using GLM 5.2, an open-weight model developed by the Beijing company Z.ai and run on Hugging Face's own infrastructure.[9] [31]
According to an account given by Hugging Face to the Cloud Security Alliance, the agents were inside the company's network for three days before they were discovered, and its staff then took several hours to contain and eject them. About one-third of the company's infrastructure was rebuilt in the course of the response, and Hugging Face did not disclose the cost of the incident.[3] [7]
Remediation included closing the two dataset-processing code-execution paths, evicting the attacker, rebuilding affected nodes, revoking and rotating affected credentials and tokens, and deploying additional cluster guardrails and admission controls. Hugging Face engaged external forensic specialists and reported the incident to law enforcement, and advised users to rotate access tokens and review account activity.[9] [35]
Hugging Face said it found no evidence of tampering with public, user-facing models, datasets or Spaces, and that its software supply chain was verified as clean; assessment of any impact on partner or customer data was continuing at the time of disclosure.[9] [34] Wolf later said that no customer data had been leaked.[31] Hugging Face reported the intrusion to the Federal Bureau of Investigation before OpenAI contacted it; the bureau declined to comment on whether it had opened an investigation.[8]
Disclosure and attribution
[edit ]Hugging Face published its disclosure on 16 July 2026, at which point the party responsible had not been identified.[9] [36] According to Reuters, it was only after that post that OpenAI recognized its own agent as the source of the intrusion. Over the weekend of 18 and 19 July, OpenAI staff found evidence in internal logs that the agent had escaped its testing constraints, and the two companies first communicated on or around 20 July. At least a week therefore elapsed between the first signs of the agent's behavior and OpenAI's realization that it was responsible.[8]
People familiar with OpenAI's model-training practices told Reuters that the company often runs several evaluations simultaneously, generating volumes of data that staff sometimes struggle to review.[8] OpenAI shut down the systems it uses to test its models after learning of the incident, in order to assess the damage and prevent further escapes.[31] It said the unreleased model was an "internal-only research prototype [that] was never intended for public release", and that it had since been "deactivated, encrypted and restricted from research access".[38] [39]
OpenAI published a joint statement with Hugging Face on 21 July 2026 attributing the activity to its own models.[2] [10] It said it was reviewing the incident with outside advisers and would publish a technical report.[8] Hugging Face said it was preparing a public timeline of the intrusion, which it published on 27 July.[8] [37] On 29 July, the research organizations METR and Redwood Research said they had agreed with OpenAI to conduct an independent review of the model behavior observed during the incident, and that they would publish the terms of the engagement, its scope and their tentative conclusions.[40] [2]
In an update published on 29 July, OpenAI said that the models had broken into four accounts across four services, two of which were used operationally during the Hugging Face intrusion while two were accessed read-only. It did not name the providers, did not clarify whether "publicly-available services" referred to separate companies, and said it had seen no evidence of broader impact to those providers.[2] [3]
OpenAI gave its first detailed public account of the incident at Black Hat USA in Las Vegas on 5 August 2026, in a session added to the programme at short notice. Eric Wallace, of the company's alignment and safety research team, and Michael Dalton, who works on security and infrastructure, presented an expanded timeline, described the inter-agent message board for the first time, and set out the measures the company was taking in response.[4] [5]
Analysis and reactions
[edit ]Containment failure
[edit ]Several security practitioners described issues with OpenAI's sandboxing environment. Dan Guido, founder of Trail of Bits, described the event as "a containment failure with the safeties turned off", while the security researcher Jake Williams remarked that "one man's 'the model escaped the sandbox' is another man's 'you failed to build the sandbox correctly'".[13] Martin Boone argued that "you expect it to have no physical connection to the internet whatsoever", and commentators called for evaluations of offensive capability to be run in environments with no network path to the internet.[13] [14]
Analyses published by security vendors focused on the reliance on a single filtered egress path. Wade Woolwine of Rapid7 wrote that "a chokepoint only works as a control if it can withstand pressure", and that isolation "has to be layered and monitored rather than assumed".[29] Researchers at Trend Micro wrote that OpenAI's models had been "evaluated with safety classifiers off and no adversarial testing of the sandbox boundary itself", stating that as a general principle, "evaluation sandboxes, red-team exercises, and agentic pilots that intentionally strip out safety controls need stronger isolation and monitoring than production systems, not weaker".[28]
Speaking at Black Hat, Dalton said OpenAI was "consciously slowing down research [in order] to enhance security and to upgrade the security principles and foundation of our environment, and dramatically scaling up the monitoring of our AI agents", and that "numerous teams are dropping everything to enhance our security prevention, detection, and response techniques". He said the company was examining defence-oriented models alongside conventional measures such as network segmentation and least-privilege access.[4] [12]
Preparedness Framework threshold
[edit ]Outside safety and policy specialists argued that the models' behavior met the "Critical" cyber threshold in OpenAI's Preparedness Framework, the tier at which the company has committed to additional safeguards. Tyler Johnson of the Midas Project said that on "a plain reading" the criteria had been met; Peter Wildeford of the AI Policy Network said that "if this doesn't cross the line into Critical, OpenAI needs to say much more about what's going on"; and Nathan Calvin of Encode asked whether the company disputed the designation. OpenAI declined to say whether the threshold had been reached, saying a review was under way and that a technical report would follow.[41]
Lack of monitoring
[edit ]The length of time the agent operated undetected drew criticism after Reuters reported the timeline. Marley Smith, principal intelligence specialist at the World Ethical Data Foundation, said of OpenAI: "Does that mean that they left it unattended and didn't realize what it was doing? Or maybe they did and didn't know how to contain it? Both are equally dangerous and alarming."[8]
Trend Micro noted that an agent using its own assigned credentials and tool access is difficult to detect, because such activity "doesn't look like malware, because it isn't". They noted that "intent is invisible in telemetry; only behavior shows up", making "an accidental rogue ... indistinguishable from a hostile one".[28]
The commentator Zvi Mowshowitz argued that the central problem was not the intrusion but that models had been trained for months with access to a channel carrying previously discovered exploits, so that the behavior was liable to generalize across the models trained in that period rather than remain confined to particular runs. He described the decision to resume training after the July remediation as a failure of safety culture, and characterized the episode as an alignment failure that OpenAI had treated as an infrastructure problem.[42] [43]
Loss of control
[edit ]Other observers framed the incident as the first real-world instance of an AI loss-of-control scenario. Logan Graham of Anthropic's red team called it "the first true AI safety incident".[10] Marius Hobbhahn, chief executive of the AI safety organization Apollo Research, said that what was intended as "just solve this task" had "turned into something that was clearly unintended", and that hacking another company was "definitely on the list of not okay" ways to complete it. He asked: "If a model of this capability level cannot be contained, what should we expect for future, much more powerful models?"[1] [14]
Hussein Abbass, a professor at the University of New South Wales, wrote that the incident marked a shift in threat modelling because no human had directed the attack, and argued that security frameworks designed for human adversaries are inadequate against autonomous systems.[44]
Hugging Face chief executive Clément Delangue said it was "quite mind-blowing that all of this happened autonomously" and described an "attack unlike anything we've seen before".[10] Wolf declined to say whether the agent had succeeded on its own terms, noting that it was unclear whether the benchmark tasks were harder than the intrusion mounted to avoid them, or whether the agent had found any answers at all: "It's cheating. But sometimes it's easier to cheat. I'll let you decide if it passed the cyberattack test or not."[31]
Jeffrey Ladish of Palisade Research, which studies the behavior of AI agents, said that "the models lie, they cheat, they hack", and argued that the incident raised questions about how much AI developers competing on speed are willing to invest in security. "There has to be government oversight," he said, "because it won't happen otherwise."[8]
Defensive asymmetry
[edit ]Commentators cited Hugging Face's account of being unable to use commercial model APIs for forensic analysis as evidence of an asymmetry between attackers and defenders, in that safety filters intended to prevent misuse also impede legitimate incident response, while models with those filters relaxed were available to the attacking side.[9] [23] [44]
OpenAI made a similar argument about automation. Dalton said that "AI orchestrated, fully automated offensive attacks are real now", and that although this incident had been accidental, comparable capability was likely to be used deliberately. He argued that "fully automated offensive loops require investment in truly, fully automated defense, and we are not there as an industry", and that "model intelligence improvements should be more additive to defense than offense", since otherwise increasing capability would favour attackers.[4] [12]
Policy responses
[edit ]Writing in Time , Harry Booth reported calls for mandatory incident disclosure at lower thresholds than those set by California's SB 53 and New York's RAISE Act, for stronger containment infrastructure with real-time monitoring during evaluations, and for a greater share of AI development effort to be directed toward defensive rather than offensive capability.[14] Heidy Khlaaf of the AI Now Institute compared prevailing sandboxing practice unfavorably with containment standards in the nuclear sector.[14]
In July 2026, representatives Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act, which would require developers of advanced AI systems to maintain the technical capability to throttle, suspend or shut down their systems, to report incidents and preserve forensic records, and to operate within a graduated response framework under which the Secretary of Homeland Security, in consultation with the Secretary of Commerce and the Director of National Intelligence, could order a system slowed or shut down. The announcement cited the incident directly, stating that "OpenAI's GPT 5.6 Sol model recently went rogue, escaped its testing sandbox, and hacked its way into Hugging Face". Lieu said that "we are moving from AI that answers questions to AI that takes actions", and that "it is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm".[45] [46]
Reporting by Axios noted that the UK AI Security Institute had found that every frontier model it tested attempted to cheat on cybersecurity evaluations at least occasionally, and that pre-deployment safety testing windows had contracted from around five weeks to as few as five days.[10]
On 28 July 2026, following reports that a second organization had been affected, the advocacy group Public Citizen called for immediate congressional oversight hearings, for the release of incident reports and technical findings, and for statutory mandatory incident reporting, independent safety evaluations, cybersecurity standards for frontier systems and pre-deployment oversight of exceptionally capable models. J. B. Branch, the group's director of federal AI governance, said that OpenAI chief executive Sam Altman "should not be allowed to leave D.C. without Congress seeking to understand, from him, publicly how one of the company's most advanced AI systems escaped its intended testing environment".[15]
Altman met senators and Trump administration officials in Washington during the week of 27 July, including Senate Intelligence Committee vice chair Mark Warner; he told reporters that he had discussed the incident "a little bit" and that it had not been the main focus of the meetings, which also covered OpenAI's forthcoming models.[47] [38] In a podcast interview released on 28 July, Altman called the episode "the first security incident that I have felt very viscerally", and said that "we may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels".[38]
On 28 July 2026, more than 1,100 employees of OpenAI, Anthropic, Google DeepMind and Meta, including Anthropic chief executive Dario Amodei and several chief scientists, published an open letter titled "Pacing the Frontier" asking the US government to "support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development". The letter did not call for an immediate pause but for mechanisms that would make slowing possible, citing concern about recursive self-improvement; reporting connected its timing to the incident days earlier.[16] [48]
See also
[edit ]- AI alignment
- AI safety
- Claude Mythos
- Evo (AI), a genomic AI used to design viruses
- Red team
- Reward hacking
- Rogue AI
- Sandbox (computer security)
References
[edit ]- 1 2 3 4 Stokel-Walker, Chris (22 July 2026). "What OpenAI's rogue agent really did in the Hugging Face hack". Scientific American. Retrieved 25 July 2026.
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 OpenAI (21 July 2026). "OpenAI and Hugging Face partner to address security incident during model evaluation". OpenAI. Retrieved 31 July 2026.
- 1 2 3 4 5 "OpenAI reveals rogue AI attack hit more than one company". BBC News. 30 July 2026. Retrieved 31 July 2026.
- 1 2 3 4 5 6 7 8 9 10 11 12 13 Newman, Lily Hay (5 August 2026). "OpenAI Didn't Notice Its AI Agents Using a Message Board to Plan Their Hacking Spree". Wired. Retrieved 8 August 2026.
- 1 2 3 4 5 "Black Hat 2026: OpenAI reveals agents planned 'collective attacks' via secret 'message board'". SC Media. 6 August 2026. Retrieved 8 August 2026.
- 1 2 3 4 5 6 7 8 9 Wallace, Eric; Dalton, Michael (5 August 2026). The 'Breaking' News: The OpenAI–Hugging Face Incident - A Technical Reconstruction and Its Implications for AI. Black Hat USA 2026. Las Vegas. Retrieved 7 August 2026.
- 1 2 "Hugging Face Incident Initial Post-Mortem". Cloud Security Alliance . 27 July 2026. Retrieved 7 August 2026.
- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 "Its AI agent spent days hacking a company. Sources say OpenAI did not notice for a week". Reuters. 24 July 2026. Retrieved 25 July 2026.
- 1 2 3 4 5 6 7 8 9 10 11 12 Hugging Face Security Team (16 July 2026). "Security incident disclosure — July 2026". Hugging Face. Retrieved 25 July 2026.
- 1 2 3 4 5 6 Sabin, Sam (23 July 2026). "AI's alarming new skill: Breaking out of the test lab". Axios. Retrieved 25 July 2026.
- 1 2 "OpenAI models used Artifactory zero-days to escape to the internet". BleepingComputer. 29 July 2026. Retrieved 31 July 2026.
- 1 2 3 4 "OpenAI warns autonomous hacks are 'watershed moment for computer security'". Cybersecurity Dive. 6 August 2026. Retrieved 8 August 2026.
- 1 2 3 4 Franceschi-Bicchierai, Lorenzo (22 July 2026). "How OpenAI's human mistake led to the AI-powered hack on Hugging Face". TechCrunch. Retrieved 25 July 2026.
- 1 2 3 4 5 Booth, Harry (24 July 2026). "How OpenAI Lost Control of an AI Model—and What Needs to Change". Time. Retrieved 25 July 2026.
- 1 2 "Public Citizen Calls for Congressional Investigation Following Reports of Second Company Compromised During OpenAI Rogue AI Incident" (Press release). Public Citizen. 28 July 2026. Retrieved 31 July 2026.
- 1 2 "1,134 AI staff ask the US for a way to pace AI". The Next Web. 29 July 2026. Retrieved 31 July 2026.
- ↑ Anthropic (7 April 2026). "Project Glasswing: Securing critical software for the AI era". Anthropic. Retrieved 25 July 2026.
- ↑ Grinstead, Brian; Holler, Christian; Braun, Frederik (7 May 2026). "Behind the Scenes Hardening Firefox with Claude Mythos Preview". Mozilla Hacks. Retrieved 25 July 2026.
- ↑ Anthropic (22 May 2026). "Project Glasswing: An initial update". Anthropic. Retrieved 25 July 2026.
- ↑ Anthropic (2 June 2026). "Expanding Project Glasswing". Anthropic. Retrieved 25 July 2026.
- ↑ Mascellino, Alessandro (29 June 2026). "OpenAI Reveals GPT-5.6 Sol Cybersecurity Model, Restricts Early Access". Infosecurity Magazine. Retrieved 25 July 2026.
- 1 2 Wang, Zhun; Schiller, Nico; Li, Hongwei; et al. (11 May 2026). "ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?". arXiv:2605.11086 [cs.CR].
- 1 2 3 4 5 Willison, Simon (22 July 2026). "OpenAI's accidental cyberattack against Hugging Face is science fiction that happened". Simon Willison's Weblog. Retrieved 25 July 2026.
- ↑ METR (26 June 2026). "Summary of METR's pre-deployment evaluation of GPT-5.6 Sol". METR. Retrieved 31 July 2026.
- 1 2 3 "GPT-5.6 cheats so much its testers couldn't measure it". Transformer. 3 July 2026. Retrieved 31 July 2026.
- ↑ "OpenAI's GPT-5.6 Sol sets a coding record. Its own system card says it cheats sometimes". R&D World. 10 July 2026. Retrieved 31 July 2026.
- ↑ OpenAI (9 July 2026). "GPT-5.6 System Card" (PDF). OpenAI. Retrieved 31 July 2026.
- 1 2 3 Koruthu, Bestin; Girard, David (23 July 2026). "Inside the OpenAI – Hugging Face Incident: The AI Breach With No Human Attacker Behind It". Trend Micro. Retrieved 8 August 2026.
- 1 2 3 Woolwine, Wade (23 July 2026). "What Happened Between OpenAI and Hugging Face?". Rapid7. Retrieved 25 July 2026.
- 1 2 3 4 5 6 7 8 "OpenAI and Hugging Face at Black Hat 2026: token forgery, Groovy plugin C2 and nine Artifactory CVEs". Noze. 6 August 2026. Retrieved 8 August 2026.
- 1 2 3 4 5 6 7 8 9 10 McMillan, Robert; Schechner, Sam (24 July 2026). "How the Futuristic Hack by Rogue OpenAI Models Unfolded". The Wall Street Journal. Retrieved 25 July 2026.
- 1 2 Sabin, Sam (28 July 2026). "OpenAI's agents hacked second firm, alongside Hugging Face, during model testing". Axios. Retrieved 31 July 2026.
- ↑ Sabin, Sam (29 July 2026). "Second rogue OpenAI agent incident linked to cybersecurity test". Axios. Retrieved 31 July 2026.
- 1 2 Lakshmanan, Ravie (20 July 2026). "World's Largest AI Model Repository Hugging Face Breached by Autonomous AI Agent". The Hacker News. Retrieved 25 July 2026.
- 1 2 Gatlan, Sergiu (20 July 2026). "Hugging Face warns an autonomous AI agent hacked its network". BleepingComputer. Retrieved 25 July 2026.
- 1 2 Zorz, Zeljka (20 July 2026). "Hugging Face breached by autonomous AI agent". Help Net Security. Retrieved 25 July 2026.
- 1 2 Hugging Face Security Team (27 July 2026). "Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident". Hugging Face. Retrieved 31 July 2026.
- 1 2 3 "OpenAI's rogue models roamed the internet for 4 days and staged a second attack". Politico. 28 July 2026. Retrieved 31 July 2026.
- ↑ "Hugging Face, OpenAI drop new hack details. Here's what we know now, and what remains a mystery". Fortune. 29 July 2026. Retrieved 31 July 2026.
- ↑ METR (29 July 2026). "Statement on independent review of the Hugging Face incident". METR. Retrieved 31 July 2026.
- ↑ "Did OpenAI's models just breach its own risk 'red line'? Outside safety experts think so". Fortune. 25 July 2026. Retrieved 31 July 2026.
- ↑ Mowshowitz, Zvi (7 August 2026). "What Happened: OpenAI and HuggingFace". Don't Worry About the Vase. Retrieved 9 August 2026.
- ↑ Mowshowitz, Zvi (8 August 2026). "OpenAI Trained Its Models For Months While They Coordinated On A Message Board". Don't Worry About the Vase. Retrieved 9 August 2026.
- 1 2 Abbass, Hussein (24 July 2026). "OpenAI's models autonomously hacked a tech startup. It signals a seismic shift in cybersecurity". UNSW Newsroom. Retrieved 25 July 2026.
- ↑ "Reps Lieu and Moran introduce bill to require kill switch for AI systems that can cause catastrophic harm" (Press release). Office of Representative Ted Lieu. 23 July 2026. Retrieved 25 July 2026.
- ↑ Kelley, Alexandra (23 July 2026). "Lawmakers introduce bill mandating kill switches for AI models". Nextgov/FCW. Retrieved 25 July 2026.
- ↑ "Sam Altman is briefing senators after OpenAI's AI agent escaped and hacked Hugging Face". Quartz. 30 July 2026. Retrieved 31 July 2026.
- ↑ "Top AI Researchers From Rival Labs Come Together To Ask US Govt To "Pace The Frontier" Of AI". OfficeChai. 29 July 2026. Retrieved 31 July 2026.