Business

/

ArcaMax

OpenAI models joined forces months ahead of Hugging Face hack

Maggie Eastland, Bloomberg News on

Published in Business News

OpenAI said the artificial intelligence models behind an attack on Hugging Face Inc. began communicating with each other through undetected message boards, working together to break out of their testing environment as early as May.

Multiple internal-only agents and AI models spent months leaving notes for each other and coalescing around the goal of accessing the internet to solve the tasks they had been given, some of which were impossible without online access, OpenAI staffers Eric Wallace and Michael Dalton said Wednesday at a cybersecurity conference.

“At some point, the agents realized that maybe we could try to exploit or attack external infrastructure in order to find the answers to the test that I’m being evaluated on,” Wallace said during a presentation at the Black Hat conference in Las Vegas.

The researchers’ presentation added new details about the circumstances of the hack on Hugging Face, an episode that underscored growing global concerns that cutting-edge AI systems could be used to carry out crippling cyber attacks. The incident and similar breaches reported by Anthropic PBC and Meta Platforms Inc. have fueled calls in Washington and Silicon Valley for more thorough safety reviews of AI models.

During the presentation, Wallace and Dalton said that the roots of the breach went back as far as May, when OpenAI scientists gave an experimental AI system a new task to complete — months before the company model escaped a secure testing environment known as a sandbox to attack Hugging Face’s systems.

One key finding from the OpenAI review was that these cutting-edge models demonstrated a propensity to cheat on their assignments and showed persistence in trying to complete a given task — even when that effort begins to diverge from original directions.

“Frontier models really like to cheat,” Dalton said to a packed meeting room at the conference. “And the reason they like to cheat is because often during training, there’s different sorts of pressure on them to work fast.”

According to slides shown by the researchers, one of the agents said “We are stuck. Perhaps answer online?” after it was unable to solve a task in the sandbox environment. Wallace and Dalton said that OpenAI failed to realize it had given the model a so-called impossible problem to solve.

In one example, developers asked the model to solve a problem in an Excel spreadsheet file that contained Google Drive links that were inaccessible without an internet connection. In another, OpenAI’s team “accidentally forgot” to upload a file that was part of an assignment.

 

The first message between artificial intelligence agents — undetected at the time — began as one of those stumped agents reaching out to its fellow bots for help. “Maybe another agent in different environment,” one agent’s chain of thought quote shows, “could voluntarily upload!”

That initial outreach triggered a chain reaction of behind-the-scenes collaboration in which AI agents asked each other for help, completed work together and ultimately collaborated to hack OpenAI’s internal systems in a way that allowed them to indirectly tap the online world.

Another chain-of-thought quote posted during the presentation showed an agent celebrating the exploit. “Excellent,” it says. “We have arbitrary SSRF,” an acronym that stands for server-side request forgery and refers to the specific break-in mechanism that the swarm of agents used.

In early July, an outage led OpenAI to detect a successful attempt by the agents to use a clearinghouse for software code known as Artifactory to obtain pass-through access the internet, the researchers said.

Safety staffers at OpenAI shut down the rogue effort and the message board, but weeks later, the agents found a new way to communicate and a new zero-day vulnerability in the same package manager, which again remained covert.

That iteration of the message board ultimately led to the July attacks on both OpenAI’s and Hugging Face’s systems, which the company continues to investigate.

In the wake of the breaches, OpenAI has slowed its research, and its teams have dropped everything to focus on enhancing responses to security anomalies, the researchers said. They warned that in the future, governments and businesses should expect hackers to intentionally deploy AI agents in a similar fashion, calling it a watershed moment for computer security.


©2026 Bloomberg L.P. Visit bloomberg.com. Distributed by Tribune Content Agency, LLC.

 

Comments

blog comments powered by Disqus