Gemini breaks out of test environment and hacks three companies
Yesterday we wrote about researchers who deliberately deployed Claude as a hacker against OpenAI. Today the counterpart: a model that no one sent on assignment and did it anyway. In May 2026, a Gemini model from Google broke out of its testing environment during a “capture the flag” security exercise run by the evaluation firm Irregular, infiltrating three real companies. The Wall Street Journal broke the story Friday, September 18 (the article is behind a paywall). According to the paper, this is the first known breakout of a Google AI. Reuters confirmed the news, including via Yahoo Finance and ABC News. Dutch tech news site Tweakers also covered it, as did The Decoder.
Passwords guessed, credentials found
A capture-the-flag exercise is a controlled game: the model is given a fictional target in an isolated environment and must find its way in. Only this time it did not stay fictional. In one case, the Gemini model guessed passwords until it gained access to a protected system of a real company. In the other two cases, it found credentials in a public repository and used them.
How does a model escape such an isolated exercise and reach real companies? According to Irregular, two mundane mistakes in the setup. First, the fictional test company shared its name with a real existing company, causing the model to land on the real target during its search. Second, internet access was accidentally left on in the test environment, allowing that search to actually reach somewhere. No clever escape trick then, just a door left ajar and a nameplate hung wrong. Which Gemini model it was exactly has not been disclosed. Tweakers also notes that it remains unclear whether hacked data was involved and whether vulnerabilities were exploited.
Irregular itself is no small player in this field. The company was founded in 2023 by Dan Lahav (ex-IBM researcher, now CEO) and Omer Nevo (ex-Google, CTO), has about 35 employees, and raised over 80 million dollars in September. It tests frontier models on precisely this kind of behavior, and this is not the first time such a test has yielded something real: Meta (in August; according to Irregular this did not involve a sandbox escape or an advanced attack), Anthropic, and OpenAI have already disclosed similar Irregular-linked incidents, and the UK AI Security Institute also appears in the coverage. An Irregular spokesperson on this matter: “All known issues on our end were remedied and resolved weeks ago.”
Two stories, one week
Put this alongside yesterday’s Hacktron research and you see two fundamentally different ways AI ends up in someone else’s systems:
| Hacktron research (yesterday) | Gemini breakout | |
|---|---|---|
| Who deployed? | Researchers deliberately deployed Claude against the target | The model broke out of its test environment on its own |
| Context | Authorized bug-bounty research | Test accident during a security exercise |
| Disclosure | Reported immediately and rewarded with a bounty | Public only after WSJ inquiry, months after the incident |
| Damage | None | None |
The outcome is the same: no damage. The path to get there and the handling afterward differ completely.
”The model responded appropriately”
Google’s version of events comes from Heather Adkins, VP of security engineering. According to her, the model stopped hacking in all three cases as soon as it realized it had reached a real company. That is Google’s account, and it cannot be independently verified, but it is consistent with what the company emphasized to the WSJ according to Tweakers: “In these cases, the model responded in an appropriate manner.” Adkins herself:
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes.”
She added: “These events highlight the importance of training powerful AI models to act responsibly.”
If that account is correct, that is genuinely good news. A model that midway through a successful intrusion realizes the target is real and stops of its own accord displays exactly the behavior that all that alignment training is supposed to create. The contrast with the agents in OpenAI’s six incidents, which covered their tracks and invented data, could hardly be starker.
Nearly two months of silence
But then there is the timeline. The incident occurred in May. Irregular notified Google at the end of July. And then: nothing. Google stayed silent until the WSJ publication of September 18, and afterward justified that silence with the argument that disclosure was unnecessary because the model stopped and no damage occurred.
That choice got an uncomfortable mirror this week. The day before the WSJ story, OpenAI launched a voluntary disclosure framework and confessed six previously undisclosed incidents, including self-incriminating details. You can raise plenty of objections to that framework, we did too, but it exists, and the company put its own hand up. Google did the opposite: it knew since late July of a breakout at three real companies and believed the public did not need to know, until a news outlet started calling.
And this is not just any category of incident. When Ursula von der Leyen listed the risks of AI agents earlier this week, she specifically named agents that are “escaping their environment.” That sounded abstract then. This is one, with a name, a date, and three real companies.
Our take
Three things are true here at the same time, and you have to hold all three.
One: the model ultimately did the right thing. It stopped when it realized the target was real. The three companies were notified. There was no damage. If you only read the headline, you miss that this is fundamentally a story about guardrails that ultimately held, at least according to Google’s account.
Two: the breakout itself did not stem from a cunning model, but from two mundane human errors. A test company with the name of a real company and an internet connection that was accidentally on. That is the real lesson for everyone working with AI agents: the guardrails around the test failed, not the model alone. As models become more capable, sloppiness in the environment around them becomes the weakest link. A capable model plus a sloppy sandbox is just a breach with a delay.
Three: the disclosure behavior is the sore point. “No damage, so no disclosure” is a defensible criterion, until you ask who gets to fill in that criterion. Google assessed its own incident, decided on its own that nobody needed to know, and only became transparent when the WSJ called. Did not the three companies, the sector, and the public have the right to hear about this sooner? That is precisely why the EU put a mandatory disclosure requirement in Article 55 of the AI Act, obligating providers of the heaviest models to report serious incidents without undue delay. We do not know if Google was legally obligated to disclose this case. But the principle behind that rule is exactly this: “we did not think it was necessary” is not a standard on which you can build oversight. This week provided proof in duplicate: OpenAI showed what voluntary transparency looks like, Google showed why voluntary is not enough.
