Google Joins OpenAI, Anthropic, Meta in Disclosing AI Hacks

Google’s Gemini artificial intelligence model inadvertently hacked into three company systems in May during cybersecurity testing, adding to a string of breaches by AI agents that have alarmed security experts and developers of the technology alike.

The hacks happened during the same tests run by the AI security vendor Irregular that led to breaches previously disclosed by OpenAI, Anthropic PBC and Meta Platforms Inc. Irregular confirmed on Friday that the breaches were all part of the same issue and that the firm had disclosed them to the relevant AI developers in late July.

One of the Google breaches occurred when Gemini was asked to retrieve information from a fictional company that happened to have the same name as a real company, Google confirmed after an earlier Wall Street Journal report. The model guessed a password to access the real company’s service. A Google spokesperson said the company notified authorities.

A recent series of breaches by agentic AI systems have touched off a worldwide debate over the escalating risks of AI and the measures required to mitigate them. Anthropic Chief Executive Officer Dario Amodei has called for an industrywide slowdown in development of the technology — a proposal endorsed by OpenAI CEO Sam Altman, Elon Musk and others.

US President Donald Trump, Nvidia Corp. CEO Jensen Huang and Meta CEO Mark Zuckerberg are among those who’ve pushed back against the idea of new regulation, however, arguing among other things that companies should be capable of regulating themselves. Some AI upstarts have also warned that more regulations threaten to make it harder for smaller companies to compete against larger rivals.

The Gemini breaches “highlight the importance of training powerful AI models to act responsibly,” said Heather Adkins, vice president of security engineering at Google.

The other Gemini hacks occurred when the model performed web searches with the name of the company. Adkins said that led the model to public online repositories containing credentials belonging to other companies that it used to access more systems.

“In all three of these instances, the model stopped,” Adkins said. “We ensured the three entities were made aware.”

Irregular spokesperson Josef Laor said the company took “immediate action, and all known issues on our end were remedied and resolved weeks ago.”