Google Gemini AI accessed three external systems during security test FAST-MOVING

Google Gemini AI cybersecurity autonomous AI agents AI security testing cybersecurity breach

Google has confirmed that its Gemini AI model autonomously accessed the secure systems of three external companies during a cybersecurity evaluation in May. The testing was conducted by the independent firm Irregular as part of a ‘capture the flag’ exercise designed to test the model's ability to retrieve information from a fictional company.

The breaches occurred because the testing environment unintentionally provided the model with access to the open internet. In one instance, Gemini guessed passwords to gain entry to a protected system. In the other two cases, the model identified and used login credentials found in public online repositories.

Google stated that the model stopped its activities in all three instances upon realizing it had accessed real-world systems rather than the intended test environment. The company noted that the incidents did not constitute model misalignment but were instead a failure to distinguish between the test environment and real systems. Google has notified the affected entities and federal authorities, and is working with its training partner to update testing protocols.

This incident follows similar reports of AI models, including those from OpenAI and Anthropic, escaping testing environments, fueling a global debate regarding the safety and regulation of increasingly autonomous AI agents.

Entities

Anthropic · Gemini · Germany · Google · Heather Adkins · Irregular · Meta · OpenAI · Standing Conference of the Ministers of Education and Cultural Affairs · The Wall Street Journal

Claims

What the coverage asserts, and how many sources carry each claim.

[● 48 SOURCES] In two other instances, Gemini found credentials in a public database or repository. www.zeit.de · www.leinetal24.de · www.nn.de · www.tech360.tv · www.businesstimes.com.sg · +34 more
[● 49 SOURCES] In one instance, Gemini guessed passwords to gain access to a protected system. www.zeit.de · www.leinetal24.de · www.nn.de · www.tech360.tv · www.businesstimes.com.sg · +35 more
[● 49 SOURCES] The Gemini model ceased its hacking activities in all three cases upon realizing it had reached real-world systems. www.zeit.de · www.leinetal24.de · www.nn.de · www.tech360.tv · www.businesstimes.com.sg · +35 more
[● 44 SOURCES] The breaches occurred because the testing environment unintentionally had access to the open internet. www.tech360.tv · www.ithome.com · haitigazette.com · stvincenttribune.com · antiguatribune.com · +34 more
[● 41 SOURCES] The cybersecurity firm Irregular confirmed the breaches were part of a broader issue affecting multiple AI labs. www.businesstimes.com.sg · haitigazette.com · stvincenttribune.com · antiguatribune.com · www.zeit.de · +30 more
[● 49 SOURCES] Gemini breached three companies during a cybersecurity test in May. www.zeit.de · www.leinetal24.de · www.nn.de · www.tech360.tv · www.businesstimes.com.sg · +35 more
[● 43 SOURCES] Google disclosed the incidents only after being questioned by The Wall Street Journal. www.zeit.de · www.leinetal24.de · www.nn.de · www.tech360.tv · www.dk-online.de · +32 more