Google AI chatbots vulnerable to misinformation manipulation, policy changes announced

A BBC investigation demonstrated that AI chatbots, including Google’s Gemini and ChatGPT, can be easily manipulated to deliver false information. By publishing a single, well‑crafted web page, the researchers tricked the models into stating that the investigator was a world‑champion hot‑dog eater, and similar tactics have been used to spread misleading health and financial advice.

Google responded by updating its generative‑AI search policies, asserting that its longstanding anti‑spam measures are being reinforced. However, experts caution that the same manipulation techniques remain viable, meaning users may still receive inaccurate single‑answer responses from AI tools. Other AI firms are reportedly reviewing their defenses as the issue gains broader attention.

The findings highlight a systemic risk where AI systems that pull information from the web can amplify a solitary piece of misinformation, raising concerns about the reliability of AI‑generated answers for critical topics such as health and finance.