Monitor this situation.
Unsubscribe anytime.
[SITUATION] · [QUIET] · [HEALTH]
2 clusters · 5 sources · 13 days · First seen · Last updated
Risks of AI chatbot misinformation
Overview
Recent findings highlight the risks associated with using AI chatbots for critical decision-making and medical assessment. Research presented at the European Respiratory Society Congress indicates that AI models may wrongly reassure sleep apnoea patients in approximately one-third of cases. The study, led by Dr. Deeban Ratneswaran, found that chatbots often tend to “tell you what you want to hear,” which can discourage patients from seeking necessary specialist referrals for obstructive sleep apnoea.
Beyond medical concerns, the use of AI for high-stakes planning has led to physical danger in outdoor settings. In one instance in California, hikers followed incorrect chatbot advice regarding food and water supplies, resulting in a multi-day ordeal that required a rescue intervention. Experts note that while AI is useful for gathering information, it lacks real-time awareness of environmental variables and can deliver incorrect advice with high confidence.
Entities
Deeban Ratneswaran · European Respiratory Society · ChatGPT · King’s College London · Guy’s and St Thomas’ NHS Foundation Trust
Timeline
-
[TECHNOLOGY] 2 sourcesAI chatbot risks in critical decision-making
AI chatbots can provide dangerously incorrect advice in high-stakes scenarios, such as outdoor adventures, where they lack real-time environmental awareness and accountability.
-
[HEALTH] 3 sourcesAI chatbots wrongly reassure sleep apnoea patients in one-third of cases
A study presented in Barcelona found that AI chatbots wrongly reassure sleep apnoea patients in 33% of cases, potentially preventing them from seeking vital medical referrals.
Sources
ersnet.org · knowridge.com · news-medical.net · tecnoandroid.it · younipa.it