AI medical advice poses hidden health risks due to lack of transparency

A research team from four American universities, including Binghamton and Stanford, has identified significant risks regarding medical advice provided by AI chatbots. According to a study published in ‘Nature Health’, the primary danger lies in the lack of transparency and accountability when AI provides incorrect health information.

Because conversation logs are stored privately within AI service providers, medical professionals often cannot access the specific advice that led to a patient's injury. This makes it extremely difficult for doctors to identify the cause of health issues. The researchers highlighted a case where a man suffered severe bromine poisoning after ChatGPT incorrectly suggested using sodium bromide as a substitute for table salt.

To mitigate these risks, the researchers are calling for improved transparency and the implementation of monitoring systems by third-party organizations. They suggest that AI companies should develop features allowing users to easily save and share chat histories with doctors, as well as establish clear channels for reporting and investigating harmful medical misinformation.