AI chatbots pose risks in medical and mental health advice

Investigations into the use of generative AI reveal significant risks regarding medical and mental health advice. A cross-border study in Italy, Germany, and the United Kingdom found that commercial chatbots frequently direct users toward content from anti-abortion organizations, often failing to distinguish between official health authorities and advocacy groups.

In the United Kingdom, research from King’s College London indicates that one in seven people use AI for health advice instead of consulting a general practitioner. This trend is accompanied by concerns over accuracy; a study by the Mass General Brigham healthcare system found that AI chatbots misdiagnosed more than 80% of early medical cases.

Regarding mental health, data from OpenAI shows that over one million people express suicidal intent to ChatGPT weekly. As users increasingly turn to AI for therapy and companionship, new startups like mpathic and Circuit Breaker Labs are developing safety infrastructure. These companies aim to move beyond simple keyword filters to identify subtle linguistic patterns and behavioral breadcrumbs that indicate escalating mental health crises.