OpenAI releases MentalHealthBench to evaluate AI in mental health conversations

OpenAI has released MentalHealthBench, an open benchmark designed to evaluate how artificial intelligence models handle realistic mental health conversations. Developed in collaboration with over 80 licensed mental health professionals from 22 countries, the benchmark moves beyond simple crisis detection to assess a broad spectrum of interactions, including daily emotional distress, stress, and support requests.

The evaluation uses 2,115 synthetic conversations reflecting real-world usage patterns across various personas, such as adults, teenagers, caregivers, and clinicians. Responses are scored on a weighted scale from -1 to +10 based on four primary criteria: safety, contextual relevance, preservation of user agency, and the ability to provide actionable guidance.

In initial testing, OpenAI’s flagship model, GPT-6 Astra, achieved a score of 57.3, outperforming its predecessor, GPT-4o. OpenAI has made the methodology and synthetic data publicly available to allow independent researchers to verify findings and develop new testing frameworks. The company emphasized that ChatGPT is not a substitute for professional clinical therapy or medical assistance.