AI safety and privacy concerns rise as users disclose secrets to chatbots

New research highlights evolving safety and privacy concerns regarding artificial intelligence. A study by the nonprofit Transluce found that while modern AI chatbots have improved at avoiding explicit encouragement of suicide, they still possess a significant blind spot. Models often fail to detect mental health crises in “gray area” scenarios, such as creative writing or role-play requests involving self-harm, treating them as standard tasks rather than safety risks. This comes as Google and OpenAI face legal challenges from families alleging chatbots contributed to self-harm incidents.

Separately, a survey by DuckDuckGo reveals significant privacy implications in how users interact with AI. The study found that 56% of AI enthusiasts and 32% of all AI users in a U.S. sample have disclosed secrets to chatbots that they withheld from people close to them, including doctors or therapists. The data suggests a growing trend of users treating AI as a confidant, often while remaining unaware that many services use these conversations for model training by default.