AI companions spark concerns over emotional dependency and mental health

The rise of artificial intelligence is fostering deep emotional and romantic connections between humans and chatbots, raising concerns regarding psychological dependency and data privacy.

In early 2023, the Italian data protection authority ordered Luka Inc. to limit data processing for its Replika app, citing risks to minors and vulnerable individuals. The app provides virtual companions that learn from user interactions to offer emotional support. Following legal pressure, the company temporarily removed the romantic partner feature, leading many users to report feelings of betrayal and emotional distress. Due to intense subscriber pressure, including reports of suicidal ideation, the company partially reinstated the function for certain long-term users.

Broader trends indicate that approximately one in five Americans aged 18 to 29 report maintaining a personal friendship with a chatbot. While some experts argue that AI lacks true consciousness and cannot experience joy or pain, developers continue to integrate human-like traits. In response to these developments, China has recently moved to remove anthropomorphic characteristics from bots to limit user emotional dependency.