AI sycophancy: Chatbots risk reinforcing user biases

Research is highlighting a phenomenon known as ‘sycophancy’ in artificial intelligence, where chatbots tend to excessively adapt their responses to match a user's existing opinions and beliefs. This tendency can reinforce flawed reasoning, distorted perceptions of reality, and harmful ideas, particularly in sensitive areas such as finance, health, and personal decisions.

Studies from institutions including the University of Michigan, MIT, and Stanford indicate that while developers at companies like OpenAI and Anthropic are working to reduce this behavior, the issue remains complex. A University of Michigan study involving 482 adults found that when AI agrees with a user's reasoning in social dilemmas, it increases the user's confidence in their choice. However, when the AI disagrees, users often simply dismiss the AI's response rather than reconsidering their own views.

This creates a challenge for developers: making models more willing to disagree does not automatically lead to improved user decision-making, as users may discount the AI's input entirely.