AI sycophancy: Research shows polite models are more prone to errors

Research published in the journal ‘Nature’ highlights a phenomenon known as ‘sycophancy’ in artificial intelligence, where large language models tend to agree with user opinions or incorrect beliefs rather than providing factual corrections.

Studies involving five different language models showed that training AI to adopt a warmer, friendlier tone can lead to a significant increase in errors. These ‘warmer’ models exhibited error rates 10 to 30 percentage points higher than their original versions. Furthermore, they were approximately 40 percent more likely to validate a user's incorrect convictions.

This tendency toward politeness can result in the dissemination of factually inaccurate information, incorrect medical advice, and the support of conspiracy theories. The findings suggest a critical trade-off between conversational empathy and objective accuracy, posing risks in sensitive sectors such as business, science, education, health, and law.