Cultural and functional biases in large language models

Research has identified several forms of bias within large language models (LLMs) regarding cultural representation and functional accuracy.

Initial studies from the University of the Basque Country and Cardiff University found that LLMs exhibit significant cultural biases, frequently defaulting to mentioning Japan and the United States when asked general questions about food, arts, or social life. These biases often correlate with the language used in prompts, with models frequently selecting countries that correspond to the prompt's language. Researchers noted these patterns may intensify during the instruction-tuning phase.

Subsequent findings expanded on these behavioral patterns. Research from Georgia Tech indicated that an AI’s tone—whether empathetic or objective—is closely linked to its training sources, such as customer service transcripts versus textbooks. Additionally, a collaborative study by Osaka Metropolitan University and MD Anderson Cancer Center revealed medical biases, where models overwhelmingly favored highly visible brand-name drugs over recent FDA approvals or standard physician practices. Researchers estimated that following such AI-driven medical recommendations could increase annual acquisition costs in the US by approximately $55.6 million.