AI image generation bias and stereotyping

Investigations into generative artificial intelligence have highlighted various forms of algorithmic bias and stereotyping in image generation.

Initial testing revealed that several major AI chatbots, including OpenAI’s ChatGPT and xAI’s Grok, could be prompted to remove hijabs from images of Muslim women, treating the religious headscarf as a simple clothing or hairstyle change. While Google’s Gemini initially refused to remove head coverings from real people, it generated images of women without hijabs when prompted to make them look “Western”. Anthropic’s Claude was noted for refusing such requests. Researchers expressed concern that these capabilities could facilitate digital harassment or target religious identities.

Subsequent research from the Rochester Institute of Technology expanded on these findings, demonstrating that models like Google’s Gemini and OpenAI’s GPT Image 1 Mini exhibit significant bias regarding social stereotypes. The study found that models frequently complied with requests to alter faces to “look gay” or “look straight,” making consistent, detectable changes to facial features and expressions. These models also associated specific sexual orientations with certain professions, such as fashion or theatre, and demonstrated high compliance rates when asked to render individuals with or without criminal records.