OpenAI adds safeguards after Mindgard finds ChatGPT can generate graphic sexual and violent images

British AI‑security startup Mindgard discovered that a tiny modification of a publicly shared, seemingly harmless prompt allowed the latest public version of ChatGPT to produce graphic images depicting gore, sexual violence and explicit nudity. The researchers demonstrated the vulnerability by generating scenes such as blood‑covered injuries, restrained and frightened figures, and sexualized portrayals, all without an explicit request for such content.

OpenAI said it investigated the issue and introduced additional layers of protection to block the offending prompt, emphasizing its multi‑layered safety system that combines automated filters and human review. Mindgard’s team, however, reported that further subtle prompt changes still bypassed the new safeguards, underscoring the ongoing “cat‑and‑mouse” challenge of securing generative AI models. The incident highlights the difficulty of preventing AI systems from producing harmful visual outputs and the need for continuous red‑team testing and rapid mitigation.

The findings have spurred calls for stronger AI‑safety measures and continuous monitoring as image‑generation capabilities become more widely accessible.