AI safety research at George Washington University

Researchers at George Washington University, led by physics professor Neil F. Johnson and doctoral candidate Frank Yingjie Huo, have developed a mathematical formula to predict when artificial intelligence chatbots might transition from helpful responses to generating harmful or uncontrolled content. Published in the journal Patterns, the study utilizes principles of physics and topology to model the AI attention mechanism as a dynamic competitive system.

The research identifies “tipping points” where specific sequences of user queries can trigger shifts toward problematic outputs, such as instructions for weapons or self-destructive behavior. This is noted as particularly critical for on-device AI and small language models (SLMs) that lack the real-time safety filters found in cloud-based services.

In subsequent testing across seven public models from three different companies, the formula successfully predicted these behavioral shifts in 18 out of 19 cases. The researchers cautioned that chatbots may provide several acceptable responses before a sudden change in behavior, which can lead to misplaced user trust. The study emphasizes that an AI response can be objectively correct yet dangerous, such as providing misleading military, legal, or medical advice.