George Washington University researchers develop AI safety formula

Researchers at George Washington University have developed a mathematical formula designed to predict when an artificial intelligence chatbot might transition from providing helpful responses to generating harmful or uncontrolled content. The study, published in the journal Patterns, was led by physics professor Neil F. Johnson and doctoral candidate Frank Yingjie Huo.

The researchers approached the problem using principles of physics and topology, treating the AI's attention mechanism as a dynamic competitive system. By modeling the algorithm's behavior, they identified “tipping points” where the sequence of user queries can trigger a shift toward problematic content, such as instructions for weapons or self-destructive behavior.

This discovery is particularly significant for the rise of on-device AI and small language models (SLMs) integrated into smartphones. Unlike large models running on cloud servers, these local models often lack the real-time safety filters and oversight provided by internet-connected services, making predictive safety mechanisms essential for offline device security.