Researchers develop mathematical formula to predict dangerous AI behavior

Physicists at George Washington University have developed a mathematical formula designed to anticipate when an artificial intelligence system might transition from providing helpful responses to generating harmful content. The research, published in the journal Patterns, focuses on the attention mechanisms of AI models to identify a “tipping point” where outputs become undesirable.

In testing across seven public models from three different companies, the formula correctly predicted these shifts in 18 out of 19 cases. Researchers noted that a chatbot may provide several acceptable responses before suddenly changing behavior, which can lead to misplaced user trust. The study highlights that a response can be objectively correct yet dangerous, such as providing misleading medical, legal, or military advice.

Separately, a study published in JAMA Network Open evaluated 21 AI models, including ChatGPT, Gemini, and Grok, regarding their clinical diagnostic capabilities. While the models were reasonably accurate with final diagnoses when provided with complete data, they failed significantly during the differential diagnosis phase—the process of determining which diseases to rule in or out. Medical professionals advise that these tools should not be used for clinical decisions without professional supervision.