AI hallucinations found in over 2,000 court decisions

Research by Damien Charlotin, a senior research fellow at HEC Paris, reveals that generative AI hallucinations, such as fake court citations, have appeared in more than 2,000 court decisions globally since April 2023. Of these cases, over 1,300 occurred in the United States. While ChatGPT has been cited in over 100 instances, Google’s Gemini and Bard models were cited in 10 cases.

Most of these incidents resulted in warnings, with fewer than 160 leading to disciplinary action. Charlotin noted that while there was an exponential rise in such cases initially, the frequency and severity of AI hallucinations appear to have reached a plateau due to improved tools and increased awareness.

Amidst these findings, Google is promoting its legal AI tools as a “complement” to legal teams rather than a replacement. Google’s general counsel, Halimah DeLaine Prado, stated that the practice of law relies on human judgment and noted that AI already assists their legal department with tasks like contract redlining, e-discovery, and litigation preparation.