AI professional responsibility and the risks of hallucinations

The use of artificial intelligence in professional fields carries significant legal and ethical risks, particularly regarding 'hallucinations'—the generation of false or non-existent information presented with certainty. Professionals who incorporate AI-generated data, citations, or conclusions remain legally responsible for their accuracy.

Legal precedents highlight these risks. In 2023, a New York court fined lawyers for presenting fabricated jurisprudence generated by ChatGPT. In Chile, a court in Concepción sanctioned a lawyer for using non-existent rulings, and the Supreme Court suspended a lawyer for one month after she used AI-generated doctrinal citations.

The impact extends beyond the legal sector. In Wyoming, a journalist resigned after publishing fabricated statements attributed to real sources via AI. Additionally, Deloitte Australia returned part of a payment following a report that contained non-existent academic references and invented judicial citations. These incidents underscore that while AI offers speed, its value depends entirely on human verification and rigorous fact-checking.