Large Language Model integration in clinical medicine

Research regarding the application of Large Language Models (LLMs) in clinical settings has highlighted both significant security risks and potential advancements in medical data processing.

An interdisciplinary study from TU Dresden and associated institutions warned that the integration of LLMs into healthcare is outpacing the implementation of safety measures. The researchers identified risks such as ‘data poisoning’ and ‘prompt injections’ that could lead to clinical errors, such as failing to detect a tumor. They emphasized concerns regarding data privacy and the lack of institutional guidelines for informal model use.

Concurrently, other research has focused on the utility of LLMs for enhancing patient safety and data extraction. Developments include using synthetic data to optimize model performance for clinical named entity recognition and testing Retrieval-Augmented Generation (RAG) models in neurosurgery to reduce hallucinations. Additionally, new methodologies are being developed to use LLMs alongside ETL pipelines to more accurately categorize medical device reports from databases like OpenFDA.