Risks in medical artificial intelligence implementation

Overview

Recent research has highlighted critical reliability and security concerns regarding the use of artificial intelligence in the medical field.

In one study, researchers from Asan Medical Center and Samsung Changwon Hospital found that generative AI-produced medical education materials contained errors even after passing automated verification. A manual review of radiology flashcards revealed an error rate of approximately 1.0%, exceeding the researchers’ safety threshold of 0.3%. The findings suggest that expert human oversight remains essential to ensure accuracy in medical education.

Separately, investigations by the Technical University of Munich, Imperial College London, and the Hasso Plattner Institute identified significant privacy vulnerabilities. The study demonstrated that Membership Inference Attacks (MIAs) could identify specific patients within certain datasets with nearly 100% probability, indicating that individual patient data remains highly susceptible to exploitation.

Entities

European Union · npj Digital Medicine · Samsung Changwon Hospital · Imperial College London · Technical University of Munich

Timeline

1 day ago

[HEALTH] 2 sources
Researchers identify privacy risks in medical AI models
Researchers from TU Munich and other institutions have found that medical AI models are vulnerable to attacks that can identify specific patients with nearly 100% accuracy.
3 days ago

[TECHNOLOGY] 2 sources
Medical AI education materials show errors despite automated verification
Researchers found that generative AI-produced medical education materials had a 1.0% error rate even after passing automated checks, highlighting the need for expert human oversight.

Sources

bokuennews.com · deutschesgesundheitsportal.de · hin.company · technologyreview.de