FDA-cleared medical AI lacks testing for real-world patient benefits

An analysis of FDA-authorized artificial intelligence medical devices reveals a significant gap in testing for real-world patient benefits. Research published in PLOS Digital Health indicates that out of 1,357 AI-based medical devices authorized by the U.S. Food and Drug Administration, only three had been tested to determine if they actually improve patient health outcomes, such as reducing death rates, strokes, or hospitalizations.

While many AI systems are used for surgical planning, cardiovascular risk calculation, and imaging interpretation, the current authorization framework often only requires developers to demonstrate “substantial equivalence” to existing devices. This requirement does not mandate proof of improved health outcomes or equitable benefits across diverse subgroups. The study found that only 34 devices were included in registered clinical trials, and most studies were conducted in highly resourced healthcare systems, frequently excluding pregnant women, adults over 75, and non-English speakers.

In a related study regarding the development of these technologies, data suggests that bringing an AI Software as a Medical Device (SaMD) through the 510(k) clearance process requires approximately 42 person-months of specialized work across eleven different roles, while a De Novo pathway requires closer to 75 person-months.