US healthcare groups urge FDA to hold AI developers accountable

National healthcare organizations, including the American Hospital Association and the American Medical Association, have submitted comments to the Food and Drug Administration (FDA) regarding the accountability of generative AI medical devices. These groups argue that developers, rather than healthcare providers, should be held responsible for the safety and performance of AI tools such as chatbots and medical information synthesizers.

Providers expressed concern that monitoring these rapidly advancing technologies would place an undue burden on medical practices, particularly small and rural ones. There are also fears that assuming responsibility could increase exposure to malpractice lawsuits. Furthermore, hospitals noted that a lack of transparency regarding how AI models function makes it difficult for providers to identify potential flaws.

Separately, regulatory guidelines clarify that the FDA regulates medical devices that utilize AI rather than the AI technology itself. Companies must navigate specific pathways such as 510(k) clearance, De Novo classification, or premarket approval. Marketing claims must strictly align with authorized indications, with cleared devices being described as ‘cleared’ rather than ‘approved’ unless they have undergone the specific premarket approval process.