AI models evaluated for clinical knowledge and ultrasound measurement

Recent research highlights advancements in applying artificial intelligence to medical diagnostics and clinical knowledge assessment. One study conducted on a Japanese cohort compared various deep learning architectures for automated carotid intima-media thickness (cIMT) measurement via ultrasound. The carotid attention-based double-stage UNet (cAtt-dsUNet) demonstrated high accuracy, achieving commercial-grade correlation and overlap, though it required more training time and GPU memory than simpler models.

In a separate development, researchers introduced MultiMedQA, a new benchmark designed to evaluate the clinical knowledge of large language models (LLMs) across professional exams and consumer queries. Testing on the 540-billion parameter PaLM model and its instruction-tuned variant, Flan-PaLM, showed that Flan-PaLM achieved state-of-the-art accuracy on multiple-choice medical datasets, including a 67.6% accuracy rate on US Medical License Exam questions. Despite these gains, human evaluations identified remaining gaps in model responses, prompting the use of instruction prompt tuning to better align LLMs with medical domains.