This paper introduces MOTOR, a multimodal retrieval and re-ranking approach for medical visual question answering (MedVQA) that uses grounded captions and optimal transport to capture relationships between queries and retrieved context, leveraging both textual and visual information. MOTOR identifies clinically relevant contexts to augment VLM input, achieving higher accuracy on MedVQA datasets. Empirical analysis shows MOTOR outperforms state-of-the-art methods by an average of 6.45%.
MBZUAI researchers developed a new approach called Multimodal Optimal Transport via Grounded Retrieval (MOTOR) to improve the accuracy of vision-language models for medical image analysis. MOTOR combines retrieval-augmented generation (RAG) with an optimal transport algorithm to retrieve and rank relevant image and textual data. Testing on two medical datasets showed that MOTOR improved average performance by 6.45%. Why it matters: This technique addresses the challenges of limited specialized medical datasets and computational costs associated with training AI models for medical image interpretation, offering a more efficient and accurate solution.
Technology Innovation Institute (TII) has appointed Alessio Figali and Pierre-Louis Lions to the Board of Advisors of its Artificial Intelligence Cross-Center Unit (AICCU). Figali is known for his work in Optimal Transport with applications to Machine Learning, while Lions is renowned for his work in Mean Field Game Theory with applications for the design of AI based Distributed Networks. Both are Fields Medal recipients and will advise the AI Cross-Center Unit in areas related to Machine Learning and Distributed AI, respectively. Why it matters: The addition of these eminent scientists will bolster TII's AI research capabilities and credibility in shaping the future of AI in the region.