This article discusses retrieval augmentation in text generation, where information retrieved from an external source is used to condition predictions. It references recent work on retrieval-augmented image captioning, showing that model size can be greatly reduced when training data is available through retrieval. The author intends to continue this work focusing on the intersection of retrieval augmentation and in-context learning, and controllable image captioning for language learning materials. Why it matters: This research direction has the potential to improve transfer learning in vision-language models, which could be especially relevant for downstream applications in Arabic NLP and multimodal tasks.
Akhil Arora from EPFL presented a framework for AI-assisted knowledge navigation, focusing on understanding and enhancing human navigation on Wikipedia. The framework includes methods for modeling navigation patterns, identifying knowledge gaps, and assessing their causal impact. He also discussed applications beyond Wikipedia, such as multimodal knowledge navigation assistants and multilingual knowledge gap mitigation. Why it matters: This research has the potential to improve information systems by making online knowledge more accessible and navigable, especially for platforms like Wikipedia that serve as critical resources for global knowledge sharing.
Researchers proposed Utility-Aligned Embeddings (UAE), a new framework designed to enhance Retrieval-Augmented Generation (RAG) by merging the precision of LLM re-ranking with the efficiency of dense vector retrieval. UAE trains a bi-encoder to imitate an LLM utility distribution using a Utility-Modulated InfoNCE objective, injecting graded utility signals directly into the embedding space. On the QASPER benchmark, UAE improved retrieval Recall@1 by 30.59% and was over 180 times faster than efficient LLM re-ranking methods while preserving competitive performance. Why it matters: This approach offers a practical way to significantly improve the accuracy and speed of RAG systems by providing more reliable contexts at scale without heavy computational cost.
This paper introduces MOTOR, a multimodal retrieval and re-ranking approach for medical visual question answering (MedVQA) that uses grounded captions and optimal transport to capture relationships between queries and retrieved context, leveraging both textual and visual information. MOTOR identifies clinically relevant contexts to augment VLM input, achieving higher accuracy on MedVQA datasets. Empirical analysis shows MOTOR outperforms state-of-the-art methods by an average of 6.45%.