June 2023

XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models

arXiv · Jun 13 · NLP LLM

MBZUAI researchers introduce XrayGPT, a conversational medical vision-language model for analyzing chest radiographs and answering open-ended questions. The model aligns a medical visual encoder (MedClip) with a fine-tuned large language model (Vicuna) using a linear transformation. To enhance performance, the LLM was fine-tuned using 217k interactive summaries generated from radiology reports.

Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models

arXiv · Jun 8 · CV LLM

Video-ChatGPT is a new multimodal model that combines a video-adapted visual encoder with a large language model (LLM) to enable detailed video understanding and conversation. The authors introduce a new dataset of 100,000 video-instruction pairs for training the model. They also develop a quantitative evaluation framework for video-based dialogue models.

Unscented Autoencoder

arXiv · Jun 8 · Research NLP

N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition

arXiv · Jun 5 · NLP Research