MBZUAI researchers demonstrated a low-latency, multilingual multimodal AI system at GITEX that integrates speech, text, and visual capabilities for more lifelike human-machine conversation. The demo, led by Dr. Hisham Cholakkal, includes a mobile app where users can point their camera at an object and ask questions, receiving spoken answers in multiple languages. They are also integrating the model into a robot dog that can respond to voice commands. Why it matters: This work addresses key challenges in deploying LLMs to real-world applications in the Middle East, such as multilingual support and real-time responsiveness.
MBZUAI will present two assistive AI prototypes at GITEX 2025: smart glasses with a camera and eye tracker that identify objects and medication, and a brain-computer interface (BCI) device integrated with robotics to control a robotic dog's movements. The smart glasses use a multimodal large language model (LLM) to help visually impaired individuals, while the BCI aims to restore hands-free communication for people with mobility limitations. Hisham Cholakkal leads the research team, which received a Meta Regional Research Grant 2025 for its work on multimodal LLM for smart wearables. Why it matters: The research demonstrates the potential of AI to improve the quality of life for vulnerable populations and addresses the challenge of providing cost-effective care for aging societies.
Hisham Cholakkal has received MBZUAI’s inaugural Award for Teaching Excellence, launched by the University’s Center for Teaching and Learning. Cholakkal, an Assistant Professor of Computer Vision who joined MBZUAI in 2020, was recognized for his innovative teaching methods and positive impact on students. The award considers course evaluations and student feedback to recognize impactful, student-centered teaching. Why it matters: This award highlights MBZUAI's commitment to recognizing and promoting excellence in AI education within the region.