MBZUAI researchers have introduced SURPRISE3D, a benchmark for evaluating 3D spatial reasoning in AI systems, along with a 3D Spatial Reasoning Segmentation (3D-SRS) task. The benchmark includes over 900 indoor scenes and 200,000 language queries paired with 3D masks, emphasizing spatial relationships over object naming. A companion paper, MLLM-For3D, explores adapting 2D multimodal LLMs for 3D reasoning. Why it matters: This work addresses a key limitation in current AI, pushing towards embodied AI that can understand and act in 3D environments based on human-like spatial reasoning.
MBZUAI researchers have developed "Tactile Skills," a new embodied AI framework enabling robots to rapidly learn complex tactile tasks. The framework combines expert process knowledge with reusable tactile control and adaptation components, reducing reliance on extensive datasets. Tested on 28 industrial tasks, the robots achieved nearly 100% success, demonstrating adaptability to changing conditions. Why it matters: This breakthrough offers a practical and scalable approach to robotic automation, potentially transforming robots into adaptable assistants across diverse industries in the GCC.
A research paper evaluates a challenge-based method for robotics awareness using an LLM-enabled humanoid robot, implemented with employees of AD Ports Group in the UAE. Participants interacted with the robot in a logistics-inspired task environment using voice commands interpreted through an LLM-based control framework. A post-event survey of 102 responses indicated high satisfaction (8.46/10), increased interest in robotics and AI (4.47/5), and improved understanding of human-robot collaboration (4.45/5). Why it matters: This study provides evidence for a replicable method of introducing robotics and embodied AI to non-specialist users in real industrial settings within the UAE, leveraging advanced LLM capabilities.
Muhammad Shafique from NYU Abu Dhabi discusses building energy-efficient and robust EdgeAI systems. The talk covers trends, challenges, and techniques for optimizing software and hardware stacks. These optimizations aim to enable embodied AI in autonomous systems, IoT-Healthcare, Industrial-IoT, and smart environments. Why it matters: The research addresses key challenges in deploying AI on resource-constrained edge devices in the GCC region, particularly regarding energy efficiency and security.
Dr. Hao Dong from Peking University presented research on addressing the challenge of limited large-scale training data in embodied AI, particularly for manipulation, task planning, and navigation. The presentation covered simulation learning and large models. Dr. Dong is a chief scientist of China's National Key Research and Development Program and an area chair/associate editor for NeurIPS, CVPR, AAAI, and ICRA. Why it matters: Overcoming data scarcity is crucial for advancing embodied AI research and enabling more sophisticated robotic applications in the region.
Ivan Laptev from INRIA Paris presented a talk at MBZUAI on embodied multi-modal visual understanding, covering advancements in video understanding tasks like question answering and captioning. The talk highlighted recent work on vision-language navigation and manipulation. He argued that detailed understanding of the physical world through vision is still in early stages, discussing open research directions related to robotics and video generation. Why it matters: The discussion of robotics applications and future research directions in embodied AI could influence the direction of AI research and development in the UAE, particularly at MBZUAI.
MBZUAI visiting professor of computer vision, Xiaojun Chang, has been named to Clarivate’s 2024 Highly Cited Researchers list, placing him in the top 1% of researchers worldwide. Chang has accumulated over 18,000 citations from around 200 papers, with his research focusing on multimodal foundation models and their applications to embodied AI and healthcare. Chang also holds positions at the University of Technology Sydney and RMIT University in Australia. Why it matters: This recognition highlights MBZUAI's growing impact and its role in fostering interdisciplinary collaboration and innovation in AI research, attracting further opportunities and collaborations.
MBZUAI Professor Ian Reid discusses his career in embodied AI, from early work on active vision at Oxford to current research. He highlights three key developments: cameras as geometric sensors, visual SLAM, and advancements in robot navigation. Reid distinguishes embodied AI from systems like ChatGPT, emphasizing its need for understanding and interaction with the physical world. Why it matters: The insights from a leading expert underscore the importance of embodied AI as the next frontier in intelligent systems and robotics in the region.