Marc Pollefeys from ETH Zurich and Microsoft Spatial AI Lab will discuss building 3D environment representations for assisting humans and robots. The talk covers visual 3D mapping, localization, spatial data access, and navigation using geometry and learning-based methods. It also explores building rich 3D semantic representations for scene interaction via open vocabulary queries leveraging foundation models. Why it matters: Advancements in spatial AI and 3D scene understanding are critical for enabling more capable robots and AI assistants in various applications within the region.
Eyal Ofek of Microsoft Research is researching how to augment users' senses and use scene understanding to create more inclusive workspaces, especially for remote work. His work involves designing applications flexible to changing environments and personalized to each user. Ofek's background includes computer vision, augmented reality, and leading research groups at Microsoft. Why it matters: This research aims to improve remote collaboration and adapt technology to individual user needs, which could enhance productivity and inclusivity in the evolving work landscape of the GCC region.
MBZUAI researchers have introduced SURPRISE3D, a benchmark for evaluating 3D spatial reasoning in AI systems, along with a 3D Spatial Reasoning Segmentation (3D-SRS) task. The benchmark includes over 900 indoor scenes and 200,000 language queries paired with 3D masks, emphasizing spatial relationships over object naming. A companion paper, MLLM-For3D, explores adapting 2D multimodal LLMs for 3D reasoning. Why it matters: This work addresses a key limitation in current AI, pushing towards embodied AI that can understand and act in 3D environments based on human-like spatial reasoning.
MBZUAI has launched a Metaverse Lab led by Hao Li, focusing on integrating computer vision, graphics, and machine learning for metaverse applications. The lab aims to develop AI algorithms for photorealistic virtual humans and dynamic environment digitization. Pinscreen, Li's AI startup, previously created avatars for Expo 2020 Dubai. Why it matters: This initiative positions MBZUAI and the UAE as key players in the development of core technologies underpinning the metaverse and digital communication.
Ian Reid, a Professor of Computer Science at the University of Adelaide, gave a talk at MBZUAI on leveraging deep learning to go beyond geometric SLAM. The talk covered using prior domain knowledge to improve map and shape estimation and enabling navigation in unvisited environments. The research aims to turn cameras into devices for flexible, large-scale situational awareness or "Spatial AI" sensors. Why it matters: Integrating deep learning with SLAM could significantly advance robotic navigation and spatial understanding, with applications for autonomous systems in various industries.
The article discusses immersive analytics, which uses VR and AR to visualize data in 3D and embed it into the user's environment, and reviews systems and techniques from the Data Visualisation and Immersive Analytics lab at Monash University. It explores the concept of "embodied sensemaking" and its potential to improve how people work with complex data. Professor Tim Dwyer directs the Data Visualisation and Immersive Analytics Lab at Monash University. Why it matters: Immersive analytics could significantly enhance data comprehension and decision-making across various sectors in the Middle East, where large-scale projects and smart city initiatives generate vast datasets.
MBZUAI's Metaverse Lab is developing AI algorithms for photorealistic virtual humans and dynamic environments. Hao Li, Director of the lab, envisions using the metaverse for immersive learning experiences related to history and culture. He is also working on tools to prevent deepfakes and other cyberthreats. Why it matters: This research at MBZUAI aims to advance AI and immersive technologies for education and address potential risks in the metaverse.
Microsoft Azure AI CTO Dr. Xuedong Huang will speak at the MBZUAI Executive Program on AI-powered communications. Huang will share his experience in advancing Microsoft's AI stack, from deep learning infrastructure to new user experiences. He has over 170 U.S. patents and has contributed to speech technology, including Windows SAPI and Azure Speech. Why it matters: This talk can help foster knowledge transfer and collaboration between a global AI leader and the UAE's flagship AI university.