MBZUAI increased faculty diversity and worked with global partners on application projects in 2023, including developing Jais (with Core42 and Cerebras) and Vicuna (with UC San Diego, UC Berkeley, CMU, and Stanford). They also launched Jais Climate, a bilingual LLM for climate intelligence, and LLM360, a framework for transparent LLM research. Why it matters: MBZUAI's involvement in open-source GenAI initiatives and partnerships positions the UAE as a key player in responsible AI development and talent creation.
Researchers from MBZUAI, UC Berkeley, CMU, Stanford, and UC San Diego collaborated to create Vicuna, an open-source chatbot that costs $300 to train, unlike ChatGPT which costs over $4 million. Vicuna achieves 90% of ChatGPT's subjective language quality while being far more energy-efficient and can run on a single GPU. It was fine-tuned from Meta AI’s LLaMA model using user-shared conversations and has gained significant traction on GitHub. Why it matters: This research demonstrates that high-quality chatbots can be developed at a fraction of the cost and environmental impact, opening up new possibilities for sustainable AI development in the region.
Paul Liang from CMU presented on machine learning foundations for multisensory AI, discussing a theoretical framework for modality interactions. The talk covered cross-modal attention and multimodal transformer architectures, and applications in mental health, pathology, and robotics. Liang's research aims to enable AI systems to integrate and learn from diverse real-world sensory modalities. Why it matters: This highlights the growing importance of multimodal AI research and its potential for advancements across various sectors in the region, including healthcare and robotics.
A CMU professor and MBZUAI affiliated faculty presented research on how LLMs store and use knowledge learned during pre-training. The study used a synthetic biography dataset to show that LLMs may not effectively use memorized knowledge at inference time, even with zero training loss. Data augmentation during pre-training can force the model to store knowledge in specific token embeddings. Why it matters: The research highlights limitations in LLM knowledge manipulation and extraction, with implications for improving model architectures and training strategies for more effective knowledge utilization in Arabic LLMs.
MBZUAI's AI Quorum launched its second workshop, "Building Ecosystems for AI at Scale," focusing on AI scalability and business applications. The first CASL workshop aims to define steps for organizations to become self-sufficient with AI and explore new use cases. Speakers include MBZUAI faculty and researchers from CMU, Stanford, KAUST, UC Berkeley, and Google. Why it matters: The workshop highlights the UAE's growing role in fostering AI innovation and bridging the gap between academic research and industry applications in the region.
A Mixture of Experts (MoE) layer is a sparsely activated deep learning layer. It uses a router network to direct each token to one of the experts. Yuanzhi Li, an assistant professor at CMU and affiliated faculty at MBZUAI, researches deep learning theory and NLP. Why it matters: This highlights MBZUAI's engagement with cutting-edge deep learning research, specifically in efficient model design.
Gus Xia, assistant professor of machine learning at MBZUAI, is exploring how teaching robots music can enhance their interaction with humans. Xia collaborates with robots on musical compositions as part of this research. He also holds affiliations at NYU Shanghai, Tandon, CILVR, and MARL, and has a Ph.D. from CMU. Why it matters: This interdisciplinary approach could lead to more intuitive and empathetic AI systems in the future.
MBZUAI President Eric Xing delivered a talk at Carnegie Mellon University on May 13, 2022, titled “From Learning, to Meta-Learning, to Lego-Learning — theory, systems, and engineering.” Xing discussed the development of a standard model for learning, inspired by the standard model in physics, which aims to unify various machine learning paradigms. Before joining MBZUAI, Xing was a professor at CMU and founder of Petuum Inc., an AI development platform company. Why it matters: This talk highlights MBZUAI's leadership in advancing theoretical frameworks for machine learning and its commitment to unifying different AI approaches.