The paper introduces FanarGuard, a bilingual moderation filter for Arabic and English language models that considers both safety and cultural alignment. A dataset of 468K prompt-response pairs was created and scored by LLM judges on harmlessness and cultural awareness to train the filter. The first benchmark targeting Arabic cultural contexts was developed to evaluate cultural alignment. Why it matters: FanarGuard advances context-sensitive AI safeguards by integrating cultural awareness into content moderation, addressing a critical gap in current alignment techniques.
Researchers introduce AraDiCE, a benchmark for Arabic Dialect and Cultural Evaluation, comprising seven synthetic datasets in various dialects and Modern Standard Arabic (MSA). The benchmark includes approximately 45,000 post-edited samples and evaluates LLMs on dialect comprehension, generation, and cultural awareness across the Gulf, Egypt, and Levant. Results show that Arabic-specific models like Jais and AceGPT outperform multilingual models on dialectal tasks, but challenges remain in dialect identification, generation, and translation. Why it matters: This benchmark and associated datasets will help improve LLMs' ability to understand and generate diverse Arabic dialects and cultural contexts, addressing a significant gap in current models.
MBZUAI researchers, in collaboration with over 70 researchers, have created the Culturally diverse Visual Question Answering (CVQA) benchmark to evaluate cultural understanding in multimodal LLMs. The CVQA dataset includes over 10,000 questions in 31 languages and 13 scripts, testing models on images of local dishes, personalities, and monuments. Testing of several multimodal LLMs on the CVQA benchmark revealed significant challenges, even for top models. Why it matters: This benchmark highlights the need for AI models to better understand diverse cultures, promoting fairness and relevance across different languages and regions.
Salim T. S. Al-Hassani from the University of Manchester presented at KAUST's 2019 Winter Enrichment Program about the contributions of Muslim civilization to science and engineering. The lecture highlighted inventions like early clocks from Muslim heritage, including Al-Jazari's elephant clock. Al-Hassani aims to address the neglect of non-European cultures' impact on humanity. Why it matters: The talk emphasizes the historical significance of Islamic contributions to science and technology, relevant for promoting STEM education and cultural awareness in the region.
Thamar Solorio of MBZUAI served as general chair of EMNLP 2024, which hosted over 4,000 attendees. MBZUAI researchers presented nearly 50 studies, including one co-authored by Solorio and Monojit Choudhury that received an Outstanding Paper Award. Key themes included cultural awareness, machine-generated content detection, and LLM empathy and cultural representation. Why it matters: MBZUAI's strong presence at EMNLP highlights its growing influence in the international NLP research community and its focus on culturally aware AI.