Researchers from MBZUAI, University of Washington, and other institutions presented studies at EMNLP 2024 exploring how LLMs represent cultures. A survey analyzed dozens of recent studies on LLMs and culture and proposes a new framework for future research. The survey found that there is no widely accepted definition of 'culture' in NLP, making it challenging to interpret how models represent culture through language. Why it matters: This highlights a key gap in the field and emphasizes the need for a more rigorous and consistent understanding of culture in AI, especially as LLMs become more globally integrated.
Todd Nims, a filmmaker born in Saudi Arabia, premiered his film "Joud" at KAUST's 2018 Winter Enrichment Program. The film, set in Saudi Arabia, explores the cycle of life in reverse and the meaning of "Joud" (generosity in the face of scarcity). Nims describes Saudi Arabia as a "magical place" due to its rich storytelling tradition. Why it matters: The article highlights KAUST's role in showcasing cultural works and supporting Saudi artists, though the AI relevance is limited.
A new dataset called ArabCulture is introduced to address the lack of culturally relevant commonsense reasoning resources in Arabic AI. The dataset covers 13 countries across the Gulf, Levant, North Africa, and the Nile Valley, spanning 12 daily life domains with 54 fine-grained subtopics. It was built from scratch by native speakers writing and validating culturally relevant questions. Why it matters: The dataset highlights the need for more culturally aware models and benchmarks tailored to the Arabic-speaking world, moving beyond machine-translated resources.
A new paper from MBZUAI introduces JEEM, a benchmark dataset for evaluating vision-language models on their understanding of images grounded in four Arabic-speaking societies (Jordan, UAE, Egypt, and Morocco) and their ability to use local dialects. The dataset comprises 2,178 images and 10,890 question-answer pairs reflecting everyday life and culturally specific scenes. Evaluation of several Arabic-capable models (Maya, PALO, Peacock, AIN, AyaV) and GPT-4o revealed that while models can generate fluent language, they struggle with genuine understanding, consistency, and relevance, especially when cultural context is important. Why it matters: This research highlights the challenges of building AI systems that can truly understand and interact with diverse cultures, emphasizing the need for culturally grounded datasets and evaluation metrics.
KAUST will host a Modern Saudi Art Exhibit from Arabian Wings (Jan 11-15), an Al-Balad 24 Photography Exhibition featuring work by Marina Kochetyga and Andrea Bachofen (Jan 11-16), and an East African Tingatinga art exhibition (Jan 18-24). The Al-Balad exhibit includes a video by Dr. Lorenzo Pareschi documenting a fire in the historic district. Why it matters: These art exhibits expose the KAUST community to diverse artistic styles and cultural perspectives, fostering cross-cultural understanding.
This paper introduces a method using Stable Diffusion XL (SDXL) fine-tuned with LoRA to generate culturally relevant coloring templates based on Emirati Al-Sadu weaving patterns for mental health therapy. The approach aims to leverage coloring therapy's stress-relieving benefits while embedding cultural resonance, potentially aiding in the treatment of Generalized Anxiety Disorder (GAD). Future research will explore the impact of Emirati heritage art on Emirati individuals using biosignals to assess engagement and effectiveness.
The paper introduces SaudiCulture, a new benchmark for evaluating the cultural competence of LLMs within Saudi Arabia, covering five major geographical regions and diverse cultural domains. The benchmark includes questions of varying complexity and distinguishes between common and specialized regional knowledge. Evaluations of five LLMs (GPT-4, Llama 3.3, FANAR, Jais, and AceGPT) revealed performance declines on region-specific questions, highlighting the need for region-specific knowledge in LLM training.