This research addresses the underexplored area of aligning Arabic Large Language Models (LLMs) with country-specific cultural behavioral preferences. The authors propose ArabPref, the first large-scale pan-Arab preference dataset designed for cultural behavior alignment conditioned on country. ArabPref includes both a culturally grounded preference dataset for training and evaluation, and a multiple-choice benchmark for assessing culturally aligned behavior across various Arab nations, with all test data validated by native speakers. Why it matters: This dataset and benchmark provide a crucial foundation for systematically studying and improving the cultural nuance and appropriateness of LLMs for the diverse Arab world.
The paper introduces FanarGuard, a bilingual moderation filter for Arabic and English language models that considers both safety and cultural alignment. A dataset of 468K prompt-response pairs was created and scored by LLM judges on harmlessness and cultural awareness to train the filter. The first benchmark targeting Arabic cultural contexts was developed to evaluate cultural alignment. Why it matters: FanarGuard advances context-sensitive AI safeguards by integrating cultural awareness into content moderation, addressing a critical gap in current alignment techniques.
MBZUAI researchers presented a method for cross-cultural transfer learning to improve language models' understanding of diverse Arab cultures. They used in-context learning and demonstration-based reinforcement (DITTO) to transfer cultural knowledge between countries. Experiments showed up to 34% improvement in performance on cultural understanding benchmarks using only a few demonstrations. Why it matters: This research addresses the gap in cultural understanding of Arabic language models, especially for smaller Arab countries, and provides a novel transfer learning approach.