Skip to content
GCC AI Research

Building Arabic NLP from the Ground Up: Twenty Years of Lessons, Failures, and Open Problems

arXiv · · Significant research

Summary

This paper reflects on two decades of building NLP resources and research infrastructure for Arabic, an historically underserved language. The first decade focused on foundational linguistic infrastructure, while the second shifted towards computational social science and socially oriented applications. The authors highlight three lessons: dataset building is a social process, communities often matter more than shared tasks, and computational social science exposes challenges beyond traditional NLP training. Why it matters: The paper argues that the most difficult problems in developing NLP for underserved communities are social, institutional, and epistemic, offering critical insights for future research directions in Arabic AI.

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

A Panoramic Survey of Natural Language Processing in the Arab World

arXiv ·

This survey paper reviews the landscape of Natural Language Processing (NLP) research and applications in the Arab world. It discusses the unique challenges posed by the Arabic language, such as its morphological complexity and dialectal diversity. The paper also presents a historical overview of Arabic NLP and surveys various research areas, including machine translation, sentiment analysis, and speech recognition. Why it matters: The survey provides a comprehensive resource for researchers and practitioners interested in the current state and future directions of Arabic NLP, a field critical for enabling AI technologies to serve Arabic-speaking communities.

A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions

arXiv ·

This paper surveys the landscape of code-switched Arabic natural language processing, covering the mixture of Modern Standard Arabic, dialects, and foreign languages. It examines current efforts, challenges, and research gaps in the field. The survey also provides recommendations for future research directions in code-switched Arabic NLP. Why it matters: Understanding code-switching is crucial for developing effective language technologies that can handle the diverse linguistic landscape of the Arab world.

The Landscape of Arabic Large Language Models (ALLMs): A New Era for Arabic Language Technology

arXiv ·

This article surveys the landscape of Arabic Large Language Models (ALLMs), tracing their evolution from early text processing systems to sophisticated AI models. It highlights the unique challenges and opportunities in developing ALLMs for the 422 million Arabic speakers across 27 countries. The paper also examines the evaluation of ALLMs through benchmarks and public leaderboards. Why it matters: ALLMs can bridge technological gaps and empower Arabic-speaking communities by catering to their specific linguistic and cultural needs.

Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps

arXiv ·

This survey paper analyzes over 40 benchmarks used to evaluate Arabic large language models, categorizing them into Knowledge, NLP Tasks, Culture and Dialects, and Target-Specific evaluations. It identifies progress in benchmark diversity but also highlights gaps like limited temporal evaluation and cultural misalignment. The paper also examines methods for creating benchmarks, including native collection, translation, and synthetic generation. Why it matters: The survey provides a comprehensive reference for Arabic NLP research and offers recommendations for future benchmark development to better align with cultural contexts.