This study investigates methods to steer Arabic Large Language Models (LLMs) towards generating specific dialects, addressing the challenge of data scarcity for dialectal Arabic. Researchers identified sparse neuron populations encoding dialect-specific features and developed a vector-steering approach using dialect-specific activation directions. These inference-time methods allow for controlling dialectal output by amplifying or suppressing neuron activity or injecting specific vectors. Why it matters: This research offers a principled, interpretability-grounded framework to improve dialectal accuracy in Arabic LLMs without fine-tuning, crucial for enhancing their utility in the diverse Arabic-speaking world.
This paper explores cross-lingual transfer in Arabic language models, which are typically pretrained on Modern Standard Arabic (MSA) but expected to generalize to diverse dialects. The study uses probing on 3 NLP tasks and representational similarity analysis to assess transfer effectiveness. Results show transfer is uneven across dialects, partially linked to geographic proximity, and models trained on all dialects exhibit negative interference. Why it matters: The findings highlight challenges in cross-lingual transfer for Arabic NLP and raise questions about dialect similarity for model training.