Skip to content
GCC AI Research

Predicting and Explaining Cross-lingual Zero-shot and Few-shot Transfer in LLMs

MBZUAI · Notable

NLP LLM Research Arabic AI Ethics

Summary

Project LITMUS explores predicting cross-lingual transfer accuracy in multilingual language models, even without test data in target languages. The goal is to estimate model performance in low-resource languages and optimize training data for desired cross-lingual performance. This research aims to identify factors influencing cross-lingual transfer, contributing to linguistically fair MMLMs. Why it matters: Improving cross-lingual transfer is vital for creating more equitable and effective multilingual AI systems, especially for languages with limited resources.

Keywords

cross-lingual transfer · multilingual models · low-resource languages · fairness · LITMUS

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

From FusHa to Folk: Exploring Cross-Lingual Transfer in Arabic Language Models

arXiv ·

This paper explores cross-lingual transfer in Arabic language models, which are typically pretrained on Modern Standard Arabic (MSA) but expected to generalize to diverse dialects. The study uses probing on 3 NLP tasks and representational similarity analysis to assess transfer effectiveness. Results show transfer is uneven across dialects, partially linked to geographic proximity, and models trained on all dialects exhibit negative interference. Why it matters: The findings highlight challenges in cross-lingual transfer for Arabic NLP and raise questions about dialect similarity for model training.