arXiv ·
A new benchmark, ViMUL-Bench, is introduced to evaluate video LLMs across 14 languages, including Arabic, with a focus on cultural inclusivity. The benchmark includes 8k manually verified samples across 15 categories and varying video durations. A multilingual video LLM, ViMUL, is also presented, along with a training set of 1.2 million samples, with both to be publicly released.
arXiv ·
This paper benchmarks multilingual and monolingual LLM performance across Arabic, English, and Indic languages, examining model compression effects like pruning and quantization. Multilingual models outperform language-specific counterparts, demonstrating cross-lingual transfer. Quantization maintains accuracy while promoting efficiency, but aggressive pruning compromises performance, particularly in larger models. Why it matters: The findings highlight strategies for scalable and fair multilingual NLP, addressing hallucination and generalization errors in low-resource languages.
MBZUAI ·
ParlaMint is a CLARIN ERIC flagship project focused on harmonizing multilingual corpora of parliamentary sessions. The newest version, published in October 2023, covers 26 European parliaments with linguistic annotations and machine translations to English. Maciej Ogrodniczuk, Head of Linguistic Engineering Group at the Institute of Computer Science, Polish Academy of Sciences, presented the project. Why it matters: While focused on European parliaments, the ParlaMint project provides a valuable model and infrastructure for creating comparable Arabic parliamentary corpora, which could enhance Arabic NLP research and political analysis in the Middle East.
MBZUAI ·
A new Bayesian matrix factorization approach is explored for performance prediction in multilingual NLP, aiming to reduce the experimental burden of evaluating various language combinations. The approach outperforms state-of-the-art methods in NLP benchmarks like machine translation and cross-lingual entity linking. It also avoids hyperparameter tuning and provides uncertainty estimates over predictions. Why it matters: Accurate performance prediction methods accelerate multilingual NLP research by reducing computational costs and improving experimental efficiency, especially valuable for Arabic NLP tasks.