Skip to content
GCC AI Research

Efficient and inclusive NLP: An instruction-based approach to improve language models

MBZUAI · Notable

NLP Research LLM Arabic AI Ethics

Summary

MBZUAI Assistant Professor Alham Fikri Aji is presenting research at EACL 2024 on efficient NLP for low-resource languages. The study uses knowledge distillation, transferring knowledge from a larger model (ChatGPT) to a smaller one using synthetic instruction data. The goal is to achieve similar performance with less computational resources, focusing on underrepresented languages. Why it matters: This work addresses the need for more accessible and inclusive NLP technologies, especially for languages lacking extensive datasets and computational resources.

Keywords

MBZUAI · NLP · low-resource languages · knowledge distillation · ChatGPT

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

Towards Inclusive NLP: Assessing Compressed Multilingual Transformers across Diverse Language Benchmarks

arXiv ·

This paper benchmarks multilingual and monolingual LLM performance across Arabic, English, and Indic languages, examining model compression effects like pruning and quantization. Multilingual models outperform language-specific counterparts, demonstrating cross-lingual transfer. Quantization maintains accuracy while promoting efficiency, but aggressive pruning compromises performance, particularly in larger models. Why it matters: The findings highlight strategies for scalable and fair multilingual NLP, addressing hallucination and generalization errors in low-resource languages.