Skip to content
GCC AI Research

Evaluation of Adversarial Robustness in Arabic Language Models

arXiv · · Significant research

NLP LLM Arabic AI Research Security

Summary

This study evaluated the adversarial robustness of five state-of-the-art Arabic Language Models against various Arabic adversarial attacks at character, word, and sentence levels. It found that diacritic insertion could reduce model accuracy by up to 92%, while manipulating Arabic conjunctions led to a 58% accuracy degradation, and paraphrasing reduced performance by an average of 76%. While adversarial training improved overall resilience, particularly for MARBERT and AraBERT, challenges against character-level noise persist. Why it matters: These findings are crucial for understanding and mitigating security vulnerabilities in Arabic AI, guiding the development of more robust and safe Arabic NLP systems.

Keywords

Adversarial Robustness · Arabic Language Models · Adversarial Attacks · NLP Security · Arabic NLP

Read original article →

Get the weekly digest

Top AI stories from the GCC region, every week.

Related

Supporting Undotted Arabic with Pre-trained Language Models

arXiv ·

The paper examines the performance of pre-trained Arabic language models on Arabic text intentionally stripped of diacritical dots to evade content classification. It proposes methods to support these "undotted" texts without retraining the models. The proposed methods achieve nearly perfect performance on one downstream task. Why it matters: The research highlights a vulnerability in Arabic NLP and offers solutions to maintain performance in the face of adversarial text manipulation.