This paper introduces GigaBERT, a customized bilingual BERT model pre-trained for Arabic NLP and English-to-Arabic zero-shot transfer learning. The study evaluates GigaBERT's performance on four information extraction tasks: named entity recognition, part-of-speech tagging, argument role labeling, and relation extraction. Results show that GigaBERT outperforms mBERT, XLM-RoBERTa, and AraBERT in both supervised and zero-shot transfer settings. Why it matters: GigaBERT advances Arabic NLP by providing a high-performing, publicly available model tailored for the complexities of the Arabic language and cross-lingual applications.
Researchers at the American University of Beirut (AUB) have released AraBERT, a BERT model pre-trained specifically for Arabic language understanding. The model was trained on a large Arabic corpus and compared against multilingual BERT and other state-of-the-art methods. AraBERT achieved state-of-the-art performance on several tested Arabic NLP tasks including sentiment analysis, named entity recognition, and question answering. Why it matters: This release provides the Arabic NLP community with a high-performing, open-source language model, facilitating further research and development.
Technology Innovation Institute (TII) in the UAE has launched Falcon 180B, an open access large language model with 180 billion parameters trained on 3.5 trillion tokens. Falcon 180B ranks first on the Hugging Face Leaderboard for pretrained LLMs, outperforming Meta's LLaMA 2 and nearing the performance of OpenAI's GPT-4 and Google's PaLM 2. The model is available for research and commercial use under the 'Falcon 180B TII License', based upon Apache 2.0. Why it matters: This release strengthens the UAE's position in AI development and promotes open access to advanced AI technology, fostering innovation and collaboration.
The paper introduces AraPoemBERT, an Arabic language model pretrained exclusively on 2.09 million verses of Arabic poetry. AraPoemBERT was evaluated against five other Arabic language models on tasks including poet's gender classification (99.34% accuracy) and poetry sub-meter classification (97.79% accuracy). The model achieved state-of-the-art results in these and other downstream tasks, and is publicly available on Hugging Face. Why it matters: This specialized model advances Arabic NLP by providing a new state-of-the-art tool tailored for the nuances of classical Arabic poetry.
G42 has launched Nanda 87B, an open-source Hindi-English LLM developed by MBZUAI in collaboration with Inception and Cerebras. Nanda 87B is built upon Llama-3.1 70B and trained on a dataset with over 65 billion Hindi tokens. The model is engineered for real-world use being fluent in formal Hindi, casual speech, and Hinglish, and is designed for translation, summarization, instruction-following, and transliteration tasks. Why it matters: This release marks a major advancement in creating inclusive AI technology tailored for one of the world's largest linguistic communities.
The paper introduces AraGPT2, a suite of pre-trained transformer models for Arabic language generation, with the largest model (AraGPT2-mega) containing 1.46 billion parameters. Trained on a large Arabic corpus of internet text and news, AraGPT2-mega demonstrates strong performance in synthetic news generation and zero-shot question answering. To address the risk of misuse, the authors also released a discriminator model with 98% accuracy in detecting AI-generated text. Why it matters: This release of both the model and discriminator fills a critical gap in Arabic NLP and encourages further research and applications in the field.