Skip to content
GCC AI Research

Search

Results for "Benchmarks"

UAE’s Falcon 40B Dominates Leaderboard: Ranks #1 Globally in Latest Hugging Face Independent Verification of Open-source AI Models

TII ·

TII's Falcon 40B, a 40-billion-parameter open-source AI model, has ranked #1 on Hugging Face's Open LLM Leaderboard, surpassing models like LLaMA and StableLM. The leaderboard uses benchmarks like AI2 Reasoning Challenge, HellaSwag, MMLU, and TruthfulQA. Trained on one trillion tokens, Falcon 40B's weights are available for research and commercial use. Why it matters: This achievement positions the UAE as a leader in generative AI and promotes transparent, inclusive AI development.

In recognition of Sheikh Khalifa’s contribution to advancing science and technology, UAE President endorses launch of K2 Think, world’s most advanced open-source reasoning model - wam.ae

WAM ·

The UAE President has endorsed the launch of K2 Think, which is described as the world’s most advanced open-source reasoning model. This launch recognizes Sheikh Khalifa’s contributions to advancing science and technology within the UAE. The announcement signifies a major national initiative in the field of artificial intelligence development. Why it matters: This positions the UAE at the forefront of open-source AI innovation and advanced reasoning capabilities, potentially setting new benchmarks for global AI development.

Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models

arXiv ·

Researchers have introduced VISE (Visual Invariance Self-Evolution), a purely unsupervised framework designed to address 'visual under-conditioning' in self-evolving Large Multimodal Models (LMMs). VISE utilizes geometric and semantic invariance-based rewards to directly regularize the model's visual conditioning, ensuring it attends to visual content rather than relying on language priors. Trained on raw unlabeled images, experiments using Qwen3-VL-2B demonstrate significant performance gains, including +16.85 CIDEr on COCO and a 5.0-point reduction in object hallucination across 18 benchmarks. Why it matters: This research from MBZUAI offers a significant advancement in improving the visual reasoning capabilities and reliability of LMMs in unsupervised settings, making them more robust for real-world applications.

How Saudi Arabia is measuring AI readiness across government - Gulf Business

SDAIA ·

Saudi Arabia is reportedly implementing frameworks and methodologies to systematically measure AI readiness across its various government entities. These initiatives aim to establish clear benchmarks and track the progress of AI adoption and capability development within the public sector. The efforts involve defining key performance indicators and developing assessment models to guide the national AI strategy. Why it matters: This systematic approach is crucial for Saudi Arabia to effectively integrate AI into public services, improve governance, and achieve its Vision 2030 goals for technological advancement and digital transformation.

RightNow-Arabic-0.5B-Turbo: An Open Sub-1B Arabic Language Model via Vocabulary Injection and Edge-First Deployment

arXiv ·

RightNow-Arabic-0.5B-Turbo is a new 518M-parameter Arabic-specialized decoder LLM, built on Qwen2.5-0.5B, designed to bridge the gap between small multilingual and large Arabic-specialized models. Its development pipeline included adding 27,032 Arabic tokens via vocabulary injection, continued pretraining on 504M Arabic tokens, and fine-tuning with supervised instruction and direct preference optimization. The model achieved a 35.9% mean accuracy on three Arabic benchmarks (COPA-ar, Arabic HellaSwag, ArabicMMLU), outperforming all same-class open models and recovering 67% of SILMA-9B's mean accuracy at 1/18 the parameters, with all code and weights publicly released. Why it matters: This model significantly advances efficient Arabic NLP by providing a powerful, specialized sub-1B LLM suitable for edge deployment, making advanced Arabic AI more accessible and performant on resource-constrained devices.

State-of-the-Art Arabic Language Modeling with Sparse MoE Fine-Tuning and Chain-of-Thought Distillation

arXiv ·

Arabic-DeepSeek-R1 is an application-driven, open-source Arabic Large Language Model (LLM) that has achieved a new state-of-the-art (SOTA) across the Open Arabic LLM Leaderboard (OALL). The model utilizes a sparse Mixture-of-Experts (MoE) backbone and a four-phase Chain-of-Thought (CoT) distillation scheme, which incorporates Arabic-specific linguistic verification and regional ethical norms. It records the highest average score on the OALL suite and outperforms proprietary frontier systems like GPT-5.1 on a majority of benchmarks evaluating comprehensive Arabic language-specific tasks. Why it matters: This work offers a validated and cost-effective framework for developing high-performing, culturally-grounded AI for under-represented languages, addressing the digital equity gap.

Are Arabic Benchmarks Reliable? QIMMA's Quality-First Approach to LLM Evaluation

arXiv ·

QIMMA is introduced as a quality-assured Arabic LLM leaderboard that places systematic benchmark validation at its core. It employs a multi-model assessment pipeline combining automated LLM judgment with human review to identify and resolve quality issues in established Arabic benchmarks. The resulting evaluation suite comprises over 52,000 samples, predominantly grounded in native Arabic content, with transparent implementation via LightEval and EvalPlus. Why it matters: This initiative provides a more reliable and reproducible foundation for evaluating Arabic Large Language Models, addressing critical quality concerns in existing benchmarks.

UAE’s Technology Innovation Institute Revolutionizes AI Language Models With New Architecture

TII ·

Technology Innovation Institute (TII) has released Falcon Mamba 7B, a new large language model and the first State Space Language Model (SSLM) in its Falcon series. Falcon Mamba 7B is the top-ranked open-source SSLM globally, outperforming Meta's Llama 3.1 8B, Llama 3 8B, and Mistral’s 7B on HuggingFace benchmarks. SSLMs excel at understanding complex, evolving situations and have applications in NLP tasks like machine translation and text summarization. Why it matters: This release strengthens the UAE's position as an AI hub, demonstrating TII's commitment to pioneering research and open-source AI development in the region.