LibrAI is an AI startup founded by Xudong Han, a University of Melbourne PhD graduate, in December 2023 after the release of ChatGPT. The company focuses on advancing AI safety and responsible AI practices, building on Han's prior work creating FairLib, an open-source toolkit for fairness in deep neural networks. LibrAI's team of seven aims to create practical solutions ensuring AI is both responsible and revolutionary. Why it matters: The establishment of a startup focused on AI safety highlights growing awareness of ethical considerations in AI development within the region.
MBZUAI researchers release OpenFactCheck, a unified framework to evaluate the factual accuracy of large language models. The framework includes modules for response evaluation, LLM evaluation, and fact-checker evaluation. OpenFactCheck is available as an open-source Python library, a web service, and via GitHub.
MBZUAI-based startup LibrAI has launched the Libra-Leaderboard, an evaluation framework for LLMs that assesses both capability and safety. The leaderboard evaluates 26 mainstream LLMs using 57 datasets, assigning scores based on bias, misinformation, and oversensitivity. LibrAI also launched the Interactive Safety Arena to engage the public and educate them on AI safety through adversarial prompt testing. Why it matters: The Libra-Leaderboard provides a benchmark for responsible AI development, emphasizing the importance of aligning AI capabilities with safety considerations in the rapidly evolving LLM landscape.
The ArabJobs dataset is a new corpus of over 8,500 Arabic job advertisements collected from Egypt, Jordan, Saudi Arabia, and the UAE. The dataset contains over 550,000 words and captures linguistic, regional, and socio-economic variation in the Arab labor market. It is available on GitHub and can be used for fairness-aware Arabic NLP and labor market research.
MBZUAI researchers introduce FAID, a fine-grained AI-generated text detection framework capable of classifying text as human-written, LLM-generated, or collaboratively written. FAID utilizes multi-level contrastive learning and multi-task auxiliary classification to capture authorship and model-specific characteristics, and can identify the underlying LLM family. The framework outperforms existing baselines, especially in generalizing to unseen domains and new LLMs, and includes a multilingual, multi-domain dataset called FAIDSet.