arXiv ·
Researchers at MBZUAI have demonstrated a method called "Data Laundering" to artificially boost language model benchmark scores using knowledge distillation. The technique covertly transfers benchmark-specific knowledge, leading to inflated accuracy without genuine improvements in reasoning. The study highlights a vulnerability in current AI evaluation practices and calls for more robust benchmarks.
MBZUAI ·
Researchers from MBZUAI, University of British Columbia, and Monash University have created LaMini-LM, a collection of small language models distilled from ChatGPT. LaMini-LM is trained on a dataset of 2.58M instructions and can be deployed on consumer laptops and mobile devices. The smaller models perform almost as well as larger counterparts while addressing security concerns. Why it matters: This work enables the deployment of LLMs in resource-constrained environments and enhances data security by reducing reliance on cloud-based LLMs.
MBZUAI ·
MBZUAI had 22 papers accepted at ICLR 2023, with faculty Kun Zhang co-authoring seven of them. Yuanzhi Li, an affiliated assistant professor at MBZUAI, received an honorable mention for his paper on knowledge distillation. Additionally, a paper co-authored by MBZUAI President Eric Xing was recognized as a top 5% paper at the conference. Why it matters: MBZUAI's strong presence at a top-tier machine learning conference like ICLR demonstrates the university's growing influence and research capabilities in the global AI landscape.
arXiv ·
The paper introduces a framework for camel farm monitoring using a combination of automated annotation and fine-tune distillation. The Unified Auto-Annotation framework uses GroundingDINO and SAM to automatically annotate surveillance video data. The Fine-Tune Distillation framework then fine-tunes student models like YOLOv8, transferring knowledge from a larger teacher model, using data from Al-Marmoom Camel Farm in Dubai.
MBZUAI ·
MBZUAI Assistant Professor Alham Fikri Aji is presenting research at EACL 2024 on efficient NLP for low-resource languages. The study uses knowledge distillation, transferring knowledge from a larger model (ChatGPT) to a smaller one using synthetic instruction data. The goal is to achieve similar performance with less computational resources, focusing on underrepresented languages. Why it matters: This work addresses the need for more accessible and inclusive NLP technologies, especially for languages lacking extensive datasets and computational resources.