The paper introduces Sparse-Quantized Representation (SpQR), a new compression format and quantization technique for large language models (LLMs). SpQR identifies outlier weights and stores them in higher precision while compressing the remaining weights to 3-4 bits. The method achieves less than 1% accuracy loss in perplexity for LLaMA and Falcon LLMs and enables a 33B parameter LLM to run on a single 24GB consumer GPU. Why it matters: This enables near-lossless compression of LLMs, making powerful models accessible on resource-constrained devices and accelerating inference without significant accuracy degradation.
Shamma Almazrouei from the Directed Energy Research Center (DERC) won the Best Oral Presentation Award at the 11th Global Conference on Material Sciences and Engineering (CMSE 2022) for her presentation on "Chromium Doped Tungsten Alloy for Plasma-Facing Components Formed by Compression Plasma Flows." Her research explored the structural and mechanical properties of chromium-doped tungsten alloys for plasma-facing components of fusion reactors. The study suggests that Cr-W alloys can be used to construct plasma-facing components that can withstand high thermal loads. Why it matters: This award recognizes impactful materials science research in the UAE related to fusion reactors, a critical area for future energy technologies.
This paper benchmarks multilingual and monolingual LLM performance across Arabic, English, and Indic languages, examining model compression effects like pruning and quantization. Multilingual models outperform language-specific counterparts, demonstrating cross-lingual transfer. Quantization maintains accuracy while promoting efficiency, but aggressive pruning compromises performance, particularly in larger models. Why it matters: The findings highlight strategies for scalable and fair multilingual NLP, addressing hallucination and generalization errors in low-resource languages.
KAUST Professor Marc Genton and his former postdoc Stefano Castruccio jointly won the 2017 Wilcoxon Award for their paper in Technometrics. Their paper, "Compressing an ensemble with statistical models: An algorithm for global 3D spatio-temporal temperature," details a data-compression scheme for climate simulations. The method reduces data-storage requirements and accelerates climate research capacity. Why it matters: This award highlights KAUST's contribution to statistical methods for climate modeling and big data analysis, particularly relevant for studying renewable energy resources in Saudi Arabia.
KAUST and TU Munich researchers have published a paper on a novel carbon capture technique. The technique focuses on converting CO2 directly from flue gas using catalytic systems, addressing the challenge of CO2 conversion requiring purification, compression, and high temperatures. Catalysts are often seen as viable green technology options to increase the renewable rates of CO2. Why it matters: This research has the potential to advance sustainable energy solutions by improving the efficiency and reducing the environmental costs associated with carbon capture and utilization.
MBZUAI graduate Daniel Gebre from Eritrea has developed iShrink, a compression pipeline that reduces the size of LLMs to run offline on mobile phones and edge devices. Gebre's master's thesis focused on enabling access to cutting-edge AI without requiring internet access, motivated by his experience in under-resourced regions. His journey to MBZUAI began with a Merit Scholarship awarded by the UAE Ministry of Education, leading him to pursue a Master of Science in Machine Learning. Why it matters: This research has the potential to democratize access to AI in areas with limited internet connectivity, particularly in regions like Gebre's homeland, Eritrea.
A presentation discusses using programmable network devices to reduce communication bottlenecks in distributed deep learning. It explores in-network aggregation and data processing to lower memory needs and increase bandwidth usage. The talk also covers gradient compression and the potential role of programmable NICs. Why it matters: Optimizing distributed deep learning infrastructure is critical for scaling AI model training in resource-constrained environments.
A CMU researcher, Dr. Hongyi Wang, presented an evaluation of gradient compression methods in distributed training, finding limited speedup in most realistic setups. The research identifies the root causes and proposes desirable properties for gradient compression methods to provide significant speedup. The talk was promoted by MBZUAI. Why it matters: Understanding the limitations of gradient compression techniques can help optimize distributed training strategies for AI models in the region.