The researchers introduce KAU-CSSL, the first continuous Saudi Sign Language (SSL) dataset focusing on complete sentences. They propose a transformer-based model using ResNet-18 for spatial feature extraction and a Transformer Encoder with Bidirectional LSTM for temporal dependencies. The model achieved 99.02% accuracy in signer-dependent mode and 77.71% in signer-independent mode, advancing communication tools for the SSL community.
Researchers from MBZUAI have developed EchoCoTr, a novel spatiotemporal deep learning method for estimating left ventricular ejection fraction (LVEF) from echocardiograms. EchoCoTr combines CNNs and vision transformers to overcome the limitations of each when applied to medical video data. The method achieves state-of-the-art results on the EchoNet-Dynamic dataset, demonstrating improved accuracy compared to existing approaches, with code available on GitHub.
MBZUAI researchers have developed an AI program using vision transformers that can learn a person's handwriting style and generate text in that style. The US Patent and Trademark Office recently granted a patent for this technology, which could aid individuals with writing impairments. The system overcomes limitations of previous GAN-based approaches by processing long-range dependencies in handwriting. Why it matters: This patented AI tool enhances personalized text generation and has potential applications in assistive technology and improving handwriting recognition models.
MBZUAI researchers developed AirCast, a novel AI model for improved air pollution forecasting, which won the best paper award at the TerraBytes workshop during ICML. AirCast fuses weather and chemistry data using a Vision Transformer and frequency-weighted MAE to better predict extreme events like Saharan dust storms. In tests across the Middle East and North Africa, AirCast reduced PM2.5 error by 33% compared to a persistence baseline and outperformed the CAMS physics model. Why it matters: Accurate air pollution forecasting is critical for public health in the GCC region, and this research demonstrates a significant advancement using AI to address this challenge.
MBZUAI researchers developed FeSViBS, a new federated split learning technique for vision transformers that addresses data scarcity and privacy concerns in healthcare image classification. The method combines federated learning and split learning to train models collaboratively without sharing sensitive patient data directly. It overcomes limitations of traditional centralized training and vulnerabilities in federated learning. Why it matters: This approach enables the development of AI-powered healthcare applications while adhering to stringent data privacy regulations, unlocking the potential of machine learning in medical imaging.
An MBZUAI team led by Ph.D. student Dmitry Demidov won the Best Student Paper Award at VISAPP 2023 for their work on fine-grained visual classification. Their paper, 'Salient Mask-Guided Vision Transformer for Fine-Grained Classification,' introduces SM-ViT, a technique using a salient mask to improve Vision Transformer accuracy. The model focuses on defining characteristics of objects, outperforming standard ViT architecture, even with fewer or lower-resolution images. Why it matters: This award recognizes MBZUAI's contribution to advancing computer vision, particularly in applications requiring nuanced object recognition, such as robotics and automated systems.
MBZUAI researchers presented "TransRadar," a study at WACV proposing new uses for radar in object identification. The study, led by Yahia Dalbah, explores fusing radar with other technologies to identify objects, particularly for autonomous vehicles. The "TransRadar" approach uses an adaptive-directional transformer for real-time multi-view radar semantic segmentation. Why it matters: This research addresses the limitations of radar by enhancing its object recognition capabilities, potentially improving the reliability of autonomous systems in adverse conditions.
MBZUAI researchers developed GroupMamba, a new set of state-space models (SSMs) for computer vision that addresses limitations in existing SSMs related to computational efficiency and optimization challenges. GroupMamba introduces a new layer called modulated group mamba, improving efficiency and stability. In benchmark tests, GroupMamba performed as well as similar SSM systems, but more efficiently, offering a backbone for tasks like image classification, object detection, and segmentation. Why it matters: This research aims to bridge the gap between vision transformers and CNNs by improving SSMs, potentially leading to more efficient and powerful computer vision models.