A recent study questions the necessity of deep ensembles, which improve accuracy and match larger models. The study demonstrates that ensemble diversity does not meaningfully improve uncertainty quantification on out-of-distribution data. It also reveals that the out-of-distribution performance of ensembles is strongly determined by their in-distribution performance. Why it matters: The findings suggest that larger, single neural networks can replicate the benefits of deep ensembles, potentially simplifying model deployment and reducing computational costs in the region.
An associate professor of Statistics at the University of Toronto gave a talk on how ensemble learning stabilizes and improves the generalization performance of an individual interpolator. The talk focused on bagged linear interpolators and introduced the multiplier-bootstrap-based bagged least square estimator. The multiplier bootstrap encompasses the classical bootstrap with replacement as a special case, along with a Bernoulli bootstrap variant. Why it matters: While the talk occurred at MBZUAI, the content is about ensemble learning which is a core area for improving AI model performance, and is of general interest to the AI research community.
The paper introduces a novel method for short-term, high-resolution traffic prediction, modeling it as a matrix completion problem solved via block-coordinate descent. An ensemble learning approach is used to capture periodic patterns and reduce training error. The method is validated using both simulated and real-world traffic data from Abu Dhabi, demonstrating superior performance compared to other algorithms.
A Mixture of Experts (MoE) layer is a sparsely activated deep learning layer. It uses a router network to direct each token to one of the experts. Yuanzhi Li, an assistant professor at CMU and affiliated faculty at MBZUAI, researches deep learning theory and NLP. Why it matters: This highlights MBZUAI's engagement with cutting-edge deep learning research, specifically in efficient model design.
Søren Brunak presented deep learning approaches for analyzing disease trajectories using data from 7-10 million patients in Denmark and the USA. The models predict future outcomes like mortality and specific diagnoses, such as pancreatic cancer, using 15-40 years of patient data. Disease trajectories and explainable AI can generate hypotheses for molecular-level investigations into causal aspects of disease progression. Why it matters: This research demonstrates the potential of large-scale patient data and AI to improve disease prediction and generate hypotheses for further investigation into disease mechanisms relevant to regional healthcare systems.
A new brain tumor segmentation method based on convolutional neural networks is proposed for the BraTS-GoAT challenge. The method employs the MedNeXt architecture and model ensembling to segment tumors in brain MRI scans from diverse populations. Experiments on the unseen validation set demonstrate promising results with an average DSC of 85.54%.
The study compares deep learning models trained via transfer learning from ImageNet (TII-models) against those trained solely on medical images (LMI-models) for disease segmentation. Results show that combining outputs from both model types can improve segmentation performance by up to 10% in certain scenarios. A repository of models, code, and over 10,000 medical images is available on GitHub to facilitate further research.
Daisuke Kihara from Purdue University presented a seminar at MBZUAI on using deep learning for biomolecular structure modeling. His lab is developing 3D structure modeling methods, especially for cryo-electron microscopy (cryo-EM) data. They are also working on RNA structure prediction and peptide docking using deep neural networks inspired by AlphaFold2. Why it matters: Applying advanced deep learning techniques to biomolecular structure prediction can accelerate drug discovery and our understanding of molecular functions.