This paper presents a benchmark study of contrastive learning (CL) methods applied to Arabic social meaning tasks like sentiment analysis and dialect identification. The study compares state-of-the-art supervised CL techniques against vanilla fine-tuning across a range of tasks. Results indicate that CL methods outperform vanilla fine-tuning in most cases and demonstrate data efficiency. Why it matters: This work highlights the potential of contrastive learning for improving performance in Arabic NLP, especially in low-resource scenarios.
This paper introduces a self-supervised contrastive learning method for segmenting the left ventricle in echocardiography images when limited labeled data is available. The approach uses contrastive pretraining to improve the performance of UNet and DeepLabV3 segmentation networks. Experiments on the EchoNet-Dynamic dataset show the method achieves a Dice score of 0.9252, outperforming existing approaches, with code available on Github.
The paper introduces TimeHUT, a new method for learning time-series representations using hierarchical uniformity-tolerance balancing of contrastive representations. TimeHUT employs a hierarchical setup to learn both instance-wise and temporal information, along with a temperature scheduler to balance uniformity and tolerance. The method was evaluated on UCR, UAE, Yahoo, and KPI datasets, demonstrating superior performance in classification tasks and competitive results in anomaly detection.
Xiao Wang from Purdue University presented research on Adversarial Contrastive Learning (AdCo) and Cooperative-adversarial Contrastive Learning (CaCo) for improved self-supervised learning. He also discussed CryoREAD, a framework for building DNA/RNA structures from cryo-EM maps, and future work in deep learning for drug discovery. Wang's algorithms have impacted molecular biology, leading to new structure discoveries published in journals like Cell and Nature Microbiology. Why it matters: The research advances AI techniques for crucial tasks in molecular biology and drug discovery, with potential applications for institutions in the GCC region focused on healthcare and biotechnology.
This paper introduces Pulmonary Embolism Detection using Contrastive Learning (PECon), a supervised contrastive pretraining strategy using both CT scans and EHR data to improve feature alignment between modalities for better PE diagnosis. PECon pulls sample features of the same class together while pushing away features of other classes. The approach achieves state-of-the-art results on the RadFusion dataset, with an F1-score of 0.913 and AUROC of 0.943.
Researchers propose a universal anatomical embedding (UAE) framework for medical image analysis to learn appearance, semantic, and cross-modality anatomical embeddings. UAE incorporates semantic embedding learning with prototypical contrastive loss, a fixed-point-based matching strategy, and an iterative approach for cross-modality embedding learning. The framework was evaluated on landmark detection, lesion tracking and CT-MRI registration tasks, outperforming existing state-of-the-art methods.
A talk introduces a computational framework for learning a compact structured representation for real-world datasets, that is both discriminative and generative. It proposes to learn a closed-loop transcription between the distribution of a high-dimensional multi-class dataset and an arrangement of multiple independent subspaces, known as a linear discriminative representation (LDR). The optimality of the closed-loop transcription can be characterized in closed-form by an information-theoretic measure known as the rate reduction. Why it matters: The framework unifies concepts and benefits of auto-encoding and GAN and generalizes them to the settings of learning a both discriminative and generative representation for multi-class visual data.