September 2023

SlimPajama-DC: Understanding Data Combinations for LLM Training

arXiv · Sep 19 · LLM Research

Researchers at MBZUAI release SlimPajama-DC, an empirical analysis of data combinations for pretraining LLMs using the SlimPajama dataset. The study examines the impact of global vs. local deduplication and the proportions of highly-deduplicated multi-source datasets. Results show that increased data diversity after global deduplication is crucial, with the best configuration outperforming models trained on RedPajama.

DGM-DR: Domain Generalization with Mutual Information Regularized Diabetic Retinopathy Classification

arXiv · Sep 18 · Research CV

This paper introduces a domain generalization (DG) method for Diabetic Retinopathy (DR) classification that maximizes mutual information using a large pretrained model. The method aims to address the challenge of domain shift in medical imaging caused by variations in data acquisition. Experiments on public datasets demonstrate that the proposed method outperforms state-of-the-art techniques, achieving a 5.25% improvement in average accuracy.

TII-SSRC-23 Dataset: Typological Exploration of Diverse Traffic Patterns for Intrusion Detection

arXiv · Sep 14 · Research NLP