The paper introduces ScoreAdv, a novel approach for generating natural adversarial examples (UAEs) using diffusion models. It incorporates an adversarial guidance mechanism and saliency maps to shift the sampling distribution and inject visual information. Experiments on ImageNet and CelebA datasets demonstrate state-of-the-art attack success rates, image quality, and robustness against defenses.
This paper introduces SemDiff, a novel method for generating unrestricted adversarial examples (UAEs) by exploring the semantic latent space of diffusion models. SemDiff uses multi-attribute optimization to ensure attack success while preserving the naturalness and imperceptibility of generated UAEs. Experiments on high-resolution datasets demonstrate SemDiff's superior performance compared to state-of-the-art methods in attack success rate and imperceptibility, while also evading defenses.
The paper introduces VENOM, a text-driven framework for generating high-quality unrestricted adversarial examples using diffusion models. VENOM unifies image content generation and adversarial synthesis into a single reverse diffusion process, enhancing both attack success rate and image quality. The framework incorporates an adaptive adversarial guidance strategy with momentum to ensure the generated adversarial examples align with the distribution of natural images.
This paper introduces Diffusion-BBO, a new online black-box optimization (BBO) framework that uses a conditional diffusion model as an inverse surrogate model. The framework employs an Uncertainty-aware Exploration (UaE) acquisition function to propose scores in the objective space for conditional sampling. The approach is shown theoretically to achieve a near-optimal solution and empirically outperforms existing online BBO baselines across 6 scientific discovery tasks.
Researchers from MBZUAI have developed XReal, a diffusion model for generating realistic chest X-ray images with precise control over anatomy and pathology location. The model utilizes an Anatomy Controller and a Pathology Controller to introduce spatial control in a pre-trained Text-to-Image Diffusion Model without fine-tuning. XReal outperforms existing X-ray diffusion models in realism, as evaluated by quantitative metrics and radiologists' ratings, and the code/weights are available.
Researchers from MBZUAI and other institutions have developed a new framework called STEREO to improve the safety of text-to-image diffusion models. STEREO uses a two-stage approach: STE (Search Thoroughly Enough) based on adversarial training and REO (Robustly Erase Once) for batch concept erasure. This framework aims to enhance safety without significantly impacting the model's performance on normal queries. Why it matters: The framework addresses vulnerabilities in AI image generation, reducing the creation of inappropriate images while preserving performance on harmless queries.
MBZUAI M.Sc. graduate Mohammad Hanan Ghani developed new techniques to improve text-to-image generation from long text prompts, combining large language models and diffusion models. Advised by Dr. Salman Khan, Ghani published three papers at ICLR, BMVC, and NeurIPS, with the ICLR paper focusing on generating images that accurately reflect detailed text descriptions. The new system improves upon existing techniques to generate images that closely follow the details of the input text. Why it matters: This research addresses a key limitation in current T2I models and advances the field of multimodal AI, potentially improving the capabilities of robots and autonomous devices.
MBZUAI faculty, researchers, and students will present 34 papers at the 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Fahad Khan is a co-author on 11 accepted papers, while Salman Khan and Shijian Lu have 10 and 9 papers, respectively. One paper focuses on person image synthesis via a denoising diffusion model, and another introduces PromptCAL for generalized novel category discovery. Why it matters: This large volume of acceptances at a top-tier conference highlights MBZUAI's growing prominence and research contributions in computer vision, with potential impact across various industries from online retail to autonomous driving.