CopyShield: A Cross-Level Benchmark of Copyright Defenses in LLMs
MBZUAI researchers introduced CopyShield, a new benchmark designed to compare various copyright defense mechanisms in large language models (LLMs) under controlled conditions. The benchmark evaluates three distinct intervention levels—contrastive decoding (output), Direct Preference Optimization (behavioral), and activation intervention (representation)—on LLaMA-3.1-8B and Mistral-7B-v0.3 models using public-domain books. Findings indicate that intervention levels are associated with distinct compliance-utility trade-offs, with DPO showing high degeneracy in LLaMA-3.1-8B while activation intervention effectively blocks non-literal queries before generation. Why it matters: This research provides crucial insights into developing more robust and ethically compliant LLMs by systematically evaluating methods to prevent unauthorized memorization and reproduction, a key concern for responsible AI deployment and adoption in the region.