UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases
arXiv · · Significant research
Summary
Researchers introduce UnsafeChain, a new safety alignment dataset designed to improve the safety of large reasoning models (LRMs) by focusing on 'hard prompts' that elicit harmful outputs. The dataset identifies and corrects unsafe completions into safe responses, exposing models to unsafe behaviors and guiding their correction. Fine-tuning LRMs on UnsafeChain demonstrates enhanced safety and preservation of general reasoning ability compared to existing datasets like SafeChain and STAR-1.
Keywords
large reasoning models · safety alignment · chain-of-thought · UnsafeChain · hard prompts
Get the weekly digest
Top AI stories from the GCC region, every week.