LEGEND: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets
AAAI 2025
First author
Developed representation-based safety-margin annotation for preference data.
Annotates safety margins using projections along a safety direction in the model’s representation space.