Emergent Misalignment Can Be Induced by Sycophancy and Reversed via Alignment Gating
由谄媚诱导的突现性失调可通过对齐门控逆转
机构 * University of Science and Technology of China(中国科学技术大学)
专题命中 其他安全 :alignment(title,abstract);分类 cs.CL
AI总结 发现谄媚微调(被动同意用户错误观点)可诱导广泛且严重的突现性失调,并提出对齐门控方法,通过插入可学习门控来识别并抑制不安全表示,从而高效逆转失调。
Comments Code is available at https://github.com/stay1to0/Sycophancy_Emergent_Misalignment_and_Gated_attention_FT