Minor First, Major Last: A Depth-Induced Implicit Bias of Sharpness-Aware Minimization
先浅后深:一种由深度诱导的sharpness-aware minimization的隐式偏见
机构 * Graduate School of AI, KAIST(韩国成均馆大学人工智能研究生院) ; Mobilint, Inc.(Mobilint公司)
AI总结 该研究探讨了在训练线性可分二分类问题时,sharpness-aware minimization (SAM) 的隐式偏见,发现对于深度L=2的情况,SAM的行为与深度L=1时不同,展示了sequential feature amplification现象。
Comments Accepted to ICLR 2026, 84 pages, 35 figures