Remember to Forget: Gated Adaptive Positional Encoding
记住遗忘:门控自适应位置编码
机构 * Department of Computer Science and Technology, University of Cambridge(计算机科学与技术系,剑桥大学) ; Department of Computer, Control and Management Engineering, Sapienza University(计算机、控制与管理工程系,萨皮恩扎大学) ; Department of Information Engineering, University of Padova(信息工程系,帕多瓦大学)
AI总结 GAPE通过引入内容感知偏差优化注意力逻辑,在保持旋转几何的前提下,分离距离抑制与token重要性,提升长上下文鲁棒性。