Sliding Window Attention Training for Efficient Large Language Models
机构 * City University of Hong Kong(香港城市大学) ; Xi’an Jiaotong University(西安交通大学) ; Jarvis Research Center, Tencent YouTu Lab(腾讯优图实验室研究部) ; Westlake University(西湖大学) ; University of Science and Technology of China(中国科学技术大学) ; Southern University of Science and Technology(南方科技大学)
Comments 14 pages, 5 figures