SPOT: Sparse Probing and Outcome Calibration for On-Policy Distillation
SPOT:面向在线策略蒸馏的稀疏探测与结果校准
Zikun Qu, Min Zhang, Mingze Kong, Zhiwei Shang, Zhengyu Chen, Yikun Ban, Shuang Qiu, Zhongxiang Dai
机构
*
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
;
East China Normal University(华东师范大学)
;
Beihang University(北京航空航天大学)
;
City University of Hong Kong(香港城市大学)
机构
*
Northeastern University, Boston, MA, USA
;
The Chinese University of Hong Kong, Hong Kong, China
;
Peking University, Beijing, China
;
Westlake University, Hangzhou, China
;
Harvard University, Cambridge, MA, USA
;
Purdue University, West Lafayette, IN, USA
;
University of Oxford, Oxford, United Kingdom
机构
*
University of California San Diego(加州大学圣地亚哥分校)
;
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
;
Peking University(北京大学)
;
University of California, Los Angeles(加州大学洛杉矶分校)
;
California Institute of Technology(加州理工学院)
;
ETH Zurich(苏黎世联邦理工学院)
Commentspost conference revision version at ICML; update: removed CGYRO due to bug in cases search. Will add back soon; Make the title consistent w/ pdf