VIA-SD: Verification via Intra-Model Routing for Speculative Decoding
VIA-SD:通过模型内路由进行推测解码的验证
Yuchen Xian, Yang He, Yunqiu Xu, Yi Yang
机构
*
ReLER, The State Key Lab of Brain Machine Intelligence, Zhejiang University(脑机智能国家重点实验室,浙江大学)
;
College of Artificial Intelligence, Zhejiang University(人工智能学院,浙江大学)
;
CFAR, Agency for Science, Technology and Research, Singapore(科学与技术研究局,新加坡)
;
National University of Singapore(新加坡国立大学)
机构
*
Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University(浙江大学控制科学与工程学院工业控制技术研究所)
;
Huzhou Institute, Zhejiang University, and Huzhou Key Laboratory of Autonomous System(浙江大学湖州研究院,湖州市自主系统重点实验室)
;
Zhejiang Zhongyan Industry Co., Ltd(浙江中烟工业有限责任公司)
;
Differential Robotics Technology Company(微分机器人技术有限公司)