ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling
ZipMoE:通过无损压缩和缓存亲和调度实现高效的设备端MoE服务
Yuchen Yang, Yaru Zhao, Pu Yang, Shaowei Wang, Zhi-Hua Zhou
机构
*
School of Electronic Science and Engineering, Nanjing University, China.(南京大学电子科学与工程学院)
;
National Key Laboratory for Novel Software Technology, Nanjing University, China.(南京大学新型软件技术国家重点实验室)
Skill Weaving: Efficient LLM Improvement via Modular Skillpacks
技能编织:通过模块化技能包实现高效的LLM改进
Zhuo Li, Guodong Du, Zesheng Shi, Weiyang Guo, Weijun Yao, Yuan Zhou, Jiabo Zhang, Jing Li
机构
*
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳))
;
The Hong Kong Polytechnic University(香港理工大学)
;
Huawei Technologies Co., Ltd.(华为技术有限公司)
;
Shanghai Jiaotong University(上海交通大学)
Huanrong Liu, Chunlin Tian, Xuyang Wei, Qingbiao Li, Li Li
机构
*
Faculty of Science and Technology, University of Macau, Macau, China(澳门大学科学与技术学院)
;
School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China(电子科技大学信息与软件工程学院)
机构
*
University of California, San Diego(加州大学圣地亚哥分校)
;
University of Hong Kong(香港大学)
;
Columbia University(哥伦比亚大学)
;
Amazon(亚马逊)
;
City University of Hong Kong(香港城市大学)
CommentsThe content has been significantly revised and the author has also changed. Therefore, the paper will be withdrawn for revision and then uploaded after the completion of the modifications