机构
*
State Key Lab of Processors, Institute of Computing Technology, CAS(处理器国家重点实验室,计算技术研究所,中国科学院)
;
Jiangsu Key Laboratory of AI for Industries, Institute of AI for Industries, CAS(江苏人工智能工业重点实验室,人工智能工业研究所,中国科学院)
;
University of Chinese Academy of Sciences(中国科学院大学)
;
Cambricon Technologies(寒武科技)
;
Intelligent Software Research Center, Institute of Software, CAS(软件研究所智能软件研究中心,中国科学院)
;
University of Science and Technology of China(中国科学技术大学)
Multi-level context Modeling for consistent expert selection in Mixture-of-Experts
用于混合专家模型中一致专家选择的多层次上下文建模
Shuhan Huang, Naifan Zhang, Yuanbo Tang, Yang Li, Wai Kin Victor Chan
机构
*
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
;
School of AI, The Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳)人工智能学院)
机构
*
State Key Laboratory of Integrated Chips and Systems(集成芯片与系统国家重点实验室)
;
Research Institute of Intelligent Complex Systems(智能复杂系统研究院)
;
Frontier Institute of Chip and System(芯片与系统前沿院)
;
School of Mathematical Sciences and Shanghai Center for Mathematical Sciences(数学科学学院和上海数学中心)
;
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
;
State Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain Science, Institute of Brain Science(医学神经生物学国家重点实验室和教育部脑科学前沿中心,脑科学研究院)
CommentsMajor revision, retitled. The quasi-martingale conditions of v1 are relaxed to weaker signed conditions; adds empirical diagnostics showing a meta-learned Beta-Bernoulli BFT satisfies the signed condition