When Does Sparse MoE Help in Vision? The Role of Backbone Compute Leverage in Sparse Routing
何时稀疏MoE在视觉中起作用?背骨计算利用在稀疏路由中的作用
机构 * Department of Computer Science and Software Engineering(计算机科学与软件工程系) ; Auburn University(阿伯拉罕大学) ; Department of Information Management(信息管理系) ; National Central University(国立中央大学)
AI总结 研究稀疏top-k路由在视觉分类中的有效性,发现计算利用模式,指出背骨架构和多专家路由对性能的影响,通过实验验证关键因素。
Comments 24 pages (main + appendix), 8 figures, 18 tables. Under review at TMLR. Code and aggregate results: https://github.com/libophd/sparse-moe-vision-rho