Late-Layer Fusion is Enough: Dual-Path Vision Token Routing for Multimodal Large Language Models under Visual Saturation
晚期融合足矣:面向视觉饱和的多模态大语言模型的双路径视觉令牌路由
机构 * School of Mechanics and Engineering Science, Peking University(北京大学力学与工程科学学院) ; Department of Automation, Tsinghua University(清华大学自动化系)
专题命中 可控生成 :text-to-image(abstract);分类 cs.CV
AI总结 针对多模态大语言模型中视觉令牌在深层饱和的问题,提出双路径视觉令牌路由(DPVR-LF),在饱和点将视觉令牌路由至单层可训练分支,仅最后层融合,以约3%可训练参数保持性能并减少计算。
Comments 18 pages, 4 figures. Submitted to Pattern Recognition