PGN: Design and Implementation of a Vision-Language Navigation System Based on Pangu Multimodal Foundation Model
PGN:基于盘古多模态基础模型的视觉语言导航系统的设计与实现
机构 * University of Electronic Science and Technology of China(电子科技大学) ; School of Information and Communication Engineering(信息与通信工程学院)
专题命中 图文多模态 :multimodal(title,abstract);multimodal foundation model(title);分类 cs.AI
AI总结 研究基于盘古多模态基础模型设计实现视觉语言导航系统PGN,训练分两阶段,先对齐视觉与语言编码器,再使模型适应专家轨迹,结合多种计算方式,在特定评估下取得一定指标,量化了离线专家动作对齐情况。
Comments 6 pages, 5 figures