多模态大模型

CAD-GPT: Synthesising CAD Construction Sequence with Spatial Reasoning-Enhanced Multimodal LLMs

Siyu Wang, Cailian Chen, Xinyi Le, Qimin Xu, Lei Xu, Yanzhou Zhang, Jie Yang

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2025 (Vol. 39, No. 8), pages 7880-7888. DOI: 10.1609/aaai.v39i8.32849

Journal ref Proc. of the AAAI Conf. on Artificial Intelligence, 39(8):7880-7888, 2025

Speculative Decoding Reimagined for Multimodal Large Language Models

Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Rongrong Ji

机构 * Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室、中华人民共和国教育部、厦门大学） 纠错

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 12 pages

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models

Woody Haosheng Gan, Deqing Fu, Julian Asilis, Ollie Liu, Dani Yogatama, Vatsal Sharan, Robin Jia, Willie Neiswanger

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Enhancing the Geometric Problem-Solving Ability of Multimodal LLMs via Symbolic-Neural Integration

Yicheng Pan, Zhenrong Zhang, Pengfei Hu, Jiefeng Ma, Jun Du, Jianshu Zhang, Quan Liu, Jianqing Gao, Feng Ma

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 5 figures

Bridging Compressed Image Latents and Multimodal Large Language Models

Chia-Hao Kao, Cheng Chien, Yu-Jen Tseng, Yi-Hsin Chen, Alessandro Gnutti, Shao-Yuan Lo, Wen-Hsiao Peng, Riccardo Leonardi

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ICLR 2025

Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, Lichao Sun

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models

Junyan Lin, Haoran Chen, Dawei Zhu, Xiaoyu Shen

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Comments Accepted to EMNLP 2024 Main Conference

Visual Grounding with Multi-modal Conditional Adaptation

Ruilin Yao, Shengwu Xiong, Yichen Zhao, Yi Rong

专题命中 其他多模态 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM MM 2024 [Oral]

VizECGNet: Visual ECG Image Network for Cardiovascular Diseases Classification with Multi-Modal Training and Knowledge Distillation

Ju-Hyeon Nam, Seo-Hyung Park, Su Jung Kim, Sang-Chul Lee

专题命中 其他多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted in International Conference on Image Processing (ICIP) 2024

The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective

Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng

专题命中 其他多模态 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Ongoing work. 21 pages. Related materials are continually maintained and available at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md

INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model

Yiwei Ma, Zhibin Wang, Xiaoshuai Sun, Weihuang Lin, Qiang Zhou, Jiayi Ji, Rongrong Ji

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM

Yixuan Wu, Yizhou Wang, Shixiang Tang, Wenhao Wu, Tong He, Wanli Ouyang, Philip Torr, Jian Wu

专题命中 其他多模态 ：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision

Hao Dong, Eleni Chatzi, Olga Fink

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ECCV 2024, code: https://github.com/donghao51/MOOSA

The Use of Multimodal Large Language Models to Detect Objects from Thermal Images: Transportation Applications

Huthaifa I. Ashqar, Taqwa I. Alhadidi, Mohammed Elhenawy, Nour O. Khanfar

专题命中 其他多模态 ：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Unified Modeling Enhanced Multimodal Learning for Precision Neuro-Oncology

Huahui Yi, Xiaofei Wang, Kang Li, Chao Li

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Multi-modal Deep Learning

Chen Yuhua

专题命中 其他多模态 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Master's thesis

Jailbreaking Attack against Multimodal Large Language Model

Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, Rong Jin

专题命中 其他多模态 ：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, Jingren Zhou

专题命中 其他多模态 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV、cs.CL

Enhancing the Spatial Awareness Capability of Multi-Modal Large Language Model

Yongqiang Zhao, Zhenyu Li, Zhi Jin, Feng Zhang, Haiyan Zhao, Chengfeng Dou, Zhengwei Tao, Xinhai Xu, Donghong Liu

专题命中 其他多模态 ：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.AI、cs.MM

ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model

Hanyao Huang, Ou Zheng, Dongdong Wang, Jiayi Yin, Zijin Wang, Shengxuan Ding, Heng Yin, Chuan Xu, Renjie Yang, Qian Zheng, Bing Shi

专题命中 其他多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CL、cs.AI

Journal ref Int J Oral Sci. 2023 Jul 28;15(1):29

Emotional Reaction Intensity Estimation Based on Multimodal Data

Shangfei Wang, Jiaqiang Wu, Feiyi Zheng, Xin Li, Xuewei Li, Suwen Wang, Yi Wu, Yanan Chang, Xiangyu Miao

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.AI

Heterogeneous Graph Learning for Multi-modal Medical Data Analysis

Sein Kim, Namkyeong Lee, Junseok Lee, Dongmin Hyun, Chanyoung Park

专题命中 其他多模态 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments AAAI 2023

A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training

Zhihao Fan, Zhongyu Wei, Jingjing Chen, Siyuan Wang, Zejun Li, Jiarong Xu, Xuanjing Huang

专题命中 其他多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Learning Abstract Representations through Lossy Compression of Multi-Modal Signals

Charles Wilmot, Gianluca Baldassarre, Jochen Triesch

专题命中 其他多模态 ：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI、cs.MM

Cross-Modal Information Maximization for Medical Imaging: CMIM

Tristan Sylvain, Francis Dutil, Tess Berthier, Lisa Di Jorio, Margaux Luck, Devon Hjelm, Yoshua Bengio

专题命中 其他多模态 ：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICASSP 2021

Learning Factorized Multimodal Representations

Yao-Hung Hubert Tsai, Paul Pu Liang, Amir Zadeh, Louis-Philippe Morency, Ruslan Salakhutdinov

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ICLR 2019

General-Purpose Multimodal Transformer meets Remote Sensing Semantic Segmentation

Nhi Kieu, Kien Nguyen, Sridha Sridharan, Clinton Fookes

专题命中 其他多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted to CVPR Workshop on Multimodal Learning for Earth and Environment 2023

Multimodal Shape Completion via Conditional Generative Adversarial Networks

Rundi Wu, Xuelin Chen, Yixin Zhuang, Baoquan Chen

专题命中 其他多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2020 (spotlight). Project page at https://chriswu1997.github.io/files/multimodal-pc/index.html

Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation

用于鲁棒多模态序列推荐的序列模态丢弃（Sequential Modality Dropout, SMD）

Guanqun Yang, Wenlong Zhang

专题命中 其他多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.MM

AI总结 本文提出序列模态丢弃（SMD），通过训练时随机擦除模态流提升多模态序列推荐器的鲁棒性，在四个骨干网络和亚马逊数据集上显著提升了缺失模态下的推荐准确率，且几乎不损失完整模态性能。

Comments Accepted at CIKM 2026. Code: https://github.com/guanqun-yang/SMD

TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory

TRAM：利用轨迹衍生辅助记忆增强多模态推理

Kang Liu, Zijing Wang, Yongkang Liu, Mengjie Zhao, Xiaocui Yang, Shi Feng, Yifei Zhang, Daling Wang

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL

AI总结 TRAM是一种无需训练的多模态推理增强方法，通过轨迹衍生辅助记忆整合推理信息，在4种MLRM变体的8个基准测试中提升了多模态推理性能。