多模态大模型 - arXivDaily 专题

2508.00332 2025-08-04 cs.CL 79%

Improving Multimodal Contrastive Learning of Sentence Embeddings with Object-Phrase Alignment

Kaiyan Zhao, Zhongtao Miao, Yoshimasa Tsuruoka

机构 * The University of Tokyo（东京大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Work in progress

2507.23188 2025-08-01 cs.CV 79%

Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space

Shiyao Yu, Zi-An Wang, Kangning Yin, Zheng Tian, Mingyuan Zhang, Weixin Si, Shihao Zou

机构 * Southern University of Science and Technology（南方科技大学）； Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）； University of Chinese Academy of Sciences（中国科学院大学）； ShanghaiTech University（上海科技大学）； Nanyang Technological University（南洋理工大学）； Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology（计算机科学与控制工程学院，深圳大学先进技术学院）

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE TMM 2025

2507.20259 2025-07-29 cs.CV 79%

L-MCAT: Unpaired Multimodal Transformer with Contrastive Attention for Label-Efficient Satellite Image Classification

Mitul Goswami, Mrinal Goswami

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2507.20189 2025-07-29 eess.SP cs.AI cs.LG q-bio.NC 79%

NeuroCLIP: A Multimodal Contrastive Learning Method for rTMS-treated Methamphetamine Addiction Analysis

Chengkai Wang, Di Wu, Yunsheng Liao, Wenyao Zheng, Ziyi Zeng, Xurong Gao, Hemmings Wu, Zhoule Zhu, Jie Yang, Lihua Zhong, Weiwei Cheng, Yun-Hsuan Chen, Mohamad Sawan

机构 * CenBRAIN Neurotech Center of Excellence, School of Engineering, Westlake University（西溪大学工程学院先进神经技术中心）； School of Data Science, Xiamen University Malaysia（马来西亚厦门大学数据科学学院）； Department of Neurosurgery, Second Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第二医院神经外科）； Department of Education and Correction, Zhejiang Gongchen Compulsory Isolated Detoxification Center（浙江省公检强制隔离戒毒所教育矫正部）； Zhejiang Liangzhu Compulsory Isolated Detoxification Center（浙江省良渚强制隔离戒毒所）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

2408.00970 2025-07-25 cs.MM 79%

Multimodal Fusion via Hypergraph Autoencoder and Contrastive Learning for Emotion Recognition in Conversation

Zijian Yi, Ziming Zhao, Zhishu Shen, Tiehua Zhang

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.MM

Comments Accepted by ACM MULTIMEDIA 2024

2507.17533 2025-07-24 cs.CV 79%

Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding

Liwen Liu, Weidong Yang, Lipeng Ma, Ben Fei

机构 * Fudan University（复旦大学）； The Chinese University of Hong Kong（香港中文大学）

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

2504.07942 2025-07-22 cs.CV 79%

MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation

Nico Catalano, Stefano Samele, Paolo Pertino, Matteo Matteucci

机构 * Politecnico di Milano（米兰理工大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2507.14738 2025-07-22 cs.CV 79%

MultiRetNet: A Multimodal Vision Model and Deferral System for Staging Diabetic Retinopathy

Jeannie She, Katie Spivakovsky

机构 * MIT（麻省理工学院）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2507.12998 2025-07-18 cs.CV cs.LG 79%

Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning

Zihua Zhao, Feng Hong, Mengxi Chen, Pengyi Chen, Benyuan Liu, Jiangchao Yao, Ya Zhang, Yanfeng Wang

机构 * Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学合作中位创新中心）； School of AI, Shanghai Jiao Tong University（上海交通大学人工智能学院）； Shanghai AI Laboratory（上海人工智能实验室）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2501.19043 2025-07-16 cs.CV 79%

Self-Supervised Cross-Modal Text-Image Time Series Retrieval in Remote Sensing

Genc Hoxha, Olivér Angyal, Begüm Demir

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2507.09174 2025-07-15 cs.CL 79%

RAMA: Retrieval-Augmented Multi-Agent Framework for Misinformation Detection in Multimodal Fact-Checking

Shuo Yang, Zijian Yu, Zhenzhe Ying, Yuqin Dai, Guoqing Wang, Jun Lan, Jinfeng Xu, Jinze Li, Edith C. H. Ngai

机构 * The University of Hong Kong（香港大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

2506.16035 2025-07-15 cs.LG cs.AI cs.IR 79%

Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding

Vishesh Tripathi, Tanmay Odapally, Indraneel Das, Uday Allu, Biddwan Ahmed

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments 11 pages, 1 Figure, 1 Table

2507.02357 2025-07-04 cs.CL 79%

Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models

Christian Jaumann, Annemarie Friedrich, Rainer Lienhart

机构 * University of Augsburg（奥格斯堡大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at 5th Workshop on Scholarly Document Processing @ ACL 2025

2507.01066 2025-07-03 cs.IR cs.CV cs.LG 79%

Embedding-based Retrieval in Multimodal Content Moderation

Hanzhong Liang, Jinghao Shi, Xiang Shen, Zixuan Wang, Vera Wen, Ardalan Mehrani, Zhiqian Chen, Yifan Wu, Zhixin Zhang

专题命中跨模态检索：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments Camera ready for SIGIR 2025

2501.01644 2025-07-01 cs.CL cs.LG 79%

Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs

Tien Dang, Viet Thanh Duy Nguyen, Minh Tuan Le, Truong-Son Hy

机构 * University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）； Washington University in St. Louis（华盛顿大学圣路易斯分校）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

2506.22056 2025-06-30 cs.AI 79%

Universal Retrieval for Multimodal Trajectory Modeling

Xuan Zhang, Ziyan Jiang, Rui Meng, Yifei Leng, Zhenbang Xiao, Zora Zhiruo Wang, Yanyi Shang, Dehan Kong

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments 18 pages, 3 figures, accepted by Workshop on Computer-use Agents @ ICML 2025

2506.21056 2025-06-27 cs.CV 79%

SAMURAI: Shape-Aware Multimodal Retrieval for 3D Object Identification

Dinh-Khoi Vo, Van-Loc Nguyen, Minh-Triet Tran, Trung-Nghia Le

机构 * University of Science, VNU-HCM（科学大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2506.20070 2025-06-26 cs.IR cs.LG cs.MM 79%

Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision

KMA Solaiman, Bharat Bhargava

机构 * Department of Computer Science Purdue University West Lafayette, IN 47906, USA（计算机科学系帕克森大学西拉法叶市，印第安纳州 47906，美国）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.MM

Comments Submitted to ICDE'24. An earlier version of this paper appeared on TechRxiv: https://www.techrxiv.org/doi/full/10.36227/techrxiv.21990284.v1, uploaded on February 05, 2023

2506.17302 2025-06-24 cs.CV cs.LG 79%

Fine-Scale Soil Mapping in Alaska with Multimodal Machine Learning

Yijun Lin, Theresa Chen, Colby Brungard, Grunwald Sabine, Sue Ives, Matt Macander, Timm Nawrocki, Yao-Yi Chiang, Nic Jelinski

机构 * University of Minnesota（明尼苏达大学）； New Mexico State University（新墨西哥州立大学）； University of Florida（佛罗里达大学）； ABR, Inc.（ABR公司）； University of Alaska-Anchorage（阿拉斯加大学安克雷奇分校）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 12 pages, Submitted to SIGSPATIAL 2025

2506.12411 2025-06-17 cs.CR cs.CV 79%

InverTune: Removing Backdoors from Multimodal Contrastive Learning Models via Trigger Inversion and Activation Tuning

Mengyuan Sun, Yu Li, Yuchen Liu, Bo Du, Yunjie Ge

机构 * 1 School of Cyber Science ； Engineering, Wuhan University 0.3em 2 School of Computer Science, Wuhan University 0.3em 3 Institute for Math \& AI, Wuhan University

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2506.11499 2025-06-16 cs.CL 79%

On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval

Seongbo Jang, Seonghyeon Lee, Dongha Lee, Hwanjo Yu

机构 * Myongji University（明州大学）； Kyungpook National University（庆北国立大学）； Yonsei University（延世大学）； Pohang University of Science and Technology（坡山科学技术大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments 9 pages, 1 figure

2408.11526 2025-06-12 cs.AI 79%

RConE: Rough Cone Embedding for Multi-Hop Logical Query Answering on Multi-Modal Knowledge Graphs

Mayank Kharbanda, Rajiv Ratn Shah, Raghava Mutharaju

机构 * IIIT-Delhi（德里印度理工学院）

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted in TKDE (June 2025) as regular paper

2408.04211 2025-06-12 cs.CL cs.IR 79%

MMREC: LLM Based Multi-Modal Recommender System

Jiahao Tian, Jinman Zhao, Zhenkai Wang, Zhicheng Ding

机构 * Georgia Institute of Technology（佐治亚理工学院）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）； University of Toronto（多伦多大学）； Columbia University（哥伦比亚大学）

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

2506.06912 2025-06-10 cs.CV 79%

Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM

Olivier Papillon, Rafik Goubran, James Green, Julien Larivière-Chartier, Caitlin Higginson, Frank Knoefel, Rébecca Robillard

机构 * Bruyère Health Research Institute（布里埃尔健康研究中心）； Institute for Mental Health Research at the Royal Ottawa Hospital（皇家渥太华医院心理健康研究所）； School of Psychology, University of Ottawa（心理学系，渥太华大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Submitted to IEEE MeMeA 2025

2407.19719 2025-06-03 cs.CV 79%

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Jiaxin Zhang, Yunqin Li, Tomohiro Fukuda, Bowen Wang

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 15 pages, 10 figures

2505.21973 2025-05-29 cs.MM 79%

Towards Structure-aware Model for Multi-modal Knowledge Graph Completion

Linyu Li, Zhi Jin, Yichi Zhang, Dongming Jin, Chengfeng Dou, Yuanpeng He, Xuan Zhang, Haiyan Zhao

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.MM

2505.19952 2025-05-27 cs.CV cs.IR 79%

Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval

Rong-Cheng Tu, Wenhao Sun, Hanzhe You, Yingjie Wang, Jiaxing Huang, Li Shen, Dacheng Tao

机构 * College of Computing and Data Science, Nanyang Technological University, Singapore（南洋理工大学计算机与数据科学学院）； School of Information Science and Technology, University of Science and Technology of China, Hefei, China（中国科学技术大学信息科学与技术学院）； Sun Yat-sen University Shenzhen Campus, School of Cyber Science and Technology, Shenzhen, China（中山大学深圳校区计算机科学与技术学院）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2505.19509 2025-05-27 cs.LG cs.AI 79%

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

Yifan Jia, Kailin Jiang, Yuyang Liang, Qihan Ren, Yi Xin, Rui Yang, Fenze Feng, Mingcai Chen, Hengyang Lu, Haozhe Wang, Xiaoye Qu, Dongrui Liu, Lizhen Cui, Yuntao Du

机构 * Joint SDU-NTU Centre for Artificial Intelligence Research&School of Software（山东大学与南京工业大学人工智能研究中心及软件学院）； University of Science and Technology of China（中国科学技术大学）； Shanghai Jiaotong University（上海交通大学）； Nanjing University（南京大学）； Nanjing University of Posts and Telecommunications（南京邮电大学）； Jiangnan University（江南大学）； The Hong Kong University of Science and Technology（香港科学与技术大学）； Shanghai AI Laboratory（上海人工智能实验室）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments The source code is available at https://github.com/MLLMKCBENCH/MLLMKC

2406.11193 2025-05-21 cs.CL 79%

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））； The Hong Kong University of Science and Technology（香港科学与技术大学）； Tongji University（同济大学）

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by the Main Conference of Empirical Methods in Natural Language Processing (EMNLP) 2024

2505.14014 2025-05-21 cs.CV 79%

EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation

Zelin Zhang, Tao Zhang, KediLI, Xu Zheng

机构 * University of Sydney（悉尼大学）； University of Technology Sydney（悉尼技术大学）； HKUST(GZ)（香港科技大学（广州））

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

1. 跨模态检索 3450 篇

Improving Multimodal Contrastive Learning of Sentence Embeddings with Object-Phrase Alignment

Multi-Modal Motion Retrieval by Learning a Fine-Grained Joint Embedding Space

L-MCAT: Unpaired Multimodal Transformer with Contrastive Attention for Label-Efficient Satellite Image Classification

NeuroCLIP: A Multimodal Contrastive Learning Method for rTMS-treated Methamphetamine Addiction Analysis

Multimodal Fusion via Hypergraph Autoencoder and Contrastive Learning for Emotion Recognition in Conversation

Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding

MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation

MultiRetNet: A Multimodal Vision Model and Deferral System for Staging Diabetic Retinopathy

Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning

Self-Supervised Cross-Modal Text-Image Time Series Retrieval in Remote Sensing

RAMA: Retrieval-Augmented Multi-Agent Framework for Misinformation Detection in Multimodal Fact-Checking

Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal Document Understanding

Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models

Embedding-based Retrieval in Multimodal Content Moderation

Multimodal Contrastive Representation Learning in Augmented Biomedical Knowledge Graphs

Universal Retrieval for Multimodal Trajectory Modeling

SAMURAI: Shape-Aware Multimodal Retrieval for 3D Object Identification

Multimodal Information Retrieval for Open World with Edit Distance Weak Supervision

Fine-Scale Soil Mapping in Alaska with Multimodal Machine Learning

InverTune: Removing Backdoors from Multimodal Contrastive Learning Models via Trigger Inversion and Activation Tuning

On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval

RConE: Rough Cone Embedding for Multi-Hop Logical Query Answering on Multi-Modal Knowledge Graphs

MMREC: LLM Based Multi-Modal Recommender System

Sleep Stage Classification using Multimodal Embedding Fusion from EOG and PSM

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Towards Structure-aware Model for Multi-modal Knowledge Graph Completion

Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval

Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation