多模态大模型

AmCLR: Unified Augmented Learning for Cross-Modal Representations

Ajay Jagannath, Aayush Upadhyay, Anant Mehta

专题命中 图文多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 2 figures

Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tags

Daiqing Qi, Handong Zhao, Zijun Wei, Sheng Li

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Main Conference at EMNLP 2024

Multi-Modal interpretable automatic video captioning

Antoine Hanna-Asaad, Decky Aspandi, Titus Zaharia

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue

Zhangpu Li, Changhong Zou, Suxue Ma, Zhicheng Yang, Chen Du, Youbao Tang, Zhenjie Cao, Ning Zhang, Jui-Hsin Lai, Ruei-Sung Lin, Yuan Ni, Xingzhi Sun, Jing Xiao, Jieke Hou, Kai Zhang, Mei Han

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

MemeCLIP: Leveraging CLIP Representations for Multimodal Meme Classification

Siddhant Bikram Shah, Shuvam Shiwakoti, Maheep Chaudhary, Haohan Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted to EMNLP 2024 (Main)

Integrated Image-Text Based on Semi-supervised Learning for Small Sample Instance Segmentation

Ruting Chi, Zhiyi Huang, Yuexing Han

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Learning Multimodal Cues of Children's Uncertainty

Qi Cheng, Mert İnan, Rahma Mbarki, Grace Grmek, Theresa Choi, Yiming Sun, Kimele Persaud, Jenny Wang, Malihe Alikhani

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments SIGDIAL 2023

Improving Multi-modal Large Language Model through Boosting Vision Capabilities

Yanpeng Sun, Huaxin Zhang, Qiang Chen, Xinyu Zhang, Nong Sang, Gang Zhang, Jingdong Wang, Zechao Li

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.MM

White-box Multimodal Jailbreaks Against Large Vision-Language Models

Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, Yu-Gang Jiang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zujun Ma, Chao Zhang

专题命中 图文多模态 ：multimodal(title)；audio-visual(abstract)；分类 cs.CV、cs.CL

Who's in and who's out? A case study of multimodal CLIP-filtering in DataComp

Rachel Hong, William Agnew, Tadayoshi Kohno, Jamie Morgenstern

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Content warning: This paper discusses societal stereotypes and sexually-explicit material that may be disturbing, distressing, and/or offensive to the reader

Journal ref Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO 2024)

Design as Desired: Utilizing Visual Question Answering for Multimodal Pre-training

Tongkun Su, Jun Li, Xi Zhang, Haibo Jin, Hao Chen, Qiong Wang, Faqin Lv, Baoliang Zhao, Yin Hu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by MICCAI2024

MultiClimate: Multimodal Stance Detection on Climate Change Videos

Jiawen Wang, Longfei Zuo, Siyao Peng, Barbara Plank

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 5 pages, 1 figure

Stop Reasoning! When Multimodal LLM with Chain-of-Thought Reasoning Meets Adversarial Image

Zefeng Wang, Zhen Han, Shuo Chen, Fan Xue, Zifeng Ding, Xun Xiao, Volker Tresp, Philip Torr, Jindong Gu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Emotion Recognition with Vision-language Prompting and Modality Dropout

Anbin QI, Zhongliang Liu, Xinyong Zhou, Jinba Xiao, Fengrun Zhang, Qi Gan, Ming Tao, Gaozheng Zhang, Lu Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning

Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Multi-Modal Adapter for Vision-Language Models

Dominykas Seputis, Serghei Mihailov, Soham Chatterjee, Zehao Xiao

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs

Vardaan Pahuja, Weidi Luo, Yu Gu, Cheng-Hao Tu, Hong-You Chen, Tanya Berger-Wolf, Charles Stewart, Song Gao, Wei-Lun Chao, Yu Su

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 figures

Towards Deconfounded Image-Text Matching with Causal Inference

Wenhui Li, Xinqi Su, Dan Song, Lanjun Wang, Kun Zhang, An-An Liu

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments ACM MM

Journal ref 2023/10/26,Proceedings of the 31st ACM International Conference on Multimedia,6264-6273

Robust Domain Generalization for Multi-modal Object Recognition

Yuxin Qiao, Keqin Li, Junhong Lin, Rong Wei, Chufeng Jiang, Yang Luo, Haoyu Yang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 2 figures. This is a preprint version of the article. The final version will be published in the proceedings of the IEEE conference

Contextual Object Detection with Multimodal Large Language Models

Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, Chen Change Loy

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments IJCV 2024

Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation

Jingjing Xie, Yuxin Zhang, Mingbao Lin, Liujuan Cao, Rongrong Ji

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACMMM2024

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models

Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Under Review

Multimodal Misinformation Detection using Large Vision-Language Models

Sahar Tahmasebi, Eric Müller-Budack, Ralph Ewerth

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted for publication in: Conference on Information and Knowledge Management (CIKM) 2024

Mismatch Quest: Visual and Textual Feedback for Image-Text Misalignment

Brian Gordon, Yonatan Bitton, Yonatan Shafir, Roopal Garg, Xi Chen, Dani Lischinski, Daniel Cohen-Or, Idan Szpektor

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Journal ref ECCV 2024

Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Markus J. Buehler

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

Yongzhu Miao, Shasha Li, Jintao Tang, Ting Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments The paper has been accepted by ICME 2023

Vision-Language Consistency Guided Multi-modal Prompt Learning for Blind AI Generated Image Quality Assessment

Jun Fu, Wei Zhou, Qiuping Jiang, Hantao Liu, Guangtao Zhai

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Signal Processing Letter

Solution for SMART-101 Challenge of CVPR Multi-modal Algorithmic Reasoning Task 2024

Jinwoo Ahn, Junhyeok Park, Min-Jun Kim, Kang-Hyeon Kim, So-Yeong Sohn, Yun-Ji Lee, Du-Seong Chang, Yu-Jung Heo, Eun-Sol Kim

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability

Yejun Yoon, Seunghyun Yoon, Kunwoo Park

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ACL 2024 (findings), 16 pages