多模态大模型

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models

Zikang Liu, Kun Zhou, Wayne Xin Zhao, Dawei Gao, Yaliang Li, Ji-Rong Wen

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments under review

MedCLIP-SAMv2: Towards Universal Text-Driven Medical Image Segmentation

Taha Koleilat, Hojat Asgariandehkordi, Hassan Rivaz, Yiming Xiao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 2 figures, 6 tables

Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations

Kangyu Zhu, Ziyuan Qin, Huahui Yi, Zekun Jiang, Qicheng Lao, Shaoting Zhang, Kang Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NAACL 2025 Main Conference

BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks

Yunhan Zhao, Xiang Zheng, Lin Luo, Yige Li, Xingjun Ma, Yu-Gang Jiang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Journal ref ICLR 2025, BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks. In Proceedings of the International Conference on Learning Representations (ICLR), 2025

Evaluating Vision-Language Models for Emotion Recognition

Sree Bhattacharyya, James Z. Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NAACL 2025 Findings

Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generation

Jihyo Kim, Seulbi Lee, Sangheum Hwang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICLR 2025. The first two authors contributed equally

MultiChartQA: Benchmarking Vision-Language Models on Multi-Chart Problems

Zifeng Zhu, Mengzhao Jia, Zhihan Zhang, Lang Li, Meng Jiang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments NAACL 2025, 19 pages, 10 figures

Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion

Marco Mistretta, Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, Andrew D. Bagdanov

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at ICLR 2025

What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning

Yifan Du, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, Jinpeng Wang, Chuyuan Wang, Mingchen Cai, Ruihua Song, Ji-Rong Wen

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by COLING2025

AquaticCLIP: A Vision-Language Foundation Model for Underwater Scene Analysis

Basit Alawode, Iyyakutti Iyappan Ganapathi, Sajid Javed, Naoufel Werghi, Mohammed Bennamoun, Arif Mahmood

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

CLIP-DQA: Blindly Evaluating Dehazed Images from Global and Local Perspectives Using CLIP

Yirui Zeng, Jun Fu, Hadi Amirpour, Huasheng Wang, Guanghui Yue, Hantao Liu, Ying Chen, Wei Zhou

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ISCAS 2025 (Oral)

Vision-Language Modeling in PET/CT for Visual Grounding of Positive Findings

Zachary Huemann, Samuel Church, Joshua D. Warner, Daniel Tran, Xin Tie, Alan B McMillan, Junjie Hu, Steve Y. Cho, Meghan Lubner, Tyler J. Bradshaw

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification

Xiangyu Sun, Xiaoguang Zou, Yuanquan Wu, Guotai Wang, Shaoting Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments This paper has been accepted for presentation at the 2025 IEEE International Symposium on Biomedical Imaging (ISBI 2025)

Large Models in Dialogue for Active Perception and Anomaly Detection

Tzoulio Chamiti, Nikolaos Passalis, Anastasios Tefas

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to International Conference of Pattern Recognition (ICPR 2024)

Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities

Shounak Datta, Dhanasekar Sundararaman

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, Nadine Chang, Karan Sapra, Amala Sanjay Deshmukh, Tuomas Rintamaki, Matthieu Le, Ilia Karmanov, Lukas Voegtle, Philipp Fischer, De-An Huang, Timo Roman, Tong Lu, Jose M. Alvarez, Bryan Catanzaro, Jan Kautz, Andrew Tao, Guilin Liu, Zhiding Yu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Global Semantic-Guided Sub-image Feature Weight Allocation in High-Resolution Large Vision-Language Models

Yuxuan Liang, Xu Li, Xiaolei Chen, Haotian Chen, Yi Zheng, Chenghang Lai, Bin Li, Xiangyang Xue

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 10 figures and tables

Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models

Jakob Krogh Petersen, Valdemar Licht, Mads Nielsen, Asbjørn Munk

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures. To be published in ISBI 2025

Pix2Cap-COCO: Advancing Visual Comprehension via Pixel-Level Captioning

Zuyao You, Junke Wang, Lingyu Kong, Bo He, Zuxuan Wu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VisMin: Visual Minimal-Change Understanding

Rabiul Awal, Saba Ahmadi, Le Zhang, Aishwarya Agrawal

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted at NeurIPS 2024. Project URL at https://vismin.net/

Patch is Enough: Naturalistic Adversarial Patch against Vision-Language Pre-training Models

Dehong Kong, Siyuan Liang, Xiaopeng Zhu, Yuansheng Zhong, Wenqi Ren

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments accepted by Visual Intelligence

Journal ref Visual Intelligence, 2024, Vol 2, article no.17

MedFILIP: Medical Fine-grained Language-Image Pre-training

Xinjie Liang, Xiangyu Li, Fanding Li, Jie Jiang, Qing Dong, Wei Wang, Kuanquan Wang, Suyu Dong, Gongning Luo, Shuo Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 5 figures, IEEE Journal of Biomedical and Health Informatics 2025

Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model

Huan Ma, Yan Zhu, Changqing Zhang, Peilin Zhao, Baoyuan Wu, Long-Kai Huang, Qinghua Hu, Bingzhe Wu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Visual question answering: from early developments to recent advances -- a survey

Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Sunil Aryal, Imran Razzak, Hakim Hacid

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 20 papers

Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective

Sahil Verma, Gantavya Bhatt, Avi Schwarzschild, Soumye Singhal, Arnav Mohanty Das, Chirag Shah, John P Dickerson, Pin-Yu Chen, Jeff Bilmes

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at TMLR (https://openreview.net/forum?id=Conma3qnaT)

From Simple to Professional: A Combinatorial Controllable Image Captioning Agent

Xinran Wang, Muxi Diao, Baoteng Li, Haiwen Zhang, Kongming Liang, Zhanyu Ma

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments A technical report. Project: https://github.com/xin-ran-w/CapAgent

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

Zhecan Wang, Junzhang Liu, Chia-Wei Tang, Hani Alomari, Anushka Sivakumar, Rui Sun, Wenhao Li, Md. Atabuzzaman, Hammad Ayyubi, Haoxuan You, Alvi Ishmam, Kai-Wei Chang, Shih-Fu Chang, Chris Thomas

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding

Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, Cha Zhang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Project link: https://zeyofu.github.io/ReFocus/

Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding

Aaron Lohner, Francesco Compagno, Jonathan Francis, Alessandro Oltramari

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Won the 'Best Paper Runner-up Award' at the 2024 IEEE International Automated Vehicle Validation Conference (IAVVC 2024). Also accepted at the 1st Workshop on Semantic Reasoning and Goal Understanding in Robotics, at the Robotics Science and Systems Conference (RSS SemRob 2024)