视觉大模型 / VLM - arXivDaily 专题

2507.21637 2025-07-30 cs.AI 79%

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Wanying Wang, Zeyu Ma, Han Zheng, Xin Tan, Mingang Chen

机构 * Shanghai Key Laboratory of Computer Software Testing and Evaluating（上海软件测试与评估 key laboratory）； Shanghai Normal University（上海Normal University）； TrustAI Pte. Ltd. ； East China Normal University（东华师范大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.AI

Comments Accepted by ACM Multimedia 2025

2507.21521 2025-07-30 cs.CV 79%

Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration

Athmanarayanan Lakshmi Narayanan, Amrutha Machireddy, Ranganath Krishnan

机构 * Intel Labs（英特尔实验室）； Intel Corporation（英特尔公司）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments International Joint Conference on Neural Networks 2025 (Accepted)

2507.21494 2025-07-30 cs.LG 79%

Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning

Wenxuan Bao, Ruxi Deng, Ruizhong Qiu, Tianxin Wei, Hanghang Tong, Jingrui He

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.LG

Comments Accepted by ICCV 2025

2507.15652 2025-07-22 cs.CV 79%

Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

Haoran Zhou, Zihan Zhang, Hao Chen

机构 * Southeast University（东南大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2405.20090 2025-07-22 cs.CV 79%

Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models

Hao Cheng, Erjia Xiao, Jiayan Yang, Jinhao Duan, Yichi Wang, Jiahang Cao, Qiang Zhang, Le Yang, Kaidi Xu, Jindong Gu, Renjing Xu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））； University of Oxford（牛津大学）； Drexel University（德雷塞尔大学）； Beijing University of Technology（北京理工大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Xi’an Jiaotong University（西安交通大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments This paper is accepted by ACM MM 2025

2507.12950 2025-07-21 cs.LG 79%

Insights into a radiology-specialised multimodal large language model with sparse autoencoders

Kenza Bouzid, Shruthi Bannur, Felix Meissen, Daniel Coelho de Castro, Anton Schwaighofer, Javier Alvarez-Valle, Stephanie L. Hyland

机构 * Microsoft Research, Health Futures, Cambridge, United Kingdom（微软研究院，健康未来，剑桥，英国）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.LG

Comments Actionable Interpretability Workshop at ICML 2025. 24 pages, 7 figures, 5 tables

2507.07709 2025-07-11 cs.CV 79%

One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models

Jiale Zhao, Xinyang Jiang, Junyao Gao, Yuhao Xue, Cairong Zhao

机构 * Tongji University（同济大学）； Microsoft Research Asia（微软亚洲研究院）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2506.23101 2025-07-08 cs.CL cs.AI cs.CY 79%

From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship

Yue Xu, Wenjie Wang

机构 * School of Information Science and Technology, ShanghaiTech University（信息科学与技术学院，上海科技大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

2409.09318 2025-07-08 cs.CL cs.CV 79%

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

Yahan Tu, Rui Hu, Jitao Sang

机构 * Beijing Jiaotong University（北京交通大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2506.23590 2025-07-01 cs.CV 79%

CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models

Qiming Li, Zekai Ye, Xiaocheng Feng, Weihong Zhong, Libo Qin, Ruihan Chen, Baohang Li, Kui Jiang, Yaowei Wang, Ting Liu, Bing Qin

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2506.22979 2025-07-01 cs.CV 79%

Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation

Jie Liu, Jiayi Shen, Pan Zhou, Jan-Jakob Sonke, Efstratios Gavves

机构 * University of Amsterdam（阿姆斯特丹大学）； Singapore Management University（新加坡管理大学）； The Netherlands Cancer Institute（荷兰癌症研究院）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments ICCV2025 Proceeding

2501.04931 2025-06-30 cs.CR cs.AI cs.CL 79%

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Shiji Zhao, Ranjie Duan, Fengxiang Wang, Chi Chen, Caixin Kang, Shouwei Ruan, Jialing Tao, YueFeng Chen, Hui Xue, Xingxing Wei

机构 * Institution1（机构1）； Institution2（机构2）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.AI

Comments ICCV2025

2506.16760 2025-06-23 cs.CL cs.CV 79%

Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

Lei Jiang, Zixun Zhang, Zizhou Wang, Xiaobing Sun, Zhen Li, Liangli Zhen, Xiaohua Xu

机构 * University of Science and Technology of China（中国科学技术大学）； The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））； Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments 15 pages, 9 figures

2501.00848 2025-06-23 cs.CV 79%

IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models

Yiming Zhang, Zicheng Zhang, Xinyi Wei, Xiaohong Liu, Guangtao Zhai, Xiongkuo Min

机构 * Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, China（图像通信与网络工程研究院，上海交通大学）

专题命中幻觉与鲁棒性：vision-language model(title)；visual language model(abstract)；分类 cs.CV

2502.10794 2025-06-18 cs.CV 79%

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

Zuopeng Yang, Jiluan Fan, Anli Yan, Erdun Gao, Xin Lin, Tao Li, Kanghua Mo, Changyu Dong

机构 * Guangzhou University（广州大学）； Shanghai Jiao Tong University（上海交通大学）； Australian Institute for Machine Learning, The University of Adelaide（澳大利亚机器学习研究所，阿德莱德大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments CVPR 2025 highlight

2506.06076 2025-06-09 cs.CV 79%

Full Conformal Adaptation of Medical Vision-Language Models

Julio Silva-Rodríguez, Leo Fillioux, Paul-Henry Cournède, Maria Vakalopoulou, Stergios Christodoulidis, Ismail Ben Ayed, Jose Dolz

机构 * ÉTS Montréal（蒙特利尔ÉTS学院）； CentraleSupélec, Université Paris-Saclay（中央理工学院，巴黎萨克雷大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments IPMI 2025. Code: https://github.com/jusiro/FCA

2506.03371 2025-06-05 cs.CV 79%

Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views

Xiaonan Wang, Bo Shao, Hansaem Kim

专题命中幻觉与鲁棒性：VLM(title)；vision-language model(abstract)；分类 cs.CV

2407.19719 2025-06-03 cs.CV 79%

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Jiaxin Zhang, Yunqin Li, Tomohiro Fukuda, Bowen Wang

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

Comments 15 pages, 10 figures

2410.02681 2025-05-30 cs.LG 79%

Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

Shuoyuan Wang, Yixuan Li, Hongxin Wei

机构 * Shuoyuan Wang（王舒元）； Yixuan Li（李奕轩）； Hongxin Wei（魏宏鑫）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.LG

Comments Accepted by ICML 2025

2505.22305 2025-05-29 cs.CV 79%

IKIWISI: An Interactive Visual Pattern Generator for Evaluating the Reliability of Vision-Language Models Without Ground Truth

Md Touhidul Islam, Imran Kabir, Md Alimoor Reza, Syed Masum Billah

机构 * Pennsylvania State University（宾夕法尼亚州立大学）； Drake University（德拉威大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Accepted at DIS'25 (Funchal, Portugal)

2505.20901 2025-05-28 cs.CL cs.AI 79%

A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models

Junhyuk Choi, Minju Kim, Yeseon Hong, Bugeun Kim

机构 * Department of Artificial Intelligence, Chung-Ang University（人工智能系， Chung-Ang 大学）

专题命中幻觉与鲁棒性：vision language model(title,abstract)；分类 cs.AI

Comments Under review

2505.20362 2025-05-28 cs.IR cs.AI 79%

VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration

Jiahui Geng, Qing Li, Zongxiong Chen, Yuxia Wang, Derui Zhu, Zhuohan Xie, Chenyang Lyu, Xiuying Chen, Preslav Nakov, Fakhri Karray

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学）； Fraunhofer Institute for Open Communication Systems（弗劳恩霍夫开放通信系统研究所）； Technical University of Munich（慕尼黑技术大学）； Alibaba International Digital Commerce（阿里巴巴国际数字商务）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.AI

2505.15425 2025-05-26 cs.CV 79%

On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?

Raza Imam, Rufael Marew, Mohammad Yaqub

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎德·穆萨大学人工智能学院）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

Comments Dataset and Code is available at https://github.com/BioMedIA-MBZUAI/RobustMedCLIP Accepted at: Medical Image Understanding and Analysis (MIUA) 2025

2505.11405 2025-05-19 cs.CV cs.CL 79%

EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models

Bohao Xing, Xin Liu, Guoying Zhao, Chengyu Liu, Xiaolan Fu, Heikki Kälviäinen

机构 * Lappeenranta-Lahti University of Technology LUT（拉佩兰塔-拉赫蒂技术大学）； University of Oulu（奥卢大学）； Southeast University（东南大学）； Shanghai Jiao Tong University（上海交通大学）； Brno University of Technology（布拉格技术大学）

专题命中幻觉与鲁棒性：multimodal large language model(title,abstract)；分类 cs.CV

2404.05046 2025-05-07 cs.CV cs.CL 79%

FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback

Liqiang Jing, Xinya Du

机构 * Department of Computer Science, The University of Texas at Dallas（计算机科学系，德克萨斯大学达拉斯分校）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2505.02971 2025-05-07 cs.CV 79%

Adversarial Robustness Analysis of Vision-Language Models in Medical Image Segmentation

Anjila Budathoki, Manish Dhakal

机构 * Department of Computer Science（计算机科学系）； Georgia State University（佐治亚州立大学）

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

2505.01851 2025-05-06 cs.CV 79%

Mitigating Group-Level Fairness Disparities in Federated Visual Language Models

Chaomeng Chen, Zitong Yu, Junhao Dong, Sen Su, Linlin Shen, Shutao Xia, Xiaochun Cao

机构 * Great Bay University（大湾大学）； Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）； Nanyang Technological University（南洋理工大学）； Beijing University of Posts and Telecommunications（北京邮电大学）； Shenzhen University（深圳大学）； Pengcheng Laboratory（鹏城实验室）； Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）

专题命中幻觉与鲁棒性：visual language model(title,abstract)；分类 cs.CV

2504.10179 2025-04-15 cs.AI cs.CL cs.ET 79%

The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance

Anwesha Mohanty, Venkatesh Balavadhani Parthasarathy, Arsalan Shahid

专题命中幻觉与鲁棒性：MLLM(title)；multimodal large language model(abstract)；分类 cs.AI

2504.01838 2025-04-03 cs.CV 79%

Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images

Nusrat Munia, Abdullah-Al-Zubaer Imran

专题命中幻觉与鲁棒性：vision-language model(title)；vision language model(abstract)；分类 cs.CV

Comments Paper accepted at International Symposium on Biomedical Imaging (ISBI 2025)

2411.16724 2025-04-02 cs.CV 79%

Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens

Zhangqi Jiang, Junkai Chen, Beier Zhu, Tingjin Luo, Yankun Shen, Xu Yang

专题命中幻觉与鲁棒性：vision-language model(title,abstract)；分类 cs.CV

1. 幻觉与鲁棒性 2246 篇

Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Models

Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibration

Latte: Collaborative Test-Time Adaptation of Vision-Language Models in Federated Learning

Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models

Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Models

Insights into a radiology-specialised multimodal large language model with sparse autoencoders

One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models

From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship

ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models

CAI: Caption-Sensitive Attention Intervention for Mitigating Object Hallucination in Large Vision-Language Models

Probabilistic Prototype Calibration of Vision-Language Models for Generalized Few-shot Semantic Segmentation

Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency

Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models

IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models

Distraction is All You Need for Multimodal Large Language Model Jailbreaking

Full Conformal Adaptation of Medical Vision-Language Models

Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views

Urban Safety Perception Assessments via Integrating Multimodal Large Language Models with Street View Images

Understanding and Mitigating Miscalibration in Prompt Tuning for Vision-Language Models

IKIWISI: An Interactive Visual Pattern Generator for Evaluating the Reliability of Vision-Language Models Without Ground Truth

A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models

VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration

On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?

EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models

FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback

Adversarial Robustness Analysis of Vision-Language Models in Medical Image Segmentation

Mitigating Group-Level Fairness Disparities in Federated Visual Language Models

The Future of MLLM Prompting is Adaptive: A Comprehensive Experimental Evaluation of Prompt Engineering Methods for Robust Multimodal Performance

Prompting Medical Vision-Language Models to Mitigate Diagnosis Bias by Generating Realistic Dermoscopic Images

Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens