视觉大模型 / VLM

Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspective

Zhaotian Weng, Zijun Gao, Jerone Andrews, Jieyu Zhao

机构 * University of Southern California（南加州大学） ； Sony AI（索尼人工智能） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models

Sangmin Woo, Donguk Kim, Jaehyuk Jang, Yubin Choi, Changick Kim

机构 * KAIST（韩国科学技术院） 纠错

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments ACL 2025 Findings; Project: https://sangminwoo.github.io/AvisC/

BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Models

Huu-Thien Tran, Thanh-Dat Truong, Khoa Luu

机构 * CVIU Lab, University of Arkansas（CVIU实验室，阿肯色大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments CVPRW 2025, 8 pages, 4 figures

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models

Kening Zheng, Junkai Chen, Yibo Yan, Xin Zou, Xuming Hu

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） ； Guangxi Zhuang Autonomous Region Big Data Research Institute（广西壮族自治区大数据研究院） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted by Findings of ACL 2025

Image Tokens Matter: Mitigating Hallucination in Discrete Tokenizer-based Large Vision-Language Models via Latent Editing

Weixing Wang, Zifeng Ding, Jindong Gu, Rui Cao, Christoph Meinel, Gerard de Melo, Haojin Yang

机构 * Hasso Plattner Institute（霍普夫纳研究所） ； University of Potsdam（波茨坦大学） ； University of Cambridge（剑桥大学） ； University of Oxford（牛津大学） ； German University of Digital Science（德国数字科学大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models

Hao Yin, Guangzong Si, Zilei Wang

机构 * University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation

Zherui Zhang, Jiaxin Wu, Changwei Wang, Rongtao Xu, Longzhao Huang, Wenhao Xu, Wenbo Xu, Li Guo, Shibiao Xu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

Yongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao, Rui Zheng, Jinlan Fu, Zhenfei Yin, Senjie Jin, Yu Qiao, Xuanjing Huang, Feng Zhao, Tao Gui, Jing Shao

机构 * University of Science and Technology of China（中国科学技术大学） ； Fudan University（复旦大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Are vision language models robust to uncertain inputs?

Xi Wang, Eric Nalisnick

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems

Yuhang Wang, Hao Zhou

机构 * Department of Civil & Environmental Engineering , University of South Florida（土木与环境工程系，佛罗里达州立大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title)；VLM(abstract)；分类 cs.CV、cs.LG

Transferable Adversarial Attacks on Black-Box Vision-Language Models

Kai Hu, Weichen Yu, Li Zhang, Alexander Robey, Andy Zou, Chengming Xu, Haoqi Hu, Matt Fredrikson

机构 * Carnegie Mellon University（卡内基梅隆大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title)；visual question answering(abstract)；分类 cs.CV、cs.LG

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models

Sangmin Woo, Kang Zhou, Yun Zhou, Shuai Wang, Sheng Guan, Haibo Ding, Lin Lee Cheong

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments NAACL 2025

Owls are wise and foxes are unfaithful: Uncovering animal stereotypes in vision-language models

Tabinda Aman, Mohammad Nadeem, Shahab Saquib Sohail, Mohammad Anas, Erik Cambria

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Object-Level Verbalized Confidence Calibration in Vision-Language Models via Semantic Perturbation

Yunpu Zhao, Rui Zhang, Junbin Xiao, Ruibo Hou, Jiaming Guo, Zihao Zhang, Yifan Hao, Yunji Chen

机构 * University of Science and Technology of China（中国科学技术大学） ； SKL of Processors, Institute of Computing Technology, CAS（中国科学院计算技术研究所处理器专项实验室） ； National University of Singapore（新加坡国立大学） ； University of Chinese Academy of Sciences（中国科学院大学） ； University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models

Chung-En, Yu, Hsuan-Chih, Chen, Brian Jalaian, Nathaniel D. Bastian

机构 * University of West Florida（西弗吉尼亚大学） ； New York University（纽约大学） ； United States Military Academy（美国军事学院） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models

Simon Schrodi, David T. Hoffmann, Max Argus, Volker Fischer, Thomas Brox

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICLR 2025 (Oral)

Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models

Zhaochen Wang, Bryan Hooi, Yiwei Wang, Ming-Hsuan Yang, Zi Huang, Yujun Cai

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Under review at COLM 2025

SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments

Yue Cao, Yun Xing, Jie Zhang, Di Lin, Tianwei Zhang, Ivor Tsang, Yang Liu, Qing Guo

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance

Qingpei Guo, Kaiyou Song, Zipeng Feng, Ziping Ma, Qinglong Zhang, Sirui Gao, Xuzheng Yu, Yunxiao Sun, Tai-Wei Chang, Jingdong Chen, Ming Yang, Jun Zhou

专题命中 幻觉与鲁棒性 ：MLLM(title,abstract)；分类 cs.AI、cs.LG

Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models

Mirko Borszukovszki, Ivo Pascal de Jong, Matias Valdenegro-Toro

专题命中 幻觉与鲁棒性 ：vision-language model(title)；visual language model(abstract)；分类 cs.CV、cs.LG

Comments 10 pages, 11 figures, TrustNLP Workshop @ NAACL 2025 Camera ready

Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models

Bowei Tian, Xuntao Lyu, Meng Liu, Hongyi Wang, Ang Li

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.AI、cs.LG

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICLR 2025 Oral

AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models

Jiaming Zhang, Junhong Ye, Xingjun Ma, Yige Li, Yunfan Yang, Yunhao Chen, Jitao Sang, Dit-Yan Yeung

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.AI、cs.LG

Comments CVPR 2025

Hyperbolic Safety-Aware Vision-Language Models

Tobia Poppi, Tejaswi Kasarla, Pascal Mettes, Lorenzo Baraldi, Rita Cucchiara

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

The in-context inductive biases of vision-language models differ across modalities

Kelsey Allen, Ishita Dasgupta, Eliza Kosoy, Andrew K. Lampinen

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments 11 pages

TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction

Chao Wang, Weiwei Fu, Yang Zhou

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models

Shicheng Xu, Liang Pang, Yunchang Zhu, Huawei Shen, Xueqi Cheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments ICLR 2025

CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models

Zihao Sheng, Zilin Huang, Yansong Qu, Yue Leng, Sruthi Bhavanam, Sikai Chen

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptation

Yuheng Ji, Yue Liu, Zhicheng Zhang, Zhao Zhang, Yuting Zhao, Xiaoshuai Hao, Gang Zhou, Xingwei Zhang, Xiaolong Zheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding

Kyungmin Min, Minbeom Kim, Kang-il Lee, Dongryeol Lee, Kyomin Jung

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments NAACL 2025 (Findings); Renamed SGD to SumGD in Summary-Guided Decoding to prevent confusion with Stochastic Gradient Descent