视觉大模型 / VLM

Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Models

Ziwei Zheng, Junyao Zhao, Le Yang, Lijun He, Fan Li

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback

Wenyi Xiao, Ziwei Huang, Leilei Gan, Wanggui He, Haoyuan Li, Zhelun Yu, Fangxun Shu, Hao Jiang, Linchao Zhu

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments repo: https://github.com/Mr-Loevan/HSA-DPO

Woodpecker: Hallucination Correction for Multimodal Large Language Models

Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, Enhong Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by Science China Information Sciences (SCIS)

Journal ref SCIENCE CHINA Information Sciences, 2024

MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models

Yichi Zhang, Yao Huang, Yitong Sun, Chang Liu, Zhe Zhao, Zhengwei Fang, Yifan Wang, Huanran Chen, Xiao Yang, Xingxing Wei, Hang Su, Yinpeng Dong, Jun Zhu

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 100 pages, 84 figures, 33 tables

Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach

Shijian Deng, Wentian Zhao, Yu-Jhe Li, Kun Wan, Daniel Miranda, Ajinkya Kale, Yapeng Tian

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models

Ali Abdollahi, Mahdi Ghaznavi, Mohammad Reza Karimi Nejad, Arash Mari Oriyad, Reza Abbasi, Ali Salesi, Melika Behjati, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Journal ref Volume 392 of ECAI 2024, Pages 729 - 736

SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models

Zonghao Ying, Aishan Liu, Siyuan Liang, Lei Huang, Jinyang Guo, Wenbo Zhou, Xianglong Liu, Dacheng Tao

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；MLLM(abstract)

Reducing Hallucinations in Vision-Language Models via Latent Space Steering

Sheng Liu, Haotian Ye, Lei Xing, James Zou

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 21 pages

$\textit{MMJ-Bench}$: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models

Fenghua Weng, Yue Xu, Chengyan Fu, Wenjie Wang

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；MLLM(abstract)

Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking

Wei Zhang, Pengfei Li, Junli Wang, Bingchuan Sun, Qihao Jin, Guangjun Bao, Shibo Rui, Yang Yu, Wenchao Ding, Peng Li, Yilun Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；MLLM(abstract)

Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models

Yubo Wang, Chaohu Liu, Yanqiu Qu, Haoyu Cao, Deqiang Jiang, Linli Xu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ACMMM 2024

CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration

Jiahui Gao, Renjie Pi, Tianyang Han, Han Wu, Lanqing Hong, Lingpeng Kong, Xin Jiang, Zhenguo Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；MLLM(abstract)

Comments 10 pages, COLM-2024

ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models

Yeji Park, Deokyeong Lee, Junsuk Choe, Buru Chang

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments First two authors contributed equally. Source code is available at https://github.com/yejipark-m/ConVis

Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques

Rishika Bhagwatkar, Shravan Nayak, Reza Bayat, Alexis Roger, Daniel Z Kaplan, Pouya Bashivan, Irina Rish

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models

Zongbo Han, Zechen Bai, Haiyang Mei, Qianli Xu, Changqing Zhang, Mike Zheng Shou

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

AAPL: Adding Attributes to Prompt Learning for Vision-Language Models

Gahyeon Kim, Sohee Kim, Seokju Lee

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to CVPR 2024 Workshop on Prompting in Vision, Project Page: https://github.com/Gahyeonkim09/AAPL

C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion

Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark Hasegawa-Johnson, Yingzhen Li, Chang D. Yoo

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2024

One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models

Lin Li, Haoyan Guan, Jianing Qiu, Michael Spratling

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2024

Evaluation and Analysis of Hallucination in Large Vision-Language Models

Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, Jitao Sang, Haoyu Tang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 11 pages, 5 figures

Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?

Cheng-En Wu, Yu Tian, Haichao Yu, Heng Wang, Pedro Morgado, Yu Hen Hu, Linjie Yang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by ICCV2023

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models

探索多语言多模态大语言模型的对抗鲁棒性与安全对齐

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

机构 * Mohamed Bin Zayed University of AI, UAE（穆罕默德·本·扎耶德人工智能大学，阿联酋） ； Khalifa University, UAE（卡比拉大学，阿联酋） ； Australian National University, Australia（澳大利亚国立大学，澳大利亚） 纠错

专题命中 幻觉与鲁棒性 ：MLLM(abstract,abstract_cn)；grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

AI总结 本研究通过梯度攻击和跨语言评估，发现多语言多模态大语言模型存在可迁移的对抗脆弱性，并揭示低资源语言因理解失败而呈现的虚假安全现象，提出深层训练整合才能实现真正的多语言安全对齐。

Journal ref The 37th British Machine Vision Conference (BMVC) 2026

MonitorVLM-v2: A Deployed Vision-Language Framework for Real-Time Safety Violation Detection

MonitorVLM-v2：用于实时安全违规检测的已部署视觉-语言框架

Jiang Wu, Sichao Wu, Yinsong Ma, Lifang Zheng, Jingliang Duan

机构 * School of Mechanical Engineering, University of Science and Technology Beijing（北京科技大学机械工程学院） ； The Laboratory for Computational Sensing and Robotics, Johns Hopkins University（约翰霍普金斯大学计算传感与机器人实验室） 纠错

专题命中 幻觉与鲁棒性 ：VLM(summary_cn,abstract)；分类 cs.CV

AI总结 本文提出MonitorVLM-v2框架，将VLM安全评估转化为有限符号空间概率推理，结合SymPO算法与熵驱动分类机制，在地下矿部署中实现19.45倍提速，确认违规量达人工检查的2.78倍，满足实时可审计工业监控需求。

Multimodal Reward Hacking in Reinforcement Learning

强化学习中的多模态奖励黑客攻击

Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, Shenghua Liu

机构 * Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） ； University of California, Merced（加州大学默塞德分校） ； Southeast University（东南大学） 纠错

专题命中 幻觉与鲁棒性 ：MLLM(abstract,abstract_cn)；VLM(abstract)；multimodal large language model(abstract)；分类 cs.AI

AI总结 研究强化学习中多模态奖励黑客攻击问题，通过引入新奖励失败率（NRFR）等方法，在多种设置下改变模型规模、算法等因素进行实验，发现仅结果奖励易导致黑客攻击，扩大规模可减少但不能消除，还得出不同算法和奖励方式对黑客攻击的影响及相关结论。

AutoMine Solution for AV2 2026 Scenario Mining Challenge

AutoMine 解决方案：面向 AV2 2026 场景挖掘挑战

Songliang Cao, Jiele Zhao, Yuru Wang, Hao Li, Daqi Liu, Zehan Zhang, Fangzhen Li, Yu Wang, Yue Zhang, Bing Wang, Guang Chen, Hao Lu, Hangjun Ye

机构 * Xiaomi EV（小米汽车） ； Huazhong University of Science and Technology（华中科技大学） 纠错

专题命中 幻觉与鲁棒性 ：VLM(summary_cn,abstract)；分类 cs.AI

AI总结 提出基于 LLM 和 VLM 的自优化场景挖掘方法 AutoMine，通过语义保持提示增强、鲁棒轨迹原子函数与 VLM 函数结合以及执行反馈优化，在 CVPR 2026 挑战赛中取得领先性能。

Comments CVPR 2026 Scenario Mining Challenge (Temporal Track Winners)

Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees

自信记忆：具有概率保证的时空记忆不确定性量化

Harry Zhang, Nicolas Gorlo, Luca Carlone

机构 * MIT（麻省理工学院） 纠错

专题命中 幻觉与鲁棒性 ：VLM(summary_cn,abstract)；分类 cs.CV

AI总结 针对机器人长期操作中VLM描述噪声大、视角不一致的问题，提出目标级语义不确定性评分，并集成到UQ-DAAAM系统中，通过主动选择高质量视图和融合多视角描述来降低不确定性，同时提供概率保证。

StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs

StructBreak: 多模态大语言模型中结构性认知过载引发的安全故障

Yang Luo, Xinran Liu, Tiantian Ji, Zhiyi Yin, Lingyun Peng, Shuyu Li

机构 * Key Laboratory of Trustworthy Distributed Computing and Service (MoE), Beijing University of Posts and Telecommunications（可信分布式计算与服务重点实验室（MoE），北京邮电大学） ； Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所） 纠错

专题命中 幻觉与鲁棒性 ：MLLM(summary_cn,abstract_cn)；multimodal large language model(abstract)；分类 cs.AI

AI总结 提出StructBreak框架，通过量化结构性认知过载（SCO）揭示一种高阶认知过载攻击范式，在六种主流MLLM上实现92%平均攻击成功率，并证明该攻击通过结构性通道绕过安全过滤器。

Comments 23 pages; accepted to Findings of ACL 2026. This paper contains examples of harmful content

Art3D: Training-Free 3D Generation from Flat-Colored Illustration

Art3D: 无需训练的从平面着色插图生成3D

Xiaoyan Cong, Jiayi Shen, Zekun Li, Rao Fu, Tao Lu, Srinath Sridhar

机构 * Brown University（布朗大学） 纠错

专题命中 幻觉与鲁棒性 ：VLM(summary_cn,abstract)；分类 cs.CV

AI总结 Art3D通过结合预训练2D图像生成模型和基于VLM的现实感评估，将平面着色插图提升为3D，解决了传统方法在处理无3D illusion的参考图像时的局限性，展示了其在多种绘画风格中的适应性。

Comments Technical Report. Project Page: https://joy-jy11.github.io/

HyCodePolicy: Hybrid Language Controllers for Multimodal Monitoring and Decision in Embodied Agents

Yibin Liu, Zhixuan Liang, Zanxin Chen, Tianxing Chen, Mengkang Hu, Wanxi Dong, Congsheng Xu, Zhaoming Han, Yusen Qin, Yao Mu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；VLM(abstract)；grounding(abstract)；multimodal large language model(abstract)

Comments Accepted to ICCV 2025 Workshop on Multi-Modal Reasoning for Agentic Intelligence

Covert Visual Prompt Injection against Commercial Multimodal Large Language Models

对抗性提示注入攻击多模态大语言模型

Meiwen Ding, Song Xia, Chenqi Kong, Xudong Jiang

机构 * Rapid-Rich Object Search Lab, School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院快速丰富目标搜索实验室） 纠错

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

AI总结 本文研究了针对强大闭源多模态大语言模型的不可察觉视觉提示注入攻击，通过界文本叠加嵌入恶意提示并优化视觉扰动，提升攻击性能。

Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit

能否用视觉-语言模型评估城市衰败：以底特律为例

Xiaohao Yang, Aohua Tian, Derek Van Berkel, Xu Qiang, Mark Lindquist

机构 * School for Environment and Sustainability（环境与可持续发展学院） ； School of Information（信息学院） ； University of Michigan（密歇根大学） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

AI总结 本研究以底特律为案例，提出基于开源大视觉-语言模型的多视角住宅衰败评估框架，经实验验证该框架可低成本跟踪住房状况，为传统调查提供补充。

Comments 19 pages, 11 figures