视觉大模型 / VLM

Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Models

Xu Han, Linghao Jin, Xuezhe Ma, Xiaofeng Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in Large Vision Language Models

Sangmin Woo, Jaehyuk Jang, Donguk Kim, Yubin Choi, Changick Kim

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

Comments Project: https://sangminwoo.github.io/RITUAL/

Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models

Yi-Lun Lee, Yi-Hsuan Tsai, Wei-Chen Chiu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Under review. Project pages: https://github.com/YiLunLee/VCD_Analysis

Efficient Prompt Tuning of Large Vision-Language Model for Fine-Grained Ship Classification

Long Lan, Fengxiang Wang, Xiangtao Zheng, Zengmao Wang, Xinwang Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments It has been accepted by TGRS

V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization

Yuxi Xie, Guanzhen Li, Xiao Xu, Min-Yen Kan

专题命中 幻觉与鲁棒性 ：vision language model(title)；vision-language model(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024 Findings; 9 pages, 6 figures, 5 tables (16 pages, 8 figures, 8 tables including references and appendices)

Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective

Yanan Zhang, Jiangmeng Li, Lixiang Liu, Wenwen Qiang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted by NeurIPS 2024

Identifying Implicit Social Biases in Vision-Language Models

Kimia Hamidieh, Haoran Zhang, Walter Gerych, Thomas Hartvigsen, Marzyeh Ghassemi

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models

Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, Wenhao Zheng, Zhaoyang Wang, Xiao Wang, Xuchao Zhang, Chetan Bansal, Marc Niethammer, Junzhou Huang, Hongtu Zhu, Yun Li, Jimeng Sun, Zongyuan Ge, Gang Li, James Zou, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2024 Datasets and Benchmarks Track

Multi-Object Hallucination in Vision-Language Models

Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F. Fouhey, Joyce Chai

专题命中 幻觉与鲁棒性 ：vision-language model(title)；vision language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024 | Project page: https://multi-object-hallucination.github.io/

Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models

Lu Yu, Haiyang Zhang, Changsheng Xu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024

A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models

Yuhan Liang, Yijun Li, Yumeng Niu, Qianhe Shen, Hangyu Liu

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

White-box Multimodal Jailbreaks Against Large Vision-Language Models

Ruofan Wang, Xingjun Ma, Hanxu Zhou, Chuanjun Ji, Guangnan Ye, Yu-Gang Jiang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

PIP: Detecting Adversarial Examples in Large Vision-Language Models via Attention Patterns of Irrelevant Probe Questions

Yudong Zhang, Ruobing Xie, Jiansheng Chen, Xingwu Sun, Yu Wang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACM Multimedia 2024 BNI track (Oral)

Safeguarding Vision-Language Models Against Patched Visual Prompt Injectors

Jiachen Sun, Changsheng Wang, Jiongxiao Wang, Yiwei Zhang, Chaowei Xiao

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages

Adversarial Prompt Tuning for Vision-Language Models

Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, Jitao Sang

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments ECCV 2024

Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models

Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, Bing Qin

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACL 2024 Main Conference. 21 pages, 20 figures

Mitigating Multilingual Hallucination in Large Vision-Language Models

Xiaoye Qu, Mingyang Song, Wei Wei, Jianfeng Dong, Yu Cheng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Vision-Language Models under Cultural and Inclusive Considerations

Antonia Karamolegkou, Phillip Rust, Yong Cao, Ruixiang Cui, Anders Søgaard, Daniel Hershcovich

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments HuCLLM @ ACL 2024

MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context

Zishan Gu, Changchang Yin, Fenglin Liu, Ping Zhang

专题命中 幻觉与鲁棒性 ：vision language model(title,abstract)；分类 cs.CV、cs.AI

An Empirical Study Into What Matters for Calibrating Vision-Language Models

Weijie Tu, Weijian Deng, Dylan Campbell, Stephen Gould, Tom Gedeon

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments ICML 2024 Camera Ready

NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models

Kai Wu, Boyuan Jiang, Zhengkai Jiang, Qingdong He, Donghao Luo, Shengzhi Wang, Qingwen Liu, Chengjie Wang

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 5 figures with supplementary material

A Survey on Hallucination in Large Vision-Language Models

Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, Wei Peng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization

Zining Chen, Weiqiu Wang, Zhicheng Zhao, Fei Su, Aidong Men, Hongying Meng

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted to CVPR2024

SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples

Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwalla, Vasudev Lal

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2024. arXiv admin note: text overlap with arXiv:2310.02988

Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, Huaxiu Yao

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted by ICLR 2024

On the Out-Of-Distribution Generalization of Multimodal Large Language Models

Xingxuan Zhang, Jiansheng Li, Wenjing Chu, Junjia Hai, Renzhe Xu, Yuqing Yang, Shikai Guan, Jiazheng Xu, Peng Cui

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

Temporal Insight Enhancement: Mitigating Temporal Hallucination in Multimodal Large Language Models

Li Sun, Liuan Wang, Jun Sun, Takayuki Okatani

专题命中 幻觉与鲁棒性 ：multimodal large language model(title,abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 7 figures

Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, Lidong Bing

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI

Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models

Shuo Chen, Jindong Gu, Zhen Han, Yunpu Ma, Philip Torr, Volker Tresp

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.LG

Comments Accepted at NeurIPS 2023 Datasets and Benchmarks Track; 9 pages, 5 figures

Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples

Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Vasudev Lal

专题命中 幻觉与鲁棒性 ：vision-language model(title,abstract)；分类 cs.CV、cs.AI