视觉大模型 / VLM

Resilience of Vision Transformers for Domain Generalisation in the Presence of Out-of-Distribution Noisy Images

Hamza Riaz, Alan F. Smeaton

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.CV

Comments 31 pages

Unleashing the Power of Pre-trained Encoders for Universal Adversarial Attack Detection

Yinghe Zhang, Chi Liu, Shuai Zhou, Sheng Shen, Peng Gui

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Mitigating Low-Level Visual Hallucinations Requires Self-Awareness: Database, Model and Training Strategy

Yinan Sun, Xiongkuo Min, Zicheng Zhang, Yixuan Gao, Yuqin Cao, Guangtao Zhai

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Multi-Reward as Condition for Instruction-based Image Editing

Xin Gu, Ming Li, Libo Zhang, Fan Chen, Longyin Wen, Tiejian Luo, Sijie Zhu

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV

Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings

Zonghao Ying, Guangyi Zheng, Yongxin Huang, Deyue Zhang, Wenxin Zhang, Quanchen Zou, Aishan Liu, Xianglong Liu, Dacheng Tao

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners

Zhi Zheng, Qian Feng, Hang Li, Alois Knoll, Jianxiang Feng

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments Accepted at ICRA 2024 Workshop on Back to the Future: Robot Learning Going Probabilistic. Website: https://sites.google.com/view/konwloop/home

Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation

Beitao Chen, Xinyu Lyu, Lianli Gao, Jingkuan Song, Heng Tao Shen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Long-horizon Embodied Planning with Implicit Logical Inference and Hallucination Mitigation

Siyuan Liu, Jiawei Du, Sicheng Xiang, Zibo Wang, Dingsheng Luo

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs

Chongjun Tu, Peng Ye, Dongzhan Zhou, Lei Bai, Gang Yu, Tao Chen, Wanli Ouyang

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Urban Safety Perception Through the Lens of Large Multimodal Models: A Persona-based Approach

Ciro Beneduce, Bruno Lepri, Massimiliano Luca

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.AI

From Screens to Scenes: A Survey of Embodied AI in Healthcare

Yihao Liu, Xu Cao, Tingting Chen, Yankai Jiang, Junjie You, Minghua Wu, Xiaosong Wang, Mengling Feng, Yaochu Jin, Jintai Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments 56 pages, 11 figures, manuscript accepted by Information Fusion

CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generation

Yue Jiang, Jiawei Chen, Dingkang Yang, Mingcheng Li, Shunli Wang, Tong Wu, Ke Li, Lihua Zhang

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment

Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan Ö. Arık, Tomas Pfister

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Published in ICLR 2025

Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks

Wenpeng Xing, Minghao Li, Mohan Li, Meng Han

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Dysca: A Dynamic and Scalable Benchmark for Evaluating Perception Ability of LVLMs

Jie Zhang, Zhongqi Wang, Mengqi Lei, Zheng Yuan, Bei Yan, Shiguang Shan, Xilin Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ICLR2025

Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation

Hieu Nguyen, Zihao He, Shoumik Atul Gandre, Ujjwal Pasupulety, Sharanya Kumari Shivakumar, Kristina Lerman

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences

Shanshan Han, Salman Avestimehr, Chaoyang He

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments arXiv admin note: text overlap with arXiv:2406.10847

MARAGE: Transferable Multi-Model Adversarial Attack for Retrieval-Augmented Generation Data Extraction

Xiao Hu, Eric Liu, Weizhou Wang, Xiangyu Guo, David Lie

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports

Romain Hardy, Sung Eun Kim, Du Hyun Ro, Pranav Rajpurkar

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Comments Accepted to AIMedHealth 10 pages, 5 figures

ViG-Bias: Visually Grounded Bias Discovery and Mitigation

Badr-Eddine Marani, Mohamed Hanini, Nihitha Malayarukil, Stergios Christodoulidis, Maria Vakalopoulou, Enzo Ferrante

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV

Comments ECCV 2024

Breaking Barriers in Physical-World Adversarial Examples: Improving Robustness and Transferability via Robust Feature

Yichen Wang, Yuxuan Chou, Ziqi Zhou, Hangtao Zhang, Wei Wan, Shengshan Hu, Minghui Li

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by AAAI2025

Craft: Cross-modal Aligned Features Improve Robustness of Prompt Tuning

Jingchen Sun, Rohan Sharma, Vishnu Suresh Lokhande, Changyou Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to WACV 2025

CRoF: CLIP-based Robust Few-shot Learning on Noisy Labels

Shizhuo Deng, Bowen Han, Jiaqi Chen, Hao Wang, Dongyue Chen, Tong Jia

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?

Shuo Chen, Zhen Han, Bailan He, Zifeng Ding, Wenqian Yu, Philip Torr, Volker Tresp, Jindong Gu

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.LG

Comments technical report; update code repo link

GenMAC: Compositional Text-to-Video Generation with Multi-Agent Collaboration

Kaiyi Huang, Yukun Huang, Xuefei Ning, Zinan Lin, Yu Wang, Xihui Liu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Comments Project website: https://karine-h.github.io/GenMAC/

WellDunn: On the Robustness and Explainability of Language Models and Large Language Models in Identifying Wellness Dimensions

Seyedali Mohammadi, Edward Raff, Jinendra Malekar, Vedant Palit, Francis Ferraro, Manas Gaur

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments Accepted in BlackboxNLP @ EMNLP 2024

Journal ref Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 364-388, November 2024, Miami, Florida, US. Association for Computational Linguistics

Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration

Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, Tao Jiang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.LG

Comments Repo: NeurIPS2024_SPV-MIA" target="_blank" rel="noopener">https://github.com/tsinghua-fib-lab/NeurIPS2024_SPV-MIA

Journal ref The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS 2024)

Leveraging Pre-trained Models for FF-to-FFPE Histopathological Image Translation

Qilai Zhang, Jiawen Li, Peiran Liao, Jiali Hu, Tian Guan, Anjia Han, Yonghong He

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at IEEE BIBM 2024

When Geoscience Meets Foundation Models: Towards General Geoscience Artificial Intelligence System

Hao Zhang, Jin-Jian Xu, Hong-Wei Cui, Lin Li, Yaowen Yang, Chao-Sheng Tang, Niklas Boers

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI

Comments accpeted by IEEE Geoscience and Remote Sensing Magazine

Journal ref IEEE Geoscience and Remote Sensing Magazine, 2024

Enhancing Visual Classification using Comparative Descriptors

Hankyeol Lee, Gawon Seo, Wonseok Choi, Geunyoung Jung, Kyungwoo Song, Jiyoung Jung

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by WACV 2025