视觉大模型 / VLM

Prioritizing Image-Related Tokens Enhances Vision-Language Pre-Training

Yangyi Chen, Hao Peng, Tong Zhang, Heng Ji

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） 纠错

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments The code will be available at https://github.com/Yangyi-Chen/PRIOR

Don't Deceive Me: Mitigating Gaslighting through Attention Reallocation in LMMs

Pengkun Jiao, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yu-Gang Jiang

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV、cs.AI

ReferGPT: Towards Zero-Shot Referring Multi-Object Tracking

Tzoulio Chamiti, Leandro Di Bella, Adrian Munteanu, Nikos Deligiannis

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV、cs.AI

Comments Accepted CVPR 2025 Workshop on Distillation of Foundation Models for Autonomous Driving

Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict

Pouya Pezeshkpour, Moin Aminnaseri, Estevam Hruschka

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

ContextIQ: A Multimodal Expert-Based Video Retrieval System for Contextual Advertising

Ashutosh Chaubey, Anoubhav Agarwaal, Sartaki Sinha Roy, Aayush Agrawal, Susmita Ghose

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Published at WACV 2025

Aligning Visual Contrastive learning models via Preference Optimization

Amirabbas Afzali, Borna Khodabandeh, Ali Rasekh, Mahyar JafariNodeh, Sepehr kazemi, Simon Gottschalk

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuning

Hao Yu, Zhuokai Zhao, Shen Yan, Lukasz Korycki, Jianyu Wang, Baosheng He, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, Hanchao Yu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

MF-CLIP: Leveraging CLIP as Surrogate Models for No-box Adversarial Attacks

Jiaming Zhang, Lingyu Qiu, Qi Yi, Yige Li, Jitao Sang, Changsheng Xu, Dit-Yan Yeung

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.AI、cs.LG

good4cir: Generating Detailed Synthetic Captions for Composed Image Retrieval

Pranavi Kolouju, Eric Xing, Robert Pless, Nathan Jacobs, Abby Stylianou

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification

Dongseob Kim, Hyunjung Shim

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025 Accepted

PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training

Cong Chen, Mingyu Liu, Chenchen Jing, Yizhou Zhou, Fengyun Rao, Hao Chen, Bo Zhang, Chunhua Shen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding

Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, Yanning Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

See It from My Perspective: How Language Affects Cultural Bias in Image Understanding

Amith Ananthram, Elias Stengel-Eskin, Mohit Bansal, Kathleen McKeown

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICLR 2025. 22 pages, 6 figures. Code/models: https://github.com/amith-ananthram/see-it-from-my-perspective

VidLBEval: Benchmarking and Mitigating Language Bias in Video-Involved LVLMs

Yiming Yang, Yangyang Guo, Hui Lu, Yan Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Can Hallucination Correction Improve Video-Language Alignment?

Lingjun Zhao, Mingyang Xie, Paola Cascante-Bonilla, Hal Daumé, Kwonjoon Lee

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions

Yuhan Fu, Ruobing Xie, Jiazhen Liu, Bangxiang Lan, Xingwu Sun, Zhanhui Kang, Xirong Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments The proposed method does not work for up-to-date MLLMs.

Understanding and Rectifying Safety Perception Distortion in VLMs

Xiaohan Zou, Jian Kang, George Kesidis, Lu Lin

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

A Multimodal Automated Interpretability Agent

Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, Antonio Torralba

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 25 pages, 13 figures

Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguity

Kaiqu Liang, Zixu Zhang, Jaime Fernández Fisac

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments NeurIPS 2024

Embodied Red Teaming for Auditing Robotic Foundation Models

Sathwik Karnik, Zhang-Wei Hong, Nishant Abhangi, Yen-Chen Lin, Tsun-Hsuan Wang, Christophe Dupuy, Rahul Gupta, Pulkit Agrawal

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.AI、cs.LG

Color in Visual-Language Models: CLIP deficiencies

Guillem Arias, Ramon Baldrich, Maria Vanrell

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 10 figures, conference, Artificial Intelligence

Journal ref in Color and Imaging Conference, 2024, pp 101 - 106

A comparison between humans and AI at recognizing objects in unusual poses

Netta Ollikka, Amro Abbas, Andrea Perin, Markku Kilpeläinen, Stéphane Deny

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments version accepted at TMLR

BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation

Pablo Morales-Álvarez, Stergios Christodoulidis, Maria Vakalopoulou, Pablo Piantanida, Jose Dolz

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments 30 pages, 5 figures, 23 tables

RAG-Check: Evaluating Multimodal Retrieval Augmented Generation Performance

Matin Mortaheb, Mohammad A. Amir Khojastepour, Srimat T. Chakradhar, Sennur Ulukus

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AI

Liang Zhang, Jionghao Lin, John Sabatini, Conrad Borchers, Daniel Weitekamp, Meng Cao, John Hollander, Xiangen Hu, Arthur C. Graesser

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI、cs.LG

RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Models

Seulki Park, Daeho Um, Hajung Yoon, Sanghyuk Chun, Sangdoo Yun, Jin Young Choi

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ECCV Synthetic Data for Computer Vision Workshop (Oral)

Gaussian Splatting Under Attack: Investigating Adversarial Noise in 3D Objects

Abdurrahman Zeybey, Mehmet Ergezer, Tommy Nguyen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to Safe Generative AI Workshop @ NeurIPS 2024: https://neurips.cc/virtual/2024/workshop/84705

Journal ref Safe Generative AI Workshop @ NeurIPS 2024

Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs

Xiaofeng Zhang, Yihao Quan, Chaochen Gu, Chen Shen, Xiaosong Yuan, Shaotian Yan, Hao Cheng, Kaijie Wu, Jieping Ye

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

A Sober Look at the Robustness of CLIPs to Spurious Features

Qizhou Wang, Yong Lin, Yongqiang Chen, Ludwig Schmidt, Bo Han, Tong Zhang

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2024; Qizhou Wang, Yong Lin, and Yongqiang Chen contributed equally; Project page: https://counteranimal.github.io

Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations

Chenyu You, Yifei Min, Weicheng Dai, Jasjeet S. Sekhon, Lawrence Staib, James S. Duncan

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted by CVPR 2024