视觉大模型 / VLM

M3GIA: A Cognition Inspired Multilingual and Multimodal General Intelligence Ability Benchmark

Wei Song, Yadong Li, Jianhua Xu, Guowei Wu, Lingfeng Ming, Kexin Yi, Weihua Luo, Houyi Li, Yi Du, Fangda Guo, Kaicheng Yu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.AI

Situated Ground Truths: Enhancing Bias-Aware AI by Situating Data Labels with SituAnnotate

Delfina Sol Martinez Pandiani, Valentina Presutti

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments Author preprint

Image Textualization: An Automatic Framework for Creating Accurate and Detailed Image Descriptions

Renjie Pi, Jianshu Zhang, Jipeng Zhang, Rui Pan, Zhekai Chen, Tong Zhang

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation

Enhui Ma, Lijun Zhou, Tao Tang, Zhan Zhang, Dong Han, Junpeng Jiang, Kun Zhan, Peng Jia, Xianpeng Lang, Haiyang Sun, Di Lin, Kaicheng Yu

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Comments Project Page: https://westlake-autolab.github.io/delphi.github.io/, 8 figures

ED-SAM: An Efficient Diffusion Sampling Approach to Domain Generalization in Vision-Language Foundation Models

Thanh-Dat Truong, Xin Li, Bhiksha Raj, Jackson Cothren, Khoa Luu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Automated Multi-level Preference for MLLMs

Mengxi Zhang, Wenhao Wu, Yu Lu, Yuxin Song, Kang Rong, Huanjin Yao, Jianbo Zhao, Fanglong Liu, Yifan Sun, Haocheng Feng, Jingdong Wang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Preprint

Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography

Nhat Chung, Sensen Gao, Tuan-Anh Vu, Jie Zhang, Aishan Liu, Yun Lin, Jin Song Dong, Qing Guo

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Comments 12 pages, 5 tables, 5 figures, work in progress

BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity

Andrew F. Luo, Margaret M. Henderson, Michael J. Tarr, Leila Wehbe

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

Comments ICLR 2024. Project page: https://www.cs.cmu.edu/~afluo/BrainSCUBA

Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness

Sibo Wang, Jie Zhang, Zheng Yuan, Shiguang Shan

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models

Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, Nicholas Carlini

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

HallE-Control: Controlling Object Hallucination in Large Multimodal Models

Bohan Zhai, Shijia Yang, Chenfeng Xu, Sheng Shen, Kurt Keutzer, Chunyuan Li, Manling Li

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Comments Our code is publicly available at https://github.com/bronyayang/HallE_Control

MemeCraft: Contextual and Stance-Driven Multimodal Meme Generation

Han Wang, Roy Ka-Wei Lee

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.AI

Comments 8 pages, 7 figures, ACM MM 2024

OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, Nenghai Yu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.CV

Comments CVPR 2024, code is available at https://github.com/shikiw/OPERA

$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations

Xiang Li, Kai Qiu, Jinglu Wang, Xiaohao Xu, Rita Singh, Kashu Yamazak, Hao Chen, Xiaonan Huang, Bhiksha Raj

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.CV

Comments Code and dataset will be released at https://github.com/lxa9867/r2bench

Rethinking Model Ensemble in Transfer-based Adversarial Attacks

Huanran Chen, Yichi Zhang, Yinpeng Dong, Xiao Yang, Hang Su, Jun Zhu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding

Haogeng Liu, Quanzeng You, Xiaotian Han, Yiqi Wang, Bohan Zhai, Yongfei Liu, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization

Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, Conghui He

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Project Website: https://opendatalab.github.io/HA-DPO, Code: https://github.com/opendatalab/HA-DPO

TeG-DG: Textually Guided Domain Generalization for Face Anti-Spoofing

Lianrui Mu, Jianhong Bai, Xiaoxuan He, Jiangnan Ye, Xiaoyu Liang, Yuchen Yang, Jiedong Zhuang, Haoji Hu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift

Jielin Qiu, Yi Zhu, Xingjian Shi, Florian Wenzel, Zhiqiang Tang, Ding Zhao, Bo Li, Mu Li

专题命中 幻觉与鲁棒性 ：visual reasoning(abstract)；分类 cs.CV

Comments Accepted by Journal of Data-centric Machine Learning Research (DMLR) 2024

VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution

Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shtedritski, Hannah Rose Kirk

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS Datasets and Benchmarks 2023. Data and code available at https://github.com/oxai/visogender

AutoRepo: A general framework for multi-modal LLM-based automated construction reporting

Hongxu Pu, Xincong Yang, Jing Li, Runhao Guo, Heng Li

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments We believe that keeping this version of the paper publicly available may lead to confusion or misinterpretation regarding our current research direction and findings

GELDA: A generative language annotation framework to reveal visual biases in datasets

Krish Kabra, Kathleen M. Lewis, Guha Balakrishnan

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 21 pages, 15 figures, 9 tables

Hallucination Augmented Recitations for Language Models

Abdullatif Köksal, Renat Aksitov, Chung-Ching Chang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Read-only Prompt Optimization for Vision-Language Few-shot Learning

Dongjun Lee, Seokwon Song, Jihee Suh, Joonmyung Choi, Sanghyeok Lee, Hyunwoo J. Kim

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at ICCV2023

Uniformly Distributed Category Prototype-Guided Vision-Language Framework for Long-Tail Recognition

Siming Fu, Xiaoxuan He, Xinpeng Ding, Yuchen Cao, Hualiang Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 11pages, 5figures

Journal ref ACM MM2023

ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification

Mohammad Reza Taesiri, Giang Nguyen, Sarra Habchi, Cor-Paul Bezemer, Anh Nguyen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2023 Track on Datasets and Benchmarks

Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association

Qiyu Wu, Mengjie Zhao, Yutong He, Lang Huang, Junya Ono, Hiromi Wakaki, Yuki Mitsufuji

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.CV

Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping

Adam Rashid, Satvik Sharma, Chung Min Kim, Justin Kerr, Lawrence Chen, Angjoo Kanazawa, Ken Goldberg

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments See the project website at: lerftogo.github.io

Implicit Neural Representation for Cooperative Low-light Image Enhancement

Shuzhou Yang, Moxuan Ding, Yanmin Wu, Zihan Li, Jian Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Gradient-Based Word Substitution for Obstinate Adversarial Examples Generation in Language Models

Yimu Wang, Peng Shi, Hongyang Zhang

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Comments 19 pages