视觉大模型 / VLM - arXivDaily 专题

2410.22194 2024-10-30 cs.AI cs.CL cs.CV 62%

ADAM: An Embodied Causal Agent in Open-World Environments

Shu Yu, Chaochao Lu

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2410.20265 2024-10-29 cs.CV cs.CY cs.LG 62%

You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models

Eric Slyman, Anirudh Kanneganti, Sanghyun Hong, Stefan Lee

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Workshop paper at NeurIPS 2024 RBFM. 6 pages, 3 figures

2410.09365 2024-10-15 cs.CV cs.LG 62%

Debiasing Vison-Language Models with Text-Only Training

Yunfan Yang, Chaoquan Jiang, Zhiyu Lin, Jinlin Xiao, Jiaming Zhang, Jitao Sang

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

2410.03176 2024-10-07 cs.CV cs.AI 62%

Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models

Yufang Liu, Tao Ji, Changzhi Sun, Yuanbin Wu, Aimin Zhou

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2024

2405.02954 2024-10-04 cs.CV cs.LG 62%

Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training

Wenyu Zhang, Li Shen, Chuan-Sheng Foo

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Extension of ICCV paper arXiv:2212.07585; Published at IJCV

2312.10813 2024-09-12 cs.CV cs.CL cs.LG 62%

Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability

Tianxiang Hao, Mengyao Lyu, Hui Chen, Sicheng Zhao, Xiaohan Ding, Jungong Han, Guiguang Ding

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

2409.02882 2024-09-05 cs.CV cs.LG 62%

Benchmarking Spurious Bias in Few-Shot Image Classifiers

Guangtao Zheng, Wenqian Ye, Aidong Zhang

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ECCV 2024

2409.01151 2024-09-04 cs.CV cs.LG 62%

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Yueqian Wang, Jianxin Liang, Yuxuan Wang, Huishuai Zhang, Dongyan Zhao

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；分类 cs.CV、cs.LG

2408.11866 2024-08-23 cs.CL cs.AI cs.LG q-bio.BM 62%

Crossing New Frontiers: Knowledge-Augmented Large Language Model Prompting for Zero-Shot Text-Based De Novo Molecule Design

Sakhinana Sagar Srinivas, Venkataramana Runkana

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments Paper was accepted at R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Foundation Models, NeurIPS-2023. Please find the links: accepted-papers?authuser=0" target="_blank" rel="noopener">https://sites.google.com/view/r0-fomo/accepted-papers?authuser=0 and https://neurips.cc/virtual/2023/workshop/66517

2408.05446 2024-08-13 cs.CV cs.LG 62%

Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness

Stanislav Fort, Balaji Lakshminarayanan

专题命中幻觉与鲁棒性：vision language model(abstract)；分类 cs.CV、cs.LG

Comments 34 pages, 25 figures, appendix

2407.17060 2024-07-25 cs.CV cs.AI cs.CL eess.IV 62%

High Efficiency Image Compression for Large Visual-Language Models

Binzhe Li, Shurun Wang, Shiqi Wang, Yan Ye

专题命中幻觉与鲁棒性：visual language model(abstract)；分类 cs.CV、cs.AI

2401.14555 2024-06-26 cs.CV cs.LG 62%

Revisiting Active Learning in the Era of Vision Foundation Models

Sanket Rajan Gupte, Josiah Aklilu, Jeffrey J. Nirschl, Serena Yeung-Levy

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to TMLR

2406.12663 2024-06-19 cs.CV cs.AI 62%

Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?

Mingqian Feng, Yunlong Tang, Zeliang Zhang, Chenliang Xu

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

2406.11331 2024-06-18 cs.CV cs.IR cs.LG 62%

They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Bias

Salma Abdel Magid, Jui-Hsien Wang, Kushal Kafle, Hanspeter Pfister

专题命中幻觉与鲁棒性：vision language model(abstract)；分类 cs.CV、cs.LG

2405.19334 2024-06-11 cs.AI cs.CL cs.CV cs.MM cs.SD 62%

LLMs Meet Multimodal Generation and Editing: A Survey

Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Runtao Liu, Ruibin Yuan, Yazhou Xing, Wenhai Wang, Jifeng Dai, Yong Zhang, Wei Xue, Qifeng Liu, Yike Guo, Qifeng Chen

专题命中幻觉与鲁棒性：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 52 Pages with 16 Figures, 12 Tables, and 545 References. GitHub Repository at: https://github.com/YingqingHe/Awesome-LLMs-meet-Multimodal-Generation

2405.16860 2024-05-28 cs.CV cs.AI 62%

Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks

Yunqi Zhang, Songda Li, Chunyuan Deng, Luyi Wang, Hui Zhao

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accept to NAACL 2024(main)

2405.11629 2024-05-21 cs.CV cs.AI 62%

Searching Realistic-Looking Adversarial Objects For Autonomous Driving Systems

Shengxiang Sun, Shenzhe Zhu

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

2405.11154 2024-05-21 cs.CV cs.AI 62%

Revisiting the Robust Generalization of Adversarial Prompt Tuning

Fan Yang, Mingxuan Xia, Sangzhou Xia, Chicheng Ma, Hui Hui

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.09921 2024-04-16 cs.CV cs.AI 62%

Zero-shot Building Age Classification from Facade Image Using GPT-4

Zichao Zeng, June Moh Goo, Xinglei Wang, Bin Chi, Meihui Wang, Jan Boehm

专题命中幻觉与鲁棒性：vision language model(abstract)；分类 cs.CV、cs.AI

2404.08958 2024-04-16 cs.CV cs.CL cs.LG 62%

AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning

Yuwei Tang, Zhenyi Lin, Qilong Wang, Pengfei Zhu, Qinghua Hu

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted by CVPR 2024

2403.07888 2024-04-03 cs.CV cs.AI 62%

Cross-modality debiasing: using language to mitigate sub-population shifts in imaging

Yijiang Pang, Bao Hoang, Jiayu Zhou

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.00312 2024-04-02 cs.CV cs.AI 62%

Bayesian Exploration of Pre-trained Models for Low-shot Image Classification

Yibo Miao, Yu Lei, Feng Zhou, Zhijie Deng

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

2303.15230 2024-03-27 cs.CV cs.CL cs.LG 62%

Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning

Siteng Huang, Biao Gong, Yutong Feng, Min Zhang, Yiliang Lv, Donglin Wang

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments CVPR 2024

2310.16226 2024-03-22 cs.CV cs.CL cs.LG 62%

TiC-CLIP: Continual Training of CLIP Models

Saurabh Garg, Mehrdad Farajtabar, Hadi Pouransari, Raviteja Vemulapalli, Sachin Mehta, Oncel Tuzel, Vaishaal Shankar, Fartash Faghri

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments ICLR 2024

2311.16101 2023-11-28 cs.CV cs.CL cs.LG 62%

How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs

Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, Cihang Xie

专题命中幻觉与鲁棒性：visual reasoning(abstract)；分类 cs.CV、cs.LG

Comments H.T., C.C., and Z.W. contribute equally. Work done during H.T. and Z.W.'s internship at UCSC, and C.C. and Y.Z.'s internship at UNC

2311.14029 2023-11-27 cs.CV cs.LG 62%

Understanding the Vulnerability of CLIP to Image Compression

Cangxiong Chen, Vinay P. Namboodiri, Julian Padget

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments R0-FoMo: Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models at NeurIPS 2023

2309.14356 2023-11-01 cs.LG cs.CL cs.CV 62%

COCO-Counterfactuals: Automatically Constructed Counterfactual Examples for Image-Text Pairs

Tiep Le, Vasudev Lal, Phillip Howard

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2023 Datasets and Benchmarks Track

2310.03182 2023-10-06 cs.CV cs.CL cs.LG 62%

Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models

An Yan, Yu Wang, Yiwu Zhong, Zexue He, Petros Karypis, Zihan Wang, Chengyu Dong, Amilcare Gentili, Chun-Nan Hsu, Jingbo Shang, Julian McAuley

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments 18 pages, 12 figures

2307.10490 2023-10-04 cs.CR cs.AI cs.CL cs.LG 62%

Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, Vitaly Shmatikov

专题命中幻觉与鲁棒性：LLaVA(abstract)；分类 cs.AI、cs.LG

2309.15251 2023-09-28 cs.CV cs.AI 62%

VPA: Fully Test-Time Visual Prompt Adaptation

Jiachen Sun, Mark Ibrahim, Melissa Hall, Ivan Evtimov, Z. Morley Mao, Cristian Canton Ferrer, Caner Hazirbas

专题命中幻觉与鲁棒性：vision-language model(abstract)；分类 cs.CV、cs.AI

1. 幻觉与鲁棒性 2251 篇

ADAM: An Embodied Causal Agent in Open-World Environments

You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models

Debiasing Vison-Language Models with Text-Only Training

Investigating and Mitigating Object Hallucinations in Pretrained Vision-Language (CLIP) Models

Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training

Towards Efficient Vision-Language Tuning: More Information Density, More Generalizability

Benchmarking Spurious Bias in Few-Shot Image Classifiers

Understanding Multimodal Hallucination with Parameter-Free Representation Alignment

Crossing New Frontiers: Knowledge-Augmented Large Language Model Prompting for Zero-Shot Text-Based De Novo Molecule Design

Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness

High Efficiency Image Compression for Large Visual-Language Models

Revisiting Active Learning in the Era of Vision Foundation Models

Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?

They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Bias

LLMs Meet Multimodal Generation and Editing: A Survey

Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks

Searching Realistic-Looking Adversarial Objects For Autonomous Driving Systems

Revisiting the Robust Generalization of Adversarial Prompt Tuning

Zero-shot Building Age Classification from Facade Image Using GPT-4

AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning

Cross-modality debiasing: using language to mitigate sub-population shifts in imaging

Bayesian Exploration of Pre-trained Models for Low-shot Image Classification

Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learning

TiC-CLIP: Continual Training of CLIP Models

How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs

Understanding the Vulnerability of CLIP to Image Compression

COCO-Counterfactuals: Automatically Constructed Counterfactual Examples for Image-Text Pairs

Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models

Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

VPA: Fully Test-Time Visual Prompt Adaptation