视觉大模型 / VLM

Improving Bilingual Capabilities of Language Models to Support Diverse Linguistic Practices in Education

Anand Syamkumar, Nora Tseng, Kaycie Barron, Shanglin Yang, Shamya Karumbaiah, Rheeya Uppal, Junjie Hu

专题命中 幻觉与鲁棒性 ：MLLM(abstract)；分类 cs.AI

TorchOpera: A Compound AI System for LLM Safety

Shanshan Han, Zijian Hu, Alay Dilipbhai Shah, Han Jin, Yuhang Yao, Dimitris Stripelis, Zhaozhuo Xu, Chaoyang He

专题命中 幻觉与鲁棒性 ：grounding(abstract)；分类 cs.AI

Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting

Xingyu Zhu, Beier Zhu, Yi Tan, Shuo Wang, Yanbin Hao, Hanwang Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2024 Spotlight

ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users

Guanlin Li, Kangjie Chen, Shudong Zhang, Jie Zhang, Tianwei Zhang

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.AI

Comments Accepted by NeurIPS 2024

HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding

Fan Yuan, Chi Qin, Xiaogang Xu, Piji Li

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at Main Conference of EMNLP 2024

Can CLIP Count Stars? An Empirical Study on Quantity Bias in CLIP

Zeliang Zhang, Zhuo Liu, Mingqian Feng, Chenliang Xu

专题命中 幻觉与鲁棒性 ：visual question answering(abstract)；分类 cs.CV

Comments Short paper. Accepted by the Findings of EMNLP 2024

CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Camera

Jingpei Lu, Zekai Liang, Tristin Xie, Florian Ritcher, Shan Lin, Sainan Liu, Michael C. Yip

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 7 pages, 5 figures, project website: https://sites.google.com/ucsd.edu/ctrnet-x

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding

Xiaoyu Liang, Jiayuan Yu, Lianrui Mu, Jiedong Zhuang, Jiaqi Hu, Yuchen Yang, Jiangnan Ye, Lu Lu, Jian Chen, Haoji Hu

专题命中 幻觉与鲁棒性 ：visual question answering(abstract)；分类 cs.CV

Comments PRCV

Can LVLMs Obtain a Driver's License? A Benchmark Towards Reliable AGI for Autonomous Driving

Yuhang Lu, Yichen Yao, Jiadong Tu, Jiangnan Shao, Yuexin Ma, Xinge Zhu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination

Dingchen Yang, Bowen Cao, Guang Chen, Changjun Jiang

专题命中 幻觉与鲁棒性 ：LLaVA(abstract)；分类 cs.CV

Image-to-Text Logic Jailbreak: Your Imagination can Help You Do Anything

Xiaotian Zou, Ke Li, Yongkang Chen

专题命中 幻觉与鲁棒性 ：visual language model(abstract)；分类 cs.CV

Physics-Inspired Generative Models in Medical Imaging: A Review

Dennis Hein, Afshin Bozorgpour, Dorit Merhof, Ge Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Conditional Prototype Rectification Prompt Learning

Haoxing Chen, Yaohui Li, Zizheng Huang, Yan Hong, Zhuoer Xu, Zhangxuan Gu, Jun Lan, Huijia Zhu, Weiqiang Wang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors

Lin Guan, Yifan Zhou, Denis Liu, Yantian Zha, Heni Ben Amor, Subbarao Kambhampati

专题命中 幻觉与鲁棒性 ：VLM(abstract)；分类 cs.AI

Comments Published as a conference paper at COLM 2024

Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs

Shi Liu, Kecheng Zheng, Wei Chen

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Data Processing Techniques for Modern Multimodal Models

Yinheng Li, Han Ding, Hang Chen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding

Quan Kong, Yuki Kawana, Rajat Saini, Ashutosh Kumar, Jingjing Pan, Ta Gu, Yohei Ozao, Balazs Opra, David C. Anastasiu, Yoichi Sato, Norimasa Kobori

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV24. Website: https://woven-visionai.github.io/wts-dataset-homepage/

Adapt2Reward: Adapting Video-Language Models to Generalizable Robotic Rewards via Failure Prompts

Yanting Yang, Minghao Chen, Qibo Qiu, Jiahao Wu, Wenxiao Wang, Binbin Lin, Ziyu Guan, Xiaofei He

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV 2024 camera-ready

LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition

Teng Wang, Lingquan Meng, Lei Cheng, Changyin Sun

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Learning Robust 3D Representation from CLIP via Dual Denoising

Shuqing Luo, Bowen Qu, Wei Gao

专题命中 幻觉与鲁棒性 ：vision language model(abstract)；分类 cs.CV

Evidential Concept Embedding Models: Towards Reliable Concept Explanations for Skin Disease Diagnosis

Yibo Gao, Zheyao Gao, Xin Gao, Yuanye Liu, Bomin Wang, Xiahai Zhuang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments accepted by MICCAI 2024

Coarse-to-Fine Concept Bottleneck Models

Konstantinos P. Panousis, Dino Ienco, Diego Marcos

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.LG

Guardrails for avoiding harmful medical product recommendations and off-label promotion in generative AI models

Daniel Lopez-Martinez

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Comments CVPR 2024 Responsible Generative AI (ReGenAI) workshop

Versatile Backdoor Attack with Visible, Semantic, Sample-Specific, and Compatible Triggers

Ruotong Wang, Hongrui Chen, Zihao Zhu, Li Liu, Baoyuan Wu

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments 23 pages, 21 figures, 18 tables

VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models

Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, Zilong Zheng

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking

Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.AI

Towards Event-oriented Long Video Understanding

Yifan Du, Kun Zhou, Yuqi Huo, Yifan Li, Wayne Xin Zhao, Haoyu Lu, Zijia Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Comments Work on progress

From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment

Yusuke Hirota, Ryo Hachiuma, Chao-Han Huck Yang, Yuta Nakashima

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Hallucination Mitigation Prompts Long-term Video Understanding

Yiwei Sun, Zhihang Liu, Chuanbin Liu, Bowei Pu, Zhihan Zhang, Hongtao Xie

专题命中 幻觉与鲁棒性 ：multimodal large language model(abstract)；分类 cs.CV

Spuriousness-Aware Meta-Learning for Learning Robust Classifiers

Guangtao Zheng, Wenqian Ye, Aidong Zhang

专题命中 幻觉与鲁棒性 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to KDD 2024