多模态大模型 - arXivDaily 专题

2501.14905 2025-01-28 cs.CV 70%

Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing

Madeline Anderson, Miriam Cha, William T. Freeman, J. Taylor Perron, Nathaniel Maidel, Kerri Cahoy

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2501.11231 2025-01-22 cs.CV 70%

KPL: Training-Free Medical Knowledge Mining of Vision-Language Models

Jiaxiang Liu, Tianxiang Hu, Jiawei Du, Ruiyuan Zhang, Joey Tianyi Zhou, Zuozhu Liu

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments AAAI(Oral)

2501.08580 2025-01-16 cs.CV 70%

Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

Jiaqi Huang, Zunnan Xu, Ting Liu, Yong Liu, Haonan Han, Kehong Yuan, Xiu Li

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI2025

2501.06430 2025-01-14 cs.CV 70%

Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs

Shan Zhang, Aotian Chen, Yanpeng Sun, Jindong Gu, Yi-Yu Zheng, Piotr Koniusz, Kai Zou, Anton van den Hengel, Yuan Xue

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

2401.17851 2025-01-03 cs.CV 70%

Instruction-Guided Scene Text Recognition

Yongkun Du, Zhineng Chen, Yuchen Su, Caiyan Jia, Yu-Gang Jiang

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by TPAMI

2412.17544 2024-12-24 cs.AI 70%

Retention Score: Quantifying Jailbreak Risks for Vision Language Models

Zaitang Li, Pin-Yu Chen, Tsung-Yi Ho

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.AI

Comments 14 pages, 8 figures, AAAI 2025

Journal ref AAAI 2025

2412.13364 2024-12-19 cs.CV 70%

Bringing Multimodality to Amazon Visual Search System

Xinliang Zhu, Michael Huang, Han Ding, Jinyu Yang, Kelvin Chen, Tao Zhou, Tal Neiman, Ouye Xie, Son Tran, Benjamin Yao, Doug Gray, Anuj Bindal, Arnab Dhua

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Journal ref Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024

2412.12492 2024-12-18 cs.CV 70%

DuSSS: Dual Semantic Similarity-Supervised Vision-Language Model for Semi-Supervised Medical Image Segmentation

Qingtao Pan, Wenhao Qiao, Jingjiao Lou, Bing Ji, Shuo Li

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

2402.12121 2024-12-17 cs.CL cs.AI cs.CV cs.MM 70%

IRR: Image Review Ranking Framework for Evaluating Vision-Language Models

Kazuki Hayashi, Kazuma Onishi, Toma Suzuki, Yusuke Ide, Seiji Gobara, Shigeki Saito, Yusuke Sakai, Hidetaka Kamigaito, Katsuhiko Hayashi, Taro Watanabe

专题命中图文多模态：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 18pages, Accepted at COLING25

2407.07485 2024-12-12 cs.CV 70%

Zero-Shot Class Unlearning in CLIP with Synthetic Samples

A. Kravets, V. Namboodiri

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Published in WACV 2025 conference

2412.06292 2024-12-10 cs.CV 70%

ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models

Bingchen Gong, Diego Gomez, Abdullah Hamdi, Abdelrahman Eldesokey, Ahmed Abdelreheem, Peter Wonka, Maks Ovsjanikov

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Project website is accessible at https://sites.google.com/view/zerokey

2409.14607 2024-12-02 cs.CV cs.LG 70%

Patch Ranking: Efficient CLIP by Learning to Rank Local Patches

Cheng-En Wu, Jinhong Lin, Yu Hen Hu, Pedro Morgado

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by WACV 2025

2409.08464 2024-12-02 cs.CV 70%

VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation

Hanning Chen, Yang Ni, Wenjun Huang, Yezi Liu, SungHeon Jeong, Fei Wen, Nathaniel Bastian, Hugo Latapie, Mohsen Imani

专题命中图文多模态：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted at WACV 2025

2411.15411 2024-11-26 cs.CV 70%

FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity

Hang Hua, Qing Liu, Lingzhi Zhang, Jing Shi, Zhifei Zhang, Yilin Wang, Jianming Zhang, Jiebo Luo

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Preprint

2411.02669 2024-11-06 cs.CV 70%

Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack

Xiaojun Jia, Sensen Gao, Qing Guo, Ke Ma, Yihao Huang, Simeng Qin, Yang Liu, Ivor Tsang Fellow, Xiaochun Cao

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2410.23370 2024-11-01 cs.CV 70%

Multilingual Vision-Language Pre-training for the Remote Sensing Domain

João Daniel Silva, Joao Magalhaes, Devis Tuia, Bruno Martins

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at ACM SIGSPATIAL 2024 - Research Papers

2312.12735 2024-10-31 cs.CV 70%

MetaSegNet: Metadata-collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images

Libo Wang, Sijun Dong, Ying Chen, Xiaoliang Meng, Shenghui Fang, Songlin Fei

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by IEEE Transactions on Geoscience and Remote Sensing

2410.08021 2024-10-28 cs.CV 70%

OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling

Linhui Xiao, Xiaoshan Yang, Fang Peng, Yaowei Wang, Changsheng Xu

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024. The project page: https://github.com/linhuixiao/OneRef

2410.11639 2024-10-17 cs.CV 70%

Efficient and Effective Universal Adversarial Attack against Vision-Language Pre-training Models

Fan Yang, Yihao Huang, Kailong Wang, Ling Shi, Geguang Pu, Yang Liu, Haoyu Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 11 pages

2410.11255 2024-10-16 cs.CV 70%

CLIP-DFGS: A Hard Sample Mining Method for CLIP in Generalizable Person Re-Identification

Huazhong Zhao, Lei Qi, Xin Geng

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by ACM TOMM

2410.09733 2024-10-15 cs.CV 70%

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

Hang Hua, Yunlong Tang, Ziyun Zeng, Liangliang Cao, Zhengyuan Yang, Hangfeng He, Chenliang Xu, Jiebo Luo

专题命中图文多模态：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 21 pages, 15 figures

2405.14156 2024-10-08 cs.CV 70%

Unveiling the Tapestry of Consistency in Large Vision-Language Models

Yuan Zhang, Fei Xiao, Tao Huang, Chun-Kai Fan, Hongyuan Dong, Jiawen Li, Jiacong Wang, Kuan Cheng, Shanghang Zhang, Haoyuan Guo

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024

2409.09582 2024-09-25 cs.CV 70%

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

Yiyi Tao, Zhuoyue Wang, Hang Zhang, Lun Wang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2407.20171 2024-08-27 cs.CV 70%

Diffusion Feedback Helps CLIP See Better

Wenxuan Wang, Quan Sun, Fan Zhang, Yepeng Tang, Jing Liu, Xinlong Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2408.09530 2024-08-20 cs.AI 70%

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

Dawei Dai, Yuanhui Zhang, Long Xu, Qianlan Yang, Xiaojing Shen, Shuyin Xia, Guoyin Wang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.AI

Comments 8 pages, 4 figs

2407.18589 2024-07-29 cs.CV 70%

HICEScore: A Hierarchical Metric for Image Captioning Evaluation

Zequn Zeng, Jianqiao Sun, Hao Zhang, Tiansheng Wen, Yudi Su, Yan Xie, Zhengjue Wang, Bo Chen

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by ACM MM2024

2311.16254 2024-07-25 cs.CV cs.AI cs.CL cs.MM 70%

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

Samuele Poppi, Tobia Poppi, Federico Cocchi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ECCV 2024

2407.14177 2024-07-22 cs.CV 70%

EVLM: An Efficient Vision-Language Model for Visual Understanding

Kaibing Chen, Dong Shen, Hanwen Zhong, Huasong Zhong, Kui Xia, Di Xu, Wei Yuan, Yifei Hu, Bin Wen, Tianke Zhang, Changyi Liu, Dewen Fan, Huihui Xiao, Jiahong Wu, Fan Yang, Size Li, Di Zhang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2403.12445 2024-07-16 cs.CV 70%

Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory

Sensen Gao, Xiaojun Jia, Xuhong Ren, Ivor Tsang, Qing Guo

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments ECCV2024. Code is available at https://github.com/SensenGao/VLPTransferAttack

2407.07403 2024-07-15 cs.CV 70%

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, Wei Hu

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

1. 图文多模态 4651 篇

Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensing

KPL: Training-Free Medical Knowledge Mining of Vision-Language Models

Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

Open Eyes, Then Reason: Fine-grained Visual Mathematical Understanding in MLLMs

Instruction-Guided Scene Text Recognition

Retention Score: Quantifying Jailbreak Risks for Vision Language Models

Bringing Multimodality to Amazon Visual Search System

DuSSS: Dual Semantic Similarity-Supervised Vision-Language Model for Semi-Supervised Medical Image Segmentation

IRR: Image Review Ranking Framework for Evaluating Vision-Language Models

Zero-Shot Class Unlearning in CLIP with Synthetic Samples

ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models

Patch Ranking: Efficient CLIP by Learning to Rank Local Patches

VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation

FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity

Semantic-Aligned Adversarial Evolution Triangle for High-Transferability Vision-Language Attack

Multilingual Vision-Language Pre-training for the Remote Sensing Domain

MetaSegNet: Metadata-collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images

OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modeling

Efficient and Effective Universal Adversarial Attack against Vision-Language Pre-training Models

CLIP-DFGS: A Hard Sample Mining Method for CLIP in Generalizable Person Re-Identification

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models

Unveiling the Tapestry of Consistency in Large Vision-Language Models

NEVLP: Noise-Robust Framework for Efficient Vision-Language Pre-training

Diffusion Feedback Helps CLIP See Better

PA-LLaVA: A Large Language-Vision Assistant for Human Pathology Image Understanding

HICEScore: A Hierarchical Metric for Image Captioning Evaluation

Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models

EVLM: An Efficient Vision-Language Model for Visual Understanding

Boosting Transferability in Vision-Language Attacks via Diversification along the Intersection Region of Adversarial Trajectory

A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends