视觉大模型 / VLM - arXivDaily 专题

2502.05178 2025-02-10 cs.CV 57%

QLIP: Text-Aligned Visual Tokenization Unifies Auto-Regressive Multimodal Understanding and Generation

Yue Zhao, Fuzhao Xue, Scott Reed, Linxi Fan, Yuke Zhu, Jan Kautz, Zhiding Yu, Philipp Krähenbühl, De-An Huang

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

Comments Tech report. Project page: https://nvlabs.github.io/QLIP/

2502.03836 2025-02-07 cs.CV 57%

Adapting Human Mesh Recovery with Vision-Language Feedback

Chongyang Xu, Buzhen Huang, Chengfang Zhang, Ziliang Feng, Yangang Wang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments 6 pages, 7 figures

2501.18698 2025-02-03 cs.CV 57%

Human Re-ID Meets LVLMs: What can we expect?

Kailash Hambarde, Pranita Samale, Hugo Proença

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2501.01097 2025-01-31 cs.CV 57%

EliGen: Entity-Level Controlled Image Generation with Regional Attention

Hong Zhang, Zhongjie Duan, Xingjun Wang, Yingda Chen, Yu Zhang

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.CV

2501.14548 2025-01-27 cs.CV 57%

Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding

Zhongyi Shui, Jianpeng Zhang, Weiwei Cao, Sinuo Wang, Ruizhe Guo, Le Lu, Lin Yang, Xianghua Ye, Tingbo Liang, Qi Zhang, Ling Zhang

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted by ICLR 2025

2403.04652 2025-01-22 cs.CL cs.AI 57%

Yi: Open Foundation Models by 01.AI

01. AI, :, Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yanpeng Li, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, Zonghong Dai

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.AI

2501.10144 2025-01-20 cs.CV 57%

A Vision-Language Framework for Multispectral Scene Representation Using Language-Grounded Features

Enes Karanfil, Nevrez Imamoglu, Erkut Erdem, Aykut Erdem

专题命中 VLM训练与架构：LLaVA(abstract)；分类 cs.CV

2501.09294 2025-01-17 cs.CV cs.CL 57%

Efficient Few-Shot Medical Image Analysis via Hierarchical Contrastive Vision-Language Learning

Harrison Fuller, Fernando Gabriela Garcia, Victor Flores

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2501.08313 2025-01-15 cs.CL cs.CV 57%

MiniMax-01: Scaling Foundation Models with Lightning Attention

MiniMax, Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, Enwei Jiao, Gengxin Li, Guojun Zhang, Haohai Sun, Houze Dong, Jiadai Zhu, Jiaqi Zhuang, Jiayuan Song, Jin Zhu, Jingtao Han, Jingyang Li, Junbin Xie, Junhao Xu, Junjie Yan, Kaishun Zhang, Kecheng Xiao, Kexi Kang, Le Han, Leyang Wang, Lianfei Yu, Liheng Feng, Lin Zheng, Linbo Chai, Long Xing, Meizhi Ju, Mingyuan Chi, Mozhi Zhang, Peikai Huang, Pengcheng Niu, Pengfei Li, Pengyu Zhao, Qi Yang, Qidi Xu, Qiexiang Wang, Qin Wang, Qiuhui Li, Ruitao Leng, Shengmin Shi, Shuqi Yu, Sichen Li, Songquan Zhu, Tao Huang, Tianrun Liang, Weigao Sun, Weixuan Sun, Weiyu Cheng, Wenkai Li, Xiangjun Song, Xiao Su, Xiaodong Han, Xinjie Zhang, Xinzhu Hou, Xu Min, Xun Zou, Xuyang Shen, Yan Gong, Yingjie Zhu, Yipeng Zhou, Yiran Zhong, Yongyi Hu, Yuanxiang Fan, Yue Yu, Yufeng Yang, Yuhao Li, Yunan Huang, Yunji Li, Yunpeng Huang, Yunzhi Xu, Yuxin Mao, Zehan Li, Zekang Li, Zewei Tao, Zewen Ying, Zhaoyang Cong, Zhen Qin, Zhenhua Fan, Zhihang Yu, Zhuo Jiang, Zijia Wu

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments A technical report from MiniMax. The authors are listed in alphabetical order. We open-sourced our MiniMax-01 at https://github.com/MiniMax-AI

2501.08168 2025-01-15 cs.AI 57%

LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinking

Yukai Ma, Tiantian Wei, Naiting Zhong, Jianbiao Mei, Tao Hu, Licheng Wen, Xuemeng Yang, Botian Shi, Yong Liu

专题命中 VLM训练与架构：visual language model(abstract)；分类 cs.AI

2409.05427 2025-01-10 cs.CV 57%

TextToucher: Fine-Grained Text-to-Touch Generation

Jiahang Tu, Hao Fu, Fengyu Yang, Hanbin Zhao, Chao Zhang, Hui Qian

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

Comments This paper has been accepted by AAAI 2025

2501.01535 2025-01-06 cs.HC cs.AI cs.CL cs.CY 57%

A Metasemantic-Metapragmatic Framework for Taxonomizing Multimodal Communicative Alignment

Eugene Yu Ji

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments 34 pages, 1 figure, 3 tables. Draft presented at 2023 ZJU Logic and AI Summit EAI Workshop

2501.00917 2025-01-03 cs.CV 57%

Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Models

Emily Johnson, Noah Wilson

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2412.18176 2024-12-31 cs.IR cs.AI 57%

Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation

Yucong Luo, Qitao Qin, Hao Zhang, Mingyue Cheng, Ruiran Yan, Kefan Wang, Jie Ouyang

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI

2412.16334 2024-12-24 cs.CV 57%

DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment

Cijo Jose, Théo Moutakanni, Dahyun Kang, Federico Baldassarre, Timothée Darcet, Hu Xu, Daniel Li, Marc Szafraniec, Michaël Ramamonjisoa, Maxime Oquab, Oriane Siméoni, Huy V. Vo, Patrick Labatut, Piotr Bojanowski

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2412.15453 2024-12-23 cs.CL cs.AI 57%

Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization

Sahil Wadhwa, Chengtian Xu, Haoming Chen, Aakash Mahalingam, Akankshya Kar, Divya Chaudhary

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments 10 pages, 6 tables, 1 figure, The First Workshop on Multilingual Counterspeech Generation (MCG) at The 31st International Conference on Computational Linguistics (COLING 2025)

Journal ref The First Workshop on Multilingual Counterspeech Generation (MCG) at The 31st International Conference on Computational Linguistics (COLING 2025)

2412.14837 2024-12-20 cs.CV 57%

ObjVariantEnsemble: Advancing Point Cloud LLM Evaluation in Challenging Scenes with Subtly Distinguished Objects

Qihang Cao, Huangxun Chen

专题命中 VLM训练与架构：VLM(abstract)；分类 cs.CV

Comments Accepted to AAAI2025

2412.14475 2024-12-20 cs.CV cs.CL 57%

MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval

Junjie Zhou, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian, Yongping Xiong

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

2412.13746 2024-12-19 cs.CL cs.AI cs.IR 57%

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment

Zhuoran Jin, Hongbang Yuan, Tianyi Men, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao

专题命中 VLM训练与架构：grounding(abstract)；分类 cs.AI

Comments 26 pages, 12 figures, 6 tables

2407.11780 2024-12-19 cs.CL cs.AI 57%

SwitchCIT: Switching for Continual Instruction Tuning

Xinbo Wu, Max Hartman, Vidhata Arjun Jayaraman, Lav R. Varshney

专题命中 VLM训练与架构：visual reasoning(abstract)；分类 cs.AI

2412.12500 2024-12-18 cs.CL cs.AI 57%

Beyond Data Quantity: Key Factors Driving Performance in Multilingual Language Models

Sina Bagheri Nezhad, Ameeta Agrawal, Rhitabrat Pokharel

专题命中 VLM训练与架构：MLLM(abstract)；分类 cs.AI

Comments Accepted at The First Workshop on Language Models for Low-Resource Languages @ COLING 2025

2311.07449 2024-12-18 cs.CV 57%

Semantically Grounded QFormer for Efficient Vision Language Understanding

Moulik Choraria, Xinbo Wu, Sourya Basu, Nitesh Sekhar, Yue Wu, Xu Zhang, Prateek Singhal, Lav R. Varshney

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments Preprint Under Review

2412.11663 2024-12-17 cs.CV cs.MM 57%

LMM-Regularized CLIP Embeddings for Image Classification

Maria Tzelepi, Vasileios Mezaris

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

Comments Accepted for publication, 26th Int. Symp. on Multimedia (IEEE ISM 2024), Tokyo, Japan, Dec. 2024. This is the authors' "accepted version"

2412.11196 2024-12-17 cs.CL cs.CV 57%

Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal

Yuhao Wang, Zhiyuan Zhu, Heyang Liu, Yusheng Liao, Hongcheng Liu, Yanfeng Wang, Yu Wang

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

2412.10843 2024-12-17 cs.CV 57%

Learning Semantic-Aware Representation in Visual-Language Models for Multi-Label Recognition with Partial Labels

Haoxian Ruan, Zhihua Xu, Zhijing Yang, Yongyi Lu, Jinghui Qin, Tianshui Chen

专题命中 VLM训练与架构：vision language model(abstract)；分类 cs.CV

Comments ACM Transactions on Multimedia Computing Communications and Applications

2412.10758 2024-12-17 cs.CV 57%

Optimizing Vision-Language Interactions Through Decoder-Only Models

Kaito Tanaka, Benjamin Tan, Brian Wong

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2412.10720 2024-12-17 cs.CV 57%

Bridging Vision and Language: Modeling Causality and Temporality in Video Narratives

Ji-jun Park, Soo-joon Choi

专题命中 VLM训练与架构：vision-language model(abstract)；分类 cs.CV

2407.15838 2024-12-17 cs.CV 57%

MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity

Yangzhou Liu, Yue Cao, Zhangwei Gao, Weiyun Wang, Zhe Chen, Wenhai Wang, Hao Tian, Lewei Lu, Xizhou Zhu, Tong Lu, Yu Qiao, Jifeng Dai

专题命中 VLM训练与架构：visual question answering(abstract)；分类 cs.CV

Comments 18 pages, 8 figures, technical report

Journal ref Sci China Inf Sci, 2024

2412.09549 2024-12-13 cs.CV 57%

Exemplar Masking for Multimodal Incremental Learning

Yi-Lun Lee, Chen-Yu Lee, Wei-Chen Chiu, Yi-Hsuan Tsai

专题命中 VLM训练与架构：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://github.com/YiLunLee/Exemplar_Masking_MCIL

2412.07119 2024-12-11 cs.CV 57%

DiffCLIP: Few-shot Language-driven Multimodal Classifier

Jiaqing Zhang, Mingxiang Cao, Xue Yang, Kai Jiang, Yunsong Li

专题命中 VLM训练与架构：visual language model(abstract)；分类 cs.CV

1. VLM训练与架构 5058 篇

QLIP: Text-Aligned Visual Tokenization Unifies Auto-Regressive Multimodal Understanding and Generation

Adapting Human Mesh Recovery with Vision-Language Feedback

Human Re-ID Meets LVLMs: What can we expect?

EliGen: Entity-Level Controlled Image Generation with Regional Attention

Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding

Yi: Open Foundation Models by 01.AI

A Vision-Language Framework for Multispectral Scene Representation Using Language-Grounded Features

Efficient Few-Shot Medical Image Analysis via Hierarchical Contrastive Vision-Language Learning

MiniMax-01: Scaling Foundation Models with Lightning Attention

LeapVAD: A Leap in Autonomous Driving via Cognitive Perception and Dual-Process Thinking

TextToucher: Fine-Grained Text-to-Touch Generation

A Metasemantic-Metapragmatic Framework for Taxonomizing Multimodal Communicative Alignment

Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Models

Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation

DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment

Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization

ObjVariantEnsemble: Advancing Point Cloud LLM Evaluation in Challenging Scenes with Subtly Distinguished Objects

MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment

SwitchCIT: Switching for Continual Instruction Tuning

Beyond Data Quantity: Key Factors Driving Performance in Multilingual Language Models

Semantically Grounded QFormer for Efficient Vision Language Understanding

LMM-Regularized CLIP Embeddings for Image Classification

Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal

Learning Semantic-Aware Representation in Visual-Language Models for Multi-Label Recognition with Partial Labels

Optimizing Vision-Language Interactions Through Decoder-Only Models

Bridging Vision and Language: Modeling Causality and Temporality in Video Narratives

MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity

Exemplar Masking for Multimodal Incremental Learning

DiffCLIP: Few-shot Language-driven Multimodal Classifier