视觉大模型 / VLM

[CLS] Token Tells Everything Needed for Training-free Efficient MLLMs

Ao Wang, Fengyuan Sun, Hui Chen, Zijia Lin, Jungong Han, Guiguang Ding

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments 12 pages,4 figures

Cognitive Architecture Toward Common Ground Sharing Among Humans and Generative AIs: Trial on Model-Model Interactions in Tangram Naming Task

Junya Morita, Tatsuya Yui, Takeru Amaya, Ryuichiro Higashinaka, Yugo Takeuchi

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.AI

Comments Proceedings of the 2023 AAAI Fall Symposium on Integrating Cognitive Architectures and Generative Models

Journal ref Proceedings of the 2023 AAAI Fall Symposia, vol.2, no. 1, 2023

Multimodal Instruction Tuning with Conditional Mixture of LoRA

Ying Shen, Zhiyang Xu, Qifan Wang, Yu Cheng, Wenpeng Yin, Lifu Huang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments 12 pages, 7 figures, ACL 2024

A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds

Shiwali Mohan, Wiktor Piotrowski, Roni Stern, Sachin Grover, Sookyung Kim, Jacob Le, Johan De Kleer

专题命中 VLM训练与架构 ：visual reasoning(abstract)；分类 cs.AI

Journal ref Artificial Intelligence (2024), Volume 334, Issue C

LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant

Yikun Liu, Pingan Chen, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiangchao Yao, Yanfeng Wang, Weidi Xie

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Efficient Multi-modal Large Language Models via Visual Token Grouping

Minbin Huang, Runhui Huang, Han Shi, Yimeng Chen, Chuanyang Zheng, Xiangguo Sun, Xin Jiang, Zhenguo Li, Hong Cheng

专题命中 VLM训练与架构 ：visual question answering(abstract)；分类 cs.CV

Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-training

Haicheng Wang, Chen Ju, Weixiong Lin, Shuai Xiao, Mengting Chen, Yixuan Huang, Chang Liu, Mingshuai Yao, Jinsong Lan, Ying Chen, Qingwen Liu, Yanfeng Wang

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

DistinctAD: Distinctive Audio Description Generation in Contexts

Bo Fang, Wenhao Wu, Qiangqiang Wu, Yuxin Song, Antoni B. Chan

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

SketchAgent: Language-Driven Sequential Sketch Generation

Yael Vinker, Tamar Rott Shaham, Kristine Zheng, Alex Zhao, Judith E Fan, Antonio Torralba

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments project page: https://sketch-agent.csail.mit.edu/

Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer

Zengqun Zhao, Yu Cao, Shaogang Gong, Ioannis Patras

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at WACV 2025 (Camera-Ready Version)

Octavius: Mitigating Task Interference in MLLMs via LoRA-MoE

Zeren Chen, Ziqin Wang, Zhen Wang, Huayang Liu, Zhenfei Yin, Si Liu, Lu Sheng, Wanli Ouyang, Yu Qiao, Jing Shao

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments 22 pages, 12 figures. Accepted in ICLR 2024

Exploring Large Language Models for Multimodal Sentiment Analysis: Challenges, Benchmarks, and Future Directions

Shezheng Song

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.AI

AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description

Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Gül Varol, Weidi Xie, Andrew Zisserman

专题命中 VLM训练与架构 ：VLM(abstract)；分类 cs.CV

Comments Project Page: https://www.robots.ox.ac.uk/~vgg/research/autoad-zero/

ORID: Organ-Regional Information Driven Framework for Radiology Report Generation

Tiancheng Gu, Kaicheng Yang, Xiang An, Ziyong Feng, Dongnan Liu, Weidong Cai

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.CV

Comments 13 pages, 11 figures, WACV2025

Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Models

Wei Wang, Zhaowei Li, Qi Xu, Linfeng Li, YiQing Cai, Botian Jiang, Hang Song, Xingcan Hu, Pengyu Wang, Li Xiao

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.CV

Multimodal Instruction Tuning with Hybrid State Space Models

Jianing Zhou, Han Li, Shuai Zhang, Ning Xie, Ruijie Wang, Xiaohan Nie, Sheng Liu, Lingyun Wang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

LG-Gaze: Learning Geometry-aware Continuous Prompts for Language-Guided Gaze Estimation

Pengwei Yin, Jingjing Wang, Guanzhong Zeng, Di Xie, Jiang Zhu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease

Francesco Chiumento, Mingming Liu

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.AI

Comments The paper has been accepted by the conference: "2024 International Conference on Big Data (IEEE Big Data 2024)"

Precision or Recall? An Analysis of Image Captions for Training Text-to-Image Generation Model

Sheng Cheng, Maitreya Patel, Yezhou Yang

专题命中 VLM训练与架构 ：vision language model(abstract)；分类 cs.CV

Comments EMNLP 2024 Findings. Code: https://github.com/shengcheng/Captions4T2I

Label Critic: Design Data Before Models

Pedro R. A. S. Bassi, Qilong Wu, Wenxuan Li, Sergio Decherchi, Andrea Cavalli, Alan Yuille, Zongwei Zhou

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

VGA: Vision GUI Assistant -- Minimizing Hallucinations through Image-Centric Fine-Tuning

Ziyang Meng, Yu Dai, Zezheng Gong, Shaoxiong Guo, Minglong Tang, Tongquan Wei

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by EMNLP2024

Leveraging LLMs for MT in Crisis Scenarios: a blueprint for low-resource languages

Séamus Lankford, Andy Way

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.AI

Comments arXiv admin note: text overlap with arXiv:2403.02370, arXiv:2403.01580

OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models

Junda Wu, Xintong Li, Ruoyu Wang, Yu Xia, Yuxin Xiong, Jianing Wang, Tong Yu, Xiang Chen, Branislav Kveton, Lina Yao, Jingbo Shang, Julian McAuley

专题命中 VLM训练与架构 ：grounding(abstract)；分类 cs.LG

Comments 10 pages

Accelerating Transformers with Spectrum-Preserving Token Merging

Hoai-Chau Tran, Duy M. H. Nguyen, Duy M. Nguyen, Trung-Tin Nguyen, Ngan Le, Pengtao Xie, Daniel Sonntag, James Y. Zou, Binh T. Nguyen, Mathias Niepert

专题命中 VLM训练与架构 ：LLaVA(abstract)；分类 cs.LG

Comments Accepted at NeurIPS 2024

Multi-Class Textual-Inversion Secretly Yields a Semantic-Agnostic Classifier

Kai Wang, Fei Yang, Bogdan Raducanu, Joost van de Weijer

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted in WACV 2025. Code link: https://github.com/wangkai930418/mc_ti

LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding

Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed Elhoseiny, Vikas Chandra

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://vision-cair.github.io/LongVU

Harmonizing Visual Text Comprehension and Generation

Zhen Zhao, Jingqun Tang, Binghong Wu, Chunhui Lin, Shu Wei, Hao Liu, Xin Tan, Zhizhong Zhang, Can Huang, Yuan Xie

专题命中 VLM训练与架构 ：MLLM(abstract)；分类 cs.CV

Comments accepted by NeurIPS 2024

CoIN: A Benchmark of Continual Instruction tuNing for Multimodel Large Language Model

Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Lianli Gao, Jingkuan Song

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

DaLPSR: Leverage Degradation-Aligned Language Prompt for Real-World Image Super-Resolution

Aiwen Jiang, Zhi Wei, Long Peng, Feiqiang Liu, Wenbo Li, Mingwen Wang

专题命中 VLM训练与架构 ：multimodal large language model(abstract)；分类 cs.CV

Improving Multi-modal Large Language Model through Boosting Vision Capabilities

Yanpeng Sun, Huaxin Zhang, Qiang Chen, Xinyu Zhang, Nong Sang, Gang Zhang, Jingdong Wang, Zechao Li

专题命中 VLM训练与架构 ：vision-language model(abstract)；分类 cs.CV