视觉大模型 / VLM

PAVE: Patching and Adapting Video Large Language Models

Zhuoming Liu, Yiquan Li, Khoi Duc Nguyen, Yiwu Zhong, Yin Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2025 Camera Ready

DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models

Yongdong Wang, Runze Xiao, Jun Younes Louhi Kasahara, Ryosuke Yajima, Keiji Nagatani, Atsushi Yamashita, Hajime Asama

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)

Comments The work was first submitted to an IEEE conference on September 15, 2024

Adaptive Keyframe Sampling for Long Video Understanding

Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, Qixiang Ye

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2025

Bayesian Low-Rank LeArning (Bella): A Practical Approach to Bayesian Neural Networks

Bao Gia Doan, Afshar Shamsi, Xiao-Yu Guo, Arash Mohammadi, Hamid Alinejad-Rokny, Dino Sejdinovic, Damien Teney, Damith C. Ranasinghe, Ehsan Abbasnejad

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This paper is accepted in AAAI'25", and the code is available at https://bnn-bella.github.io/BNN-Bella/

Reasoning-Enhanced Object-Centric Learning for Videos

Jian Li, Pu Ren, Yang Liu, Hao Sun

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimization

Yue Zhang, Liqiang Jing, Vibhav Gogate

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by AAAI 2025

VQA-Levels: A Hierarchical Approach for Classifying Questions in VQA

Madhuri Latha Madaka, Chakravarthy Bhagvati

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data

Jeremy Andrew Irvin, Emily Ruoyu Liu, Joyce Chuyi Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, Stefano Ermon

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published at ICLR 2025

Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy

Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Bhargava Kumar, Amit Agarwal, Ishan Banerjee, Srikant Panda, Tejaswini Kumar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

NoisyEQA: Benchmarking Embodied Question Answering Against Noisy Queries

Tao Wu, Chuhao Zhou, Yen Heng Wong, Lin Gu, Jianfei Yang

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Uni-Mlip: Unified Self-supervision for Medical Vision Language Pre-training

Ameera Bawazir, Kebin Wu, Wenbin Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 15 pages, 2 figures, accepted by BMVC'24

Preserving Pre-trained Representation Space: On Effectiveness of Prefix-tuning for Large Multi-modal Models

Donghoon Kim, Gusang Lee, Kyuhong Shim, Byonghyo Shim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Findings of EMNLP 2024

Visual Text Matters: Improving Text-KVQA with Visual Text Entity Knowledge-aware Large Multimodal Assistant

Abhirama Subramanyam Penamakuri, Anand Mishra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to EMNLP (Main) 2024

Sparks of Artificial General Intelligence(AGI) in Semiconductor Material Science: Early Explorations into the Next Frontier of Generative AI-Assisted Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Sreeja Gangasani, Chidaksh Ravuru, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published at Deployable AI (DAI) Workshop at AAAI-2024

A Multimodal Memes Classification: A Survey and Open Research Issues

Tariq Habib Afridi, Aftab Alam, Muhammad Numan Khan, Jawad Khan, Young-Koo Lee

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This is a survey paper on recent state of the art VL models that can be used for memes classification. it has 15 pages and 2 figures

Journal ref SCA 2020

Towards Human-Level Understanding of Complex Process Engineering Schematics: A Pedagogical, Introspective Multi-Agent Framework for Open-Domain Question Answering

Sagar Srinivas Sakhinana, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Our paper is accepted for publication at ML4CCE workshop at ECML PKDD 2024

Foundational Model for Electron Micrograph Analysis: Instruction-Tuning Small-Scale Language-and-Vision Assistant for Enterprise Adoption

Sakhinana Sagar Srinivas, Chidaksh Ravuru, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Our paper is published at ICML 2024 Workshop ML for Life and Material Science: From Theory to Industry Applications, Vienna, Austria

Leveraging Large Language Models for Scalable Vector Graphics-Driven Image Understanding

Mu Cai, Zeyi Huang, Yuheng Li, Utkarsh Ojha, Haohan Wang, Yong Jae Lee

专题命中 视觉问答 ：visual reasoning(abstract)；分类 cs.CV、cs.AI、cs.LG

Composition Vision-Language Understanding via Segment and Depth Anything Model

Mingxiao Huo, Pengliang Ji, Haotian Lin, Junchen Liu, Yixiao Wang, Yijun Chen

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Reverse Image Retrieval Cues Parametric Memory in Multimodal LLMs

Jialiang Xu, Michael Moor, Jure Leskovec

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)

SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes

Soubhik Sanyal, Partha Ghosh, Jinlong Yang, Michael J. Black, Justus Thies, Timo Bolkart

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Updated to camera ready version of CVPR 2024

Advancing Multimodal Medical Capabilities of Gemini

Lin Yang, Shawn Xu, Andrew Sellergren, Timo Kohlberger, Yuchen Zhou, Ira Ktena, Atilla Kiraly, Faruk Ahmed, Farhad Hormozdiari, Tiam Jaroensri, Eric Wang, Ellery Wulczyn, Fayaz Jamil, Theo Guidroz, Chuck Lau, Siyuan Qiao, Yun Liu, Akshay Goel, Kendall Park, Arnav Agharwal, Nick George, Yang Wang, Ryutaro Tanno, David G. T. Barrett, Wei-Hung Weng, S. Sara Mahdavi, Khaled Saab, Tao Tu, Sreenivasa Raju Kalidindi, Mozziyar Etemadi, Jorge Cuadros, Gregory Sorensen, Yossi Matias, Katherine Chou, Greg Corrado, Joelle Barral, Shravya Shetty, David Fleet, S. M. Ali Eslami, Daniel Tse, Shruthi Prabhakara, Cory McLean, Dave Steiner, Rory Pilgrim, Christopher Kelly, Shekoofeh Azizi, Daniel Golden

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation

Jaemin Cho, Yushi Hu, Roopal Garg, Peter Anderson, Ranjay Krishna, Jason Baldridge, Mohit Bansal, Jordi Pont-Tuset, Su Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments ICLR 2024; Project website: https://google.github.io/dsg

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, Jiaqi Li, Xiaoze Liu, Jeff Z. Pan, Ningyu Zhang, Huajun Chen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Ongoing work; 41 pages (Main Text), 55 pages (Total), 11 Tables, 13 Figures, 619 citations; Paper list is available at https://github.com/zjukg/KG-MM-Survey

Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology

Nur Yildirim, Hannah Richardson, Maria T. Wetscherek, Junaid Bajwa, Joseph Jacob, Mark A. Pinnock, Stephen Harris, Daniel Coelho de Castro, Shruthi Bannur, Stephanie L. Hyland, Pratik Ghosh, Mercy Ranjit, Kenza Bouzid, Anton Schwaighofer, Fernando Pérez-García, Harshita Sharma, Ozan Oktay, Matthew Lungren, Javier Alvarez-Valle, Aditya Nori, Anja Thieme

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)

Comments to appear at CHI 2024

Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models

Didi Zhu, Zhongyi Sun, Zexi Li, Tao Shen, Ke Yan, Shouhong Ding, Kun Kuang, Chao Wu

专题命中 视觉问答 ：LLaVA(abstract)；visual question answering(abstract)

Improving Automatic VQA Evaluation Using Large Language Models

Oscar Mañas, Benno Krojer, Aishwarya Agrawal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at AAAI 2024 (main track)

VQA-based Robotic State Recognition Optimized with Genetic Algorithm

Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)

Comments Accepted at ICRA2023

Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoning

Enna Sachdeva, Nakul Agarwal, Suhas Chundi, Sean Roelofs, Jiachen Li, Mykel Kochenderfer, Chiho Choi, Behzad Dariush

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering

Chenlu Zhan, Peng Peng, Hongsen Wang, Tao Chen, Hongwei Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG