多模态大模型

Question-Answering Dense Video Events

Hangyu Qin, Junbin Xiao, Angela Yao

机构 * National University of Singapore（新加坡国立大学） 纠错

专题命中 视频多模态 ：MLLM(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to SIGIR'25

ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Data

Chengsen Wang, Qi Qi, Zhongwen Rao, Lujia Pan, Jingyu Wang, Jianxin Liao

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） ； Huawei Noah’s Ark Lab（华为诺亚实验室） 纠错

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)

VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation

Chaofan Zhang, Peng Hao, Xiaoge Cao, Xiaoshuai Hao, Shaowei Cui, Shuo Wang

机构 * State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院） ； Samsung R&D Institute China–Beijing（三星中国北京研发中心） ； Beijing Academy of Artificial Intelligence（北京人工智能研究院） 纠错

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)

VidCtx: Context-aware Video Question Answering with Image Models

Andreas Goulas, Vasileios Mezaris, Ioannis Patras

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted in IEEE ICME 2025. This is the authors' accepted version

Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1

Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Lu Qiu, Ying Shan, Xihui Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Technical Report (In Progress); Code released at: https://github.com/TencentARC/SEED-Bench-R1

Towards Visual-Prompt Temporal Answering Grounding in Medical Instructional Video

Bin Li, Yixuan Weng, Bin Sun, Shutao Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 8 pages, 6 figures, 3 tables

Shushing! Let's Imagine an Authentic Speech from the Silent Video

Jiaxin Ye, Hongming Shan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、eess.AS

Comments Project Page: https://imagintalk.github.io

Valley: Video Assistant with Large Language model Enhanced abilitY

Ruipu Luo, Ziwang Zhao, Min Yang, Zheming Yang, Minghui Qiu, Tao Wang, Zhongyu Wei, Yanhao Wang, Cen Chen

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Can Large Language Models Grasp Concepts in Visual Content? A Case Study on YouTube Shorts about Depression

Jiaying "Lizzy" Liu, Yiheng Su, Praneel Seth

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)

Comments 11 pages

Journal ref CHI Conference on Human Factors in Computing Systems (CHI EA 2025)

PreMind: Multi-Agent Video Understanding for Advanced Indexing of Presentation-style Videos

Kangda Wei, Zhengyu Zhou, Bingqing Wang, Jun Araki, Lukas Lange, Ruihong Huang, Zhe Feng

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matching

Yongqi Wang, Wenxiang Guo, Rongjie Huang, Jiawei Huang, Zehan Wang, Fuming You, Ruiqi Li, Zhou Zhao

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM、eess.AS

Comments accepted by NeurIPS 2024

VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection

Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, Si Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 14 pages, 14 figures

LLaVA-OneVision: Easy Visual Task Transfer

Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Homepage: https://llava-vl.github.io/blog/2024-08-05-llava-onevision/

Large Body Language Models

Saif Punjwani, Larry Heck

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

TraveLER: A Modular Multi-LMM Agent Framework for Video Question-Answering

Chuyi Shang, Amos You, Sanjay Subramanian, Trevor Darrell, Roei Herzig

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2024 (Main)

Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos

Jianrui Zhang, Mu Cai, Yong Jae Lee

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project Page: https://vinoground.github.io

NL-Eye: Abductive NLI for Images

Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding

Xiao Wang, Jianlong Wu, Zijia Lin, Fuzheng Zhang, Di Zhang, Liqiang Nie

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Under peer review

Top-down Activity Representation Learning for Video Question Answering

Yanan Wang, Shuichiro Haruta, Donghuo Zeng, Julio Vizcarra, Mori Kurokawa

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments presented at MIRU2024

Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input

Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, Jie Hu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline

Qi Jia, Baoyu Fan, Cong Xu, Lu Liu, Liang Jin, Guoguang Du, Zhenhua Guo, Yaqian Zhao, Xuanjing Huang, Rengang Li

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Towards Event-oriented Long Video Understanding

Yifan Du, Kun Zhou, Yuqi Huo, Yifan Li, Wayne Xin Zhao, Haoyu Lu, Zijia Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Work on progress

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 38 pages, 44 figures

TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioning

Quang Minh Dinh, Minh Khoi Ho, Anh Quan Dang, Hung Phong Tran

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 7134-7143

An Empirical Study of Excitation and Aggregation Design Adaptions in CLIP4Clip for Video-Text Retrieval

Xiaolun Jing, Genke Yang, Jian Chu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 20 pages

Vision Mamba: A Comprehensive Survey and Taxonomy

Xiao Liu, Chenxu Zhang, Lei Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments https://github.com/lx6c78/Vision-Mamba-A-Comprehensive-Survey-and-Taxonomy

Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition

Xiao Wang, Qian Zhu, Jiandong Jin, Jun Zhu, Futian Wang, Bo Jiang, Yaowei Wang, Yonghong Tian

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Parameter Efficient Fine-Tuning Strategy for Video-based Pedestrian Attribute Recognition

Can I Trust Your Answer? Visually Grounded Video Question Answering

Junbin Xiao, Angela Yao, Yicong Li, Tat Seng Chua

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to CVPR'24. (Compared with preprint version, we mainly improve the presentation, discuss more related works, and extend experiments in Appendix.)

HawkEye: Training Video-Text LLMs for Grounding Text in Videos

Yueqian Wang, Xiaojun Meng, Jianxin Liang, Yuxuan Wang, Qun Liu, Dongyan Zhao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling

Jiaqi Xu, Bo Liu, Yunkuo Chen, Mengli Cheng, Xing Shi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.MM