多模态大模型

Foundation Models and Adaptive Feature Selection: A Synergistic Approach to Video Question Answering

Sai Bhargav Rongali, Mohamad Hassan N C, Ankit Jha, Neha Bhargava, Saurabh Prasad, Biplab Banerjee

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Journal ref WACV2025

LinVT: Empower Your Image-level Large Language Model to Understand Videos

Lishuai Gao, Yujie Zhong, Yingsen Zeng, Haoxian Tan, Dengjie Li, Zheng Zhao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

What's the Move? Hybrid Imitation Learning via Salient Points

Priya Sundaresan, Hengyuan Hu, Quan Vuong, Jeannette Bohg, Dorsa Sadigh

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization

Yuliang Liu, Mingxin Huang, Hao Yan, Linger Deng, Weijia Wu, Hao Lu, Chunhua Shen, Lianwen Jin, Xiang Bai

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VideoICL: Confidence-based Iterative In-context Learning for Out-of-Distribution Video Understanding

Kangsan Kim, Geon Park, Youngwan Lee, Woongyeong Yeo, Sung Ju Hwang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Long-Term Ad Memorability: Understanding & Generating Memorable Ads

Harini SI, Somesh Singh, Yaman K Singla, Aanisha Bhattacharyya, Veeky Baths, Changyou Chen, Rajiv Ratn Shah, Balaji Krishnamurthy

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Published in WACV-2025

Look Every Frame All at Once: Video-Ma$^2$mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing

Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Project page: https://ivy-lvlm.github.io/Video-MA2MBA/

On Moving Object Segmentation from Monocular Video with Transformers

Christian Homeyer, Christoph Schnörr

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments WICCV2023

Journal ref Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 (880--891)

TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability

Shimin Chen, Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Generative Timelines for Instructed Visual Assembly

Alejandro Pardo, Jui-Hsien Wang, Bernard Ghanem, Josef Sivic, Bryan Russell, Fabian Caba Heilbron

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

HourVideo: 1-Hour Video-Language Understanding

Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, Li Fei-Fei

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2024 Datasets and Benchmarks Track; 28 pages

StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

Junming Lin, Zheng Fang, Chi Chen, Zihao Wan, Fuwen Luo, Peng Li, Yang Liu, Maosong Sun

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Simple and Effective Temporal Grounding Pipeline for Basketball Broadcast Footage

Levi Harris

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities

Md Farhan Ishmam, Md Sakib Hossain Shovon, M. F. Mridha, Nilanjan Dey

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Posture-Informed Muscular Force Learning for Robust Hand Pressure Estimation

Kyungjin Seo, Junghoon Seo, Hanseok Jeong, Sangpil Kim, Sang Ho Yoon

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024. Project Page Link: https://pimforce.hcitech.org/

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, Lidong Bing

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ZC, SL, HZ, YX, and XL contributed equally to this project. Code: https://github.com/DAMO-NLP-SG/VideoLLaMA2

Transformer for Object Re-Identification: A Survey

Mang Ye, Shuoyi Chen, Chenyue Li, Wei-Shi Zheng, David Crandall, Bo Du

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by International Journal of Computer Vision (IJCV) in October 2024

CinePile: A Long Video Question Answering Dataset and Benchmark

Ruchit Rawal, Khalid Saifullah, Miquel Farré, Ronen Basri, David Jacobs, Gowthami Somepalli, Tom Goldstein

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Project page with all the artifacts - https://ruchitrawal.github.io/cinepile/. Updated version with adversarial refinement pipeline and more model evaluations

Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence

Andres Karjus

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Generating Signed Language Instructions in Large-Scale Dialogue Systems

Mert İnan, Katherine Atwell, Anthony Sicilia, Lorna Quandt, Malihe Alikhani

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024) Industry Track

SatSwinMAE: Efficient Autoencoding for Multiscale Time-series Satellite Imagery

Yohei Nakayama, Jiawei Su, Luis M. Pazos-Outón

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Sparse Prototype Network for Explainable Pedestrian Behavior Prediction

Yan Feng, Alexander Carballo, Kazuya Takeda

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs

Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, Yunhe Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Tech report

InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation

Andrew Lee, Ian Chuang, Ling-Yuan Chen, Iman Soltani

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at Conference on Robot Learning (CoRL) 2024

VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models

Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 9 pages, 4 figures

The Solution for Temporal Action Localisation Task of Perception Test Challenge 2024

Yinan Han, Qingyuan Jiang, Hongming Mei, Yang Yang, Jinhui Tang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning

Thong Nguyen, Yi Bin, Xiaobao Wu, Xinshuai Dong, Zhiyuan Hu, Khoi Le, Cong-Duy Nguyen, See-Kiong Ng, Luu Anh Tuan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ECCV 2024

Temporal Reasoning Transfer from Text to Video

Lei Li, Yuanxin Liu, Linli Yao, Peiyuan Zhang, Chenxin An, Lean Wang, Xu Sun, Lingpeng Kong, Qi Liu

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://video-t3.github.io

RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation

Hanxiao Jiang, Binghao Huang, Ruihai Wu, Zhuoran Li, Shubham Garg, Hooshang Nayyeri, Shenlong Wang, Yunzhu Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://jianghanxiao.github.io/roboexp-web/

AIM 2024 Challenge on Video Saliency Prediction: Methods and Results

Andrey Moskalenko, Alexey Bryncev, Dmitry Vatolin, Radu Timofte, Gen Zhan, Li Yang, Yunlong Tang, Yiting Liao, Jiongzhi Lin, Baitao Huang, Morteza Moradi, Mohammad Moradi, Francesco Rundo, Concetto Spampinato, Ali Borji, Simone Palazzo, Yuxin Zhu, Yinan Sun, Huiyu Duan, Yuqin Cao, Ziheng Jia, Qiang Hu, Xiongkuo Min, Guangtao Zhai, Hao Fang, Runmin Cong, Xiankai Lu, Xiaofei Zhou, Wei Zhang, Chunyu Zhao, Wentao Mu, Tao Deng, Hamed R. Tavakoli

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM

Comments ECCVW 2024