多模态大模型

ViT LoS V2X: Vision Transformers for Environment-aware LoS Blockage Prediction for 6G Vehicular Networks

Ghazi Gharsallah, Georges Kaddoum

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Action-Based ADHD Diagnosis in Video

Yichun Li, Yuxing Yang, Syed Nohsen Naqvi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 31st European Symposium on Artificial Neural Networks

RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation

Yonglin Li, Jing Zhang, Xiao Teng, Long Lan, Xinwang Liu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Grounding

Kaijing Ma, Haojian Huang, Jin Chen, Haodong Chen, Pengliang Ji, Xianghao Zang, Han Fang, Chao Ban, Hao Sun, Mulin Chen, Xuelong Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Ongoing work: 28pages, 19 figures, 7 tables. Code is available at: https://kaijing.space/SRAM/

On the Efficacy of Text-Based Input Modalities for Action Anticipation

Apoorva Beedu, Harish Haresamudram, Karan Samel, Irfan Essa

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses

Chaolei Tan, Zihang Lin, Junfu Pu, Zhongang Qi, Wei-Yi Pei, Zhi Qu, Yexin Wang, Ying Shan, Wei-Shi Zheng, Jian-Fang Hu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM MM 2024. Project page: https://synopground.github.io/

DeVAn: Dense Video Annotation for Video-Language Models

Tingkai Liu, Yunzhe Tao, Haogeng Liu, Qihang Fan, Ding Zhou, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Published in 62nd ACL (2024) Main Track

Edit As You Wish: Video Caption Editing with Multi-grained User Control

Linli Yao, Yuanmeng Zhang, Ziheng Wang, Xinglin Hou, Tiezheng Ge, Yuning Jiang, Xu Sun, Qin Jin

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM MM 2024

Comparing Optical Flow and Deep Learning to Enable Computationally Efficient Traffic Event Detection with Space-Filling Curves

Tayssir Bouraffa, Elias Kjellberg Carlson, Erik Wessman, Ali Nouri, Pierre Lamart, Christian Berger

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 27th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2024)

LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding

Haoning Wu, Dongxu Li, Bei Chen, Junnan Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 29 pages

Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs

Jinmin Li, Kuofeng Gao, Yang Bai, Jingyun Zhang, Shu-Tao Xia

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments arXiv admin note: substantial text overlap with arXiv:2403.13507

Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment

Hao Fei, Shengqiong Wu, Meishan Zhang, Min Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by IEEE TPAMI 2024

Journal ref [J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

Unified and Dynamic Graph for Temporal Character Grouping in Long Videos

Xiujun Shu, Wei Wen, Liangsheng Xu, Ruizhi Qiao, Taian Guo, Hanjun Li, Bei Gan, Xiao Wang, Xing Sun

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Towards Holistic Language-video Representation: the language model-enhanced MSR-Video to Text Dataset

Yuchen Yang, Yingxuan Duan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies

Hung-Ting Su, Chun-Tong Chao, Ya-Ching Hsu, Xudong Lin, Yulei Niu, Hung-Yi Lee, Winston H. Hsu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://ander1119.github.io/TiM

A Survey of Video Datasets for Grounded Event Understanding

Kate Sanders, Benjamin Van Durme

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

Qiang Sun, Yuanyi Luo, Wenxiao Zhang, Sirui Li, Jichunyang Li, Kai Niu, Xiangrui Kong, Wei Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Augmented Commonsense Knowledge for Remote Object Grounding

Bahram Mohammadi, Yicong Hong, Yuankai Qi, Qi Wu, Shirui Pan, Javen Qinfeng Shi

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Video sentence grounding with temporally global textual knowledge

Cai Chen, Runzhong Zhang, Jianjun Gao, Kejun Wu, Kim-Hui Yap, Yi Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Context-Enhanced Video Moment Retrieval with Large Language Models

Weijia Liu, Bo Miao, Jiuxin Cao, Xuelin Zhu, Bo Liu, Mehwish Nasim, Ajmal Mian

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

MADRL-Based Rate Adaptation for 360° Video Streaming with Multi-Viewpoint Prediction

Haopeng Wang, Zijian Long, Haiwei Dong, Abdulmotaleb El Saddik

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI、cs.MM

Comments Accepted by IEEE Internet of Things Journal

Mamba-FETrack: Frame-Event Tracking via State Space Model

Ju Huang, Shiao Wang, Shuai Wang, Zhe Wu, Xiao Wang, Bo Jiang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

HDBN: A Novel Hybrid Dual-branch Network for Robust Skeleton-based Action Recognition

Jinfu Liu, Baiqiao Yin, Jiaying Lin, Jiajun Wen, Yue Li, Mengyuan Liu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting

Fengyi Fu, Shancheng Fang, Weidong Chen, Zhendong Mao

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 10 figures, ACM Transactions on Multimedia Computing, Communications and Applications, 2024

Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection

Yang Liu, Tongfei Shen, Dong Zhang, Qingying Sun, Shoushan Li, Guodong Zhou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICMR 2024

LITA: Language Instructed Temporal-Localization Assistant

De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, Jan Kautz

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Number Sense as an Emergent Property of the Manipulating Brain

Neehar Kondapaneni, Pietro Perona

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 5 figures, 15 supplemental figures

Journal ref Scientific reports, 14(6858) 2024

VidLA: Video-Language Alignment at Scale

Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan, Son Tran, Benjamin Z. Yao, Belinda Zeng, Mubarak Shah, Trishul Chilimbi

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2024

MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators

Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, Wanli Ouyang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 18 pages, 8 figures, accepted by AAAI 2024

WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs

Deshun Yang, Luhui Hu, Yu Tian, Zihao Li, Chris Kelly, Bang Yang, Cindy Yang, Yuexian Zou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 2 figures, 2 tables