多模态大模型

MultiIoT: Benchmarking Machine Learning for the Internet of Things

Shentong Mo, Louis-Philippe Morency, Russ Salakhutdinov, Paul Pu Liang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams

Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, Xiaojie Jin

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

3rd Place Solution for MeViS Track in CVPR 2024 PVUW workshop: Motion Expression guided Video Segmentation

Feiyu Pan, Hao Fang, Xiankai Lu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Hybrid-Learning Video Moment Retrieval across Multi-Domain Labels

Weitong Cai, Jiabo Huang, Shaogang Gong

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by BMVC2022

Human-Centered Automation

Carlos Toxtli

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.AI

Comments 12 pages, 0 figures

Unified Video-Language Pre-training with Synchronized Audio

Shentong Mo, Haofan Wang, Huaxia Li, Xu Tang

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Pegasus-v1 Technical Report

Raehyuk Jung, Hyojun Go, Jaehyuk Yi, Jiho Jang, Daniel Kim, Jay Suh, Aiden Lee, Cooper Han, Jae Lee, Jeff Kim, Jin-Young Kim, Junwan Kim, Kyle Park, Lucas Lee, Mars Ha, Minjoon Seo, Abraham Jo, Ed Park, Hassan Kianinejad, SJ Kim, Tony Moon, Wade Jeong, Andrei Popescu, Esther Kim, EK Yoon, Genie Heo, Henry Choi, Jenna Kang, Kevin Han, Noah Seo, Sunny Nguyen, Ryan Won, Yeonhoo Park, Anthony Giuliani, Dave Chung, Hans Yoon, James Le, Jenny Ahn, June Lee, Maninder Saini, Meredith Sanders, Soyoung Lee, Sue Kim, Travis Couture

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models

Wei Han, Hui Chen, Min-Yen Kan, Soujanya Poria

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 13 pages, 7 figures, accepted to Findings of NAACL 2024

AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contexts

Jun Yu, Zerui Zhang, Zhihong Wei, Gongpeng Zhao, Zhongpeng Cai, Yongqi Wang, Guochen Xie, Jichao Zhu, Wangyuan Zhu

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV

Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow

Hanyu Zhou, Yi Chang, Zhiwei Shi, Luxin Yan

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Unmasked Teacher: Towards Training-Efficient Video Foundation Models

Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, Yu Qiao

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments ICCV2023

From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities

Chaochao Lu, Chen Qian, Guodong Zheng, Hongxing Fan, Hongzhi Gao, Jie Zhang, Jing Shao, Jingyi Deng, Jinlan Fu, Kexin Huang, Kunchang Li, Lijun Li, Limin Wang, Lu Sheng, Meiqi Chen, Ming Zhang, Qibing Ren, Sirui Chen, Tao Gui, Wanli Ouyang, Yali Wang, Yan Teng, Yaru Wang, Yi Wang, Yinan He, Yingchun Wang, Yixu Wang, Yongting Zhang, Yu Qiao, Yujiong Shen, Yurong Mou, Yuxi Chen, Zaibin Zhang, Zhelun Shi, Zhenfei Yin, Zhipin Wang

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Beyond the Frame: Single and mutilple video summarization method with user-defined length

Vahid Ahmadi Kalkhorani, Qingquan Zhang, Guanqun Song, Ting Zhu

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV

Temporal Adaptive RGBT Tracking with Modality Prompt

Hongyu Wang, Xiaotao Liu, Yifan Li, Meng Sun, Dian Yuan, Jing Liu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

MapFormer: Boosting Change Detection by Using Pre-change Information

Maximilian Bernhard, Niklas Strauß, Matthias Schubert

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments accepted at ICCV 2023

Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition

Chengyou Jia, Minnan Luo, Xiaojun Chang, Zhuohang Dang, Mingfei Han, Mengmeng Wang, Guang Dai, Sizhe Dang, Jingdong Wang

专题命中 视频多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

VTimeLLM: Empower LLM to Grasp Video Moments

Bin Huang, Xin Wang, Hong Chen, Zihan Song, Wenwu Zhu

专题命中 视频多模态 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

E-ViLM: Efficient Video-Language Model via Masked Video Modeling with Semantic Vector-Quantized Tokenizer

Jacob Zhiyuan Fang, Skyler Zheng, Vasu Sharma, Robinson Piramuthu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

VLANet: Video-Language Alignment Network for Weakly-Supervised Video Moment Retrieval

Minuk Ma, Sunjae Yoon, Junyeong Kim, Youngjoon Lee, Sunghun Kang, Chang D. Yoo

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 16 pages, 6 figures, European Conference on Computer Vision, 2020

Bidirectional Correlation-Driven Inter-Frame Interaction Transformer for Referring Video Object Segmentation

Meng Lan, Fu Rong, Zuchao Li, Wei Yu, Lefei Zhang

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interaction

Ganglai Wang, Peng Zhang, Junwen Xiong, Feihan Yang, Wei Huang, Yufei Zha

专题命中 视频多模态 ：cross-modal(abstract)；audio-visual(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2203.05178

Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models

Junting Pan, Ziyi Lin, Yuying Ge, Xiatian Zhu, Renrui Zhang, Yi Wang, Yu Qiao, Hongsheng Li

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation

Zhuoyan Luo, Yicheng Xiao, Yong Liu, Shuyan Li, Yitong Wang, Yansong Tang, Xiu Li, Yujiu Yang

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending

Xingjian He, Sihan Chen, Fan Ma, Zhicheng Huang, Xiaojie Jin, Zikang Liu, Dongmei Fu, Yi Yang, Jing Liu, Jiashi Feng

专题命中 视频多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video

Zenan Xu, Xiaojun Meng, Yasheng Wang, Qinliang Su, Zexuan Qiu, Xin Jiang, Qun Liu

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CL

Comments Accepted by IJCAI-2023

Temporal Sentence Grounding in Videos: A Survey and Future Directions

Hao Zhang, Aixin Sun, Wei Jing, Joey Tianyi Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

Weakly-Supervised Temporal Article Grounding

Long Chen, Yulei Niu, Brian Chen, Xudong Lin, Guangxing Han, Christopher Thomas, Hammad Ayyubi, Heng Ji, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2022, https://github.com/zjuchenlong/WSAG

Tracking Objects and Activities with Attention for Temporal Sentence Grounding

Zeyu Xiong, Daizong Liu, Pan Zhou, Jiahao Zhu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments accepted by ICASSP2023

MRTNet: Multi-Resolution Temporal Network for Video Sentence Grounding

Wei Ji, Long Chen, Yinwei Wei, Yiming Wu, Tat-Seng Chua

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments work in progress

Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion

Shruti Agarwal, Liwen Hu, Evonne Ng, Trevor Darrell, Hao Li, Anna Rohrbach

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted in WACV 2023