多模态大模型

ViLA: Efficient Video-Language Alignment for Video Question Answering

Xijun Wang, Junbang Liang, Chun-Kai Wang, Kenan Deng, Yu Lou, Ming Lin, Shan Yang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Comments ECCV 2024

LLM4Brain: Training a Large Language Model for Brain Video Understanding

Ruizhe Zheng, Lichao Sun

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments ECCV2024 Workshop

Vision-Language Models Assisted Unsupervised Video Anomaly Detection

Yalong Jiang, Liquan Mao

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment

Yuxiao Chen, Kai Li, Wentao Bao, Deep Patel, Yu Kong, Martin Renqiang Min, Dimitris N. Metaxas

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

Towards Child-Inclusive Clinical Video Understanding for Autism Spectrum Disorder

Aditya Kommineni, Digbalay Bose, Tiantian Feng, So Hyun Kim, Helen Tager-Flusberg, Somer Bishop, Catherine Lord, Sudarsana Kadiri, Shrikanth Narayanan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments 5 pages, 2 figures, 2 tables

NaviQAte: Functionality-Guided Web Application Navigation

Mobina Shahbandeh, Parsa Alian, Noor Nashid, Ali Mesbah

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL

JARViS: Detecting Actions in Video Using Unified Actor-Scene Context Relation Modeling

Seok Hwan Lee, Taein Son, Soo Won Seo, Jisong Kim, Jun Won Choi

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Comments 31 pages, 10 figures, update references

Prior Knowledge Integration via LLM Encoding and Pseudo Event Regulation for Video Moment Retrieval

Yiyang Jiang, Wengyu Zhang, Xulu Zhang, Xiaoyong Wei, Chang Wen Chen, Qing Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted to ACM Multimedia 2024

Optimizing 4D Lookup Table for Low-light Video Enhancement via Wavelet Priori

Jinhong He, Minglong Xue, Wenhai Wang, Mingliang Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Depth on Demand: Streaming Dense Depth from a Low Frame Rate Active Sensor

Andrea Conti, Matteo Poggi, Valerio Cambareri, Stefano Mattoccia

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted for publication at the European Conference on Computer Vision (ECCV) 2024

PiTe: Pixel-Temporal Alignment for Large Video-Language Model

Yang Liu, Pengxiang Ding, Siteng Huang, Min Zhang, Han Zhao, Donglin Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

STLLM-DF: A Spatial-Temporal Large Language Model with Diffusion for Enhanced Multi-Mode Traffic System Forecasting

Zhiqi Shao, Haoning Xi, Haohui Lu, Ze Wang, Michael G. H. Bell, Junbin Gao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.AI

Comments 26 pages, 11 figures

HowToCaption: Prompting LLMs to Transform Video Annotations at Scale

Nina Shvetsova, Anna Kukleva, Xudong Hong, Christian Rupprecht, Bernt Schiele, Hilde Kuehne

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments https://github.com/ninatu/howtocaption

Unified Framework with Consistency across Modalities for Human Activity Recognition

Tuyen Tran, Thao Minh Le, Hung Tran, Truyen Tran

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted to BMVC 2024

MMA-MRNNet: Harnessing Multiple Models of Affect and Dynamic Masked RNN for Precise Facial Expression Intensity Estimation

Dimitrios Kollias, Andreas Psaroudakis, Anastasios Arsenos, Paraskevi Theofilou, Chunchang Shao, Guanyu Hu, Ioannis Patras

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Dissecting Temporal Understanding in Text-to-Audio Retrieval

Andreea-Maria Oncescu, João F. Henriques, A. Sophia Koepke

专题命中 视频多模态 ：multimodal(abstract)；分类 eess.AS

Comments 9 pages, 5 figures, ACM Multimedia 2024, https://www.robots.ox.ac.uk/~vgg/research/audio-retrieval/dtu/

Exploiting temporal information to detect conversational groups in videos and predict the next speaker

Lucrezia Tosato, Victor Fortier, Isabelle Bloch, Catherine Pelachaud

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted to Pattern Recognition Letter, 8 pages, 10 figures

Journal ref Pattern Recognition Letters Volume 177, January 2024, Pages 164 168

Training-free Video Temporal Grounding using Large-scale Pre-trained Models

Minghang Zheng, Xinhao Cai, Qingchao Chen, Yuxin Peng, Yang Liu

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by ECCV 2024

Training-Free Action Recognition and Goal Inference with Dynamic Frame Selection

Ee Yeo Keat, Zhang Hao, Alexander Matyasko, Basura Fernando

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Sec2Sec Co-attention for Video-Based Apparent Affective Prediction

Mingwei Sun, Kunpeng Zhang

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.MM

Comments 5 pages, 3 figures

Towards Real-world Event-guided Low-light Video Enhancement and Deblurring

Taewoo Kim, Jaeseok Jeong, Hoonhee Cho, Yuhwan Jeong, Kuk-Jin Yoon

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted in ECCV2024

Personalized Video Summarization using Text-Based Queries and Conditional Modeling

Jia-Hong Huang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Ph.D. thesis, 137 pages

Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos

Qirui Chen, Shangzhe Di, Weidi Xie

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation

Xu Zheng, Yuanhuiyi Lyu, Jiazhou Zhou, Lin Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

Data-Centric Machine Learning for Earth Observation: Necessary and Sufficient Features

Hiba Najjar, Marlon Nuske, Andreas Dengel

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI

Comments Accepted at MACLEAN workshop, ECML/PKDD 2024

The Instance-centric Transformer for the RVOS Track of LSVOS Challenge: 3rd Place Solution

Bin Cao, Yisi Zhang, Hanyi Wang, Xingjian He, Jing Liu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2406.13939

CoSEC: A Coaxial Stereo Event Camera Dataset for Autonomous Driving

Shihan Peng, Hanyu Zhou, Hao Dong, Zhiwei Shi, Haoyue Liu, Yuxing Duan, Yi Chang, Luxin Yan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Comments This work has been submitted to the IEEE for possible publication

HeadsetOff: Enabling Photorealistic Video Conferencing on Economical VR Headsets

Yili Jin, Xize Duan, Fangxin Wang, Xue Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.MM

Comments Accepted by ACM Multimedia 2024

Multi-Modality Co-Learning for Efficient Skeleton-based Action Recognition

Jinfu Liu, Chen Chen, Mengyuan Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV

Disentangle and denoise: Tackling context misalignment for video moment retrieval

Kaijing Ma, Han Fang, Xianghao Zang, Chao Ban, Lanxiang Zhou, Zhongjiang He, Yongxiang Li, Hao Sun, Zerun Feng, Xingsong Hou

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV