多模态大模型

ICANet: A Method of Short Video Emotion Recognition Driven by Multimodal Data

Xuecheng Wu, Mengmeng Tian, Lanhang Zhai

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MTSA-SNN: A Multi-modal Time Series Analysis Model Based on Spiking Neural Network

Chengzhi Liu, Zheng Tao, Zihong Luo, Chenghao Liu

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 6 pages, 6 figures, published to International Conference on Computer Supported Cooperative Work in Design

Journal ref International Conference on Pattern Recognition 2024

Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model

Benlin Liu, Yuhao Dong, Yiqin Wang, Zixian Ma, Yansong Tang, Luming Tang, Yongming Rao, Wei-Chiu Ma, Ranjay Krishna

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments project page: https://coarse-correspondence.github.io

Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration

Wei Ji, Li Li, Zheqi Lv, Wenqiao Zhang, Mengze Li, Zhen Wan, Wenqiang Lei, Roger Zimmermann

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Efficient Large Multi-modal Models via Visual Context Compression

Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, Alan Yuille

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments NeurIPS 2024 Camera Ready; Code is available at https://github.com/Beckschen/LLaVolta

MMSummary: Multimodal Summary Generation for Fetal Ultrasound Video

Xiaoqing Guo, Qianhui Men, J. Alison Noble

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments MICCAI 2024

Multi-modal Motion Prediction using Temporal Ensembling with Learning-based Aggregation

Kai-Yin Hong, Chieh-Chih Wang, Wen-Chieh Lin

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024), accepted by IROS2024

Multimodal Learning for Embryo Viability Prediction in Clinical IVF

Junsik Kim, Zhiyi Shi, Davin Jeong, Johannes Knittel, Helen Y. Yang, Yonghyun Song, Wanhua Li, Yicong Li, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to MICCAI 2024

RoMemes: A multimodal meme corpus for the Romanian language

Vasile Păiş, Sara Niţă, Alexandru-Iulius Jerpelea, Luca Pană, Eric Curea

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 12 pages, 7 tables, 1 figure, submitted to The 19th International Conference on Linguistic Resources and Tools for Natural Language Processing (ConsILR 2024)

Temporal-Enhanced Multimodal Transformer for Referring Multi-Object Tracking and Segmentation

Changcheng Xiao, Qiong Cao, Yujie Zhong, Xiang Zhang, Tao Wang, Canqun Yang, Long Lan

专题命中 视频多模态 ：multimodal(title)；cross-modal(abstract)；分类 cs.CV

TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement

Zhiwei Lin, Hongbo Jin, Yongtao Wang, Yufei Wei, Nan Dong

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by ECAI2024

GEM-VPC: A dual Graph-Enhanced Multimodal integration for Video Paragraph Captioning

Eileen Wang, Caren Han, Josiah Poon

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MA-FSAR: Multimodal Adaptation of CLIP for Few-Shot Action Recognition

Jiazheng Xing, Chao Xu, Mengmeng Wang, Guang Dai, Baigui Sun, Yong Liu, Jingdong Wang, Jian Zhao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

MapsTP: HD Map Images Based Multimodal Trajectory Prediction for Automated Vehicles

Sushil Sharma, Arindam Das, Ganesh Sistu, Mark Halton, Ciarán Eising

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments This paper is a preprint of a paper submitted to the 26th Irish Machine Vision and Image Processing Conference (IMVIP 2024). If accepted, the copy of record will be available at IET Digital Library

Journal ref Proceedings of the Irish Machine Vision and Image Processing Conference 2024

X-Prompt: Multi-modal Visual Prompt for Video Object Segmentation

Pinxue Guo, Wanyun Li, Hao Huang, Lingyi Hong, Xinyu Zhou, Zhaoyu Chen, Jinglun Li, Kaixun Jiang, Wei Zhang, Wenqiang Zhang

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments ACMMM'2024

GS-EVT: Cross-Modal Event Camera Tracking based on Gaussian Splatting

Tao Liu, Runze Yuan, Yi'ang Ju, Xun Xu, Jiaqi Yang, Xiangting Meng, Xavier Lagorce, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

EventDance++: Language-guided Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition

Xu Zheng, Lin Wang

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2403.14082

Portrait Video Editing Empowered by Multimodal Generative Priors

Xuan Gao, Haiyao Xiao, Chenglai Zhong, Shimin Hu, Yudong Guo, Juyong Zhang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by SIGGRAPH Asia 2024. Project Page: https://ustc3dv.github.io/PortraitGen/

Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild

Nicolas Richet, Soufiane Belharbi, Haseeb Aslam, Meike Emilie Schadt, Manuela González-González, Gustave Cortal, Alessandro Lameiras Koerich, Marco Pedersoli, Alain Finkel, Simon Bacon, Eric Granger

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 14 pages, 3 figures, ECCVw 2024

Video Token Sparsification for Efficient Multimodal LLMs in Autonomous Driving

Yunsheng Ma, Amr Abdelraouf, Rohit Gupta, Ziran Wang, Kyungtae Han

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 10 pages, 3 figures, 4 tables

Video Emotion Open-vocabulary Recognition Based on Multimodal Large Language Model

Mengying Ge, Dongkai Tang, Mingyang Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

LITE: Modeling Environmental Ecosystems with Multimodal Large Language Models

Haoran Li, Junqi Liu, Zexian Wang, Shiyuan Luo, Xiaowei Jia, Huaxiu Yao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments COLM camera-ready version. Code is released at https://github.com/hrlics/LITE

Towards Effective Fusion and Forecasting of Multimodal Spatio-temporal Data for Smart Mobility

Chenxing Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 4 pages

OE-BevSeg: An Object Informed and Environment Aware Multimodal Framework for Bird's-eye-view Vehicle Semantic Segmentation

Jian Sun, Yuqi Dai, Chi-Man Vong, Qing Xu, Shengbo Eben Li, Jianqiang Wang, Lei He, Keqiang Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Multi-Granularity and Multi-modal Feature Interaction Approach for Text Video Retrieval

Wenjun Li, Shudong Wang, Dong Zhao, Shenghui Xu, Zhaoming Pan, Zhimin Zhang

专题命中 视频多模态 ：multi-modal(title)；cross-modal(abstract)；分类 cs.CV

Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition

Masashi Hatano, Ryo Hachiuma, Ryo Fujii, Hideo Saito

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ECCV'24

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments ECCV-24; Project page: videoagent.github.io; First two authors contributed equally

Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs

Hao-Tien Lewis Chiang, Zhuo Xu, Zipeng Fu, Mithun George Jacob, Tingnan Zhang, Tsang-Wei Edward Lee, Wenhao Yu, Connor Schenck, David Rendleman, Dhruv Shah, Fei Xia, Jasmine Hsu, Jonathan Hoech, Pete Florence, Sean Kirmani, Sumeet Singh, Vikas Sindhwani, Carolina Parada, Chelsea Finn, Peng Xu, Sergey Levine, Jie Tan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Multimodal Language Models for Domain-Specific Procedural Video Summarization

Nafisa Hussain

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 6 pages, 3 figures

Multi-Modal Video Dialog State Tracking in the Wild

Adnen Abdessaied, Lei Shi, Andreas Bulling

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments ECCV 2024