多模态大模型

An Empirical Evaluation of Neural and Neuro-symbolic Approaches to Real-time Multimodal Complex Event Detection

Liying Han, Mani B. Srivastava

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Exploring the dynamic interplay of cognitive load and emotional arousal by using multimodal measurements: Correlation of pupil diameter and emotional arousal in emotionally engaging tasks

C. Kosel, S. Michel, T. Seidel, M. Foerster

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments The first two authors contributed equally to the manuscript

Global Tropical Cyclone Intensity Forecasting with Multi-modal Multi-scale Causal Autoregressive Model

Xinyu Wang, Kang Chen, Lei Liu, Tao Han, Bin Li, Lei Bai

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog

Adnen Abdessaied, Manuel von Hochmeister, Andreas Bulling

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments COLING 2024

MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces

Tianyu Zheng, Ge Zhang, Xingwei Qu, Ming Kuang, Stephen W. Huang, Zhaofeng He

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation

Hongcheng Liu, Pingjie Wang, Yu Wang, Yanfeng Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Monitoring of Urban Changes with multi-modal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23

Georg Zitzlsberger, Michal Podhoranyi

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats

Aaron Mullen, Samuel E. Armstrong, Jasmine Perdeh, Bjorn Bauer, Jeffrey Talbert, V. K. Cody Bumgardner

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding

Yunze Liu, Changxi Chen, Zifan Wang, Li Yi

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Journal ref ICRA2024

M2-CLIP: A Multimodal, Multi-task Adapting Framework for Video Action Recognition

Mengmeng Wang, Jiazheng Xing, Boyuan Jiang, Jun Chen, Jianbiao Mei, Xingxing Zuo, Guang Dai, Jingdong Wang, Yong Liu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Journal ref AAAI2024

Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation

Chen Liang, Yu Wu, Tianfei Zhou, Wenguan Wang, Zongxin Yang, Yunchao Wei, Yi Yang

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Champion solution in YouTube-VOS 2021 Track 3. Extended version published in https://ieeexplore.ieee.org/abstract/document/10083244

Multi-view Distillation based on Multi-modal Fusion for Few-shot Action Recognition(CLIP-$\mathrm{M^2}$DF)

Fei Guo, YiKang Wang, Han Qi, WenPing Jin, Li Zhu

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Cross-Modal Semi-Dense 6-DoF Tracking of an Event Camera in Challenging Conditions

Yi-Fan Zuo, Wanting Xu, Xia Wang, Yifu Wang, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by IEEE Transactions on Robotics (T-RO). arXiv admin note: text overlap with arXiv:2202.02556

Self-supervised Event-based Monocular Depth Estimation using Cross-modal Consistency

Junyu Zhu, Lina Liu, Bofeng Jiang, Feng Wen, Hongbo Zhang, Wanlong Li, Yong Liu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by IROS2023

InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, Conghui He, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, Yu Qiao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Data and Code: https://github.com/OpenGVLab/InternVideo/tree/main/Data/InternVid

Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models

Xinpeng Ding, Jinahua Han, Hang Xu, Xiaodan Liang, Wei Zhang, Xiaomeng Li

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

A graph-based multimodal framework to predict gentrification

Javad Eshtiyagh, Baotong Zhang, Yujing Sun, Linhui Wu, Zhao Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref International Conference on Urban Informatics 2023 - Best Paper Award 3rd Place

Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding

Haifeng Huang, Yang Zhao, Zehan Wang, Yan Xia, Zhou Zhao

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Parameterized Decision-making with Multi-modal Perception for Autonomous Driving

Yuyang Xia, Shuncheng Liu, Quanlin Yu, Liwei Deng, You Zhang, Han Su, Kai Zheng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments IEEE International Conference on Data Engineering (ICDE2024)

MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration

Quentin Herau, Nathan Piasco, Moussab Bennehar, Luis Roldão, Dzmitry Tsishkou, Cyrille Migniot, Pascal Vasseur, Cédric Demonceaux

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at IROS2023 Project site: https://qherau.github.io/MOISST/

X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfer

Linglin Jing, Ying Xue, Xu Yan, Chaoda Zheng, Dong Wang, Ruimao Zhang, Zhigang Wang, Hui Fang, Bin Zhao, Zhen Li

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

A multi-modal table tennis robot system

Andreas Ziegler, Thomas Gossard, Karl Vetter, Jonas Tebbe, Andreas Zell

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Comments Accepted for RoboLetics: Workshop on Robot Learning in Athletics @CoRL 2023

Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognition

Sumin Lee, Sangmin Woo, Muhammad Adi Nugroho, Changick Kim

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2208.11314

MuST: Multimodal Spatiotemporal Graph-Transformer for Hospital Readmission Prediction

Yan Miao, Lequan Yu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Video-Helpful Multimodal Machine Translation

Yihang Li, Shuichiro Shimizu, Chenhui Chu, Sadao Kurohashi, Wei Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 Main Conference (long paper)

$\mathbb{VD}$-$\mathbb{GR}$: Boosting $\mathbb{V}$isual $\mathbb{D}$ialog with Cascaded Spatial-Temporal Multi-Modal $\mathbb{GR}$aphs

Adnen Abdessaied, Lei Shi, Andreas Bulling

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments WACV 2024

Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection

Linyan Huang, Zhiqi Li, Chonghao Sima, Wenhai Wang, Jingdong Wang, Yu Qiao, Hongyang Li

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2023

MST-GAT: A Multimodal Spatial-Temporal Graph Attention Network for Time Series Anomaly Detection

Chaoyue Ding, Shiliang Sun, Jing Zhao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Information Fusion 2023 accepted

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Ziqi Gao, Yuntao Wang, Jianguo Chen, Junliang Xing, Shwetak Patel, Xin Liu, Yuanchun Shi

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling

Haogeng Liu, Qihang Fan, Tingkai Liu, Linjie Yang, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV