多模态大模型

MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer

Fudong Lin, Summer Crawford, Kaleb Guillot, Yihe Zhang, Yan Chen, Xu Yuan, Li Chen, Shelby Williams, Robert Minvielle, Xiangming Xiao, Drew Gholson, Nicolas Ashwell, Tri Setiyono, Brenda Tubana, Lu Peng, Magdy Bayoumi, Nian-Feng Tzeng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref ICCV 2023

SeisCLIP: A seismology foundation model pre-trained by multi-modal data for multi-purpose seismic feature extraction

Xu Si, Xinming Wu, Hanlin Sheng, Jun Zhu, Zefeng Li

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments 27 pages, 9 figures, 4 tables

Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers

Zhiyu Zhu, Junhui Hou, Dapeng Oliver Wu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by ICCV

Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor

Xinyang Liu, Yijin Li, Yanbin Teng, Hujun Bao, Guofeng Zhang, Yinda Zhang, Zhaopeng Cui

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to ICCV 2023 (Oral). Project Page: https://zju3dv.github.io/tof_slam/

Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection

Alessandro Flaborea, Luca Collorone, Guido D'Amely, Stefano D'Arrigo, Bardh Prenkaj, Fabio Galasso

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ICCV2023

MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation

Najmeh Sadoughi, Xinyu Li, Avijit Vajpayee, David Fan, Bing Shuai, Hector Santos-Villalobos, Vimal Bhat, Rohith MV

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments ICCV 2023 accepted

Cross-Modal Learning with 3D Deformable Attention for Action Recognition

Sangwon Kim, Dasom Ahn, Byoung Chul Ko

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by ICCV2023

Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration

Juan Del Aguila Ferrandis, João Moura, Sethu Vijayakumar

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Cross-modal Manifold Cutmix for Self-supervised Video Representation Learning

Srijan Das, Michael S. Ryoo

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at MVA 2023

Estudio de la Experiencia de Usuario mediante un Sistema de Dashboards de Análisis de Aprendizaje Multimodal

Álvaro Becerra, Roberto Daza, Ruth Cobos, Aythami Morales, Julian Fierrez

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments Accepted in "XXIII CONGRESO INTERNACIONAL DE INTERACCIÓN PERSONA-ORDENADOR 2023". Article in Spanish language. The abstract in English and Spanish. There is an extended abstract of 2 pages in English

MUVF-YOLOX: A Multi-modal Ultrasound Video Fusion Network for Renal Tumor Diagnosis

Junyu Li, Han Huang, Dong Ni, Wufeng Xue, Dongmei Zhu, Jun Cheng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments MICCAI 2023

Cross-modal Place Recognition in Image Databases using Event-based Sensors

Xiang Ji, Jiaxin Wei, Yifu Wang, Huiliang Shang, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

S-Omninet: Structured Data Enhanced Universal Multimodal Learning Architecture

Ye Xue, Diego Klabjan, Jean Utke

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer

Jiaxin Deng, Dong Shen, Shiyao Wang, Xiangyu Wu, Fan Yang, Guorui Zhou, Gaofeng Meng

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modeling

Xinlu Zhang, Shiyang Li, Zhiyu Chen, Xifeng Yan, Linda Petzold

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Interpretable Multimodal Emotion Recognition using Facial Features and Physiological Signals

Puneet Kumar, Xiaobai Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted for Oral Presentation in DAI 2023 (https://rbcdsai.iitm.ac.in/DAI-2023/program.html)

Multi-Modal Deep Learning for Multi-Temporal Urban Mapping With a Partly Missing Optical Modality

Sebastian Hafner, Yifang Ban

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 4 pages, 2 figures, accepted for publication in the IGARSS 2023 Proceedings

A Multi-Modal Transformer Network for Action Detection

Matthew Korban, Scott T. Acton, Peter Youngs

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref Pattern Recognition 2023

MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization

Wujiang Xu, Runzhong Wang, Xiaobo Guo, Shaoshuai Li, Qiongxu Ma, Yunan Zhao, Sheng Guo, Zhenfeng Zhu, Junchi Yan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation

Yunyao Mao, Wengang Zhou, Zhenbo Lu, Jiajun Deng, Houqiang Li

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments To appear in ECCV 2022 (Oral)

M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education

Álvaro Becerra, Roberto Daza, Ruth Cobos, Aythami Morales, Mutlu Cukurova, Julian Fierrez

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted in "Workshop on Open Education Resources (OER) of COMPSAC 2023"

Multimodal Short Video Rumor Detection System Based on Contrastive Learning

Yuxing Yang, Junhao Zhao, Siyi Wang, Xiangyu Min, Pengchao Wang, Haizhou Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System

Junke Wang, Dongdong Chen, Chong Luo, Xiyang Dai, Lu Yuan, Zuxuan Wu, Yu-Gang Jiang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments work in progress

Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild

Yufeng Yin, Minh Tran, Di Chang, Xinrui Wang, Mohammad Soleymani

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 8 pages, 7 figures, 5 tables

Multi-modal Facial Affective Analysis based on Masked Autoencoder

Wei Zhang, Bowen Ma, Feng Qiu, Yu Ding

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Multi-modal Prompting for Low-Shot Temporal Action Localization

Chen Ju, Zeqian Li, Peisen Zhao, Ya Zhang, Xiaopeng Zhang, Qi Tian, Yanfeng Wang, Weidi Xie

专题命中 视频多模态 ：multi-modal(title)；image-text(abstract)；分类 cs.CV

Efficient End-to-End Video Question Answering with Pyramidal Multimodal Transformer

Min Peng, Chongyang Wang, Yu Shi, Xiang-Dong Zhou

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by AAAI 2023

Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning

Yuchong Sun, Hongwei Xue, Ruihua Song, Bei Liu, Huan Yang, Jianlong Fu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2022

A Flexible Multi-view Multi-modal Imaging System for Outdoor Scenes

Meng Zhang, Wenxuan Guo, Bohao Fan, Yifan Chen, Jianjiang Feng, Jie Zhou

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Modality Mixer for Multi-modal Action Recognition

Sumin Lee, Sangmin Woo, Yeonju Park, Muhammad Adi Nugroho, Changick Kim

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023