多模态大模型 - arXivDaily 专题

2304.03022 2023-04-07 cs.IR 78%

TagGPT: Large Language Models are Zero-shot Multimodal Taggers

Chen Li, Yixiao Ge, Jiayong Mao, Dian Li, Ying Shan

专题命中音频语音多模态：multimodal(title,abstract)

Comments 13 pages, 6 figures

2302.07547 2023-02-16 stat.AP cs.HC 78%

Multimodal N-of-1 trials: A Novel Personalized Healthcare Design

Jingjing Fu, Shuheng Liu, Siqi Du, Siqiao Ruan, Xuliang Guo, Weiwei Pan, Abhishek Sharma, Stefan Konigorski

专题命中音频语音多模态：multimodal(title,abstract)

2302.03227 2023-02-08 cs.LG eess.SP q-bio.NC 78%

Automatic Sleep Stage Classification with Cross-modal Self-supervised Features from Deep Brain Signals

Chen Gong, Yue Chen, Yanan Sui, Luming Li

专题命中音频语音多模态：cross-modal(title,abstract)

Comments 4 pages, 5 figures, 11th International IEEE EMBS Conference on Neural Engineering (NER)

2205.02670 2023-01-10 cs.LG 78%

Unsupervised Mismatch Localization in Cross-Modal Sequential Data with Application to Mispronunciations Localization

Wei Wei, Huang Hengguan, Gu Xiangming, Wang Hao, Wang Ye

专题命中音频语音多模态：cross-modal(title,abstract)

2207.12566 2022-12-02 cs.HC 78%

Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment

Xiaoyan Zhou, Adam S. Williams, Francisco R. Ortega

专题命中音频语音多模态：multimodal(title,abstract)

2009.06591 2022-08-09 cs.HC 78%

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Adam S. Williams, Francisco R. Ortega

专题命中音频语音多模态：multimodal(title,abstract)

Journal ref Proceedings of the ACM on Human-Computer Interaction Volume 4 Issue ISS November 2020

2202.01439 2022-07-06 cs.HC 78%

Sensing the Breath: A Multimodal Singing Tutoring Interface with Breath Guidance

Ziyue Piao, Gus Xia

专题命中音频语音多模态：multimodal(title,abstract)

Comments New Interfaces for Musical Expression 2022 Demo Paper

2203.13932 2022-03-29 cs.MM cs.CV cs.SD eess.AS 78%

A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals

Yuanchao Li, Catherine Lai

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments 5 pages, 2 figures, submitted to INTERSPEECH 2022

2203.01977 2022-03-07 cs.MM cs.CV cs.RO cs.SD eess.AS 78%

Audio-Visual Object Classification for Human-Robot Collaboration

A. Xompero, Y. L. Pang, T. Patten, A. Prabhakar, B. Calli, A. Cavallaro

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments 5 pages, 2 figures, 1 table; accepted at ICASSP 2022; Challenge webpage, see https://corsmal.eecs.qmul.ac.uk/challenge.html

2109.04833 2022-02-21 cs.LG 78%

Multimodal Federated Learning on IoT Data

Yuchen Zhao, Payam Barnaghi, Hamed Haddadi

专题命中音频语音多模态：multimodal(title,abstract)

Comments 12 pages, IoTDI '22, May 3-6, 2022, Milan, Italy

2202.01639 2022-02-04 cs.HC 78%

Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory substitution

Rynhardt Kruger, Febe de Wet, Thomas Niesler

专题命中音频语音多模态：audio-visual(title,abstract)

2201.02237 2022-01-10 cs.RO cs.HC 78%

Multi-modal data fusion of Voice and EMG data for Robotic Control

Tauheed Khan Mohd, Jackson Carvalho, Ahmad Y Javaid

专题命中音频语音多模态：multi-modal(title,abstract)

2110.04634 2021-10-12 cs.RO 78%

Multimodal Sensory Learning for Real-time, Adaptive Manipulation

Ahalya Prabhakar, Stanislas Furrer, Lorenzo Panchetti, Maxence Perret, Aude Billard

专题命中音频语音多模态：multimodal(title,abstract)

Comments Presented at AI-HRI symposium as part of AAAI-FSS 2021 (arXiv:2109.10836)

2110.03660 2021-10-08 cs.HC cs.LG 78%

Developing Medical AI : a cloud-native audio-visual data collection study

Sagi Schein, Greg Arutiunian, Vitaly Burshtein, Gal Sadeh, Michelle Townshend, Bruce Friedman, Shada Sadr-azodi

专题命中音频语音多模态：audio-visual(title,abstract)

2010.09948 2021-10-05 cs.RO 78%

Object Permanence Through Audio-Visual Representations

Fanjun Bu, Chien-Ming Huang

专题命中音频语音多模态：audio-visual(title)；multimodal(abstract)

Comments 8 pages, 4 figures, 2 tables, published in IEEE Access

Journal ref in IEEE Access, vol. 9, pp. 131574-131582, 2021

2109.10571 2021-09-23 cs.RO 78%

Audio-Visual Grounding Referring Expression for Robotic Manipulation

Yefei Wang, Kaili Wang, Yi Wang, Di Guo, Huaping Liu, Fuchun Sun

专题命中音频语音多模态：audio-visual(title,abstract)

2102.03796 2021-08-04 cs.HC 78%

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Akinari Onishi

专题命中音频语音多模态：multimodal(title,abstract)

2105.13132 2021-06-23 cs.LG cs.CY 78%

Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution

Yang Li, Zinc Zhang, Hutchin Huang

专题命中音频语音多模态：multimodal(title)；multi-modal(abstract)

Comments Our code is available at: https://github.com/yangland/hatefulchallenge

2105.05182 2021-05-13 cs.HC 78%

PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

Yaohua Bu, Tianyi Ma, Weijun Li, Hang Zhou, Jia Jia, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, Kun Li, Zhiyong Wu, Yuanchun Shi, Xiaobo Lu, Ziwei Liu

专题命中音频语音多模态：audio-visual(title,abstract)

2105.04489 2021-05-11 cs.CV cs.CL cs.LG cs.SD eess.AS 78%

Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions

Mathew Monfort, SouYoung Jin, Alexander Liu, David Harwath, Rogerio Feris, James Glass, Aude Oliva

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.CL、eess.AS

Comments To appear at CVPR 2021

2105.00708 2021-05-04 cs.SD cs.CV cs.MM eess.AS 78%

Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation

Yan-Bo Lin, Yu-Chiang Frank Wang

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments AAAI'21

2101.09725 2021-03-15 cs.CR 78%

Audio-Visual Biometric Recognition and Presentation Attack Detection: A Comprehensive Survey

Hareesh Mandalapu, P N Aravinda Reddy, Raghavendra Ramachandra, K Sreenivasa Rao, Pabitra Mitra, S R Mahadeva Prasanna, Christoph Busch

专题命中音频语音多模态：audio-visual(title,abstract)

Journal ref in IEEE Access, vol. 9, pp. 37431-37455, 2021

2103.05309 2021-03-10 cs.HC cs.CY 78%

Trade-offs in the Design of Multimodal Interaction for Older Adults

Gianluca Schiavo, Ornella Mich, Michela Ferron, Nadia Mana

专题命中音频语音多模态：multimodal(title,abstract)

Journal ref Behaviour & Information Technology, 2020

2012.01583 2020-12-04 cs.RO 78%

Multimodal Contact Detection using Auditory and Force Features for Reliable Object Placing in Household Environments

Jaime Maldonado, Asil Kaan Bozcuoğlu, Christoph Zetzsche

专题命中音频语音多模态：multimodal(title,abstract)

2012.00922 2020-12-03 cs.HC 78%

Cross-Modal Terrains: Navigating Sonic Space through Haptic Feedback

Gabriella Isaac, Lauren Hayes, Todd Ingalls

专题命中音频语音多模态：cross-modal(title)；multimodal(abstract)

Journal ref Proceedings of the International Conference on New Interfaces for Musical Expression, 2017

2004.05488 2020-09-03 cs.NE cs.LG q-bio.NC 78%

Brain-inspired self-organization with cellular neuromorphic computing for multimodal unsupervised learning

Lyes Khacef, Laurent Rodriguez, Benoit Miramond

专题命中音频语音多模态：multimodal(title,abstract)

Comments Preprint

2008.13510 2020-09-01 cs.HC 78%

Multi-Modal End-User Programming of Web-Based Virtual Assistant Skills

Michael H. Fischer, Giovanni Campagna, Euirim Choi, Monica S. Lam

专题命中音频语音多模态：multi-modal(title,abstract)

2008.11834 2020-08-28 cs.HC 78%

Conversations On Multimodal Input Design With Older Adults

Adam S. Williams, Sarah Coler, Francisco Ortega

专题命中音频语音多模态：multimodal(title,abstract)

Comments Presented at the CHI 2020 Designing Interactions for the Ageing Populations Addressing Global Challenges Workshop

2005.08271 2020-08-12 cs.CV cs.CL cs.LG cs.SD eess.AS 78%

A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer

Vladimir Iashin, Esa Rahtu

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.CL、eess.AS

Comments Accepted by BMVC 2020. More experiments. Code: https://github.com/v-iashin/bmt Project page: https://v-iashin.github.io/bmt

2006.03813 2020-06-09 cs.HC 78%

Multimodal Systems: Taxonomy, Methods, and Challenges

Muhammad Zeeshan Baig, Manolya Kavakli

专题命中音频语音多模态：multimodal(title,abstract)

1. 音频语音多模态 4562 篇

TagGPT: Large Language Models are Zero-shot Multimodal Taggers

Multimodal N-of-1 trials: A Novel Personalized Healthcare Design

Automatic Sleep Stage Classification with Cross-modal Self-supervised Features from Deep Brain Signals

Unsupervised Mismatch Localization in Cross-Modal Sequential Data with Application to Mispronunciations Localization

Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Sensing the Breath: A Multimodal Singing Tutoring Interface with Breath Guidance

A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals

Audio-Visual Object Classification for Human-Robot Collaboration

Multimodal Federated Learning on IoT Data

Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory substitution

Multi-modal data fusion of Voice and EMG data for Robotic Control

Multimodal Sensory Learning for Real-time, Adaptive Manipulation

Developing Medical AI : a cloud-native audio-visual data collection study

Object Permanence Through Audio-Visual Representations

Audio-Visual Grounding Referring Expression for Robotic Manipulation

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution

PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions

Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation

Audio-Visual Biometric Recognition and Presentation Attack Detection: A Comprehensive Survey

Trade-offs in the Design of Multimodal Interaction for Older Adults

Multimodal Contact Detection using Auditory and Force Features for Reliable Object Placing in Household Environments

Cross-Modal Terrains: Navigating Sonic Space through Haptic Feedback

Brain-inspired self-organization with cellular neuromorphic computing for multimodal unsupervised learning

Multi-Modal End-User Programming of Web-Based Virtual Assistant Skills

Conversations On Multimodal Input Design With Older Adults

A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer

Multimodal Systems: Taxonomy, Methods, and Challenges