多模态大模型

When did you become so smart, oh wise one?! Sarcasm Explanation in Multi-modal Multi-party Dialogues

Shivani Kumar, Atharva Kulkarni, Md Shad Akhtar, Tanmoy Chakraborty

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted in ACL 2022. 13 pages, 4 figures, 12 tables

Toddler-Guidance Learning: Impacts of Critical Period on Multimodal AI Agents

Junseok Park, Kwanyoung Park, Hyunseok Oh, Ganghun Lee, Minsu Lee, Youngki Lee, Byoung-Tak Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ICMI2021 Oral Presentation, 9 pages, 9 figures

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams

Tuomo Hiippala, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, John A. Bateman

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 24 pages; revised version submitted to Language Resources & Evaluation

Journal ref Language Resources and Evaluation 55(3), 2021, pp. 661-688

UNITER-Based Situated Coreference Resolution with Rich Multimodal Input

Yichen Huang, Yuchen Wang, Yik-Cheung Tam

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text

Christopher Clark, Jordi Salvador, Dustin Schwenk, Derrick Bonafilia, Mark Yatskar, Eric Kolve, Alvaro Herrasti, Jonghyun Choi, Sachin Mehta, Sam Skjonsberg, Carissa Schoenick, Aaron Sarnat, Hannaneh Hajishirzi, Aniruddha Kembhavi, Oren Etzioni, Ali Farhadi

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments In EMNLP 2021

Metric-based multimodal meta-learning for human movement identification via footstep recognition

Muhammad Shakeel, Katsutoshi Itoyama, Kenji Nishida, Kazuhiro Nakadai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

An AutoML-based Approach to Multimodal Image Sentiment Analysis

Vasco Lopes, António Gaspar, Luís A. Alexandre, João Cordeiro

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Challenges in Procedural Multimodal Machine Comprehension:A Novel Way To Benchmark

Pritish Sahu, Karan Sikka, Ajay Divakaran

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media

Anargyros Chatzitofis, Leonidas Saroglou, Prodromos Boutis, Petros Drakoulis, Nikolaos Zioulis, Shishir Subramanyam, Bart Kevelham, Caecilia Charbonnier, Pablo Cesar, Dimitrios Zarpalas, Stefanos Kollias, Petros Daras

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref IEEE Access, 8, 176241-176262, 2020

SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

Satwik Kottur, Seungwhan Moon, Alborz Geramifard, Babak Damavandi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 7 figures, 5 tables

An animated picture says at least a thousand words: Selecting Gif-based Replies in Multimodal Dialog

Xingyao Wang, David Jurgens

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Findings of EMNLP 2021; 30 pages

MFEViT: A Robust Lightweight Transformer-based Network for Multimodal 2D+3D Facial Expression Recognition

Hanting Li, Mingzhe Sui, Zhaoqing Zhu, Feng Zhao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9pages,6 figures,5 tables

IITP at WAT 2021: System description for English-Hindi Multimodal Translation Task

Baban Gain, Dibyanayan Bandyopadhyay, Asif Ekbal

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions

David Curto, Albert Clapés, Javier Selva, Sorina Smeureanu, Julio C. S. Jacques Junior, David Gallardo-Pujol, Georgina Guilera, David Leiva, Thomas B. Moeslund, Sergio Escalera, Cristina Palmero

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to the 2021 ICCV Workshop on Understanding Social Behavior in Dyadic and Small Group Interactions

Towards Expressive Communication with Internet Memes: A New Multimodal Conversation Dataset and Benchmark

Zhengcong Fei, Zekang Li, Jinchao Zhang, Yang Feng, Jie Zhou

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering

Corentin Dancette, Remi Cadene, Damien Teney, Matthieu Cord

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICCV 2021. Code is available at https://github.com/cdancette/detect-shortcuts

Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach

Chuanbo Hu, Minglei Yin, Bin Liu, Xin Li, Yanfang Ye

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by CIKM 2021

PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling

Xiaoxue Zang, Lijuan Liu, Maria Wang, Yang Song, Hao Zhang, Jindong Chen

专题命中 多模态评测 ：image-text(title,abstract)；分类 cs.CL、cs.AI

A Multimodal Machine Learning Framework for Teacher Vocal Delivery Evaluation

Hang Li, Yu Kang, Yang Hao, Wenbiao Ding, Zhongqin Wu, Zitao Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments AIED'21: The 22nd International Conference on Artificial Intelligence in Education, 2021

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Paul Pu Liang, Terrance Liu, Anna Cai, Michal Muszynski, Ryo Ishii, Nicholas Allen, Randy Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments ACL 2021. arXiv admin note: substantial text overlap with arXiv:2012.02359

"Subverting the Jewtocracy": Online Antisemitism Detection Using Multimodal Deep Learning

Mohit Chandra, Dheeraj Pailla, Himanshu Bhatia, Aadilmehdi Sanchawala, Manish Gupta, Manish Shrivastava, Ponnurangam Kumaraguru

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Knowledge distillation from multi-modal to mono-modal segmentation networks

Minhao Hu, Matthis Maillard, Ya Zhang, Tommaso Ciceri, Giammarco La Barbera, Isabelle Bloch, Pietro Gori

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments MICCAI 2020

Journal ref MICCAI 2020

A Two-stage Multi-modal Affect Analysis Framework for Children with Autism Spectrum Disorder

Jicheng Li, Anjana Bhat, Roghayeh Barmaki

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 8 pages including reference; 8 figures

Journal ref The AAAI-21 Workshop On Affective Content Analysis; 2021

A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods

Gullal S. Cheema, Sherzod Hakimov, Eric Müller-Budack, Ralph Ewerth

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted in Workshop on Multi-ModalPre-Training for Multimedia Understanding (MMPT 2021), co-located with ICMR 2021

Sejong Face Database: A Multi-Modal Disguise Face Database

Usman Cheema, Seungbin Moon

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Database Access Link: https://github.com/usmancheema89/SejongFaceDatabase

Journal ref Computer Vision and Image Understanding, Volumes 208-209, 2021

Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP

John Chen, Ian Berlot-Attwell, Safwan Hossain, Xindi Wang, Frank Rudzicz

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Best paper award at 3rd Clinical Natural Language Processing Workshop at EMNLP 2020

Journal ref Proceedings of the 3rd Clinical Natural Language Processing Workshop (2020), pages 301--312

Adversarial Evaluation of Multimodal Models under Realistic Gray Box Assumption

Ivan Evtimov, Russel Howes, Brian Dolhansky, Hamed Firooz, Cristian Canton Ferrer

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Deep Learning Framework for Image Popularity Prediction on Social Media

Fatma S. Abousaleh, Wen-Huang Cheng, Neng-Hao Yu, Yu Tsao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments 14 pages, 11 figures, 7 tables

Journal ref IEEE Transactions on Cognitive and Developmental Systems. 2020 Nov 9

MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task

Yiqun Yao, Michalis Papakostas, Mihai Burzo, Mohamed Abouelenien, Rada Mihalcea

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments NAACL 2021 accepted

Multimodal Entity Linking for Tweets

Omar Adjali, Romaric Besançon, Olivier Ferret, Herve Le Borgne, Brigitte Grau

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Journal ref In: Jose J. et al. (eds) Advances in Information Retrieval. ECIR 2020. Lecture Notes in Computer Science, vol 12035. Springer, Cham