多模态大模型

Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning

Richard J. Chen, Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Jana Lipkova, Muhammad Shaban, Maha Shady, Mane Williams, Bumjin Joo, Zahra Noor, Faisal Mahmood

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Demo: http://pancancer.mahmoodlab.org

Multimodal Few-Shot Learning with Frozen Language Models

Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. M. Ali Eslami, Oriol Vinyals, Felix Hill

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate

Austin Botelho, Bertie Vidgen, Scott A. Hale

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Please note the paper contains examples of hateful content

Journal ref Findings of ACL, 2021

Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation

Zhiyong Wu, Lingpeng Kong, Wei Bi, Xiang Li, Ben Kao

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments To appear at ACL 2021 main conference

Cross-lingual Visual Pre-training for Multimodal Machine Translation

Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Madhyastha, Erkut Erdem, Aykut Erdem, Lucia Specia

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to EACL 2021 (Camera-ready version)

Probing Multimodal Embeddings for Linguistic Properties: the Visual-Semantic Case

Adam Dahlgren Lindström, Suna Bensch, Johanna Björklund, Frank Drewes

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Submitted July 1 2020, COLING 2020 main conference

Multimodal In-bed Pose and Shape Estimation under the Blankets

Yu Yin, Joseph P. Robinson, Yun Fu

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Conjugate Mixture Models for Clustering Multimodal Data

Vasil Khalidov, Florence Forbes, Radu Horaud

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、eess.AS

Journal ref Neural Computation, 23(2), 2011

Transductive Zero-Shot Learning using Cross-Modal CycleGAN

Patrick Bordes, Eloi Zablocki, Benjamin Piwowarski, Patrick Gallinari

专题命中 其他多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Learning more expressive joint distributions in multimodal variational methods

Sasho Nedelkoski, Mihail Bogojeski, Odej Kao

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, Accepted and presented at LOD 2020

Detecting depression in dyadic conversations with multimodal narratives and visualizations

Joshua Y. Kim, Greyson Y. Kim, Kalina Yacef

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages

Journal ref AI 2019: Advances in Artificial Intelligence. AI 2019 vol 11919

Cross-modal Subspace Learning via Kernel Correlation Maximization and Discriminative Structure Preserving

Jun Yu, Xiao-Jun Wu

专题命中 其他多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments The paper is under consideration at Multimedia Tools and Applications

Granular Multimodal Attention Networks for Visual Dialog

Badri N. Patro, Shivansh Patel, Vinay P. Namboodiri

专题命中 其他多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments ICCV Workshop

Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network

Anmol Sharma, Ghassan Hamarneh

专题命中 其他多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication in IEEE Transactions on Medical Imaging

Unsupervised Discovery of Multimodal Links in Multi-image, Multi-sentence Documents

Jack Hessel, Lillian Lee, David Mimno

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Code and data available at http://www.cs.cornell.edu/~jhessel/multiretrieval/multiretrieval.html

Journal ref EMNLP 2019

Controlling for Confounders in Multimodal Emotion Classification via Adversarial Learning

Mimansa Jaiswal, Zakaria Aldeneh, Emily Mower Provost

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments 10 pages, ICMI 2019

Multimodal Grounding for Language Processing

Lisa Beinborn, Teresa Botschen, Iryna Gurevych

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments The paper has been published in the Proceedings of the 27 Conference of Computational Linguistics. Please refer to this version for citations: https://www.aclweb.org/anthology/papers/C/C18/C18-1197/

Multi-level Multimodal Common Semantic Space for Image-Phrase Grounding

Hassan Akbari, Svebor Karaman, Surabhi Bhargava, Brian Chen, Carl Vondrick, Shih-Fu Chang

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted in CVPR 2019

Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing

Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, Hongsheng Li

专题命中 其他多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by CVPR 2019

MMKG: Multi-Modal Knowledge Graphs

Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, David S. Rosenblum

专题命中 其他多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments ESWC 2019

Multi-Modal Coreference Resolution with the Correlation between Space Structures

Qibin Zheng, Xingchun Diao, Jianjun Cao, Xiaolei Zhou, Yi Liu, Hongmei Li

专题命中 其他多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 6 figures

Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration

Yipeng Hu, Marc Modat, Eli Gibson, Wenqi Li, Nooshin Ghavami, Ester Bonmati, Guotai Wang, Steven Bandula, Caroline M. Moore, Mark Emberton, Sébastien Ourselin, J. Alison Noble, Dean C. Barratt, Tom Vercauteren

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted manuscript in Medical Image Analysis

Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer

Xin Wang, Geoffrey Oxholm, Da Zhang, Yuan-Fang Wang

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2017

Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description

Desmond Elliott, Stella Frank, Loïc Barrault, Fethi Bougares, Lucia Specia

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Journal ref Proceedings of the Second Conference on Machine Translation, 2017, pp. 215--233

RegNet: Multimodal Sensor Registration Using Deep Neural Networks

Nick Schneider, Florian Piewak, Christoph Stiller, Uwe Franke

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments published in IEEE Intelligent Vehicles Symposium, 2017

Applying the Wizard-of-Oz Technique to Multimodal Human-Robot Dialogue

Matthew Marge, Claire Bonial, Brendan Byrne, Taylor Cassidy, A. William Evans, Susan G. Hill, Clare Voss

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Presented at the 2016 IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Interactive Session, August 26-31, 2016

Robot gains Social Intelligence through Multimodal Deep Reinforcement Learning

Ahmed Hussain Qureshi, Yutaka Nakamura, Yuichiro Yoshikawa, Hiroshi Ishiguro

专题命中 其他多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments The paper is published in IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2016

Multimodal image registration for effective thermographic fever screening

C. Y. N. Dwith, Pejhman Ghassemi, Joshua Pfefer, Jon Casamento, Quanzeng Wang

专题命中 其他多模态 ：multimodal(title,journal_ref)；multi-modal(abstract)；分类 cs.CV

Journal ref Proceedings Volume 10057, Multimodal Biomedical Imaging XII 100570S, 2017

Cross-Modal Knowledge Distillation from Spatial Transcriptomics to Histology

从空间转录组到组织学的跨模态知识蒸馏

Arbel Hizmi, Artemii Bakulin, Shai Bagon, Nir Yosef

机构 * Weizmann Institute of Science（魏茨曼科学研究所） ； Reichman University（赖希曼大学） 纠错

专题命中 其他多模态 ：cross-modal(title,abstract)；分类 cs.CV

AI总结 本文提出利用空间转录组与H&E数据进行跨模态蒸馏，将转录组学的 niches 结构转移到仅依赖组织学的模型中，提升与转录组学 niches 结构的一致性，并通过细胞类型分析验证生物意义的邻近组成。

Comments Accepted to the CVMI Workshop at CVPR 2026. Project page: https://cross-modal-distillation.github.io/

Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-In-The-Loop LLM

Jiachen Li, Xiwen Li, Justin Steinberg, Akshat Choube, Bingsheng Yao, Xuhai Xu, Dakuo Wang, Elizabeth Mynatt, Varun Mishra

机构 * Northeastern University（东北大学） ； Columbia University（哥伦比亚大学） 纠错

专题命中 其他多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments Jiachen Li, Xiwen Li, Justin Steinberg, Akshat Choube, Bingsheng Yao, Xuhai Xu, Dakuo Wang, Elizabeth Mynatt, and Varun Mishra. 2025. Vital Insight: Assisting Experts' Context-Driven Sensemaking of Multi-modal Personal Tracking Data Using Visualization and Human-in-the-Loop LLM. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 9, 3, Article 101 (September 2025), 37 pages

Journal ref Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 9 (2025) 101:1-37