多模态大模型 - arXivDaily 专题

2109.05812 2022-02-16 cs.CL 79%

UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation

Zhengkun Zhang, Xiaojun Meng, Yasheng Wang, Xin Jiang, Qun Liu, Zhenglu Yang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at AAAI2022

2101.01881 2021-10-25 cs.CV cs.LG 79%

MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding

Woojeong Jin, Maziar Sanjabi, Shaoliang Nie, Liang Tan, Xiang Ren, Hamed Firooz

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to EMNLP 2021 Findings

2108.05863 2021-08-13 cs.CV 79%

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision

Xiaoshi Wu, Hadar Averbuch-Elor, Jin Sun, Noah Snavely

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Published in ICCV 2021; Project webpage: https://www.cs.cornell.edu/projects/babel/

2106.02400 2021-06-07 cs.CV cs.IR cs.LG 79%

A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval

Manh-Duy Nguyen, Binh T. Nguyen, Cathal Gurrin

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2103.16022 2021-03-31 cs.CV 79%

Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays

Xiaosong Wang, Ziyue Xu, Leo Tam, Dong Yang, Daguang Xu

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

1601.03916 2021-02-03 cs.CL 79%

Multimodal Pivots for Image Caption Translation

Julian Hitschler, Shigehiko Schamoni, Stefan Riezler

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Final version, accepted at ACL 2016. New section on Human Evaluation

2007.08883 2021-02-02 cs.CV 79%

Consensus-Aware Visual-Semantic Embedding for Image-Text Matching

Haoran Wang, Ying Zhang, Zhong Ji, Yanwei Pang, Lin Ma

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2020, Code is publicly available at: https://github.com/BruceW91/CVSE

2003.03669 2020-07-23 cs.CV 79%

Adaptive Offline Quintuplet Loss for Image-Text Matching

Tianlang Chen, Jiajun Deng, Jiebo Luo

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2020. Code is available at https://github.com/sunnychencool/AOQ

1806.08251 2020-07-08 cs.CV 79%

Learning Multimodal Representations for Unseen Activities

AJ Piergiovanni, Michael S. Ryoo

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Journal ref WACV 2020

2004.01894 2020-04-07 cs.CL 79%

Evaluating Multimodal Representations on Visual Semantic Textual Similarity

Oier Lopez de Lacalle, Ander Salaberria, Aitor Soroa, Gorka Azkune, Eneko Agirre

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted in ECAI-2020, 8 pages, 6 tables, 6 figures

2004.00277 2020-04-02 cs.CV 79%

Graph Structured Network for Image-Text Matching

Chunxiao Liu, Zhendong Mao, Tianzhu Zhang, Hongtao Xie, Bin Wang, Yongdong Zhang

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by CVPR2020

2002.08510 2020-02-21 cs.CV 79%

Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching

Tianlang Chen, Jiebo Luo

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by AAAI-20

1911.09042 2019-11-26 cs.CV 79%

Learning Cross-modal Context Graph for Visual Grounding

Yongfei Liu, Bo Wan, Xiaodan Zhu, Xuming He

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CV

Comments AAAI-2020

1910.05752 2019-10-15 cs.CL 79%

VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning

Ziqi Zhang, Yaya Shi, Jiutong Wei, Chunfeng Yuan, Bing Li, Weiming Hu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CL

1909.02701 2019-09-09 cs.CV 79%

Visual Semantic Reasoning for Image-Text Matching

Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, Yun Fu

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted to ICCV 2019 (Oral)

1908.07519 2019-08-22 cs.CV cs.HC cs.LG eess.IV eess.SP 79%

Multi-Modal Recognition of Worker Activity for Human-Centered Intelligent Manufacturing

Wenjin Tao, Ming C. Leu, Zhaozheng Yin

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments 17 pages, 8 figures, 6 tables

1907.09748 2019-07-24 cs.CL cs.IR cs.LG 79%

Position Focused Attention Network for Image-Text Matching

Yaxiong Wang, Hao Yang, Xueming Qian, Lin Ma, Jing Lu, Biao Li, Xin Fan

专题命中图文多模态：image-text(title,abstract)；分类 cs.CL

Comments Accepted by IJCAI 2019

1905.08181 2019-05-31 cs.CL 79%

A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks

Álvaro Peris, Francisco Casacuberta

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments ACL 2019 - System demonstrations

1905.07841 2019-05-21 cs.CV 79%

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

Jun Yu, Jing Li, Zhou Yu, Qingming Huang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments submitted to a journal

1905.02963 2019-05-09 cs.CV 79%

Multimodal Semantic Attention Network for Video Captioning

Liang Sun, Bing Li, Chunfeng Yuan, Zhengjun Zha, Weiming Hu

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments 6 pages, 4 figures, accepted by IEEE International Conference on Multimedia and Expo (ICME) 2019

1810.06233 2018-10-16 cs.CL 79%

UMONS Submission for WMT18 Multimodal Translation Task

Jean-Benoit Delbrouck, Stéphane Dupont

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

1707.01009 2018-06-01 cs.CL 79%

Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation

Jean-Benoit Delbrouck, Stéphane Dupont, Omar Seddati

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to GLU 2017. arXiv admin note: text overlap with arXiv:1707.00995

Journal ref Proc. GLU 2017 International Workshop on Grounding Language Understanding

1710.02718 2017-12-15 cs.CL 79%

OSU Multimodal Machine Translation System Report

Mingbo Ma, Dapeng Li, Kai Zhao, Liang Huang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments 5, WMT 2017

1708.02478 2017-10-23 cs.CV 79%

From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning

Jingkuan Song, Yuyu Guo, Lianli Gao, Xuelong Li, Alan Hanjalic, Heng Tao Shen

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

1702.05658 2017-08-11 cs.CV 79%

MAT: A Multimodal Attentive Translator for Image Captioning

Chang Liu, Fuchun Sun, Changhu Wang, Feng Wang, Alan Yuille

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

1707.04550 2017-07-17 cs.CL cs.NE 79%

CUNI System for the WMT17 Multimodal Translation Task

Jindřich Helcl, Jindřich Libovický

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments 8 pages; Camera-ready submission to WMT17

1703.08084 2017-03-24 cs.CL 79%

Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation

Jean-Benoit Delbrouck, Stephane Dupont

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Submitted to ICLR Workshop 2017

1611.05592 2016-11-18 cs.CV 79%

Multimodal Memory Modelling for Video Captioning

Junbo Wang, Wei Wang, Yan Huang, Liang Wang, Tieniu Tan

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

1609.03976 2016-09-14 cs.CL cs.NE 79%

Multimodal Attention for Neural Machine Translation

Ozan Caglayan, Loïc Barrault, Fethi Bougares

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments 10 pages, under review COLING 2016

2606.28724 2026-06-30 cs.CV cs.AI 79%

CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation

CCRC：面向图像变化描述与分割的变化感知描述与推理链

Jinhong Hu, Xiaoping Wang, Shuyin Huang, Guojin Zhong, Kaitai Liu, Kai Lu

机构 * College of Computer Science and Electronic Engineering, Hunan University, China（湖南大学计算机科学与电子工程学院，中国）； Guangxi Minzu University, China（广西民族大学，中国）； National University of Defense Technology, China（国防科学技术大学，中国）

专题命中图文多模态：MLLM(abstract,abstract_cn)；multimodal(abstract)；分类 cs.CV、cs.AI

AI总结提出变化感知描述与推理链（CCRC），通过双链框架解耦语义推理与空间分割，实现图像变化描述与分割（ICCS）任务，在合成和真实基准上达到最优性能。

1. 图文多模态 4644 篇

UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation

MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision

A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval

Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays

Multimodal Pivots for Image Caption Translation

Consensus-Aware Visual-Semantic Embedding for Image-Text Matching

Adaptive Offline Quintuplet Loss for Image-Text Matching

Learning Multimodal Representations for Unseen Activities

Evaluating Multimodal Representations on Visual Semantic Textual Similarity

Graph Structured Network for Image-Text Matching

Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching

Learning Cross-modal Context Graph for Visual Grounding

VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning

Visual Semantic Reasoning for Image-Text Matching

Multi-Modal Recognition of Worker Activity for Human-Centered Intelligent Manufacturing

Position Focused Attention Network for Image-Text Matching

A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

Multimodal Semantic Attention Network for Video Captioning

UMONS Submission for WMT18 Multimodal Translation Task

Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation

OSU Multimodal Machine Translation System Report

From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning

MAT: A Multimodal Attentive Translator for Image Captioning

CUNI System for the WMT17 Multimodal Translation Task

Multimodal Compact Bilinear Pooling for Multimodal Neural Machine Translation

Multimodal Memory Modelling for Video Captioning

Multimodal Attention for Neural Machine Translation

CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation