多模态大模型

MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images

Weihao Liu, Fangyu Lei, Tongxu Luo, Jiahe Lei, Shizhu He, Jun Zhao, Kang Liu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

DIME-FM: DIstilling Multimodal and Efficient Foundation Models

Ximeng Sun, Pengchuan Zhang, Peizhao Zhang, Hardik Shah, Kate Saenko, Xide Xia

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CV

Comments Accepted to ICCV 2023

MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild

Yuanyuan Liu, Wei Dai, Chuanxu Feng, Wenbin Wang, Guanghao Yin, Jiabei Zeng, Shiguang Shan

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments This paper has been accepted by ACM MM'22

Foundation Model is Efficient Multimodal Multitask Model Selector

Fanqing Meng, Wenqi Shao, Zhanglin Peng, Chonghe Jiang, Kaipeng Zhang, Yu Qiao, Ping Luo

专题命中 图文多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Text-based Person Search without Parallel Image-Text Data

Yang Bai, Jingyao Wang, Min Cao, Chen Chen, Ziqiang Cao, Liqiang Nie, Min Zhang

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted by ACM MM 2023

Unified Medical Image-Text-Label Contrastive Learning With Continuous Prompt

Yuhao Wang

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Caption Anything: Interactive Image Description with Diverse Multimodal Controls

Teng Wang, Jinrui Zhang, Junjie Fei, Hao Zheng, Yunlong Tang, Zhe Li, Mingqi Gao, Shanshan Zhao

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Tech-report

A Survey of Vision-Language Pre-training from the Lens of Multimodal Machine Translation

Jeremy Gwinnup, Kevin Duh

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 10 pages

Leverage Points in Modality Shifts: Comparing Language-only and Multimodal Word Representations

Aleksey Tikhonov, Lisa Bylinina, Denis Paperno

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted for StarSEM 2023

Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination

Hao Fei, Qian Liu, Meishan Zhang, Min Zhang, Tat-Seng Chua

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments ACL 2023

Scene Text Recognition with Image-Text Matching-guided Dictionary

Jiajun Wei, Hongjian Zhan, Xiao Tu, Yue Lu, Umapada Pal

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted at ICDAR2023

Improving Image Recognition by Retrieving from Web-Scale Image-Text Data

Ahmet Iscen, Alireza Fathi, Cordelia Schmid

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2023

Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairs

Junbum Cha, Jonghwan Mun, Byungseok Roh

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments CVPR 2023 camera-ready

Increasing Textual Context Size Boosts Medical Image-Text Matching

Idan Glassberg, Tom Hope

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

FER-former: Multi-modal Transformer for Facial Expression Recognition

Yande Li, Mingjie Wang, Minglun Gong, Yonggang Lu, Li Liu

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV

ContextCLIP: Contextual Alignment of Image-Text pairs on CLIP visual representations

Chanda Grover, Indra Deep Mastan, Debayan Gupta

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments 11 Pages, 7 Figures, 2 Tables, ICVGIP

Journal ref ICVGIP, 2022

The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images

Philip Müller, Georgios Kaissis, Daniel Rueckert

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Comments NeurIPS 2022 Workshop: Self-Supervised Learning - Theory and Practice (Reason for updated version: correction of a typo in Eq. (2) and (3))

LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Retrieval

Ziyang luo, Pu Zhao, Can Xu, Xiubo Geng, Tao Shen, Chongyang Tao, Jing Ma, Qingwen lin, Daxin Jiang

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

It's Just a Matter of Time: Detecting Depression with Time-Enriched Multimodal Transformers

Ana-Maria Bucur, Adrian Cosma, Paolo Rosso, Liviu P. Dinu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at ECIR 2023

USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval

Yan Zhang, Zhong Ji, Di Wang, Yanwei Pang, Xuelong Li

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV

Transferring General Multimodal Pretrained Models to Text Recognition

Junyang Lin, Xuancheng Ren, Yichang Zhang, Gao Liu, Peng Wang, An Yang, Chang Zhou

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV

Clustering-Induced Generative Incomplete Image-Text Clustering (CIGIT-C)

Dongjin Guo, Xiaoming Su, Jiatai Wang, Limin Liu, Zhiyong Pei, Zhiwei Xu

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.AI

Comments 13 pages,12 figures

UniColor: A Unified Framework for Multi-Modal Colorization with Transformer

Zhitong Huang, Nanxuan Zhao, Jing Liao

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by SIGGRAPH Asia 2022. Project page: https://luckyhzt.github.io/unicolor

Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training

Jong Hak Moon, Hyungyung Lee, Woncheol Shin, Young-Hak Kim, Edward Choi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Journal of Biomedical and Health Informatics

Journal ref IEEE Journal of Biomedical and Health Informatics 2022

Cross-Modal Graph with Meta Concepts for Video Captioning

Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at IEEE Transactions on Image Processing

Semi-Supervised Cross-Modal Salient Object Detection with U-Structure Networks

Yunqing Bao, Hang Dai, Abdulmotaleb Elsaddik

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV

Support-set based Multi-modal Representation Enhancement for Video Captioning

Xiaoya Chen, Jingkuan Song, Pengpeng Zeng, Lianli Gao, Heng Tao Shen

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV

X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning

Zhihao Yuan, Xu Yan, Yinghong Liao, Yao Guo, Guanbin Li, Zhen Li, Shuguang Cui

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments To appear in CVPR2022

On Vision Features in Multimodal Machine Translation

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, JingBo Zhu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Long paper accepted by ACL2022 main conference

Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer

Woojeong Jin, Dong-Ho Lee, Chenguang Zhu, Jay Pujara, Xiang Ren

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2022, 13 pages, 4 figures