多模态大模型

Beyond Embeddings: The Promise of Visual Table in Visual Reasoning

Yiwu Zhong, Zi-Yuan Hu, Michael R. Lyu, Liwei Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Project page: https://github.com/LaVi-Lab/Visual-Table

CMC-Bench: Towards a New Paradigm of Visual Signal Compression

Chunyi Li, Xiele Wu, Haoning Wu, Donghui Feng, Zicheng Zhang, Guo Lu, Xiongkuo Min, Xiaohong Liu, Guangtao Zhai, Weisi Lin

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding

Xintong Wang, Jingheng Pan, Liang Ding, Chris Biemann

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to Findings of ACL 2024

Towards Retrieval-Augmented Architectures for Image Captioning

Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, Rita Cucchiara

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments ACM Transactions on Multimedia Computing, Communications and Applications (2024)

HARIS: Human-Like Attention for Reference Image Segmentation

Mengxi Zhang, Heqing Lian, Yiming Liu, Jie Chen

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

VILA: On Pre-training for Visual Language Models

Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, Song Han

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024

Instance-free Text to Point Cloud Localization with Relative Position Awareness

Lichao Wang, Zhihao Yuan, Jinke Ren, Shuguang Cui, Zhen Li

专题命中 图文多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 12 pages, 10 figures, conference

Medical Vision-Language Pre-Training for Brain Abnormalities

Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CL

Bridging Vision and Language Spaces with Assignment Prediction

Jungin Park, Jiyoung Lee, Kwanghoon Sohn

专题命中 图文多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments ICLR 2024 Camera-ready

ViTamin: Designing Scalable Vision Models in the Vision-Language Era

Jieneng Chen, Qihang Yu, Xiaohui Shen, Alan Yuille, Liang-Chieh Chen

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024; https://github.com/Beckschen/ViTamin

DreamLIP: Language-Image Pre-training with Long Captions

Kecheng Zheng, Yifei Zhang, Wei Wu, Fan Lu, Shuailei Ma, Xin Jin, Wei Chen, Yujun Shen

专题命中 图文多模态 ：MLLM(abstract)；image-text(abstract)；分类 cs.CV

Towards More Unified In-context Visual Understanding

Dianmo Sheng, Dongdong Chen, Zhentao Tan, Qiankun Liu, Qi Chu, Jianmin Bao, Tao Gong, Bin Liu, Shengwei Xu, Nenghai Yu

专题命中 图文多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

RESTORE: Towards Feature Shift for Vision-Language Prompt Learning

Yuncheng Yang, Chuyan Zhang, Zuopeng Yang, Yuting Gao, Yulei Qin, Ke Li, Xing Sun, Jie Yang, Yun Gu

专题命中 图文多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 18 pages, 5 figures

VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing

Zhiyuan Chang, Mingyang Li, Junjie Wang, Cheng Li, Qing Wang

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 12pages, 3 figures

Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, Sergey Tulyakov

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024. Project Page: https://snap-research.github.io/Panda-70M

BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models

Xueliang Zhao, Xinting Huang, Tingchen Fu, Qintong Li, Shansan Gong, Lemao Liu, Wei Bi, Lingpeng Kong

专题命中 图文多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CL

Comments Preprint

CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation

Wenxuan Wang, Jing Liu, Xingjian He, Yisi Zhang, Chen Chen, Jiachen Shen, Yan Zhang, Jiangyun Li

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

GS-CLIP: Gaussian Splatting for Contrastive Language-Image-3D Pretraining from Real-World Data

Haoyuan Li, Yanpeng Zhou, Yihan Zeng, Hang Xu, Xiaodan Liang

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments The content of the technical report needs to be updated and retracted to avoid other impacts

Object-Centric Instruction Augmentation for Robotic Manipulation

Junjie Wen, Yichen Zhu, Minjie Zhu, Jinming Li, Zhiyuan Xu, Zhengping Che, Chaomin Shen, Yaxin Peng, Dong Liu, Feifei Feng, Jian Tang

专题命中 图文多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments accepted to ICRA2024

Exploring Diverse In-Context Configurations for Image Captioning

Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen, Xin Geng

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by NeurIPS2023

Bootstrapping OTS-Funcimg Pre-training Model (Botfip) -- A Comprehensive Symbolic Regression Framework

Tianhao Chen, Pengbo Xu, Haibiao Zheng

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.AI

DAP: Domain-aware Prompt Learning for Vision-and-Language Navigation

Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

专题命中 图文多模态 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 4 pages. arXiv admin note: substantial text overlap with arXiv:2309.03661

Image Captioners Are Scalable Vision Learners Too

Michael Tschannen, Manoj Kumar, Andreas Steiner, Xiaohua Zhai, Neil Houlsby, Lucas Beyer

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2023. v2 adds SugarCrepe results and more ablations, v3 has minor fixes. v4 adds a code link ( https://github.com/google-research/big_vision ). v5 has minor fixes

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Haodong Duan, Songyang Zhang, Shuangrui Ding, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Code and models are available at https://github.com/InternLM/InternLM-XComposer

OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization

Dongchen Han, Xiaojun Jia, Yang Bai, Jindong Gu, Yang Liu, Xiaochun Cao

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes

Xuying Zhang, Bo-Wen Yin, Yuming Chen, Zheng Lin, Yunheng Li, Qibin Hou, Ming-Ming Cheng

专题命中 图文多模态 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets

Marcella Cornia, Lorenzo Baraldi, Giuseppe Fiameni, Rita Cucchiara

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to IJCV

Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, Liang-Chieh Chen

专题命中 图文多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments NeurIPS 2023 camera ready. code and model available at https://github.com/bytedance/fc-clip

Data Filtering Networks

Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, Vaishaal Shankar

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.AI

Improved baselines for vision-language pre-training

Enrico Fini, Pietro Astolfi, Adriana Romero-Soriano, Jakob Verbeek, Michal Drozdzal

专题命中 图文多模态 ：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments TMLR, featured certification; changelog at https://openreview.net/forum?id=a7nvXxNmdV

Journal ref Transactions on Machine Learning Research, 10/2023, issn 2835-8856