多模态大模型 - arXivDaily 专题

2408.09085 2024-08-20 cs.CV 70%

Segment Anything with Multiple Modalities

Aoran Xiao, Weihao Xuan, Heli Qi, Yun Xing, Naoto Yokoya, Shijian Lu

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Project page: https://xiaoaoran.github.io/projects/MM-SAM

2408.05749 2024-08-13 cs.CV cs.LG 70%

Efficient and Versatile Robust Fine-Tuning of Zero-shot Models

Sungyeon Kim, Boseung Jeong, Donghyun Kim, Suha Kwak

专题命中多模态训练与对齐：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

2402.19348 2024-08-09 cs.LG cs.AI 70%

Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook

Xingchen Zou, Yibo Yan, Xixuan Hao, Yuehong Hu, Haomin Wen, Erdong Liu, Junbo Zhang, Yong Li, Tianrui Li, Yu Zheng, Yuxuan Liang

专题命中多模态训练与对齐：multimodal(abstract)；multi-modal(abstract)；分类 cs.AI

Journal ref Inform.Fusion.113(2025)102606

2404.06214 2024-07-30 cs.CL 70%

[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

Leshem Choshen, Ryan Cotterell, Michael Y. Hu, Tal Linzen, Aaron Mueller, Candace Ross, Alex Warstadt, Ethan Wilcox, Adina Williams, Chengxu Zhuang

专题命中多模态训练与对齐：multimodal(abstract)；image-text(abstract)；分类 cs.CL

2407.18244 2024-07-26 cs.CV 70%

RefMask3D: Language-Guided Transformer for 3D Referring Segmentation

Shuting He, Henghui Ding

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments ACM MM 2024, Code: https://github.com/heshuting555/RefMask3D

2308.03135 2024-07-25 cs.CV 70%

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, Lin Wang

专题命中多模态训练与对齐：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ECCV 2024 Accepted. Camera-ready version with supplementary

2407.02846 2024-07-08 cs.CV 70%

Multi-Task Domain Adaptation for Language Grounding with 3D Objects

Penglei Sun, Yaoxian Song, Xinglin Pan, Peijie Dong, Xiaofei Yang, Qiang Wang, Zhixu Li, Tiefeng Li, Xiaowen Chu

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

2406.15707 2024-06-25 cs.CV 70%

psPRF:Pansharpening Planar Neural Radiance Field for Generalized 3D Reconstruction Satellite Imagery

Tongtong Zhang, Yuanxiang Li

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

2406.12355 2024-06-19 cs.CV 70%

LiCAF: LiDAR-Camera Asymmetric Fusion for Gait Recognition

Yunze Deng, Haijun Xiong, Bin Feng

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by ICIP2024

2405.19009 2024-06-17 cs.CV 70%

Enhancing Vision-Language Model with Unmasked Token Alignment

Jihao Liu, Jinliang Zheng, Boxiao Liu, Yu Liu, Hongsheng Li

专题命中多模态训练与对齐：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by TMLR; Code and models are available at https://github.com/jihaonew/UTA

2406.09366 2024-06-14 cs.LG cs.CV q-bio.NC 70%

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

Rylan Schaeffer, Victor Lecomte, Dhruv Bhandarkar Pai, Andres Carranza, Berivan Isik, Alyssa Unell, Mikail Khona, Thomas Yerxa, Yann LeCun, SueYeon Chung, Andrey Gromov, Ravid Shwartz-Ziv, Sanmi Koyejo

专题命中多模态训练与对齐：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2305.08386 2024-05-30 cs.CV 70%

PLIP: Language-Image Pre-training for Person Representation Learning

Jialong Zuo, Jiahao Hong, Feng Zhang, Changqian Yu, Hanyu Zhou, Changxin Gao, Nong Sang, Jingdong Wang

专题命中多模态训练与对齐：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2405.17824 2024-05-29 cs.CV 70%

mTREE: Multi-Level Text-Guided Representation End-to-End Learning for Whole Slide Image Analysis

Quan Liu, Ruining Deng, Can Cui, Tianyuan Yao, Vishwesh Nath, Yucheng Tang, Yuankai Huo

专题命中多模态训练与对齐：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2405.16108 2024-05-28 cs.CV 70%

OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All

Yuanhuiyi Lyu, Xu Zheng, Dahun Kim, Lin Wang

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2311.15727 2024-05-22 cs.CV 70%

RISAM: Referring Image Segmentation via Mutual-Aware Attention Features

Mengxi Zhang, Yiming Liu, Xiangjun Yin, Huanjing Yue, Jingyu Yang

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2405.12018 2024-05-21 cs.CV 70%

Continuous Sign Language Recognition with Adapted Conformer via Unsupervised Pretraining

Neena Aloysius, Geetha M, Prema Nedungadi

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2405.03351 2024-05-07 cs.CV 70%

Modality Prompts for Arbitrary Modality Salient Object Detection

Nianchang Huang, Yang Yang, Qiang Zhang, Jungong Han, Jin Huang

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 13 pages, 7 Figures, 3 Tables

2404.16635 2024-04-26 cs.CV 70%

TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning

Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, Fei Huang

专题命中多模态训练与对齐：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 13 pages, 11 figures

2312.05278 2024-04-15 cs.CL 70%

Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects

Junyu Lu, Dixiang Zhang, Songxin Zhang, Zejian Xie, Zhuoyang Song, Cong Lin, Jiaxing Zhang, Bingyi Jing, Pingjian Zhang

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CL

2403.17702 2024-04-03 cs.CV 70%

The Solution for the CVPR 2023 1st foundation model challenge-Track2

Haonan Xu, Yurui Huang, Sishun Pan, Zhihao Guan, Yi Xu, Yang Yang

专题命中多模态训练与对齐：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2403.17334 2024-03-27 cs.CV 70%

OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation

Ganlong Zhao, Guanbin Li, Weikai Chen, Yizhou Yu

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

2403.07560 2024-03-15 cs.CV 70%

Unleashing Network Potentials for Semantic Scene Completion

Fengyun Wang, Qianru Sun, Dong Zhang, Jinhui Tang

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments accepted by CVPR2024

2302.00389 2024-03-04 cs.AI 70%

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

Muhammad Arslan Manzoor, Sarah Albarri, Ziting Xian, Zaiqiao Meng, Preslav Nakov, Shangsong Liang

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.AI

2402.02045 2024-02-06 cs.CV 70%

MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning

Zhe Li, Laurence T. Yang, Bocheng Ren, Xin Nie, Zhangyang Gao, Cheng Tan, Stan Z. Li

专题命中多模态训练与对齐：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2401.03145 2024-01-18 cs.CV 70%

Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection

Yuanpeng Tu, Boshen Zhang, Liang Liu, Yuxi Li, Xuhai Chen, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Cai Rong Zhao

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

2312.02433 2023-12-06 cs.CV 70%

Lenna: Language Enhanced Reasoning Detection Assistant

Fei Wei, Xinyu Zhang, Ailing Zhang, Bo Zhang, Xiangxiang Chu

专题命中多模态训练与对齐：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

2311.09540 2023-12-05 cs.CV 70%

FedFusion: Manifold Driven Federated Learning for Multi-satellite and Multi-modality Fusion

DaiXun Li, Weiying Xie, Yunsong Li, Leyuan Fang

专题命中多模态训练与对齐：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2311.17952 2023-12-01 cs.CV 70%

Synchronizing Vision and Language: Bidirectional Token-Masking AutoEncoder for Referring Image Segmentation

Minhyeok Lee, Dogyoon Lee, Jungho Lee, Suhwan Cho, Heeseung Choi, Ig-Jae Kim, Sangyoun Lee

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

2207.11900 2023-11-23 cs.MM 70%

GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection

Jiang Li, Xiaoping Wang, Guoqing Lv, Zhigang Zeng

专题命中多模态训练与对齐：multimodal(abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted by IEEE Transactions on Affective Computing

2311.09737 2023-11-17 cs.CV 70%

Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation

Bingnan Li, Zhitong Gao, Xuming He

专题命中多模态训练与对齐：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments 9 pages, Machine Learning for Health (ML4H) 2023

1. 多模态训练与对齐 6897 篇

Segment Anything with Multiple Modalities

Efficient and Versatile Robust Fine-Tuning of Zero-shot Models

Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook

[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

RefMask3D: Language-Guided Transformer for 3D Referring Segmentation

EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

Multi-Task Domain Adaptation for Language Grounding with 3D Objects

psPRF:Pansharpening Planar Neural Radiance Field for Generalized 3D Reconstruction Satellite Imagery

LiCAF: LiDAR-Camera Asymmetric Fusion for Gait Recognition

Enhancing Vision-Language Model with Unmasked Token Alignment

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations

PLIP: Language-Image Pre-training for Person Representation Learning

mTREE: Multi-Level Text-Guided Representation End-to-End Learning for Whole Slide Image Analysis

OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All

RISAM: Referring Image Segmentation via Mutual-Aware Attention Features

Continuous Sign Language Recognition with Adapted Conformer via Unsupervised Pretraining

Modality Prompts for Arbitrary Modality Salient Object Detection

TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning

Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objects

The Solution for the CVPR 2023 1st foundation model challenge-Track2

OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representation

Unleashing Network Potentials for Semantic Scene Completion

Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications

MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning

Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection

Lenna: Language Enhanced Reasoning Detection Assistant

FedFusion: Manifold Driven Federated Learning for Multi-satellite and Multi-modality Fusion

Synchronizing Vision and Language: Bidirectional Token-Masking AutoEncoder for Referring Image Segmentation

GA2MIF: Graph and Attention Based Two-Stage Multi-Source Information Fusion for Conversational Emotion Detection

Gradient-Map-Guided Adaptive Domain Generalization for Cross Modality MRI Segmentation