多模态大模型

LESS: Label-Efficient and Single-Stage Referring 3D Segmentation

Xuexun Liu, Xiaoxu Xu, Jinlong Li, Qiudan Zhang, Xu Wang, Nicu Sebe, Lin Ma

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Knowledge-aware Text-Image Retrieval for Remote Sensing Images

Li Mi, Xianjie Dai, Javiera Castillo-Navarro, Devis Tuia

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by IEEE TGRS

ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

Han Yu, Peikun Guo, Akane Sano

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

JointMotion: Joint Self-Supervision for Joint Motion Prediction

Royden Wagner, Omer Sahin Tas, Marvin Klemp, Carlos Fernandez

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments CoRL'24 camera-ready

Towards a Reliable Offline Personal AI Assistant for Long Duration Spaceflight

Oliver Bensch, Leonie Bensch, Tommy Nilsson, Florian Saling, Wafa M. Sadri, Carsten Hartmann, Tobias Hecking, J. Nathan Kutz

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments 75th International Astronautical Congress (IAC), Milan, Italy, 14-18 October 2024

RoRA-VLM: Robust Retrieval-Augmented Vision Language Models

Jingyuan Qi, Zhiyang Xu, Rulin Shao, Yang Chen, Jin Di, Yu Cheng, Qifan Wang, Lifu Huang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks

Isaac R. Galatzer-Levy, David Munday, Jed McGiffin, Xin Liu, Danny Karmon, Ilia Labzovsky, Rivka Moroshko, Amir Zait, Daniel McDuff

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

Marcin Chrapek, Anjo Vahldiek-Oberwagner, Marcin Spoczynski, Scott Constable, Mona Vij, Torsten Hoefler

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

GS-Hider: Hiding Messages into 3D Gaussian Splatting

Xuanyu Zhang, Jiarui Meng, Runyi Li, Zhipei Xu, Yongbing Zhang, Jian Zhang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2024, 3DGS steganography

A Foundation Model for the Solar Dynamics Observatory

James Walsh, Daniel G. Gass, Raul Ramos Pollan, Paul J. Wright, Richard Galvez, Noah Kasmanoff, Jason Naradowsky, Anne Spalding, James Parr, Atılım Güneş Baydin

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

EUFCC-CIR: a Composed Image Retrieval Dataset for GLAM Collections

Francesc Net, Lluis Gomez

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments ECCV Workshop (AI4DH2024)

Cacophony: An Improved Contrastive Audio-Text Model

Ge Zhu, Jordan Darefsky, Zhiyao Duan

专题命中 跨模态检索 ：image-text(abstract)；分类 eess.AS

Comments Accepted at IEEE/ACM Transactions on Audio, Speech, and Language Processing

CLSP: High-Fidelity Contrastive Language-State Pre-training for Agent State Representation

Fuxian Huang, Qi Zhang, Shaopeng Zhai, Jie Wang, Tianyi Zhang, Haoran Zhang, Ming Zhou, Yu Liu, Yu Qiao

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Towards Graph Prompt Learning: A Survey and Beyond

Qingqing Long, Yuchen Yan, Peiyan Zhang, Chen Fang, Wentao Cui, Zhiyuan Ning, Meng Xiao, Ning Cao, Xiao Luo, Lingjun Xu, Shiyue Jiang, Zheng Fang, Chong Chen, Xian-Sheng Hua, Yuanchun Zhou

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments I have decided to temporarily withdraw this draft as I am in the process of making further revisions to improve its content

Brain-Cognition Fingerprinting via Graph-GCCA with Contrastive Learning

Yixin Wang, Wei Peng, Yu Zhang, Ehsan Adeli, Qingyu Zhao, Kilian M. Pohl

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Bundle Fragments into a Whole: Mining More Complete Clusters via Submodular Selection of Interesting webpages for Web Topic Detection

Junbiao Pang, Anjing Hu, Qingming Huang

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments 10

A Closer Look at the Explainability of Contrastive Language-Image Pre-training

Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, Xiaomeng Li

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments 30 pages, 11 figures, under review

StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction

Shengkun Wang, Taoran Ji, Linhan Wang, Yanshen Sun, Shang-Ching Liu, Amit Kumar, Chang-Tien Lu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro Astolfi, Kyunghyun Cho, Yann LeCun

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding

Xiaoyu Liang, Jiayuan Yu, Lianrui Mu, Jiedong Zhuang, Jiaqi Hu, Yuchen Yang, Jiangnan Ye, Lu Lu, Jian Chen, Haoji Hu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments PRCV

GS-PT: Exploiting 3D Gaussian Splatting for Comprehensive Point Cloud Understanding via Self-supervised Learning

Keyi Liu, Yeqi Luo, Weidong Yang, Jingyi Xu, Zhijun Li, Wen-Ming Chen, Ben Fei

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV

MOSMOS: Multi-organ segmentation facilitated by medical report supervision

Weiwei Tian, Xinyu Huang, Junlin Hou, Caiyue Ren, Longquan Jiang, Rui-Wei Zhao, Gang Jin, Yuejie Zhang, Daoying Geng

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

Comments 14 pages, 7 figures

Optimizing CLIP Models for Image Retrieval with Maintained Joint-Embedding Alignment

Konstantin Schall, Kai Uwe Barthel, Nico Hezel, Klaus Jung

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV

The Compressor-Retriever Architecture for Language Model OS

Yuan Yang, Siheng Xiong, Ehsan Shareghi, Faramarz Fekri

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL

Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery

Long Bai, Guankun Wang, Mobarakol Islam, Lalithkumar Seenivasan, An Wang, Hongliang Ren

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by Information Fusion. Code and data availability: https://github.com/longbai1006/Surgical-VQLAPlus

Collaborative Group: Composed Image Retrieval via Consensus Learning from Noisy Annotations

Xu Zhang, Zhedong Zheng, Linchao Zhu, Yi Yang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV

Comments Accepted by Knowledge-Based Systems (KBS)

Journal ref Knowl. Based. Syst. 300 (2024) 112135

Explicit Modelling of Theory of Mind for Belief Prediction in Nonverbal Social Interactions

Matteo Bortoletto, Constantin Ruhdorfer, Lei Shi, Andreas Bulling

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.AI

Comments ECAI 2024

QD-VMR: Query Debiasing with Contextual Understanding Enhancement for Video Moment Retrieval

Chenghua Gao, Min Li, Jianshuo Liu, Junxing Ren, Lin Chen, Haoyu Liu, Bo Meng, Jitao Fu, Wenwen Su

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.AI

Comments 9 pages, 4 figures, 4 tables

Tell Codec What Worth Compressing: Semantically Disentangled Image Coding for Machine with LMMs

Jinming Liu, Yuntao Wei, Junyan Lin, Shengyang Zhao, Heming Sun, Zhibo Chen, Wenjun Zeng, Xin Jin

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

SSL-SoilNet: A Hybrid Transformer-based Framework with Self-Supervised Learning for Large-scale Soil Organic Carbon Prediction

Nafiseh Kakhani, Moien Rangzan, Ali Jamali, Sara Attarchi, Seyed Kazem Alavipanah, Michael Mommert, Nikolaos Tziolas, Thomas Scholten

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV

Comments Accepted for publication in IEEE Transactions on Geoscience and Remote Sensing (TGRS)