RAG / 检索增强生成

A Multimodal Multi-Agent Framework for Radiology Report Generation

Ziruo Yi, Ting Xiao, Mark V. Albert

机构 * University of North Texas（北卡罗来纳州立大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Facilitating Video Story Interaction with Multi-Agent Collaborative System

Yiwen Zhang, Jianing Hao, Zhan Wang, Hongling Sheng, Wei Zeng

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Prepared and submitted in 2024

Retrieval-augmented in-context learning for multimodal large language models in disease classification

Zaifu Zhan, Shuang Zhou, Xiaoshan Zhou, Yongkang Xiao, Jun Wang, Jiawen Deng, He Zhu, Yu Hou, Rui Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 17 Pages, 1 figure, 7 tables

MRAMG-Bench: A Comprehensive Benchmark for Advancing Multimodal Retrieval-Augmented Multimodal Generation

Qinhan Yu, Zhiyou Xiao, Binghui Li, Zhengren Wang, Chong Chen, Wentao Zhang

机构 * Peking University（北京大学） ； Huawei Cloud BU（华为云业务单元） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments Published as a conference paper at SIGIR 2025; 11 pages

On the Feasibility of Using MultiModal LLMs to Execute AR Social Engineering Attacks

Ting Bi, Chenghang Ye, Zheyu Yang, Ziyi Zhou, Cui Tang, Jun Zhang, Zui Tao, Kailong Wang, Liting Zhou, Yang Yang, Tianlong Yu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Reasoning LLMs for User-Aware Multimodal Conversational Agents

Hamed Rahimi, Jeanne Cattoni, Meriem Beghili, Mouad Abrini, Mahdi Khoramshahi, Maribel Pino, Mohamed Chetouani

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Agentic Multimodal AI for Hyperpersonalized B2B and B2C Advertising in Competitive Markets: An AI-Driven Competitive Advertising Framework

Sakhinana Sagar Srinivas, Akash Das, Shivam Gupta, Venkataramana Runkana

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

CUE-M: Contextual Understanding and Enhanced Search with Multimodal Large Language Model

Dongyoung Go, Taesun Whang, Chanhee Lee, Hwa-Yeon Kim, Sunghoon Park, Seunghwan Ji, Jinho Kim, Dongchan Kim, Young-Bum Kim

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments Preprint. Under review

RAD: Retrieval-Augmented Decision-Making of Meta-Actions with Vision-Language Models in Autonomous Driving

Yujin Wang, Quanfeng Liu, Zhengxin Jiang, Tianyi Wang, Junfeng Jiao, Hongqing Chu, Bingzhao Gao, Hong Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

On-Board Vision-Language Models for Personalized Autonomous Vehicle Motion Control: System Design and Real-World Validation

Can Cui, Zichong Yang, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Cong Zhang, Yunsheng Ma, Xu Cao, Wenqian Ye, Yiheng Feng, Jitesh Panchal, Lingxi Li, Yaobin Chen, Ziran Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2410.15281

Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation

Liwen Sun, James Zhao, Megan Han, Chenyan Xiong

专题命中 多模态RAG ：RAG(abstract)；retriever(abstract)；分类 cs.CL

Comments NAACL 2025 main

Characterizing Network Structure of Anti-Trans Actors on TikTok

Maxyn Leitner, Rebecca Dorn, Fred Morstatter, Kristina Lerman

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 11 pages, 4 figures. 2 tables

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Jingwei Yi, Junhao Yin, Ju Xu, Peng Bao, Yongliang Wang, Wei Fan, Hao Wang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents

Jun Chen, Dannong Xu, Junjie Fei, Chun-Mei Feng, Mohamed Elhoseiny

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments the correct arxiv version

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

Yi Ren, HanZhi Zhang, Weibin Li, Jun Fu, Diandong Liu, Tianyi Zhang, Jie He, Licheng Jiao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

ML-Promise: A Multilingual Dataset for Corporate Promise Verification

Yohei Seki, Hakusen Shu, Anaïs Lhuissier, Hanwool Lee, Juyeon Kang, Min-Yuh Day, Chung-Chi Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 6 pages

G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

Pengyue Jia, Yiding Liu, Xiaopeng Li, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments Accepted to NeurIPS2024

OpenResearcher: Unleashing AI for Accelerated Scientific Research

Yuxiang Zheng, Shichao Sun, Lin Qiu, Dongyu Ru, Cheng Jiayang, Xuefeng Li, Jifan Lin, Binjie Wang, Yun Luo, Renjie Pan, Yang Xu, Qingkai Min, Zizhao Zhang, Yiwen Wang, Wenjie Li, Pengfei Liu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments Accepted to Demo track of EMNLP 2024

ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text

Han Yu, Peikun Guo, Akane Sano

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

IPL: Leveraging Multimodal Large Language Models for Intelligent Product Listing

Kang Chen, Qingheng Zhang, Chengbao Lian, Yixin Ji, Xuwei Liu, Shuguang Han, Guoqiang Wu, Fei Huang, Jufeng Chen

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud

Marcin Chrapek, Anjo Vahldiek-Oberwagner, Marcin Spoczynski, Scott Constable, Mona Vij, Torsten Hoefler

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

OrthoDoc: Multimodal Large Language Model for Assisting Diagnosis in Computed Tomography

Youzhu Jin, Yichen Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 8 pages, 1 figure

A Learnable Agent Collaboration Network Framework for Personalized Multimodal AI Search Engine

Yunxiao Shi, Min Xu, Haimin Zhang, Xing Zi, Qiang Wu

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.IR

Comments ACMMM 2024 MMGR WORKSHOP

SearchLVLMs: A Plug-and-Play Framework for Augmenting Large Vision-Language Models by Searching Up-to-Date Internet Knowledge

Chuanhao Li, Zhen Li, Chenchen Jing, Shuo Liu, Wenqi Shao, Yuwei Wu, Ping Luo, Yu Qiao, Kaipeng Zhang

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Comments 13 pages, 6 figures, a plug-and-play framework to augment large vision-language models with up-to-date internet knowledge

LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

Ran Chen, Xueqi Yao, Xuhui Jiang

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.AI

Towards Unlocking Insights from Logbooks Using AI

Antonin Sulc, Alex Bien, Annika Eichler, Daniel Ratner, Florian Rehm, Frank Mayet, Gregor Hartmann, Hayden Hoschouer, Henrik Tuennermann, Jan Kaiser, Jason St. John, Jennefer Maldonado, Kyle Hazelwood, Raimund Kammering, Thorsten Hellert, Tim Wilksen, Verena Kain, Wan-Lin Hu

专题命中 多模态RAG ：retrieval augmented generation(abstract)；RAG(abstract)；分类 cs.CL

Comments 5 pages, 1 figure, 15th International Particle Accelerator Conference

OmniVIC: A Self-Improving Variable Impedance Controller with Vision-Language In-Context Learning for Safe Robotic Manipulation

Heng Zhang, Wei-Hsing Huang, Gokhan Solak, Arash Ajoudani

机构 * Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genoa, Italy（人机交互实验室，意大利理工学院，热那亚） ； Georgia Institute of Technology, Atlanta, USA（佐治亚理工学院，美国亚特兰大） 纠错

专题命中 多模态RAG ：RAG(abstract,comments)；retrieval-augmented generation(abstract)

Comments Code, video and RAG dataset are available at \url{https://sites.google.com/view/omni-vic}

UEmbed: Unified Sparse and Dense Multimodal Embeddings

UEmbed：统一稀疏与稠密多模态嵌入

Tingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Zhijie Nie, Yilun Zhao, Shu Wu

机构 * CASIA（中国科学院自动化研究所） ； Alibaba Group（阿里巴巴集团） ； University of Chinese Academy of Sciences（中国科学院大学） ； Yale University（耶鲁大学） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；分类 cs.IR、cs.CL、cs.AI

AI总结 UEmbed是一种仅解码器的多模态嵌入模型，可单次前向传播生成稀疏与稠密表示，在MMEB-v2和BEIR上表现优异，统一了两类嵌入并支持多模态智能体应用。

PRIMA: Pre-Training with Risk-Integrated Image--Metadata Alignment for Medical Diagnosis with LLM-Based Feature Aggregation

PRIMA：通过大语言模型进行风险集成图像-元数据对齐的医学诊断预训练

Yiqing Wang, Chunming He, Ziyun Yang, Maria Woodward, Ming-Chen Lu, Mercy Pawar, Leslie Niziol, Sina Farsiu

机构 * Department of Biomedical Engineering, Duke University（杜克大学生物医学工程系） ； Department of Ophthalmology and Visual Sciences, University of Michigan（密歇根大学眼科与视觉科学系） 纠错

专题命中 多模态RAG ：retrieval-augmented generation(abstract)；RAG(abstract)

AI总结 提出PRIMA框架，通过检索增强生成策展风险-疾病关联专家语料库优化文本编码器，用双编码器预训练策略及四个互补损失函数弥合模态差距，融合特征用于疾病分类，性能优于其他方法。

Superpixel-Based QUBO for Scalable Quantum-Enhanced Medical Image Segmentation

基于超像素的二次无约束二进制优化用于可扩展量子增强医学图像分割

Mohammad Chalhoub, Mahdi Chehimi, Laia Domingo, Omar Alhussein, Ahmed Farouk, Saif Al-Kuwari

机构 * American University of Beirut（贝鲁特美国大学） ； Centre de Visió per Computador (CVC)（计算机视觉中心 (CVC)） ； Ingenii Inc.（英格尼公司） ； Hamad Bin Khalifa University（哈马德·本·哈利法大学） 纠错

专题命中 多模态RAG ：RAG(abstract,abstract_cn)

AI总结 研究针对QUBO用于医学图像分割时的可扩展性挑战，提出基于超像素的QUBO框架，用简单线性迭代聚类分组像素，在区域邻接图上公式化分割。实验表明该方法提升了分割质量、计算速度并减小问题规模，还符合量子退火器连接限制。

Comments 7 pages, 2 figures