视觉大模型 / VLM

PP-DocBee: Improving Multimodal Document Understanding Through a Bag of Tricks

Feng Ni, Kui Huang, Yao Lu, Wenyu Lv, Guanzhong Wang, Zeyu Chen, Yi Liu

机构 * PaddlePaddle Team, Baidu Inc.（百度公司） 纠错

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Understanding Generative AI Capabilities in Everyday Image Editing Tasks

Mohammad Reza Taesiri, Brandon Collins, Logan Bolton, Viet Dac Lai, Franck Dernoncourt, Trung Bui, Anh Totti Nguyen

专题命中 文档图表理解 ：VLM(abstract)；分类 cs.CV、cs.AI

Comments Code and qualitative examples are available at: https://psrdataset.github.io

Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval

Alexander Most, Joseph Winjum, Ayan Biswas, Shawn Jones, Nishath Rajiv Ranasinghe, Dan O'Malley, Manish Bhattarai

机构 * Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室） ； Gianforte School of Computing（吉安福特计算学院） ； Montana State University Bozeman（蒙大拿州立大学博兹曼分校） 纠错

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding

De-An Huang, Subhashree Radhakrishnan, Zhiding Yu, Jan Kautz

机构 * NVIDIA（英伟达） 纠错

专题命中 文档图表理解 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Towards a Multimodal Document-grounded Conversational AI System for Education

Karan Taneja, Anjali Singh, Ashok K. Goel

机构 * Georgia Institute of Technology（佐治亚理工学院） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 4 figures, AIED 2025

MathWriting: A Dataset For Handwritten Mathematical Expression Recognition

Philippe Gervais, Anastasiia Fadeeva, Andrii Maksai

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.LG

OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations

Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, Conghui He

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR2025

ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understanding

Zhengzhuo Xu, Bowen Qu, Yiyan Qi, Sinan Du, Chengjin Xu, Chun Yuan, Jian Guo

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs

Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Dongping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, Xiangliang Zhang

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments WWW'25

MFTF: Mask-free Training-free Object Level Layout Control Diffusion Model

Shan Yang

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 7 figures

M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understanding

Jaemin Cho, Debanjan Mahata, Ozan Irsoy, Yujie He, Mohit Bansal

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments Project webpage: https://m3docrag.github.io

TableGPT2: A Large Multimodal Model with Tabular Data Integration

Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Gang Chen, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, Haoze Li, Haoxuan Lan, Jiaming Tian, Jing Yuan, Junbo Zhao, Junlin Zhou, Kaizhe Shou, Liangyu Zha, Lin Long, Liyao Li, Pengzuo Wu, Qi Zhang, Qingyi Huang, Saisai Yang, Tao Zhang, Wentao Ye, Wufang Zhu, Xiaomeng Hu, Xijun Gu, Xinjie Sun, Xiang Li, Yuhang Yang, Zhiqing Xiao

专题命中 文档图表理解 ：visual language model(abstract)；分类 cs.AI、cs.LG

OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

Qingyun Li, Zhe Chen, Weiyun Wang, Wenhai Wang, Shenglong Ye, Zhenjiang Jin, Guanzhou Chen, Yinan He, Zhangwei Gao, Erfei Cui, Jiashuo Yu, Hao Tian, Jiasheng Zhou, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhang, Pinlong Cai, Licheng Wen, Xiangchao Yan, Zhenxiang Li, Pei Chu, Yi Wang, Min Dou, Changyao Tian, Xizhou Zhu, Lewei Lu, Yushi Chen, Junjun He, Zhongying Tu, Tong Lu, Yali Wang, Limin Wang, Dahua Lin, Yu Qiao, Botian Shi, Conghui He, Jifeng Dai

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

TRINS: Towards Multimodal Language Models that Can Read

Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, Xiang Bai

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.CV、cs.AI

DUBLIN -- Document Understanding By Language-Image Network

Kriti Aggarwal, Aditi Khandelwal, Kumar Tanmay, Owais Mohammed Khan, Qiang Liu, Monojit Choudhury, Hardik Hansrajbhai Chauhan, Subhojit Som, Vishrav Chaudhary, Saurabh Tiwary

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Enhancing Document Information Analysis with Multi-Task Pre-training: A Robust Approach for Information Extraction in Visually-Rich Documents

Tofik Ali, Partha Pratim Roy

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.LG

Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study

Mariya Hendriksen, Svitlana Vakulenko, Ernst Kuiper, Maarten de Rijke

专题命中 文档图表理解 ：VLM(abstract)；分类 cs.CV、cs.LG

Comments 18 pages, accepted as a reproducibility paper at ECIR 2023

Towards Complex Document Understanding By Discrete Reasoning

Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, Tat-Seng Chua

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Comments ACM MM 2022

ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules

Zhi-Qi Cheng, Qi Dai, Siyao Li, Jingdong Sun, Teruko Mitamura, Alexander G. Hauptmann

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Hierarchical multimodal transformers for Multi-Page DocVQA

Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny

专题命中 文档图表理解 ：visual question answering(abstract)；分类 cs.CV、cs.AI

Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model

Nico Daheim, David Thulke, Christian Dugast, Hermann Ney

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Accepted to Findings of EMNLP 2022

Cascaded Span Extraction and Response Generation for Document-Grounded Dialog

Nico Daheim, David Thulke, Christian Dugast, Hermann Ney

专题命中 文档图表理解 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Accepted by 1st DialDoc Workshop at ACL-IJCNLP 2021

Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs

超越视觉证据：揭示并缓解文档多模态大语言模型中的关系隐私泄露

Beining Xu, Hairui Wang, Jiaxin Wang, Changsheng Chen, Anirban Chakraborty

机构 * Faculty of Engineering, Shenzhen MSU-BIT University（深圳北理莫斯科大学工程学院） ； Faculty of CMC, Shenzhen MSU-BIT University（深圳北理莫斯科大学计算机、数学与力学学院） ； Department of CDS, Indian Institute of Science（印度科学学院计算机与数据科学系） 纠错

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV

AI总结 本文针对文档理解MLLMs的KIE任务，揭示其在视觉证据不足时会通过记忆的字段关系泄露隐私，提出DRUF框架与DocPrivacyBench基准，实验显示DRUF可提升泄露抑制效果并维持KIE性能。

Comments ACM mm 2026

DATAREEL: Automated Data-Driven Video Story Generation with Animations

DATAREEL：基于动画的自动化数据驱动视频故事生成

Ridwan Mahbub, Syem Aziz, Mizanur Rahman, Mahir Ahmed, Shadikur Rahman, Shafiq Joty, Enamul Hoque

机构 * York University（约克大学） ； Bangladesh University of Business and Technology（孟加拉国商业与技术大学） ； RBC, Canada（加拿大RBC） ； Nanyang Technological University（南洋理工大学） ； Salesforce AI Research（Salesforce人工智能研究） 纠错

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.AI

AI总结 本文提出DataReel基准，通过328个真实故事评估模型生成动画数据视频故事的能力，采用多智能体框架提升自动化生成效果。

Comments Under Review

ArtChart: Faithful Artistic Chart Generation with Integrated Text Rendering

ArtChart：一个用于忠实生成艺术图表并集成文本渲染的基准测试

Meijia Huang, Yingjie Yin, Shihao Wang, Chenguang Ma

机构 * Ant Group（蚂蚁集团） 纠错

专题命中 文档图表理解 ：VLM(abstract)；分类 cs.CV

AI总结 研究旨在解决艺术图表生成难题，提出ArtChart框架，含特定即插即用模块及强化学习等策略，构建基准测试和评估套件，实验证明该框架能生成兼具美观与数学准确性的图表，优于开源基线。

A Survey of OCR Evaluation Methods and Metrics and the Invisibility of Historical Documents

OCR评估方法和指标及历史文件的不可见性综述

Fitsum Sileshi Beyene, Christopher L. Dancy

机构 * The Pennsylvania State University（宾夕法尼亚州立大学） 纠错

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV

AI总结 本文探讨OCR系统在历史文档中的评估问题，指出现代文档评估方法忽视了历史档案的布局、字体和材料退化，导致结构性不可见和代表性伤害。

Comments This manuscript is the author's submitted version to the ACM Conference on Fairness, Accountability, and Transparency (FAccT 2026). Please cite the final published version via ACM Digital Library when available

Journal ref FAccT '26: The 2026 ACM Conference on Fairness, Accountability, and Transparency

Roomer: Reflective Object-Grounded Model Editing and Repair for 3D Indoor Layout Synthesis

Roomer：用于3D室内布局合成的基于对象的反射式模型编辑与修复

Lingwei Dang, Ziyan Qiu, Jiajia Cheng, Shishuo Shang, Zhenhao Zhang, Yufei Zhu, Qingxin Xiao, Pan Liu, Shenghui Huang, Yun Hao, Juntong Li, Qingyao Wu

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.CV

AI总结 针对现有室内布局生成器的局部违规问题，提出Roomer框架，通过视觉语言模型规划器与确定性求解器实现基于对象的局部修复，提升布局物理有效性与可用性且可跨生成器迁移。

Decoupling semantics from vision: A framework for faithful visual-text compression evaluation

将语义与视觉解耦：一种用于可靠视觉-文本压缩评估的框架

Yonghan Gao, Zehong Chen, Lijian Xu, Jingzhi Chen, Jingwei Guan, Xingyu Zeng

机构 * Shenzhen University of Advanced Technology（深圳先进技术大学） ； Shenzhen Technology University（深圳技术大学） 纠错

专题命中 文档图表理解 ：multimodal large language model(abstract)；分类 cs.CV

AI总结 本研究针对现有视觉-文本压缩评估依赖下游任务性能的缺陷，提出解耦语义与视觉的评估框架，引入ZeroSense基准消除文本依赖，实验证实VTC质量与下游任务准确率存在显著差异。

A report-grounded vision-language foundation model for colonoscopy from 280000 routine reports

基于28万份常规报告的肠镜报告驱动的视觉-语言基础模型

Jia Yu, Yan Zhu, Yili He, Zilong Wang, Xinyang Jiang, Peiyao Fu, Ruijie Yang, Tianyi Chen, Siyuan Li, Zhihua Wang, Fei Wu, Quanlin Li, Xian Yang, Pinghong Zhou, Shuo Wang

机构 * Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心） ； Shanghai Collaborative Innovation Center of Endoscopy（上海内镜诊疗协同创新中心） ； Zhejiang University（浙江大学） ； Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院） ； Alliance Manchester Business School, The University of Manchester（曼彻斯特大学联盟曼彻斯特商学院） ； Data Science Institute, Imperial College London（伦敦帝国理工学院数据科学研究所） ； Microsoft Research Asia（微软亚洲研究院） 纠错

专题命中 文档图表理解 ：vision-language model(abstract)；分类 cs.AI

AI总结 该研究开发了肠镜视觉-语言基础模型EndoCLIP，利用28万份常规肠镜记录恢复的图像-文本对训练，在多项任务中优于通用模型，良恶性分类性能接近专家，可实现临床目标的语言指定。