arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 3717
2607.24744 2026-07-28 cs.RO cs.CV 新提交

Data Pyramid for Embodied Manipulation

用于具身操纵的数据金字塔

Yifan Ye, Yankai Fu, Yaoxu Lv, Bohan Hou, Jun Cen, Lingdong Kong, Duo Zheng, Tianxing Chen, Jiaming Liu, Ziang Cao, Yunfan Lou, Wei Chow, Xian Sun, Yingshuo Wang, Kuangzhi Ge, Xiaowei Chi, Xidong Zhang, Zhibo Pang, Yiwu Zhong, Sirui Han, Zhihe Lu, Weihao Yuan, Qifeng Chen, Michael Yu Wang, Yao Mu, Ziwei Liu, Jianfei Yang, Ping Luo, Shanghang Zhang

机构 * PKU(北京大学) NTU(南洋理工大学) HKUST(香港科技大学) NUS(新加坡国立大学) CUHK(香港中文大学) HKU(香港大学) Duke(杜克大学) UCB(加州大学伯克利分校) GBU(未提及具体中文名的机构) NJU(南京大学) SJTU(上海交通大学)

AI总结 研究围绕具身操纵数据生态系统展开,构建跨越五个互补数据源的“数据金字塔”,通过数据配方分析具身基础模型,将数据组成与多种能力联系起来,并讨论了六个开放挑战,为下一代具身系统设计奠定基础。

Comments Awesome Embodied Data Pyramid; Project Page at https://jasper-aaa.github.io/embodied-data-pyramid/ GitHub Repo at https://github.com/worldbench/awesome-embodied-data-pyramid

详情
AI中文摘要

多模态基础模型通过利用整个互联网学会了看和说。具身智能体没有这样的捷径,因为它们需要将观察与物理状态和动作相结合的数据。这些信号可以由多个数据源在不同程度上提供。在这项工作中,我们将具身数据生态系统组织成一个跨越五个互补数据源的“金字塔”:真实机器人数据、UMI 风格数据、自我中心和外中心数据、模拟数据以及通用视觉语言数据。我们围绕可扩展性和机器人对齐之间的矛盾构建金字塔,并根据数据质量、多样性、可重用性和物理保真度对每个数据源进行进一步刻画。然后,我们通过它们的数据配方分析最近的具身基础模型,研究在预训练期间如何选择、对齐和混合不同的数据源。对于具身大脑模型、视觉语言动作模型和世界动作模型,我们将数据组成与感知、推理、规划、动作生成和世界预测的能力联系起来。最后,我们讨论了六个开放挑战:构建大规模触觉数据集、收集失败和恢复数据、开发可扩展的数据收集管道、跨实例对齐动作、利用自我中心数据进行灵巧操纵以及为机器人学习设计有原则的数据配方算法。我们希望这项工作为下一代具身系统的设计奠定基础。

英文摘要

Multimodal foundation models learned to see and to speak by consuming the whole internet. Embodied agents admit no such shortcut, since they require data that couple observations with physical states and actions. These signals can be provided, to varying degrees, by multiple data sources. In this work, we organize the embodied data ecosystem as a "pyramid" spanning five complementary sources: real-robot data, UMI-style data, egocentric and exocentric data, simulation data, and general vision-language data. We organize the pyramid around the tension between scalability and robot alignment, and further characterize each source in terms of data quality, diversity, reusability, and physical fidelity. We then analyze recent embodied foundation models through the lens of their data recipes, examining how different sources are selected, aligned, and mixed during pretraining. For embodied brain models, vision-language-action models, and world-action models alike, we relate data composition to capabilities in perception, reasoning, planning, action generation, and world prediction. We close by discussing six open challenges: building large-scale tactile datasets, collecting failure and recovery data, developing scalable data-collection pipelines, aligning actions across embodiments, leveraging egocentric data for dexterous manipulation, and designing principled data recipes for robot learning. We hope this work paves the foundation for the design of next-generation embodied systems.

URL PDF HTML 收藏
2607.24743 2026-07-28 cs.CV cs.AI cs.CL 新提交

ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding

ClinFusion:用于整体医学理解的以视觉为中心的多模态大语言模型系统

Hangjie Yuan, Yichen Qian, Zhiwei Tang, Xianzhe Xu, Lirong Wu, Sicheng Yang, Jinwang Wang, Pengju Wang, Zhitao Zeng, Yizeng Han, Yan Xing, Shengxuan Luo, Tao Feng, Qing Xie, Weigen Yao, Yi Yang, Zuozhu Liu, Jiasheng Tang, Shaocheng Wang, Jitao Wang, Jiahong Dong, Weihua Chen, Feng Xu, Fan Wang

机构 * DAMO Academy, Alibaba Group(达摩院,阿里巴巴集团) Hupan Laboratory(湖畔实验室) College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院) Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系) Department of Radiology, The Affiliated Yangming Hospital of Ningbo University(宁波大学附属阳明医院放射科) Zhejiang University-University of Illinois Urbana-Champaign Institute, Zhejiang University(浙江大学伊利诺伊大学厄巴纳香槟校区联合学院,浙江大学) Hepato-Pancreato-Biliary Center, Beijing Tsinghua Changgung Hospital, School of Clinical Medicine, Tsinghua Medicine, Tsinghua University(清华长庚医院肝胆胰中心,清华大学临床医学院,清华医学,清华大学) School of Software, Tsinghua University(清华大学软件学院) Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)

AI总结 研究针对多模态大语言模型在医学领域部署的挑战,提出ClinFusion,采用组合级联视觉编码器架构和视觉基础评估框架,在多模态医学基准测试中表现优异,超越开源和专有模型,经专家盲评验证效果良好。

Comments Code: https://github.com/alibaba-damo-academy/ClinFusion; Models: https://huggingface.co/collections/Alibaba-DAMO-Academy/clinfusion

详情
AI中文摘要

多模态大语言模型在革新临床实践方面潜力巨大,但在医学领域部署存在以视觉为中心的挑战。本文介绍ClinFusion,它是专为整体医学理解设计的以视觉为中心的多模态大语言模型。提出组合式和级联式视觉编码器架构,含级联空间感知局部融合算子。还引入视觉基础评估框架。ClinFusion在多模态医学基准测试中表现出色,超越领先开源模型,在部分基准测试中优于专有模型,还可通过智能工具用于临床工作流程。经放射科医生盲评,ClinFusion生成的报告排名最高,验证了基于感兴趣区域的指标与专家判断相关性最强。

英文摘要

Multimodal large language models (MLLMs) hold immense potential to revolutionize clinical practice, yet deploying them in the medical domain is fundamentally a vision-centric challenge: models must absorb knowledge from heterogeneous 2D and 3D medical images, and evaluation protocols must align with radiologists' clinical practice and provide an accurate, fine-grained and factualness-driven assessment. In this paper, we introduce ClinFusion, a vision-centric MLLM designed for holistic medical understanding that systematically addresses these limitations. We propose a compositional and cascaded vision encoder architecture featuring a Cascade Spatial-Aware Locality Fusion operator that unifies diverse 2D and native 3D medical image understanding within a fused encoder. We further introduce a vision-grounded evaluation framework, including MedIF-Bench for instruction-following assessment and a region-of-interest-grounded method for clinically aligned and factualness-driven report generation evaluation. We show that ClinFusion sets a new state-of-the-art across a comprehensive suite of 2D and 3D multimodal medical benchmarks---spanning visual question answering, report generation, and instruction following---as well as textual medical tasks, outperforming leading open-source medical MLLMs (\textit{e.g.}, Hulu-Med, Lingshu) on 20 out of 24 benchmarks and demonstrating multimodal capabilities better than powerful proprietary models such as GPT-5.2 and Gemini-3-Flash on 13 out of 16 benchmarks, and can be further augmented with agentic tool use for retrieval-augmented and tool-assisted clinical workflows. A blinded evaluation by board-certified radiologists confirms that ClinFusion produces the highest-ranked reports, and validates our RoI-grounded metric as achieving the strongest correlation with expert judgment among all automatic evaluation metrics examined.

URL PDF HTML 收藏
2607.24731 2026-07-28 cs.CV cs.AI cs.LG 新提交

Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation

重新思考策略性扩散蒸馏中的无分类器引导

Bingnan Li, Haozhe Wang, Haozhong Xiong, Fangtai Wu, Jinpeng Yu, Yang Shi, Jiaming Liu, Ruihua Huang

机构 * Qwen Business Unit of Alibaba(阿里巴巴的通义千问业务部)

AI总结 研究策略性扩散蒸馏在无分类器引导下的表现,指出现有方法存在负分支不对称问题,引入正向匹配方法,通过分支感知分别约束正预测和条件方向,应用于视频控制实现更有效知识转移。

详情
AI中文摘要

策略性蒸馏(OPD)通过在当前学生模型生成的轨迹上查询教师模型来调整扩散模型,但对于现代扩散系统的默认组件无分类器引导(CFG)下它应如何表现,人们仍知之甚少。现有OPD方法自然地将速度匹配扩展到由CFG组成的预测,直接匹配教师和学生的引导速度。我们表明,此目标在分支级别未得到充分识别:正分支和负分支误差可在引导预测中相互补偿。通过两个对比案例,我们发现,在共享负条件下,简单匹配仍然有效,此时两个分支误差会共同减小。然而,当模型的原生CFG模式在教师的负分支中保留了学生无法获得的特权信息时,这种共同减小就会失效,并且组合目标会引发对抗性分支误差动态,减小正分支误差同时增加负分支误差。我们将这种失败模式称为负分支不对称(NBA)。为解决NBA,我们引入正向匹配(PDM),这是一种分支感知的OPD目标,它分别约束正预测和CFG条件方向。我们将PDM应用于从密集到稀疏的视频控制,其中简单的引导匹配对推理引导尺度高度敏感,而分支感知监督能够实现更强大且有效的知识转移。

英文摘要

On-policy distillation (OPD) adapts diffusion models by querying a teacher along trajectories generated by the current student, but how it should behave under classifier-free guidance (CFG), a default component of modern diffusion systems, remains poorly understood. Existing OPD methods naturally extend velocity matching to the CFG-composed prediction, directly matching teacher and student guided velocities. We show that this objective is under-identified at the branch level: positive- and negative-branch errors can compensate in the guided prediction. Through two contrasting cases, we find that naive matching remains effective under shared negative conditioning, where both branch errors decrease jointly. When the model's native CFG schema retains privileged information in the teacher's negative branch that is unavailable to the student, however, this joint reduction breaks down and the composed objective induces antagonistic branch-error dynamics, reducing the positive-branch error while increasing the negative-branch error. We term this failure mode Negative Branch Asymmetry (NBA). To address NBA, we introduce Positive--Direction Matching (PDM), a branch-aware OPD objective that separately constrains the positive prediction and the CFG conditional direction. We apply PDM to dense-to-sparse video control, where naive guided matching is highly sensitive to inference guidance scales, while branch-aware supervision enables more robust and effective knowledge transfer.

URL PDF HTML 收藏
2607.24730 2026-07-28 cs.CV cs.AI 新提交

KANEx: Translating Kolmogorov-Arnold Networks' Interpretability to Medical Explainability

KANEx:将柯尔莫哥洛夫-阿诺德网络的可解释性转化为医学可解释性

Krithi Shailya, Ananya Lakshmi Ravi, Venkatanathan K. V., Sowmya S. Sundaram, Gokul S. Krishnan, Aditi Anand, Balaraman Ravindran

机构 * Indian Institute of Technology Madras(印度理工学院马德拉斯分校) Vanderbilt University School of Medicine(范德堡大学医学院)

AI总结 研究针对医学应用中视觉模型黑箱问题,提出KANEx框架,利用柯尔莫哥洛夫-阿诺德网络的符号透明度为VLM推理奠基,设计KAN-Map热图生成方法,经实验验证该方法能提升语义相似度、视觉定位及推理质量,为可信医学人工智能发展助力。

Comments MICCAI 2026

详情
AI中文摘要

计算机视觉模型在医学应用中已非常有效,但其黑箱性质削弱了临床医生的信任。在临床工作流程中,胸部X光分类器 increasingly 与视觉语言模型(VLM)结合以生成自然语言解释。然而,这些系统增加了语言流畅性,却未解决视觉模型的潜在不透明性。随着基于样条的组件提供固有可解释功能单元的柯尔莫哥洛夫-阿诺德网络(KAN)的出现,研究能否利用其架构透明度来产生更可信的文本解释。介绍了KANEx,首个利用KAN的符号透明度为VLM推理奠定基础的框架。还设计了KAN-Map,一种直接从KAN模型而非梯度近似得出的新型热图生成方法。将这些有基础的上下文输入下游VLM以增强可解释性。在MIMIC-CXR数据集上进行基准测试,结果表明基于KAN的架构与ResNet/ViT基线相比,语义相似度提高,同时生成更忠实的显著性图,视觉定位和下游推理质量提高了10%。研究结果表明,将语言解释和视觉归因建立在数学可解释单元上是迈向可信医学人工智能的必要一步。

英文摘要

Computer vision models have become highly effective for medical applications, yet their black-box nature continues to undermine clinician trust. In clinical workflows, chest X-ray classifiers are increasingly paired with Vision-Language Models (VLMs) to generate natural-language explanations. However, these systems add linguistic fluency without addressing the underlying opacity of the visual model. With the emergence of Kolmogorov-Arnold Networks (KANs), whose spline-based components provide inherently interpretable functional units, we investigate whether this architectural transparency can be leveraged to produce more trustworthy textual explanations. We introduce KANEx, the first ever framework that leverages the symbolic transparency of KANs to ground VLM reasoning. This interpretability also made it possible to design KAN-Map, a novel heatmap generation method derived directly from KAN models rather than gradient approximations. We feed these grounded contexts into downstream VLMs for enhanced explainability. Benchmarked on the MIMIC-CXR dataset, we demonstrate that KAN-based architectures with ResNet/ViT baselines demonstrate improved semantic similarity while producing significantly more faithful saliency maps. KAN architectures improve visual localization and downstream reasoning quality by 10%. Our findings suggest that grounding linguistic explanations and visual attributions in mathematically interpretable units is a necessary step toward trustworthy medical AI.

URL PDF HTML 收藏
2607.24729 2026-07-28 cs.CV 新提交

MicroZoom: Structure-Preserving Detail Synthesis at Extreme Scale

MicroZoom:极端尺度下的结构保留细节合成

Huy Huynh, Jingwei Ma, Brian Curless, Ira Kemelmacher-Shlizerman, Steven M. Seitz

机构 * University of Washington(华盛顿大学)

AI总结 MicroZoom旨在解决微观尺度下基于参考的极端尺度超分辨率问题,通过两阶段级联设计,第一阶段恢复全局图案连贯性,第二阶段细化局部纹理细节,辅以分割掩码指导合成,实现了全局连贯、基于物质的千兆像素图像合成。

Comments Project page: https://microzoom-sr.github.io/

详情
AI中文摘要

我们介绍了MicroZoom,这是一个用于微观尺度下千兆像素图像合成的生成框架。给定一张标准照片和一组稀疏的消费级显微镜特写,MicroZoom能合成出无缝的、基于真实参考资料物质特征的千兆像素分辨率图像,实现对物体整个空间范围微观纹理的探索性可视化。我们的目标是合理合成而非精确重建。我们专注于全图像、基于参考、高达350倍放大倍数的极端尺度超分辨率,此设置带来两个主要挑战:从模糊物质边界附近的高损耗输入中恢复特定纹理细节,以及在数百万个局部预测中保留正确的大规模图案结构。我们通过两阶段级联设计解决这些问题,第一阶段恢复全局图案连贯性,第二阶段细化局部纹理细节,并辅以分割掩码来指导模糊边界处的合成。我们在一组自拍摄的日常物体上验证了我们的方法,并展示了全局连贯、基于物质的千兆像素图像。

英文摘要

We introduce MicroZoom, a generative framework for gigapixel image synthesis at the microscopic scale. Given a standard photograph and a sparse set of consumer-grade microscope close-ups, MicroZoom synthesizes a seamless, gigapixel-resolution image grounded in the material character of the real references, enabling exploratory visualization of microscopic texture across the full spatial extent of an object. Our goal is plausible synthesis, not exact reconstruction. We focus on full-image, reference-based, extreme-scale super-resolution at magnification levels of up to 350x, a setting that introduces two major challenges: (1) recovering texture-specific detail from highly lossy inputs near ambiguous material boundaries, and (2) preserving correct large-scale pattern structure, such as the repeating geometry of a fabric weave, across millions of local predictions. We address these with a two-stage cascaded design, where the first stage recovers global pattern coherence and the second refines local texture detail, supplemented by a segmentation mask to guide synthesis at ambiguous boundaries. We verify our approach on a collection of self-captured everyday objects and demonstrate globally coherent, materially grounded gigapixel imagery.

URL PDF HTML 收藏
2607.24721 2026-07-28 cs.CV 新提交

DreamStyle3D: Efficient 3D Stylized Asset Generation via Dual-Attention Disentanglement

DreamStyle3D:通过双注意力解缠实现高效3D风格化资产生成

Kai Wang, Ziheng Ouyang, Xuying Zhang, Ming-Ming Cheng, Qibin Hou

机构 * VCIP, Nankai University(视觉计算与图像处理实验室,南开大学) JD Explore Academy(京东探索研究院) AAIS, Nankai University(人工智能学院,南开大学)

AI总结 针对高效生成风格化3D资产的需求,DreamStyle3D基于解耦双交叉注意力机制,明确分离几何与风格特征,采用轻量级训练策略,构建数据集和数据管道,能在10秒内生成高质量、几何一致的风格化3D资产,提升了效率并为3D创作提供新方案。

Comments ACM MM 2026

详情
AI中文摘要

随着游戏、动画和虚拟现实行业的发展,对高效生成风格化3D资产的需求迅速增长。现有方法在保持风格保真度、几何一致性和生成效率方面仍存在困难。为此,我们提出了DreamStyle3D,一个基于解耦双交叉注意力机制的高效风格化3D资产生成框架。该方法明确分离几何和风格特征,采用轻量级训练策略。我们构建了自动数据管道和数据集。实验表明,DreamStyle3D能在10秒内生成高质量、几何一致的风格化3D资产,提高了效率并提供了新的3D内容创作解决方案。

英文摘要

With the growth of gaming, animation, and virtual reality industries, the demand for efficient generation of stylized 3D assets is rapidly increasing. However, existing approaches still struggle to jointly preserve style fidelity, geometric consistency, and generation efficiency, as most of them still rely on indirect 2D-to-3D stylization pipelines. This motivates a native 3D stylization framework that can explicitly disentangle style from geometry while remaining efficient. To this end, we propose DreamStyle3D, an efficient framework for stylized 3D asset generation built on a Decoupled Dual Cross-Attention mechanism. Our method explicitly separates geometric and stylistic features to enable efficient style injection while preserving structural consistency, and further adopts a lightweight training strategy to enhance style consistency and model generalization. In addition, we build an automated data pipeline and construct a dataset of about 15K content-style-stylized triplets for training and evaluation. Extensive experiments demonstrate that our DreamStyle3D can generate high-fidelity, geometrically consistent stylized 3D assets within 10 seconds, substantially improving efficiency while maintaining superior style quality and offering a new solution for 3D content creation. The code and data are available at https://github.com/HVision-NKU/DreamStyle3D.

URL PDF HTML 收藏
2607.24720 2026-07-28 cs.CL cs.AI cs.LG 新提交

The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation

多轮长期规划的物理学:通过单教师和多教师策略蒸馏从预训练到训练后

Tianyi Men, Zhuoran Jin, Kang Liu, Jun Zhao

机构 * Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) University of Chinese Academy of Sciences(中国科学院大学)

AI总结 研究多轮长期规划问题,通过引入统一可控环境,利用预训练、GRPO、OPD及MOPD等方法,研究规划能力在不同阶段的获取、塑造与整合,展示了多教师策略蒸馏对跨环境能力整合的作用。

详情
AI中文摘要

多轮长期规划对基础模型智能体至关重要,但如何从根本上改进尚不清楚。现有模型在不可控且不透明的互联网数据上训练,难以确定规划能力如何获得、塑造和整合。为此引入统一可控的多轮环境。研究了预训练阶段规划能力的获取,包括数据格式、分布和质量等;通过GRPO和OPD在训练后塑造规划能力,区分通用规划模式和特定任务规划知识;通过MOPD在训练后整合规划能力,展示了多教师策略蒸馏通过收敛到跨环境的共享规划模式来整合能力。

英文摘要

Multi-turn long-horizon planning is critical for foundation model agents, yet how to fundamentally improve it remains unclear. Existing models are trained on uncontrollable and opaque Internet data, making it difficult to identify how planning ability is acquired, shaped, and integrated. To address this challenge, we introduce a unified and controlled multi-turn environment that enables precise control. It allows systematically study long-horizon planning across three stages. (1) Planning ability acquisition during pre-training. We study data format, distribution, and quality. Explicit world model construction through CoT state transition modeling yields stronger long-horizon generalization. Atomic skills alone are insufficient for compositional generalization, whereas a litte long-horizon data works. Moreover, suboptimal trajectories severely impair performance because errors amplify over long horizons. (2) Planning ability shaping via GRPO and OPD post-training. Through mutual information, we distinguish general planning patterns from task-specific planning knowledge. For planning patterns, we identify three application regions of post-training: unnecessary, effective, and unsupported. OPD has a broader effective region than GRPO under low-quality and long-horizon settings, as it provides more consistent update directions. For planning knowledge, distilling unseen procedures from a teacher with different knowledge may impair student's prior world modeling without fully establishing new knowledge. (3) Planning ability integration through MOPD post-training. We show that multi-teacher on-policy distillation (MOPD) integrates capabilities by converging to shared planning-pattern across environments. Compatible patterns enable cross-environment generalization, partially shared patterns support continual learning, while completely conflicting patterns cause severe interference.

URL PDF HTML 收藏
2607.24707 2026-07-28 cs.AI cs.CV cs.DB 新提交

ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

ERUnderstand:在结构化实体关系图上评估视觉语言模型

Ali Ansari, Yasmin Mohammadi, Farnoush Nili, Parsa Esmaeilkhani, Longin Jan Latecki, Eduard Dragut

机构 * Temple University(天普大学)

AI总结 研究针对实体关系图仅为图像限制AI辅助数据库工程的问题,引入ERUnderstand基准测试。通过对2960个图表及标准化表示评估视觉语言模型,发现常见元素恢复良好,但特定元素性能差,推理增强模型可提升性能,为评估多模态理解提供基准。

详情
AI中文摘要

实体关系图(ERD)是概念数据库设计的核心,但通常仅以渲染图像形式存在,而非机器可读模式,这限制了人工智能辅助数据库工程。我们引入了ERUnderstand,这是首个用于结构化理解ER图的大规模基准测试,包含从精心挑选的教育资源、真实世界模式以及跨越不同领域、符号、复杂度级别和扩展实体关系(EER)构造的合成示例中收集的2960个图表。每个图表都与标准化的机器可读表示配对,用于对模式元素进行细粒度评估。评估当前最先进视觉语言模型(VLM)时发现,虽然常见ERD元素能可靠恢复(F1>0.74),但在弱实体(F1低至0.28)、多值属性(0.14 F1)和N元关系(0.07 F1)上性能大幅下降。推理增强模型将整体性能提高了15%-25%,但仍对语言先验和图表复杂度增加敏感。ERUnderstand为评估概念数据库模式的多模态理解提供了标准化基准。基准测试、数据集、评估工具包和生成代码可在指定网址公开获取。

英文摘要

Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering. We introduce ERUnderstand, the first large-scale benchmark for structured understanding of ER diagrams, comprising 2,960 diagrams collected from curated educational sources, real-world schemas, and synthetically generated examples spanning diverse domains, notations, complexity levels, and Extended Entity-Relationship (EER) constructs. Each diagram is paired with a standardized machine-readable representation for fine-grained evaluation of schema elements. Evaluating state-of-the-art Vision-Language Models (VLMs), we find that while common ERD elements are recovered reliably (F1 > 0.74), performance drops sharply on weak entities (as low as 0.28 F1), multivalued attributes (0.14 F1), and N-ary relationships (0.07 F1). Reasoning-augmented models improve overall performance by 15-25% but remain sensitive to linguistic priors and increasing diagram complexity. ERUnderstand provides a standardized benchmark for evaluating multimodal understanding of conceptual database schemas. The benchmark, dataset, evaluation toolkit, and generation code are publicly available at https://github.com/salinaria/ERUnderstand.

URL PDF HTML 收藏
2607.24703 2026-07-28 cs.CV 新提交

Panda: Unsupervised Pelvic Anomaly Detection for Real-Time MR Imaging

Panda:用于实时磁共振成像的无监督盆腔异常检测

Anika Knupfer, Maximilian Lindholz, Johanna Paula Müller, Jordina Aviles Verdera, Smiti Tripathy, Susanne Schulz-Heise, Jana Hutter

机构 * Institute of Information Processing, Leibniz University Hannover(莱布尼茨汉诺威大学信息处理研究所) CAIMED, L3S(CAIMED,L3S) Department of Radiology, Charité Universitätsmedizin Berlin(柏林夏里特大学医学中心放射科) Image Data Exploration and Analysis Lab, Friedrich-Alexander University Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学图像数据探索与分析实验室) Institute for Radiology, University Hospital Erlangen(埃尔朗根大学医院放射研究所)

AI总结 针对女性盆腔疾病实时异常检测难题,提出基于Dinomaly的无监督框架,利用冻结的DINOv3视觉Transformer编码器等,在子宫肌瘤数据集上评估,实现高像素级AUROC和帧级特异性,满足实时临床需求,为医生决策和协议调整提供支持。

Comments 10 pages, 5 figures

详情
AI中文摘要

女性盆腔疾病研究不足,诊断常延迟。盆腔MRI虽能提供软组织对比,但实时异常检测因生理运动、组织变形和仪器伪影而具挑战性。现有监督方法不实用。本文提出基于Dinomaly的无监督异常检测框架,从健康病例学习规范表示,无需标签标记偏差。利用冻结的DINOv3视觉Transformer编码器等,通过编码器和解码器表示间的逐令牌余弦距离定位异常,生成空间异常图。在子宫肌瘤数据集上评估,框架实现了88.06%的像素级AUROC和95.45%的帧级高特异性,每秒40.5切片,满足实时临床部署要求,能为放射科医生决策和自适应协议调整提供支持。

英文摘要

Female pelvic diseases remain an under researched area characterized by often delayed diagnosis. While pelvic MRI offers superior soft-tissue contrast for diagnosis and image-guided procedures, real-time anomaly detection remains challenging due to physiological motion, tissue deformation, and instrument artifacts. Existing supervised approaches are impractical, as adverse events are rare, heterogeneous, and difficult to annotate. We present a Dinomaly-based unsupervised anomaly detection framework adapted for pelvic MRI that learns normative representations from healthy cases and flags deviations without requiring labels. Our approach leverages a frozen DINOv3 Vision Transformer encoder combined with a noisy MLP bottleneck and Linear Attention decoder to prevent identity mapping while maintaining computational efficiency. Anomalies are localized via per-token cosine distance between encoder and decoder representations, yielding spatial anomaly maps that provide immediate feedback at the scanner to support radiologist decision-making and adaptive protocol adjustment. Evaluated on a curated subset of the Uterine Myoma Dataset, the framework achieves a pixel-level AUROC of 88.06% and high specificity (95.45%) at frame level at 40.5 slices/s, meeting real-time clinical deployment requirements. The spatial anomaly maps and frame-level scores provide immediate, localized feedback at the scanner to support radiologist decision-making and adaptive protocol adjustment during active procedures.

URL PDF HTML 收藏
2607.24701 2026-07-28 cs.CV 新提交

Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Tracking

用于模态缺失的RGBT跟踪的时空条件去噪Transformer

Andong Lu, Ziyi Zha, Jiandong Jin, Shihao Li, Chenglong Li, Jin Tang, Bin Luo

机构 * School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院) School of Artificial Intelligence, Anhui University(安徽大学人工智能学院)

AI总结 针对RGBT跟踪中模态缺失致性能下降问题,提出时空条件去噪Transformer(SCDT)。它整合时空线索,通过利用不同时间线索及噪声调制适应机制,在统一框架下实现缺失模态信息重建等,实验证明其性能优于现有方法。

Comments Accepted by CVPR2026

详情
AI中文摘要

在RGBT跟踪中,模态缺失往往会导致多模态特征表示不完整和不稳定,从而严重降低性能。现有方法通常试图从可用模态中恢复缺失模态,但在具有挑战性的场景中生成的数据质量可能不尽人意。此外,当前方法在处理缺失和完整数据时灵活性有限。为克服这些限制,我们提出了时空条件去噪Transformer(SCDT),它在统一框架中整合空间线索和时间上下文,以自适应地对缺失模态进行信息重建和对弱模态进行特征增强,用于稳健的模态缺失RGBT跟踪。具体而言,SCDT利用近期历史帧的短期时间线索捕捉细粒度时间相关性,利用编码模态演变的长期时间线索捕捉全局上下文。通过联合利用长期和短期时间上下文作为条件,SCDT逐步引导可用模态的噪声特征学习可靠且时间一致的多模态表示。此外,SCDT引入了噪声调制适应机制,可根据模态可用性动态调整其行为,使单个框架能够在模态缺失和完整场景下统一特征学习,而无需更改架构或参数。在三个公共基准数据集上的大量实验表明,我们的方法始终优于现有方法。代码可在此处获取。

英文摘要

Missing modalities in RGBT tracking often lead to incomplete and unstable multimodal feature representations that greatly degrade the performance. Existing methods typically attempt to recover missing modalities from available ones, but the quality of data generated in challenging scenarios might be unsatisfactory. In addition, current approaches exhibit limited flexibility in processing both missing and complete data. To overcome these limitations, we propose a Spatio-temporal Conditional Denoising Transformer (SCDT), which integrates the spatial cues and the temporal context to adaptively perform information reconstruction of missing modalities and feature enhancement of weak modalities in a unified framework, for robust modality-missing RGBT tracking. In particular, SCDT leverages the short-term temporal cues from recent historical frames to capture the fine-grained temporal correlations and the long-term temporal cues encoding modality evolution to capture the global context. By jointly exploiting long short-term temporal contexts as the conditions, SCDT progressively guides noisy features of available modalities to learn reliable and temporally consistent multimodal representations. Furthermore, SCDT introduces a noisemodulated adaptation mechanism that dynamically adjusts its behavior according to the modal availability, enabling a single framework to unify feature learning under both modality-missing and complete scenarios without changing the architecture or parameters. Extensive experiments on three public benchmark datasets demonstrate that our method consistently outperforms state-of-the-art methods. The code is available here.

URL PDF HTML 收藏
2607.24683 2026-07-28 cs.CV cs.AI cs.LG 新提交

Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification

多模态分类中用于缺失任意模态的协同学习

Francisco Mena, Dino Ienco, Roberto Interdonato, Cassio F. Dantas, Simon Besnard

机构 * GFZ Helmholtz Center for Geosciences(德国波茨坦地学研究中心亥姆霍兹中心) INRAE, UMR TETIS, University of Montpellier(法国蒙彼利埃大学农业环境与生态研究院、蒙彼利埃大学TETIS联合研究单位) INRIA, EVERGREEN, University of Montpellier(法国蒙彼利埃大学信息与自动化研究所、EVERGREEN实验室) CIRAD, UMR TETIS, University of Montpellier(法国蒙彼利埃大学国际农业研究磋商组织、蒙彼利埃大学TETIS联合研究单位)

AI总结 针对多模态分类中因操作约束导致训练和推理时模态可用性不一致、存在缺失任意模态的问题,提出多模态协同学习框架,引入特征和决策层面利用信息的两种方法,实验证明在不同缺失模态条件下有显著鲁棒性提升。

Comments Accepted at Discovery Science 2026

详情
AI中文摘要

多模态分类利用不同数据源的互补信息来提高预测性能。但现实场景中,如传感器故障或隐私限制等操作约束,会导致训练和推理时模态可用性不一致。以往研究主要针对双峰数据集且侧重于设计鲁棒融合过程。本文采用多模态协同学习框架,优先进行模态间协作而非融合。考虑到任何模态子集都可能缺失,不假设预定义的缺失模态模式(即缺失任意模态)。为此引入两种在特征和决策层面利用信息的方法。在两个多模态分类基准上的实验表明,在各种缺失模态条件下有显著的鲁棒性提升。第一种方法在单模态缺失的最小缺失条件下表现更鲁棒,第二种在除一个模态外所有模态都缺失的极端缺失条件下性能更好。

英文摘要

Multi-modal classification leverages complementary information across diverse data sources to enhance predictive performance. However, real-world scenarios subject to operational constraints, such as sensor failures or privacy restrictions, lead to inconsistent modality availability between training and inference times. To handle missing modalities, prior studies have mainly covered bimodal data setups and focused on designing robust fusion processes. Instead, we adopt a multi-modal co-learning framework that prioritizes inter-modal collaboration rather than multi-modal fusion. Specifically, we consider that any subset of modalities may be absent, without assuming predefined missing-modality patterns, an inference scenario we refer to as missing arbitrary modalities. To address this challenge, we introduce two alternative approaches that leverage information at both feature- and decision-level. Experiments on two multi-modal classification benchmarks demonstrate significant robustness gains in various missing modality conditions. The first method shows more robust behavior under minimal missing conditions, where a single modality is absent, whereas the second performs better under extreme missing conditions, where all-but-one modalities are missing. Our code is available at https://github.com/fmenat/Co4Miss.

URL PDF HTML 收藏
2607.24673 2026-07-28 cs.LG stat.ME 新提交

Causal-TS: A Python Library for Causal Discovery in High-Dimensional and Nonstationary Time Series

因果时间序列(Causal-TS):用于高维非平稳时间序列因果发现的Python库

Mohammad Fesanghary

机构 * Bloomberg LP(彭博有限合伙企业)

AI总结 介绍用于高维非平稳时间序列因果发现的Python库Causal-TS,它含多种算法及包装器,有统一CI测试层并支持GPU加速,通过状态发现管道检测断点,还有命令行等工具提供端到端管道,可pip安装且经测试。

Comments 4 page, Intro paper

详情
AI中文摘要

我们描述了因果时间序列(Causal-TS),这是一个用于高维非平稳多元时间序列因果发现的开源Python库。它提供了四种专门算法(CDNOTS、CDNOTS+、CEDAR和GRACE)以及GES、格兰杰、LASSO-VAR和LGES的包装器,通过PyTorch实现GPU加速的统一条件独立性(CI)测试层。一个状态发现管道通过可插拔的变点检测器检测结构断点,并使用特定状态参数对每个状态进行发现。命令行界面、合成数据生成器和可选的DoWhy集成提供了从原始时间序列到因果效应估计的端到端管道。该库可通过pip安装,在Python 3.10 - 3.12上进行了测试,可通过此https链接获取。

英文摘要

We describe Causal-TS, an open-source Python library for causal discovery in high-dimensional and nonstationary multivariate time series. Causal-TS provides four specialized algorithms-CDNOTS, CDNOTS+, CEDAR, and GRACE-along with wrappers for GES, Granger, LASSO-VAR, and LGES, all sharing a unified conditional independence (CI) test layer with GPU acceleration via PyTorch. A regime discovery pipeline detects structural breaks via pluggable changepoint detectors and runs discovery per regime with regime-specific parameters. A command-line interface, synthetic data generators, and optional DoWhy integration provide an end-to-end pipeline from raw time series to causal effect estimates. The library is pip-installable, tested on Python 3.10--3.12, and available at https://github.com/bloomberg/causal-ts.

URL PDF HTML 收藏
2607.24667 2026-07-28 cs.AI 新提交

Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

作为估计的逐出:测试时内存的固定延迟平滑视图,以及测量何时胜过累积

Maruthi Vemula, Neeraj Praneeth Gajula

机构 * University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校) New York University(纽约大学)

AI总结 研究语言模型内存管理,将存储项保留决策转化为对项目是否被重用的估计问题,提出固定延迟平滑策略RMM。在受控设置中有优势,在第三方基准测试中表现一般,贡献了框架及测量与累积效果对比的映射。

Comments 8 pages, 3 figures, 3 tables

详情
AI中文摘要

具有有限工作内存的语言模型必须反复决定保留哪些存储项。每种已部署的方法都在项目到达时做出决定,要么基于过去(StreamingLLM、H2O),要么基于对未来的猜测(SnapKV)。我们将此选择重新表述为关于隐藏信号(项目是否会被重用)的估计问题,将现有方法置于一个轴上,即提交延迟\(H\):在线过滤器和学习到的预测器在\(H = 0\)时提交,而Belady的离线最优值处于已知整个未来的位置。中间缺失的状态,即固定延迟平滑,会等待有限数量的步骤,观察正确的近期预测关注了哪些项目,然后才提交。这种测量证明了其效用,将Belady不可观察到的未来请求转化为我们从模型本身读取的内容。我们将其实例化为一种无需训练的策略RMM,它是H2O的严格推广,当测量均匀时恰好简化为H2O。在重用是内生且在时间上分离的受控设置中,证明的效用比累积注意力能更好地识别已使用的内存,并且小的有限内存表现得像大得多的内存。但在独立的第三方基准测试中,在NVIDIA的KVPress框架内与它自己的SnapKV、H2O和StreamingLLM实现进行比较时,优势大多消失:在单轮问答中,RMM与H2O相当,在流式多轮设置中输给了H2O和SnapKV。原因很简单:在自然文本上,模型对大多数令牌是正确的,所以按正确性加权注意力几乎不会改变它,并且证明的效用会退化为累积注意力,除非重用是明显且内生的,而标准基准测试并未体现这一点。我们的贡献是该框架以及关于测量何时胜过累积的真实映射,而不是新的技术水平。

英文摘要

A language model with a bounded working memory must repeatedly decide which stored items to keep. Every deployed method decides the moment an item arrives, from the past (StreamingLLM, H2O) or from a guess about the future (SnapKV). We recast the choice as an estimation problem on a hidden signal, whether an item will be reused, placing existing methods on one axis, the commit lag $H$: online filters and learned predictors commit at $H=0$, while Belady's offline optimum sits where the whole future is known. The missing regime in between, fixed-lag smoothing, waits a bounded number of steps, observes which items a correct near-future prediction attended to, and only then commits. This measurement, demonstrated utility, turns Belady's unobservable future request into something we read off the model itself. We instantiate it as a training-free policy, RMM, a strict generalization of H2O that reduces to it exactly when the measurement is uniform. In controlled settings where reuse is endogenous and separated in time, demonstrated utility identifies used memory far better than accumulated attention, and a small bounded memory behaves like a much larger one. But on independent third-party benchmarks, run inside NVIDIA's KVPress harness against its own SnapKV, H2O, and StreamingLLM implementations, the advantage mostly disappears: RMM is on par with H2O for single-turn question answering and loses to both H2O and SnapKV in a streaming multi-turn setting. The cause is simple: on natural text the model is correct about most tokens, so weighting attention by correctness barely changes it, and demonstrated utility collapses onto accumulated attention unless reuse is sharp and endogenous, which standard benchmarks do not exercise. Our contribution is the framework and an honest map of when measuring beats accumulating, not a new state of the art.

URL PDF HTML 收藏
2607.24665 2026-07-28 cs.CV cs.GR cs.LG 新提交

MMOE: Modernizing Diffusion Transformers with Efficient Expert Design

MMOE:通过高效专家设计使扩散变压器现代化

Yanhao Jia, Jiepeng Wang, Haibin Huang, Chi Zhang, Erik Cambria, Xuelong Li

机构 * College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院) Institute of Artificial Intelligence of China Telecom (TeleAI)(中国电信人工智能研究院)

AI总结 研究AIGC基础模型中扩散变压器未平衡质量与成本问题,提出MMOE对其现代化改造,将多种专家设计应用于AIGC生成,实验表明MMOE能达更好质量成本平衡,使AIGC基础模型可循大语言模型平衡扩展路径。

Comments 13 pages, 5 figures

详情
AI中文摘要

现代大语言模型通过将容量增长与效率相结合来成功扩展,控制每令牌和部署成本。AIGC基础模型,尤其是扩散变压器主干,已开始采用稀疏专家,但近期工作大多增加了总参数数量和稀疏率,却未引入使大语言模型扩展实用的效率机制,导致生成质量与训练和部署成本失衡。本文提出疑问:高效大语言模型扩展背后的架构原则能否以更平衡的方式应用于AIGC基础模型?为此引入ModernMOE(MMOE),对SiT风格的扩散变压器进行现代化改造,系统地将路由专家、共享和轻量级专家、门残差路由以及注意力残差信息重用应用于AIGC生成。通过在单个八GPU H100节点上以批量大小256训练400k步的实验,MMOE在每个记录的检查点达到更低的FID,比密集和中间稀疏专家基线收敛更快,在稀疏变体中实现了最佳的质量成本平衡。路由分析还显示了跨深度的稳定专家专业化、轻量级路由的大量使用以及去噪过程中适度的逐步骤路由变化。这些结果表明,AIGC基础模型可以通过引入经过验证的效率设计来遵循大语言模型的平衡扩展路径,而不是简单地增加总参数和稀疏率。

英文摘要

Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows. AIGC Foundation Models (AFMs), especially diffusion-transformer backbones, have begun to adopt sparse experts, but recent efforts mostly enlarge total parameter counts and sparsity ratios without importing the efficiency mechanisms that made LLM scaling practical, so generation quality is seldom balanced against training and deployment cost. This raises a natural question: can the architectural principles behind efficient LLM scaling be adapted to AFMs in a more balanced way? We introduce ModernMOE (MMOE), a modernization of SiT-style diffusion transformers that systematically adapts routed experts, shared and lightweight experts, gate-residual routing, and attention-residual information reuse to AIGC generation. Rather than treating MoE as a single plug-in replacement, MMOE studies how different modern expert components affect convergence, efficiency, and generation quality when composed inside a diffusion transformer. Every experiment in this paper is trained on a single eight-GPU H100 node with batch size 256 for 400k steps, an accessible single-machine budget. Under matched training and sampling protocols and at this budget, MMOE reaches lower FID at every recorded checkpoint, that is, it converges faster per training step, than dense and intermediate sparse-expert baselines, and among the sparse variants it attains the best quality-cost balance. Routing analysis further shows stable expert specialization across depth, substantial use of lightweight routes, and modest step-to-step routing changes during denoising. These results suggest that AFMs can follow the balanced scaling path of LLMs by importing proven efficiency designs, rather than by simply increasing total parameters and sparsity ratios.

URL PDF HTML 收藏
2607.24662 2026-07-28 cs.LG cs.SI 新提交

When Can You Correct Distribution Drift in Temporal Graph Generation? A Sharpening--Drift Tension and an Impossibility for Observation-Based Correction

何时能够校正时间图生成中的分布漂移?锐化-漂移张力与基于观测校正的不可能性

Tianpeng Li, Xuan Guo, Wenjun Wang, Wang Zhang, Pengfei Jiao

机构 * College of New Media and Communication, Tianjin University(天津大学新媒体与传播学院) School of Artificial Intelligence, Tianjin University(天津大学人工智能学院)

AI总结 研究时间图生成模型中分布漂移问题,通过掩码流匹配损失分析其不可修复性,给出实验结果,证明基于观测的校正器效果不佳,预言机消除误差效果最好,外推比不处理还差。

详情
AI中文摘要

时间图的生成模型在演化网络的一段上进行训练,然后部署到下一段,在这个间隔中性能会严重下降。我们表明这种下降是可推导的、普遍的,且无法从观测中修复。掩码流匹配损失在无独立性假设下精确分解为不可约熵加上一个散度,其沿训练路径的导数对于训练期间罕见而部署时常见的结构为正,随着训练概率趋于零而发散。实验上这种权衡是指数为 -0.605 的幂律(\(R^2 = 0.9977\)),漂移提高了采样器的误差下限且不改变达到该下限所需的步数。我们证明任何基于过去观测可测量的校正器至少会留下它所跟踪统计量的条件方差,并且只有当\(\mu^2 > v(1 - 2\rho)\)时趋势外推才优于仅信任最后一次观测。一个预言机可消除60%的误差,基于观测的最佳校正器能恢复其中的5.7%,外推比不做任何处理更差。

英文摘要

Generative models of temporal graphs are trained on one stretch of an evolving network and deployed on the next, and they degrade badly in the gap. We show this degradation is derivable, general, and not fixable from observations. The masked flow-matching loss decomposes exactly, with no independence assumption, into an irreducible entropy plus a divergence whose derivative along the training path is positive precisely for structures rare during training and common at deployment, diverging as their training probability goes to zero. Empirically the trade-off is a power law with exponent $-0.605$ ($R^2=0.9977$), and drift raises the sampler's error floor without changing how many steps reach it: across seven well-powered conditions the drift-period marginal error varies by at most $6\%$ over a $50\times$ range of sampling budgets, while the floor sits $2.2\times$ to $34.3\times$ above the in-period floor. Because the deployment period is observed, correction looks like a matter of measurement. It is not. We prove that any corrector measurable with respect to past observations leaves at least the conditional variance of the statistic it tracks, and that trend extrapolation beats trusting the last observation only when $μ^2>v(1-2ρ)$. Both premises are measurable and both go the wrong way: the drift is trendless and mean-reverting, with a one-step innovation as large as the drift itself. An oracle removes $60\%$ of the error, the best observation-based corrector recovers $5.7\%$ of that, and extrapolation is strictly worse than doing nothing clever.

URL PDF HTML 收藏
2607.24651 2026-07-28 cs.CV cs.CL cs.IR 新提交

Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

无坐标或区域标签的视觉文档理解中的证据归因

Zhuchenyang Liu, Yao Zhang, Yu Xiao

机构 * Aalto University(阿尔托大学)

AI总结 研究视觉文档理解中无坐标或区域标签时的证据归因问题,通过对比坐标与语言接口,发现语言接口能提升证据召回率、降低幻觉率。基于此用引述和检索管道作训练框架,引入GRPO方法,提高了模型严格归因准确率,找到改善归因的实用路径。

详情
AI中文摘要

可靠的视觉文档理解需要模型将每个答案归因于支持它的证据区域。近期基准和系统通过坐标接口表达这一步骤,然而视觉语言模型常在此接口下出现归因幻觉。本文研究该失败是否部分受模型通过坐标表达能力的限制。在经过验证的双语CiteVQA子集上,将坐标接口与语言接口(模型仅输出文本引述证据,多模态检索器返回引述位置)对比,六个开放视觉语言模型参与。结果显示与坐标接口相比,证据召回率提高,幻觉率降低,答案质量变化小。基于此,利用相同引述和检索管道作为训练框架,引入GRPO方法,在无区域标签情况下训练模型更好地引述证据,提高了严格归因准确率。这些发现表明了在无坐标接口和无昂贵区域级监督情况下改善归因的实用途径。

英文摘要

Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.

URL PDF HTML 收藏
2607.24649 2026-07-28 cs.AI 新提交

Reason-Mediated Behavioral Models for Auditing LLM Social Simulators

用于审计大语言模型社会模拟器的基于推理的行为模型

Atharva Pandey, Gautam Jajoo

机构 * Kairosity

AI总结 研究大语言模型作为社会模拟器的评估问题,通过防晒霜概念测试将人类理由映射到推理状态Z,发现人类理由能提升购买意图预测,大语言模型模拟理由较脆弱,贡献了社会模拟器评估框架。

详情
AI中文摘要

大语言模型越来越多地被用作社会模拟器,包括作为合成调查对象。大多数评估询问模拟结果是否类似于人类结果。我们认为这是必要的,但不够充分:模拟器可能在使用错误的基于推理的推理模式时匹配最终答案。我们通过一项94人的防晒霜概念测试来研究这个问题,其中每个受访者评估三个产品概念并撰写开放式理由。我们将这些理由映射到有符号的推理状态Z,其中正号支持采用,负号阻碍采用。这给出了一个实际的审计:固定受访者描述符D、类别上下文K和概念处理X,人类基于推理的理由是否有助于预测行为Y,以及大语言模型能否在不看到人类理由或结果的情况下模拟相同的推理状态?人类基于推理的理由显著提高了对购买意图的预测。大语言模型模拟的理由更脆弱:它们听起来往往合理,但经常呼应概念板而不是恢复受访者的接受或拒绝路径。本文为社会模拟器贡献了一个评估框架。推理状态本身并不能识别自然因果效应,但它们提供了一个可解释的测试,以检验模拟器陈述的理由是否与人类证据一致。

英文摘要

Large language models are increasingly used as social simulators, including as synthetic survey respondents. Most evaluations ask whether simulated outcomes resemble human outcomes. We argue that this is necessary but too weak: a simulator can match the final answer while using the wrong rationale-derived reason pattern. We study this problem through a 94-person sunscreen concept test in which each respondent evaluated three product concepts and wrote open-ended rationales. We map those rationales into signed reason states $Z$, where positive signs support adoption and negative signs block it. This gives a practical audit: holding respondent descriptors $D$, category context $K$, and concept treatment $X$ fixed, do human rationale-derived reasons help predict behavior $Y$, and can an LLM simulate the same reason state without seeing the human rationale or outcome? Human rationale-derived reasons substantially improve held-out prediction of purchase intent. LLM-simulated reasons are more brittle: they often sound plausible, but frequently echo the concept board rather than recover the respondent's acceptance or rejection path. The paper contributes an evaluation framework for social simulators. Reason states do not identify natural causal effects by themselves, but they provide an interpretable test of whether a simulator's stated reasons align with human evidence.

URL PDF HTML 收藏
2607.24647 2026-07-28 cs.AI cs.LG 新提交

Efficiency Matters in Autonomous Research

自主研究中的效率问题

Haiqian Yang, Yuan Cao

机构 * MIT(麻省理工学院)

AI总结 研究自主研究中解决方案搜索过程的效率问题,提出用帕累托前沿曲线下面积评估AR系统,比较多种搜索算法,发现无单一最优结构,且效率与结果质量是不同维度,引入流体搜索,该方法在评估任务中总体搜索效率最高。

详情
AI中文摘要

人工智能驱动的自主研究(AR)系统在广泛任务中日益有效,但其性能仍主要按最终结果质量评估。本文认为解决方案搜索过程的效率是同样重要却常被忽视的性能维度。随着AR从数学和编码等低成本验证领域扩展到需昂贵物理实验评估解决方案的现实科学环境,搜索效率愈发重要。为考量此维度,我们提议用帕累托前沿曲线下面积(AUC)评估AR系统,同时兼顾最终结果质量。我们比较了包括爬山法、束搜索、树搜索和进化搜索在内的几类搜索算法在十二个系统优化任务中的表现。发现没有单一搜索结构始终最有效,且搜索效率和最终结果质量是不同性能维度。因最有效的搜索策略通常事先未知,我们引入名为流体搜索的自适应程序,它使用组合式策略在搜索过程森林中动态分配固定评估预算。在评估任务中,流体搜索实现了最高总体搜索效率,与提前为每个任务给定最佳搜索结构的任务特定预言机性能相近。

英文摘要

AI-driven autonomous research (AR) systems are becoming increasingly effective across a broad range of tasks. Their performance, however, is still evaluated primarily by the quality of the final outcome. In this paper, we argue that the efficiency of the solution-search process is an equally important but often overlooked dimension of performance. A strong AR system should not only produce high-quality results, but also reach them with as small a budget as possible. Search efficiency will become increasingly important as AR expands from domains with inexpensive verification, such as mathematics and coding, to real-world scientific settings in which solution evaluation may require costly physical experiments. To capture this dimension, we propose evaluating AR systems using the area under the curve (AUC) of the Pareto frontier, alongside final outcome quality. We compare several families of search algorithms, including hill climbing, beam search, tree search, and evolutionary search, across twelve systems-optimization tasks. We find that no single search structure is consistently the most efficient. We also show that search efficiency and final outcome quality are distinct performance dimensions: a method that eventually achieves the best result may nevertheless improve slowly and consume substantially more evaluation budget before reaching that result. Because the most effective search policy is generally unknown in advance, we introduce an adaptive procedure called fluid search, which uses a portfolio bandit to dynamically allocate a fixed evaluation budget across a forest of search processes. Across the evaluated tasks, fluid search achieves the highest overall search efficiency, closely matching the performance of a per-task oracle that is given the best search structure for each task in advance.

URL PDF HTML 收藏
2607.24611 2026-07-28 cs.CV 新提交

Test-Time Adaptation via Dual Distillation for Videos Under Severe Distribution Shifts

通过双重蒸馏实现严重分布偏移下视频的测试时自适应

André Sacilotti, Samuel Felipe dos Santos, Jurandy Almeida

机构 * University of São Paulo(圣保罗大学) Federal University of São Carlos(圣卡洛斯联邦大学)

AI总结 研究针对视频在严重分布偏移下测试时自适应难题,提出双重蒸馏的测试时自适应框架TADD,依赖轻量级投影适配器及互补损失适应目标域,在多个视频动作识别基准测试中优于现有TTA基线,显著提升识别准确率。

详情
AI中文摘要

深度学习模型在计算机视觉任务中取得了先进性能,但应用于现实场景时因分布偏移会性能严重下降。测试时自适应(TTA)试图利用目标域无标签数据在推理时动态适应测试分布。然而,适应视频等连续、时间相关数据及目标域有严重偏移时,TTA仍是难题,相关研究少。为此提出通过双重蒸馏的测试时自适应(TADD)框架,依赖轻量级投影适配器弥合域差距。适配器模块在源域预训练,再用互补损失适应目标域,包括零样本蒸馏和目标蒸馏。基于冻结的CLIP主干,该方法在推理时仅引入轻量级投影适配器作为可更新组件。在三个视频动作识别基准上评估,结果表明该方法优于现有TTA基线,在UCF-HMDB、Daily-DA和Sports-DA上相比之前方法分别提升高达+3.81%、+2.63%和+3.03%。

英文摘要

Deep learning models have achieved state-of-the-art performance in several computer vision tasks. However, they experience severe performance degradation when applied to real-world scenarios due to unanticipated distribution shifts. Test-Time Adaptation (TTA) attempts to solve this problem by using unlabeled data from the target domain to dynamically adapt to the test distribution at inference time, without access to the source data. However, TTA remains a challenging problem when adapting to continuous, temporally correlated data, such as videos, and in scenarios where the target domain contains severe domain shifts. For this reason, few works in the literature explore TTA for videos under such extreme conditions. To overcome these limitations, we propose Test-time Adaptation via Dual Distillation (TADD), an online adaptation framework that relies on a lightweight projection adapter to bridge the domain gap. The adapter module is pre-trained on the source domain and then adapted to the target using our proposed complementary losses: (i) zero-shot distillation, which encourages alignment with the domain-agnostic features from a pre-trained vision-language model (VLM); and (ii) target distillation, which retains the source domain discriminative knowledge encoded in the pre-trained adapter. Built upon a frozen CLIP backbone, our method introduces this lightweight projection adapter as the sole updatable component during inference. We conducted extensive evaluations on three well-known video action recognition benchmarks: UCF-HMDB, Daily-DA, and Sports-DA. Our experiments in the closed-set scenario demonstrate that our method consistently outperforms state-of-the-art TTA baselines. Notably, our TTA approach improves upon previous methods by up to +3.81% on UCF-HMDB, +2.63% on Daily-DA, and +3.03% on Sports-DA.

URL PDF HTML 收藏
2607.24608 2026-07-28 cs.LG 新提交

Attribution and Uncertainty Behavior of Learned Residual Gyro Correction for Gyro-Stellar Estimation

用于陀螺恒星估计的学习残差陀螺校正的归因与不确定性行为

Mariela De Lucas Álvarez, Melvin Laux, Arthur de Freitas Precht, Maurice Martin, Edoardo Caroselli, Frank Kirchner, Alexander Fabisch

机构 * Robotics Innovation Center, German Research Center for AI (DFKI GmbH)(机器人创新中心,德国人工智能研究中心(DFKI有限公司)) Airbus Defence and Space GmbH(空客防务与航天有限公司)

AI总结 研究基于深度学习的陀螺仪偏差校正框架的不确定性分解与可解释性,用一维卷积神经网络预测校正,通过模型集合估计认知不确定性,经实验揭示特定轴行为及扰动影响,表明两种不确定性协同良好,为状态估计和故障检测提供见解。

Comments 21 pages, 9 Figures, EASi-Explimed Workshop within IJCAI 2026

详情
AI中文摘要

这项工作研究了基于深度学习的陀螺仪偏差校正框架中的不确定性分解和可解释性。训练一维卷积神经网络从包括陀螺仪和星跟踪器测量在内的多传感器输入中预测残余角速率校正。偏差校正被发送到具有飞行代表性的陀螺恒星估计器。该网络产生平均校正和与输入相关的(异方差)偶然不确定性,而认知不确定性则通过独立训练的模型集合来估计。所提出的方法在标称条件下进行训练,并在包括加性和时间相关噪声的标称和结构化扰动中进行评估。基于梯度的归因方法应用于校正和不确定性输出,从而能够分解驱动状态更新和不确定性估计的证据。通过汇总旋转轴和状态下的归因模式,我们揭示了特定轴的行为,并表征了结构化扰动如何影响偶然和认知不确定性之间的协作。不确定性分析表明,偶然不确定性随扰动强度增加,但分布重叠且不同状态下的校准不一致。另一方面,认知不确定性给出了一个清晰的信号,随着分布偏移的发生,这个信号会变得更清晰,表示模型之间的分歧更大。这些结果表明,偶然和认知不确定性协同工作良好,并且认知不确定性在区分标称和扰动操作条件方面表现更好。这些结果为基于混合学习的状态估计组件的行为提供了见解,并激发了将不确定性用于下游监测和故障检测的应用。

英文摘要

This work investigates uncertainty decomposition and explainability in a deep learning-based framework for gyroscope bias correction. A 1-D Convolutional Neural Network is trained to predict residual angular rate corrections from multi-sensor inputs, including gyroscope and star tracker measurements. The bias corrections are sent to a flight-representative Gyro-Stellar Estimator. The network produces both mean corrections and input-dependent (heteroscedastic) aleatoric uncertainty, while epistemic uncertainty is estimated via an ensemble of independently trained models. The proposed approach is trained under nominal conditions and evaluated in both nominal and structured perturbations that include additive and temporally correlated noise. Gradient-based attribution methods are applied to both the correction and uncertainty outputs, enabling a decomposition of the evidence that drives state updates and uncertainty estimates. By aggregating attribution patterns across rotational axes and regimes, we reveal axis-specific behaviors and characterize how structured perturbations influence the collaboration between aleatoric and epistemic uncertainty. Uncertainty analysis shows that aleatoric uncertainty increases with perturbation intensity, but the distributions overlap and the calibration is not consistent across regimes. On the other hand, epistemic uncertainty gives a clear signal that gets clearer as the distributional shift happens, showing that the models disagree more. These results show that aleatoric and epistemic uncertainty work well together and that epistemic uncertainty is better at distinguishing between nominal and perturbed operating conditions. The results provide insight into the behavior of hybrid learning-based state estimation components and motivate the use of uncertainty for downstream monitoring and fault detection.

URL PDF HTML 收藏
2607.24604 2026-07-28 cs.CL cs.AI 新提交

Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair

循环并非可靠性保障:智能代码修复的状态约束证据与类型化修订契约

Xueping Gao, Jianwei Yang, Qiang Yang

机构 * Alibaba Cloud(阿里云)

AI总结 研究编码智能体中生成-测试-修订循环的可靠性问题,通过实验揭示相关差距及问题,提出证据约束的类型化循环契约并实例化其可机械执行子集,实现规范与工件,非证明修复能力提升等。

Comments 11 pages, 4 figures, 6 tables

详情
AI中文摘要

在编码智能体中,生成-测试-修订循环很常见,但仅靠重复无法保证可靠性。我们研究了找到正确补丁与保留、验证并提交补丁之间的差距。通过对30个人类评估修复的密封五种子研究产生了900条三修订轨迹。强制修订下,当前正确性随当前轨迹从一次修订后的0.820降至两次修订后的0.673,不过一直正确的比例升至0.847。两项常见状态研究使用相同冻结程序的2430个分支以消除后处理风险集偏差。在预指定的14B复制中,陈旧轨迹对135个正确起始中的34个造成损害,而当前轨迹为4个,增加了22.2个百分点。前瞻性的540次推出策略消除了观察到的正确起始损害,但减少了错误起始修复且未达到联合标准。对24个错误和四个编码员堆栈的存储库实验揭示了下限效应和组件异质性但无显著影响。因此我们分离了准入、保存、有根据的认证、能力和活性。我们得出了一个证据约束的类型化循环契约,并在一个参考实现中实例化其可机械执行的子集,该实现将验证者证据与精确代码状态绑定,保留已验证的检查点,并发出可审计的准入收据。该实现是一个可执行规范和一致性工件,而非修复能力提高或校准验证者依赖性的证据。

英文摘要

Generate--test--revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95\% CI $[8.9,37.0]$, exact Holm $p=0.0337$). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.

URL PDF HTML 收藏
2607.24598 2026-07-28 cs.CV 新提交

QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment

QueenVIS:通过查询增强对视频实例分割仅图像训练的重新思考

Arian Kheirandish, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi

机构 * Amirkabir University of Technology (AUT)(伊朗阿米尔卡比尔理工大学) The University of Tokyo(日本东京大学)

AI总结 研究针对视频实例分割中仅图像训练方法的不足,提出QueenVIS框架,通过单帧训练时用辅助头丰富查询,推理时丢弃辅助头,以无训练方案保持时间身份,实验证明该方法有效提升分割精度,凸显增强查询稳定性对VIS的重要性。

Comments Project website: https://aut-aisl.github.io/QueenVIS-Project-Page/

详情
AI中文摘要

视频实例分割(VIS)要求模型在帧间检测、分割并跟踪对象身份,多数方法通过视频级监督强化时间一致性。以MinVIS为代表的仅图像训练方法对此提出挑战,将帧视为独立图像,仅在推理时关联实例,无需视频训练即可达到有竞争力的VIS。然而该领域已转向更复杂的视频训练跟踪器,依赖昂贵的身份一致注释,使仅图像方向未得到充分探索。诊断分析表明对象查询质量是瓶颈:仅在帧内定位对象训练的查询在帧间会漂移,破坏关联。QueenVIS引入以查询为中心的框架来强化仅图像训练的VIS。单帧训练期间,用两个辅助头丰富Mask2Former查询:一个特征预测损失使每个查询与其实例的池化主干描述符对齐,一个中心预测损失注入空间结构。推理时丢弃这两个头,不增加参数,通过无训练的查询传播和内存库方案保持时间身份。在YouTube-VIS和OVIS上,使用ResNet-50主干的QueenVIS比MinVIS有改进,在YouTube-VIS上提升高达+6.7 AP,在OVIS上提升+4.8 AP,在长序列YouTube-VIS分割上提升+10.3 AP。QueenVIS在YouTube-VIS上达到50.9 AP,与近期视频监督的最先进方法竞争,训练期间不处理单个视频片段。研究结果表明,增强对象查询的判别力和时间稳定性是VIS一个重要但未充分探索的方向。

英文摘要

Video instance segmentation (VIS) requires models to detect, segment, and track object identities across frames, and most methods enforce temporal consistency through video-level supervision. Image-only training approaches, with MinVIS as one prominent example, have challenged this assumption, reaching competitive VIS without video training by treating frames as independent images and associating instances only at inference. The field has nonetheless moved toward ever more elaborate video-trained trackers, which depend on costly identity-consistent annotations, leaving the image-only direction under-explored. A diagnostic analysis identifies object query quality as the bottleneck: queries trained only to localize objects within a frame drift apart across frames and destabilize association. QueenVIS introduces a query-centric framework for strengthening image-trained VIS. During single-frame training, we enrich Mask2Former queries with two auxiliary heads: a feature-prediction loss that aligns each query with the pooled backbone descriptor of its instance, and a center-prediction loss that injects spatial structure. Both heads are discarded at inference, adding zero parameters, and temporal identity is maintained by a training-free query-propagation and memory-bank scheme. On YouTube-VIS and OVIS with a ResNet-50 backbone, QueenVIS improves over MinVIS, up to +6.7 AP on YouTube-VIS, +4.8 AP on OVIS, and +10.3 AP on the long-sequence YouTube-VIS split. QueenVIS achieves 50.9 AP on YouTube-VIS and remains competitive with recent video-supervised state-of-the-art, without processing a single video clip during training. Our findings suggest that strengthening the discriminative power and temporal stability of object queries is an important, underexplored axis for VIS. Code and models: https://github.com/ArianKheir/QueenVIS

URL PDF HTML 收藏
2607.24591 2026-07-28 cs.CV 新提交

CameraAnything: Refilming Videos with Arbitrary Camera Control

CameraAnything:使用任意相机控制重新拍摄视频

Yixuan Li, Yanhong Zeng, Ka Leong Cheng, Jiayi Zhu, Hanlin Wang, Wen Wang, Yihao Meng, Hao Ouyang, Qiuyu Wang, Yue Yu, ZiDong Wang, Yiyuan Zhang, Yujun Shen, Dahua Lin

机构 * The Chinese University of Hong Kong(香港中文大学) Ant Group(蚂蚁集团) Tsinghua University(清华大学) Zhejiang University(浙江大学) The Hong Kong University of Science and Technology(香港科技大学)

AI总结 CameraAnything是用于相机控制视频编辑的统一框架,采用逐像素普吕克光线注入等方法联合控制相机参数,开发合成管道克服数据稀缺,能在单过程中实现多种视频编辑操作,为视频制作提供实用价值。

Comments Project page: https://yixuanli98.github.io/cameraanything/

详情
AI中文摘要

我们介绍了CameraAnything,这是首个用于相机控制视频编辑的统一框架,可对相机的内参和外参进行联合控制。现有方法要么依赖昂贵的3D重建来实现完整相机功能,要么将编辑限制在外参操作上。此外,内参和外参对视频外观的耦合影响使得解缠建模颇具挑战。为此,我们采用逐像素普吕克光线注入以及自注意力中分辨率感知的3D RoPE,在目标潜空间上构建相机条件和空间位置编码,以联合控制相机位置、焦距和原生分辨率编辑,无需裁剪或补绘。为克服配对训练数据的稀缺性,我们进一步开发了一个可扩展的合成管道,通过结构化多相机记录构建多样动态场景,并生成具有不同相机配置的同步视频。通过定制的正交训练策略,CameraAnything能够在单个生成过程中进行具有任意视点控制、焦距调整、分辨率适配和多镜头过渡的富有表现力的视频重新拍摄,为电影视频编辑和视频制作中的跨平台内容适配提供了强大的实用价值。

英文摘要

We introduce CameraAnything, the first unified framework for camera controlled video editing that enables joint control of both intrinsic and extrinsic camera parameters. Existing approaches either rely on expensive 3D reconstruction to achieve full camera functionality or restrict editing to extrinsic parameter manipulation. Moreover, the coupled influence of intrinsic and extrinsic parameters on video appearance makes disentangled modeling particularly challenging. To address this, we adopt per-pixel Plücker ray injection alongside resolution-aware 3D RoPE in self-attention, building both camera conditioning and spatial positional encoding on the target latent to jointly control camera position, focal length, and native resolution editing without cropping or outpainting. To overcome the scarcity of paired training data, we further develop a scalable synthetic pipeline that constructs diverse dynamic scenes through structured multi-camera recording and generates synchronized videos with varied camera configurations. With a tailored orthogonal training strategy, CameraAnything enables expressive video reshooting with arbitrary viewpoint control, focal length adjustment, resolution adaptation, and multi-shot transitions within a single generation process, offering strong practical value for cinematic video editing and cross-platform content adaptation in video production.

URL PDF HTML 收藏
2607.24589 2026-07-28 cs.AI 新提交

Artificial Intelligence and Innovation Ecosystem: Evolutionary Developments, Challenges, and Future Directions

人工智能与创新生态系统:进化发展、挑战及未来方向

Zhimin Zhang, Chengzhen Ma, Jia Chai, Rongxin Zhan, Huansheng Ning, Lingfeng Mao, Dan Zhang, Suiping Jiang

机构 * Beijing Institute of Computer Technology and Applications(北京计算机技术及应用研究所) School of Cyberspace Science and Technology, Beijing Institute of Technology(北京理工大学网络空间安全学院) School of Computer and Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院) Beijing Engineering Research Center for Cyberspace Data Analysis and Applications(北京网络空间数据分析与应用工程技术研究中心)

AI总结 研究人工智能与创新生态系统的关系,综合IE概念框架,引入AIIE概念,用进化视角分析AI在AIIE不同阶段作用,通过实例验证其定义及发展,最后审视AIIE未来挑战,为其发展确定新研究方向。

详情
AI中文摘要

创新生态系统(IE)的发展为经济整合、协同进步和共享成果提供了新范式。人工智能(AI)的兴起极大加速了全球数字化、信息化和智能化进程。探讨AI如何利用自身特性影响IE发展轨迹值得深入研究。鉴于AI在IE中日益突出的地位和作用,本文分析这种新形式,考察其对IE的独特贡献与潜在挑战。首先综合IE的概念框架并分解到物理、社会和思维空间的表现形式;从空间视角引入人工智能创新生态系统(AIIE)概念并探究AI赋予IE的特性;接着用进化视角分析AI在AIIE不同发展阶段的作用;然后通过企业发展实例验证AIIE定义的可行性、有效性和合理性,并从进化角度分析其发展;最后鉴于AI的固有局限,从四个角度审视AIIE未来面临的潜在挑战,旨在为其进一步发展确定新研究方向。

英文摘要

The development of the Innovative Ecosystem (IE) presents a new paradigm for economic integration, collaborative advancement, and shared achievements. The rise of Artificial Intelligence (AI) has significantly accelerated the global processes of digitization, informatization, and intelligence. Exploring how AI can leverage inherent characteristics to influence the development trajectory of IE is a topic that warrants further investigation. Given AI's increasing prominence and role within IE, the paper analyzes this new form, examining both AI's unique contributions to IE and its potential challenges. Firstly, the paper synthesizes the conceptual frameworks surrounding IE, decomposing them into manifestations in physical, social, and thinking spaces. Furthermore, the concept of Artificial Intelligence IE (AIIE) is introduced from a spatial perspective, with an exploration of the characteristics AI contributes to IE. Subsequently, the paper employs an evolutionary perspective to analyze the roles provided by AI during different development periods of AIIE. The paper then verifies the feasibility, effectiveness, and rationality of the AIIE's definition and analyzes AIIE development from an evolutionary perspective using enterprise development examples. Finally, acknowledging AI's inherent limitations, the paper examines potential challenges facing AIIE in the future from four perspectives, aiming to identify new research avenues for the further development of AIIE.

URL PDF HTML 收藏
2607.24588 2026-07-28 cs.AI 新提交

SIREN: Towards End-to-End Extreme-Weather Early Warning with Experience-Grounded LLM Agents

SIREN:借助基于经验的大语言模型智能体实现端到端极端天气预警

Hang Ni, Weijia Zhang, Fan Liu, Mengqian Lu, Hao Liu

机构 * The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)) The Hong Kong University of Science and Technology(香港科技大学) The World Sustainable Development Institute(世界可持续发展研究所)

AI总结 研究借助大语言模型智能体实现端到端极端天气预警。开发了包含多任务问答实例的SIREN-Bench基准,发现现有框架差距后,构建基于经验的SIREN框架,结合多种技术,实验证明其在预警程序和预警链上优于基线。

详情
AI中文摘要

极端天气预警对于减轻恶劣天气事件带来的社会、经济和环境风险至关重要。然而,以专家为中心的预警工作流程成本高昂、劳动密集且在整个预警到行动过程中难以扩展。尽管大语言模型智能体的最新进展实现了与天气相关任务的自动化,但现有研究仍集中在孤立的科学任务上,忽视了极端天气预警业务所需的相互依存流程链。为弥补这一差距,本研究通过大语言模型智能体研究自动化的端到端极端天气预警。首先开发了SIREN-Bench,这是一个全面的基准,包含19项任务中的600个问答实例,涵盖四个单独的预警程序和一个端到端预警链。对SIREN-Bench的评估揭示了现有天气智能体框架中的重大能力差距。这促使我们开发SIREN,一个受专家使用历史案例启发的基于经验的智能体框架,它将整合异构天气证据和工具的智能执行环境与一系列通过检索、技能提炼和预测建模利用历史案例的智能体 harnesses 相结合。大量实验表明,SIREN在单独的预警程序和端到端预警链上均优于天气智能体基线。

英文摘要

Early warning of extreme weather is essential for mitigating the societal, economic, and environmental risks posed by hazardous weather events. However, expert-centered warning workflows are costly, labor-intensive, and difficult to scale throughout the warning-to-action process. Although recent advances in Large Language Model (LLM) agents have enabled the automation of weather-related tasks, existing studies remain centered on isolated scientific tasks and overlook the chain of interdependent processes required for operational extreme-weather early warning. To bridge this gap, this study investigates automated end-to-end extreme-weather early warning through LLM agents. We first develop SIREN-Bench, a comprehensive benchmark comprising 600 question-answer instances across 19 tasks, and covering four individual warning procedures and an end-to-end warning chain. Evaluation on SIREN-Bench reveals substantial capability gaps in existing weather agent frameworks. This motivates us to develop SIREN, an experience-grounded agent framework inspired by experts' use of historical cases, which combines an agentic execution environment integrating heterogeneous weather evidence and tools with a family of agent harnesses that exploit historical cases through retrieval, skill distillation, and predictive modeling. Extensive experiments demonstrate that SIREN outperforms weather-agent baselines on both individual warning procedures and end-to-end warning chains.

URL PDF HTML 收藏
2607.24586 2026-07-28 cs.CL cs.AI 新提交

D-Score: A Spectral Hidden-State Signal for Hallucination Detection in Large Language Models

D分数:一种用于大语言模型中幻觉检测的谱隐藏状态信号

Bianca Raimondi, Davide Evangelista, Maurizio Gabbrielli, Elena Loli Piccolomini

机构 * Department of Computer Science and Engineering, University of Bologna(博洛尼亚大学计算机科学与工程系)

AI总结 研究大语言模型幻觉检测,提出基于隐藏激活几何结构的D分数,通过单次前向传播计算,用作幻觉分数。经实验验证,该分数是强大的隐藏状态信号,检测时无需外部验证器等,为幻觉检测提供新方法。

Comments Preprint. Under review

详情
AI中文摘要

大语言模型可能生成错误、无证据支持或与模型内部表示不一致的流畅文本。我们从隐藏激活的几何结构研究幻觉检测,并引入D分数,它是通过单次前向传播计算出的简单谱统计量。对于固定模型、层和容差参数,D分数计算隐藏激活矩阵中奇异值接近主导奇异值的奇异方向数量。我们将此量用作幻觉分数,当输入文本的D分数大于预定义量时将其分类为幻觉。动机是当模型处理与自身内部状态信息冲突的文本时,隐藏表示可能编码断言内容和某种反证据等,导致隐藏轨迹分布在更多奇异方向。我们通过轻量级谱论证形式化此直觉,并在FAVA注释和RAGTruth上评估结果检测器。实验表明D分数是用于幻觉检测的强大隐藏状态信号,无需外部验证器、检索步骤和多轮生成。

英文摘要

Large Language Models can produce fluent text that is false, unsupported by the available evidence, or inconsistent with information that appears to be internally represented by the model. We study hallucination detection from the geometry of hidden activations and introduce the D-Score, a simple spectral statistic computed from a single forward pass. For a fixed model, layer, and tolerance parameter, the D-Score counts how many singular directions of the hidden activation matrix have singular values that remain close to the leading one. We use this quantity as a hallucination score, classifying an input text as hallucinated when its D-Score is larger than a pre-defined quantity. The motivation is that, when a model processes a text that conflicts with information available in its own internal state, the hidden representation may encode both the asserted content and some form of counter-evidence, uncertainty, correction, or lack of support; this can make the hidden trajectory spread across additional singular directions. We formalize this intuition through a lightweight spectral argument and evaluate the resulting detector on FAVA-Annotation and RAGTruth. The experiments indicate that the D-Score is a strong hidden-state signal for hallucination detection, while requiring no external verifier, no retrieval step, and no multiple generations.

URL PDF HTML 收藏
2607.24577 2026-07-28 cs.LG cs.SE 新提交

Evaluating Fuzz Testing for Reinforcement Learning Agents

评估强化学习智能体的模糊测试

Zhibin Kang, Hanmo You, Dong Wang, Haiming Zheng, Junjie Chen

机构 * College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)

AI总结 研究针对强化学习智能体模糊测试评估差异问题,通过在三种环境下统一配置对五种方法及随机测试进行基准测试,从多角度评估,揭示不同方法特点,表明模糊测试能提升智能体鲁棒性等,还给出可操作指导。

详情
AI中文摘要

强化学习智能体越来越多地应用于机器人技术、自动驾驶和无人机控制等安全关键领域,意外行为可能导致严重后果。模糊测试是探索强化学习智能体庞大状态空间并暴露崩溃的一种有前途的方法。尽管已经提出了许多强化学习模糊测试方法,但现有研究在评估设置、基线和指标方面往往存在差异。为填补这一空白,我们进行了首次全面实证研究,从有效性、多样性、效率和实际效用四个互补角度系统评估强化学习模糊测试方法。我们在三个复杂度递增的环境(MountainCar、BipedalWalker和CARLA)中,在统一配置下对五种先进方法和随机测试进行基准测试,并进一步评估检测到的崩溃对智能体鲁棒性改进和安全监控的下游有用性。结果揭示了几个关键见解,如MDPFuzz等面向吞吐量的方法在崩溃发现方面具有卓越的有效性和效率,而SeqDivFuzz等明确旨在鼓励探索的方法在发现多样崩溃行为方面表现出色。我们还表明,模糊测试生成的崩溃可以显著提高智能体鲁棒性,并通过强大的跨方法泛化实现准确的安全监控。此外,我们为研究人员和从业者提炼了可操作的指导,强调了结合互补模糊测试策略和采用多层次多样性分析以实现更全面和实际的强化学习测试的好处。

英文摘要

Reinforcement Learning (RL) agents are increasingly deployed in safety-critical domains such as robotics, autonomous driving, and drone control, where unexpected behaviors may lead to severe real-world consequences. Fuzz testing has recently emerged as a promising method for exploring the vast state spaces of RL agents and exposing crashes. Although numerous RL fuzzing methods have been proposed, existing studies often differ in evaluation settings, baselines, and metrics, making it difficult to draw reliable conclusions about their relative effectiveness and practical usefulness. To address this gap, we present the first comprehensive empirical study that systematically evaluates RL fuzzing methods from four complementary perspectives: effectiveness, diversity, efficiency, and practical utility. We benchmark five state-of-the-art methods alongside random testing under unified configurations across three environments of increasing complexity (MountainCar, BipedalWalker, and CARLA), and further assess the downstream usefulness of detected crashes for agent robustness improvement and safety monitoring. Our results reveal several key insights. For instance,throughput-oriented methods like MDPFuzz demonstrate superior effectiveness and efficiency in crash discovery, while methods explicitly designed to encourage exploration like SeqDivFuzz excel at uncovering diverse crash behaviors. We also show that fuzzing-generated crashes can meaningfully improve agent robustness and enable accurate safety monitoring with strong cross-method generalization. Beyond these empirical findings, we distill actionable guidance for both researchers and practitioners, highlighting the benefits of combining complementary fuzzing strategies and adopting multi-level diversity analysis to achieve more comprehensive and practical RL testing.

URL PDF HTML 收藏
2607.24573 2026-07-28 cs.AI 新提交

LLM-SoccerArena: Benchmarking LLMs on Real-World Predictions in Sports

LLM - 足球竞技场:在体育领域的现实世界预测中对大语言模型进行基准测试

Jonas Schröder, Jonas Schweisthal, Oliver Müller, Markus Weinmann, Stefan Feuerriegel

机构 * MCML & LMU Munich(慕尼黑机器学习中心及慕尼黑大学) Paderborn University(帕德博恩大学) University of Cologne(科隆大学)

AI总结 研究旨在评估大语言模型预测现实世界体育赛事的能力,核心方法是引入LLM - 足球竞技场这一前瞻性实时基准,通过对2026年世界杯的评估,详细分析模型性能各维度表现,为LLMs预测性能提供新证据及开源平台。

详情
AI中文摘要

大语言模型(LLMs)越来越多地支持对不确定未来事件的决策,但评估其预测现实世界结果的能力仍然困难。现有基准通常是静态和回顾性的,无法测试LLMs如何在不确定性下综合信息预测未来事件。我们引入LLM - 足球竞技场,这是一个前瞻性实时基准,用于评估LLMs在结果未知前对现实世界体育赛事的预测能力。它提供了前瞻性实时基准协议、公共开源平台以及因子基准设计和赛事相关问题。通过对2026年国际足联世界杯的大规模评估展示了该基准,对模型在信息获取、提示策略和预测范围等方面的性能进行了详细分析,为最先进的LLMs的预测性能提供了新证据。例如,有网络访问权限的LLMs比没有的表现稍好。总体而言,LLM - 足球竞技场为未解决事件的前瞻性基准测试提供了灵活的开源平台,将持续更新并可直接应用于未来赛事。

英文摘要

Large language models (LLMs) increasingly support decisions about uncertain future events, yet evaluating their ability to forecast real-world outcomes remains difficult. In particular, existing benchmarks are typically static and retrospective, and therefore cannot test how information is synthesized by LLMs to predict future events under uncertainty. We introduce LLM-SoccerArena (https://llm-soccerarena.com), a prospective live benchmark that evaluates how well LLMs forecast real-world sports events before the outcomes are known. LLM-SoccerArena provides (1) a prospective live benchmark protocol, (2) a public open-source platform, and (3) a factorial benchmark design together with tournament-related questions (e.g., which team will win). LLM-SoccerArena automatically records timestamped, schema-validated forecasts of unresolved events, together with prompts, model versions, tool traces, and costs. The factorial design varies along four dimensions: (1) model version (e.g., GPT-5.5, Claude Opus 4.8); (2) information access; (3) prompting strategy, and (4) forecast horizon. We demonstrate LLM-SoccerArena through a large-scale evaluation of the 2026 FIFA World Cup, in which seven LLMs generated forecasts for all 104 matches and 15 tournament-related questions. We provide a detailed analysis of model performance across information access, prompting strategy, and forecast horizon. As a result, LLM-SoccerArena provides new evidence about the forecasting performance of state-of-the-art LLMs. For example, LLMs with web access outperform those without, but only by a small margin (i.e., a 0.023 improvement in Brier score). Overall, LLM-SoccerArena provides a flexible, open-source platform for prospective benchmarking of unresolved events. LLM-SoccerArena will be continuously updated, and can be directly applied to future national and international tournaments and league competitions.

URL PDF HTML 收藏
2607.24570 2026-07-28 cs.CV cs.AI 新提交

The Visual Bottleneck: Sparse-Frame Adaptation of MLLMs for Joint Spatial-Temporal Video Grounding

视觉瓶颈:用于联合时空视频定位的多模态大语言模型的稀疏帧适配

Jiameng Zhang, Srikanth Madikeri

机构 * University of Zurich(苏黎世大学)

AI总结 研究针对多模态大语言模型在视频定位中训练与部署条件不匹配问题,通过实验表明视觉特征提取是稀疏帧输入瓶颈,提出适配特定层及边界感知采样策略,证明训练策略对稀疏帧视频定位比模型规模更关键。

Comments 15 pages, 5 figures

详情
AI中文摘要

大规模视频平台每小时处理数百万上传内容,需要能定位视频中违规发生时间和地点的审核系统。处理每个帧在大规模下不可行,系统限于每个视频8到16帧的稀疏输入。然而,最先进的多模态大语言模型在数百帧的密集序列上预训练,导致训练和部署条件不匹配,性能严重下降。本文对训练策略进行系统实证研究以弥合时空视频定位差距。结果表明视觉特征提取是稀疏帧输入下的主要瓶颈。仅适配最后三个ViT层(占总参数4%),能达到68.8%的时间平均交并比,超过零样本8B密集输入模型12.8分。语言模型微调收益可忽略或为负。有时间边界时,边界感知采样策略Hybrid16比均匀采样进一步提高时间平均交并比26分。结论是对于稀疏帧视频定位,训练策略比模型规模更重要,微调后的2B模型始终优于零样本8B模型。

英文摘要

Large-scale video platforms process millions of uploads hourly, requiring moderation systems that can localize when and where policy violations occur within each video. Processing every frame is infeasible at scale, so systems are constrained to sparse inputs of 8 to 16 frames per video. Yet state-of-the-art multimodal large language models (MLLMs) are pretrained on dense sequences of hundreds of frames, creating a fundamental mismatch between training and deployment conditions. This mismatch causes severe performance collapse: the Qwen3-VL 8B model drops from 56.0% to 22.3% temporal mIoU when frames are reduced to 16, a 60.2% relative degradation. We present a systematic empirical study of training strategies to close this gap for spatial-temporal video grounding. Our results suggest that visual feature extraction is the dominant bottleneck under sparse-frame inputs. Adapting only the final three ViT layers, 4% of total parameters, achieves 68.8% temporal mIoU and surpasses a zero-shot 8B model using dense inputs by 12.8 points. Language model fine-tuning, by contrast, offers negligible or negative returns. A boundary-aware sampling strategy, Hybrid16, further improves temporal mIoU by 26 points over uniform sampling when temporal boundaries are available. We conclude that for sparse-frame video grounding, training strategy dominates model scale: a fine-tuned 2B model consistently outperforms a zero-shot 8B model, with or without dense frame access.

URL PDF HTML 收藏
2607.24563 2026-07-28 cs.AI cs.CR 新提交

TRACE-CTI: Auditable Post-Extraction Governance of TTP Claims with Knowledge Graphs

TRACE-CTI:利用知识图谱对TTP声明进行可审计的提取后治理

Federico Valletta, Giacomo Longo, Enrico Russo, Alessio Merlo

机构 * CASD – School of Advanced Defense Studies(高级国防研究学院)

AI总结 研究针对网络威胁情报报告提取器输出不可靠且缺乏相关依据的问题,提出TRACE-CTI框架,通过保留预测、聚合断言等操作进行治理。经实验评估,该框架能提升精度,还能直接回答多个相关问题,支持对TTP声明的可审计治理。

详情
AI中文摘要

安全运营中心越来越依赖于将网络威胁情报报告自动映射到MITRE ATT&CK,但提取器输出仍可能出错,且存储时往往缺少判断单个映射是否可信所需的证据、出处和验证历史。我们提出了TRACE-CTI,这是一个提取后声明治理框架,它保留运行级预测,将其聚合为配置级图断言,将设置去重后的确证具体化为共识断言,只公开有合规政策验证依据支持的图断言。该框架保留了原生证据粒度、完整提取出处、版本化信任决策和无损撤销历史。我们在两个包含65份报告和5303个句子的公共CTI语料库上评估TRACE-CTI,使用检索器和生成器家族的2×3受控矩阵,跨六个图版本逐步摄入。所有设置都无需修改模式即可合并;出处路径保持完整,操作范围保持不相交,每个可信图断言都有有效的合格验证依据。跨生成器家族设置对的输出多样性比同家族对更大。在最终图状态下,将设置支持从k≥1增加到六个设置一致,使与黄金标准对齐的精度从25.3%提高到90.6%,而召回率从88.2%降至16.3%。该图还直接回答了关于出处、信任、版本控制、依赖、分歧和审查队列的七个问题,而评估的最小扁平输出在不进行丰富或重新处理的情况下无法完全回答这些问题。这些结果支持对提取的TTP声明进行明确、可审计的治理;观察到的确证轨迹具有描述性,但未建立统计独立性或因果模型家族效应。

英文摘要

Security Operations Centers increasingly rely on automated mapping of Cyber Threat Intelligence reports to MITRE ATT&CK, yet extractor outputs remain fallible and are often stored without the evidence, provenance, and validation history needed to decide whether an individual mapping should be trusted. We present TRACE- CTI, a post-extraction claim-governance framework that preserves run-level Predictions, aggregates them into configuration-level GraphAssertions, materializes setup-deduplicated corroboration as ConsensusAssertions, and exposes only GraphAssertions backed by policy-compliant validation grounds. The framework retains native evidence granularity, complete extraction provenance, versioned trust decisions, and non-destructive revocation history. We evaluate TRACE-CTI on two public CTI corpora comprising 65 reports and 5,303 sentences, using a controlled 2 x 3 matrix of retrievers and generator families, incrementally ingested across six GraphVersions. All setups are incorporated without schema modification; provenance paths remain complete, operational scopes remain disjoint, and every trusted GraphAssertion has an active qualifying validation ground. Cross-generator-family setup pairs exhibit greater output diversity than same-family pairs. At the final graph state, increasing setup support from k >= 1 to six-setup unanimity raises gold-aligned precision from 25.3% to 90.6%, while recall decreases from 88.2% to 16.3%. The graph also directly answers seven questions about provenance, trust, versioning, dependency, disagreement, and review-queue that the evaluated minimal flat output cannot fully answer without enrichment or reprocessing. These results support explicit, auditable governance of extracted TTP claims; the observed corroboration trajectory is descriptive and does not establish statistical independence or a causal model-family effect.

URL PDF HTML 收藏