arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 1621
2607.09661 2026-07-13 cs.CV 新提交

PanoWorld: Real-World Panoramic Generation

全景世界:真实世界全景生成

Haoyuan Li, Dizhe Zhang, Yuemei Zhou, Xiangkai Zhang, Haoran Feng, Xiaofan Lin, Wenjie Jiang, Bo Du, Ming-Hsuan Yang, Lu Qi

发表机构 * Insta360 Research(影石创新科技研究院) Institute of Automation Chinese Academy of Sciences(中国科学院自动化研究所) Tsinghua University(清华大学) Wuhan University(武汉大学) UC Merced(加州大学默塞德分校)

AI总结 该研究旨在解决全景世界模型的远程记忆挑战,提出PanoWorld,利用旋转等变特性简化相机轨迹,通过DPRC和GMA进行建模与记忆增强,经三阶段训练优化,构建World360数据集验证其有效性,优于其他方法且将公开模型、代码和数据集。

Comments Project page: https://lihaoy-ux.github.io/panoworld-page/ Code:https://github.com/Insta360-Research-Team/PanoWorld

详情
AI中文摘要

在这项工作中,我们旨在通过利用全向表示的旋转等变特性来应对全景世界模型中的远程记忆挑战,其中旋转可视为隐式几何变换。基于此,我们提出了PanoWorld,它通过固定航向将相机轨迹简化为平移,用于当前动作建模和远程记忆,借助密集全景光线条件(DPRC)和几何感知记忆增强(GMA)。然后,引入了一个三阶段训练管道来逐步优化每个组件。为了在现有数据集相对稳定的大规模空间变化和多样光照条件下更好地评估物理一致性,我们构建了World360数据集,它由通过全景无人机收集的真实世界视频片段和高质量模拟片段组成。在World360上的实验证明了PanoWorld的有效性,其性能大幅优于替代方法。模型、训练代码和数据集将公开可用。更多信息可在项目页面查看。

英文摘要

In this work, we aim to address the challenge of long-range memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing datasets are relatively stable, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.

URL PDF HTML
2607.09655 2026-07-13 cs.CV 新提交

OpenLongTail: Generative Scaling of Long-Tail Driving Data

OpenLongTail:长尾驾驶数据的生成式扩展

Lulin Liu, Nuo Chen, Yan Wang, Bangya Liu, Wenyan Cong, Hezhen Hu, Boris Ivanovic, Hao Wang, Ziyao Zeng, Xinyu Gong, Yang Zhou, Zixiang Xiong, Dilin Wang, Zhangyang Wang, Weisong Shi, Ruohan Zhang, Marco Pavone, Zhiwen Fan

发表机构 * Texas A&M University(德克萨斯农工大学) NVIDIA(英伟达) UW–Madison(威斯康星大学麦迪逊分校) UT Austin(德克萨斯大学奥斯汀分校) Yale University(耶鲁大学) Adobe(奥多比公司) Meta(元公司) University of Delaware(特拉华大学) Stanford University(斯坦福大学)

AI总结 研究针对长尾驾驶数据稀缺影响策略扩展的问题,提出开源生成数据引擎OpenLongTail,通过姿态外推视图合成管道及普吕克射线几何增强,合成异构数据提升闭环驾驶稳健性,验证了其多方面有效性。

Comments Project page: https://openlongtail.github.io/

详情
AI中文摘要

扩展稳健的驾驶策略从根本上受到策划数据集中边缘情况稀缺的限制。现实世界不断捕捉这些关键事件,但从异构源收集时,此类长尾事件仍未得到充分利用。具体而言,多样但有价值的野外长尾视频缺乏训练策略模型所需的全视图覆盖,常缺少多视图姿态或仅来自单目行车记录仪。这种模态差距阻碍了这些普遍观察结果转化为用于长尾泛化的可扩展训练数据。我们引入了OpenLongTail,一个用于在长尾事件下扩展自动驾驶策略的开源生成数据引擎。为了将异构数据源转换为对策略学习有用的视图对齐且时间连贯的多视图资产,我们开发了一个基于姿态的外推视图合成管道来生成缺失视图。我们还通过将普吕克射线几何注入可扩展生成引擎,进一步增强新生成视图的跨视图一致性和时间对齐。通过合成异构长尾数据,我们观察到在处理长尾事件时闭环驾驶稳健性有显著提高。通过测量外推视图合成和姿态指标,我们验证了OpenLongTail在视觉保真度、跨视图一致性和自我轨迹恢复方面的有效性。

英文摘要

Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.

URL PDF HTML
2607.09654 2026-07-13 cs.CV cs.AI 新提交

Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models

十年视觉语言人工智能模型中准确性和视觉认知错误的演变

Shravan Murlidaran, Miguel P. Eckstein

发表机构 * Psychological & Brain Sciences, University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校心理与脑科学系) Department of Computer Science, University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校计算机科学系) Department of Electrical and Computer Engineering, University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校电气与计算机工程系)

AI总结 研究十年间视觉语言模型进展,引入CSB数据集,评估模型在其上及MS-COCO样本中的准确性与视觉认知错误类型,发现MLLM消除简单与复杂场景描述准确性差距,几乎消除多数错误类型,为模型发展提供全面评估。

详情
AI中文摘要

在过去十年中,视觉语言模型(VLM)在视觉推理方面取得了显著进展。大多数评估使用的是简单场景(MS-COCO),未展示复杂的人类互动或行为,仅以少数未经策划的人类描述作为基准,且未关注模型的错误类型。本文引入了包含100张描绘复杂社会互动/行为图像的复杂社会行为(CSB)数据集。分析了2017年至2025年十年间VLM(四个预多模态大语言模型、MLLM和五个MLLM)场景描述的进展。在CSB数据集和MS-COCO样本上评估了模型和20个人类描述相对于黄金标准的准确性。分析了五种视觉认知错误类型:对象检测、识别、幻觉、场景理解和空间依赖性。CSB数据集在场景描述准确性方面比MS-COCO有更显著的提高,预MLLM的准确性远低于排名垫底的人类描述,而MLLM的准确性与排名靠前的人类描述相似。表明MLLM消除了简单MS-COCO场景和描绘复杂行为(CSB)场景之间在场景描述准确性上的差距。MLLM几乎消除了测试数据集中的所有错误类型,除了偶尔在场景描述中依赖与人类不同的图像区域(空间依赖性错误)。还表明检测、识别和幻觉错误对场景描述准确性影响最大。这些发现更全面地评估了视觉语言模型在过去十年中的进展。

英文摘要

Vision language models (VLMs) have made remarkable progress in visual reasoning during the last decade. Most evaluations have used simple scenes (MS-COCO) that do not showcase complex human interactions or behaviors, only a handful of non-curated human descriptions as a benchmark, and have not focused on understanding the model's error types. Here, we introduce the Complex Social Behavior (CSB) dataset, containing 100 images depicting complex social interactions/behaviors. We analyze the progression of scene descriptions over a decade (2017-2025) of VLMs (four pre-Multimodal Large Language Models, MLLMs, and five MLLMs). We evaluate the accuracy of the models and 20 human descriptions relative to a gold standard on the CSB dataset and on a sample from MS-COCO. We analyzed five visual-cognitive error types: object detection, recognition, hallucination, scene understanding, and spatial dependence. The CSB dataset showed a more pronounced improvement than MS-COCO in scene description accuracy, with pre-MLLMs achieving much lower accuracy than the bottom-ranked human descriptions and MLLMs attaining accuracies similar to the top-ranked human descriptions. We show that MLLMs have eliminated the gap in scene description accuracy between simpler MS-COCO scenes and scenes depicting complex behaviors (CSB). MLLMs have almost eliminated all error types in our tested datasets, except for occasionally relying on different image regions for scene descriptions than humans do (spatial dependence error). We also show that detection, recognition, and hallucination errors have the highest impact on scene description accuracy. Together, our findings provide a more thorough evaluation of how visual language models have advanced over the last decade.

URL PDF HTML
2607.09650 2026-07-13 cs.CV 新提交

Revisiting Euler-Angle Regression with Kolmogorov-Arnold Networks

用柯尔莫哥洛夫 - 阿诺德网络重新审视欧拉角回归

Yangting Sun, Zijun Cui, Yufei Zhang

发表机构 * Michigan State University(密歇根州立大学)

AI总结 研究欧拉角回归难题,提出结合范围感知欧拉建模与柯尔莫哥洛夫 - 阿诺德网络的新框架,经理论分析和实验验证,该框架在控制旋转回归等多方面能提升精度、收敛性和效率。

详情
AI中文摘要

在许多现实世界系统中,如关节机器人和生物力学模型,旋转在关节空间中定义并由有界范围的欧拉角自然参数化。然而,回归欧拉角仍具挑战性,因其不连续性和奇异性常使训练不稳定。本文重新审视欧拉角回归,表明其有效性关键取决于旋转表示、回归架构和域约束间的相互作用。引入新框架,将范围感知欧拉建模与柯尔莫哥洛夫 - 阿诺德网络(KAN)结合,KAN用可学习单变量函数取代固定节点激活。理论分析表明有界欧拉范围促使回归函数具有近加性结构,有利于KAN的加性函数形式,实验也证实了这一趋势。在控制旋转回归、物体姿态估计及机器人和人类逆运动学上的大量实验表明在精度、收敛性和效率上有持续改进。代码将公开。

英文摘要

In many real-world systems, including articulated robots and biomechanical models, rotations are defined in joint space and naturally parameterized by Euler angles with bounded ranges. Yet regressing Euler angles remains challenging, as their discontinuities and singularities often destabilize training. In this work, we revisit Euler-angle regression and show that its effectiveness depends critically on the interaction between rotation representation, regression architecture, and domain constraints. We introduce a new framework that combines range-aware Euler modeling with Kolmogorov-Arnold Networks (KAN), which replace fixed node-wise activations with learnable univariate functions on edges. We further provide theoretical analysis indicating that bounded Euler ranges motivate a near-additive structure in the regression function, which favors the additive functional form of KAN, and we confirm this trend empirically. Extensive experiments on controlled rotation regression, object pose estimation, and robotic and human inverse kinematics demonstrate consistent improvements in accuracy, convergence, and efficiency. The code will be publicly available.

URL PDF HTML
2607.09649 2026-07-13 cs.AI 新提交

ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI

ConceptSMILE:审计基于概念的可解释人工智能的可信度

Mohadeseh Mollapour, Koorosh Aslansefat, Zeinab Dehghani, Bhupesh Kumar Mishra, Tejal Shah, Zhibao Mian

发表机构 * University of Hull(赫尔大学) Newcastle University(纽卡斯尔大学)

AI总结 研究基于概念的可解释人工智能中概念级输出的可信度问题,提出ConceptSMILE审计框架,通过扰动输入区域等方法评估可靠性,在视网膜眼底图像上评估发现不同概念和路径可靠性有差异,为评估此类人工智能可信度提供独立审计层。

详情
AI中文摘要

基于概念的可解释人工智能能使模型推理更易理解,但概念级输出并非自动可信。我们引入ConceptSMILE,这是一个基于模型无关扰动的审计框架,用于评估基于概念的解释的可靠性。它扩展了基于扰动的逻辑,从特征或区域级归因到对人类可理解的概念解释的审计。通过对输入区域进行扰动、测量概念响应变化等步骤来评估可靠性。我们在视网膜眼底图像上评估,结果表明不同概念和路径的可靠性不同,ConceptSMILE为评估基于概念的可解释人工智能的可信度提供了独立审计层。

英文摘要

Concept-based explainable artificial intelligence (AI) can make model reasoning more human-understandable, but concept-level outputs are not automatically trustworthy. We introduce ConceptSMILE, a model-agnostic perturbation-based auditing framework for evaluating the reliability of concept-based explanations. Rather than replacing SMILE, ConceptSMILE extends its perturbation-based logic from feature- or region-level attribution to the auditing of human-understandable concept explanations. The framework perturbs input regions, measures concept-response shifts, applies locality weighting, and fits an XGBoost surrogate to approximate local concept behaviour. Reliability is assessed through attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency. We evaluate ConceptSMILE on retinal fundus images by comparing MedSAM-derived visual concepts with VLM-based semantic concepts. Results show that reliability varies across concepts and pathways: MedSAM achieves stronger spatial attribution and the highest surrogate fidelity ($R^2 = 0.8503$, $R_w^2 = 0.8465$), while the VLM pathway shows stronger vessel faithfulness and stronger stability under selected artefact conditions. ConceptSMILE provides an independent audit layer for evaluating the trustworthiness of concept-based XAI.

URL PDF HTML
2607.09648 2026-07-13 cs.RO 新提交

B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations

B样条策略:通过B样条动作表示加速操纵策略

Xiaoshen Han, Haoyu Xiong, Haonan Chen, Chaoqi Liu, Antonio Torralba, Yuke Zhu, Yilun Du

发表机构 * Harvard(哈佛大学) MIT(麻省理工学院) UT Austin(德克萨斯大学奥斯汀分校)

AI总结 研究提出B样条策略,将动作参数化为连续B样条曲线,可集成到标准策略学习管道,通过直接预测参数实现。实验表明该策略能显著减少任务完成时间,在模拟和现实任务中优于基线方法且成功率高。

详情
AI中文摘要

在这项工作中,我们提出了B样条策略(BSP),一种为加速机器人操纵策略而设计的动作表示。BSP不是预测离散时间动作块,而是将动作参数化为由一组节点和控制点定义的连续B样条曲线。这种表示产生平滑、时间连续的轨迹,可在时间上缩放并由低级控制器以更高频率和速度执行。我们表明,通过直接预测B样条参数,B样条参数化动作可无缝集成到标准策略学习管道中。在模拟和现实世界任务上的实验表明,BSP显著减少任务完成时间,在保持高成功率的同时比基线方法有大幅改进。

英文摘要

In this work, we present B-spline Policy (BSP), an action representation designed for accelerating robot manipulation policies. Rather than predicting discrete-time action chunks, BSP parameterizes actions as continuous B-spline curves defined by a set of knots and control points. This representation yields smooth, time-continuous trajectories that can be temporally scaled and executed by low-level controllers at higher frequencies and speeds. We show that B-spline-parameterized actions can be seamlessly integrated into standard policy learning pipelines by directly predicting B-spline parameters. Experiments on simulated and real-world tasks demonstrate that BSP significantly reduces task completion time, achieving substantial improvements over baseline methods while maintaining strong success rates. More results: https://b-spline-policy.github.io

URL PDF HTML
2607.09641 2026-07-13 cs.LG cs.AI 新提交

Semantic Pareto-DQN: A Multi-Objective Reinforcement Learning Framework for Financial Anomaly Detection

语义帕累托深度Q网络:一种用于金融异常检测的多目标强化学习框架

Cláudio Lúcio do Val Lopes, Lucca Machado da Silva

发表机构 * A3Data(A3数据公司)

AI总结 针对金融异常检测的类不平衡问题,提出语义帕累托深度Q网络框架,合成交易特征,优化向量奖励,映射帕累托前沿,经实验验证可打破零召回陷阱,提升少数类召回率,为金融异常发现提供新途径。

Comments BRACIS 2026 - 36th Brazilian Conference on Intelligent Systems

详情
AI中文摘要

金融异常检测面临极端类不平衡问题,导致传统单目标算法出现“欺诈崩溃”,默认选择多数类,无法平衡异常拦截与客户摩擦。为避免扭曲数据重采样来克服此问题,我们提出语义帕累托深度Q网络(Semantic Pareto-DQN)这一多目标强化学习框架。该方法将异构交易特征合成连贯的自然语言叙述,由大语言模型编码,产生稳健、尺度不变的状态表示。智能体优化明确解耦金融效能、操作摩擦和语义发现的向量奖励。通过映射连续帕累托前沿,系统动态应对错过异常与误报的不对称成本。在电子商务欺诈和UCI信用数据集上的实证评估表明,语义帕累托深度Q网络成功打破零召回陷阱,与标量化基线相比,实现了更高的少数类召回率,为金融异常发现提供了一种替代方案,以换取有限的操作摩擦。

英文摘要

Financial anomaly detection suffers from extreme class imbalance, causing traditional single-objective algorithms to exhibit ``fraud collapse'', defaulting to the majority class and failing to balance anomaly interdiction with customer friction. To overcome this without distortive data resampling, we propose the Semantic Pareto-DQN, a multi-objective reinforcement learning framework. Our approach synthesizes heterogeneous transaction features into cohesive natural-language narratives, encoded by large language models, thereby producing a robust, scale-invariant state representation. The agent optimizes a vectorial reward that explicitly decouples financial efficacy, operational friction, and semantic discovery. By mapping the continuous Pareto frontier, the system dynamically navigates the asymmetric costs of missed anomalies versus false positives. Empirical evaluations across E-Commerce fraud and UCI Credit datasets show that semantic Pareto-DQN successfully shatters the zero-recall trap. It achieves superior minority-class recall compared to scalarized baselines, providing an alternative to trade bounded operational friction for financial anomaly discovery.

URL PDF HTML
2607.09630 2026-07-13 cs.CV 新提交

The Effects of Synthetic Data and Label Distribution on Canola Branch Counting

合成数据和标签分布对油菜分枝计数的影响

Amirsalar Darvishpour, Mikolaj Cieslak, Adam Runions

发表机构 * University of Calgary(卡尔加里大学)

AI总结 研究合成数据和标签分布对油菜分枝计数的影响,通过校准的L系统植物模型训练ResNet - 18模型,独立改变合成与真实图像比例及标签分布,发现不同比例和分布对性能有不同影响,得出最佳参数和简单替代方案。

Comments 5 pages, 4 figures, submitted to EPA 2026

详情
AI中文摘要

收集带注释的植物图像用于自动表型分析通常既缓慢又昂贵。模拟生长发育的植物模型可以生成带有精确标签的无限合成图像。然而,先前的研究表明,纳入合成数据是否能提高性能取决于合成图像与真实图像的比例以及合成数据集的标签分布。为了系统地量化这两个因素,我们使用校准的L系统植物模型在油菜分枝计数任务上训练ResNet - 18模型。我们独立改变每个因素。合成与真实图像比例在1:5到1:22之间时性能普遍提高;最佳比例(1:7)相比仅使用真实数据训练,平均绝对差降低了7.6%。对于标签分布,均匀的合成分布效果很差(绝对差约为1.70);向真实分布插值90%时绝对差为0.927,而对真实标签分布进行高斯平滑得到最佳总体结果(绝对差0.912,相比仅使用真实数据提高了14.7%)。每个标签最少10张合成图像是一种有适度提升的简单替代方案,而每个标签100张则会过度校正并损害性能。

英文摘要

Collecting annotated plant images for automated phenotyping is often slow and expensive. Plant models simulating growth and development can generate unlimited synthetic images with exact labels. However, previous work has established that whether incorporating synthetic data improves performance depends on the ratio of synthetic to real images and the label distribution of the synthetic dataset. To systematically quantify both factors, we train ResNet-18 models on a canola branch-counting task using a calibrated L-system plant model. We vary each factor independently. Synthetic-to-real ratios of 1:5 to 1:22 broadly improve performance; the best ratio (1:7) reduces mean absolute difference by 7.6% over real-only training. For label distribution, a uniform synthetic distribution is strongly suboptimal (abs. diff. of approximately 1.70); interpolating 90% toward the real distribution yields abs. diff. 0.927, whereas Gaussian smoothing of the real label distribution yields the best overall result (abs. diff. 0.912, a 14.7% improvement over real-only). A minimum of 10 synthetic images per label offers a simpler alternative with modest gains, while 100 per label over-corrects and hurts performance.

URL PDF HTML
2607.09623 2026-07-13 cs.CL cs.AI 新提交

Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026

通过置信度校准和增量推理实现特定任务的多模态问答代理,用于QANTA 2026

Nirjhar Das, Md. Al-Mamun Provath

发表机构 * Department of Computer Science Engineering, Chittagong University of Engineering \& Technology, Chattogram, Bangladesh

AI总结 针对QANTA 2026共享挑战,开发特定任务双代理架构,抢答代理用带置信度校准的模型及数值推理策略,加分代理用多种推理整合信息,强调仅托管环境下的高效推理与校准,系统取得高分,证明轻量级策略在资源受限问答中性能强。

Comments 10 pages, 1 figure. Accepted at the EMM-QA 2026 Workshop, ICML 2026 (Non-Archival). Rank #1 overall system in the QANTA 2026 Challenge

详情
AI中文摘要

我们提交了在ICML 2026高效多模态问答研讨会上的QANTA 2026共享挑战的成果。QANTA评估多模态知识竞赛系统,该系统在现实效率约束下根据增量显示的文本和图像回答金字塔式问题。挑战包括两个不同任务:抢答问题,需在不确定时决定何时回答;加分问题,强调准确答案选择和人工采纳。为实现不同目标,我们开发特定任务的双代理架构。抢答代理使用带有置信度校准回答的GPT-4o-mini-class模型及特定领域数值推理策略减少过度自信预测;加分代理使用带有导入感知推理、结构化关系推理和多模态证据整合的GPT-4o-class模型。我们的方法强调在仅托管环境中的高效推理策略和置信度校准。我们的系统在排行榜上获得最高分0.402,包括抢答分数0.238和加分效应分数0.164。结果表明轻量级、特定任务推理策略能在资源受限的多模态问答基准测试中表现出色。

英文摘要

We present our submission to the QANTA 2026 shared challenge at the ICML 2026 Workshop on Efficient Multimodal Question Answering (EMM-QA). Quanta evaluates multimodal quizbowl systems that answer pyramid-style questions from incrementally revealed text and accompanying images while operating under realistic efficiency constraints. The challenge consists of two distinct tasks: Tossup questions, which require deciding when to answer under uncertainty, and Bonus questions, which emphasize accurate answer selection and human adoption. To address these differing objectives, we develop a task-specific two-agent architecture. Our Tossup agent utilizes a GPT-4o-mini-class model (referred to as GPT-4.1-mini in the competition logs) with confidence-calibrated answering and a domain-specific numeric reasoning policy that reduces overconfident predictions from isolated quantitative clues. Our Bonus agent uses GPT-4o-class model (referred to as GPT-4.1) with leadin-aware reasoning, structured relational reasoning, and multimodal evidence integration to improve exact answer selection. Rather than relying on a retrieval pipeline or model ensembles, our approach emphasizes efficient reasoning policies and confidence calibration within a hosted-only environment. Our system achieved the highest overall leaderboard score of 0.402, including a Tossup score of 0.238 and a Bonus Effect score of 0.164. The results demonstrate that lightweight, task-specific reasoning strategies can provide strong performance on resource-constrained multimodal question answering benchmarks.

URL PDF HTML
2607.09600 2026-07-13 cs.AI cs.CL 新提交

Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation

Agora:通过基于拍卖的任务分配增强大语言模型智能体推理

Kaiji Zhou, Ales Leonardis, Yue Feng

发表机构 * University of Birmingham(伯明翰大学)

AI总结 研究旨在增强LLM智能体推理能力,提出Agora框架,通过基于拍卖的机制动态分配任务,将推理步骤视为可交易项目让智能体依纠正能力投标,实验表明该框架在多基准测试中表现优且能实现成本-质量权衡。

Comments Preprint. 12 pages, 4 figures

详情
AI中文摘要

增强大语言模型(LLM)智能体的推理能力需要有效编排各种专家模型和工具。然而,现有框架通常基于任务与专家模型或工具功能之间的粗粒度匹配来调用应用程序编程接口(API),而忽略了功能相似替代方案之间的性能可变性和成本效率等关键因素。为解决此问题,我们提出了Agora框架,该框架引入了一种激励兼容的拍卖机制,用于将任务动态分配给专家模型和工具。通过将推理步骤视为可交易的项目,Agora使智能体能够根据其纠正后的能力进行投标,确保关键逻辑被路由到最有能力的求解器,而不是最过度自信的求解器。在五个基准测试中的评估表明,在可比候选池下,Agora优于匹配的单模型、路由和级联基线,同时通过单个拍卖参数展现出可控的成本-质量权衡。

英文摘要

Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools. However, existing frameworks typically call APIs based on coarse-grained matching between tasks and the functions of expert models or tools, while overlooking critical factors such as performance variability and cost efficiency among functionally similar alternatives. To address this, we propose Agora, a framework that introduces an incentive-compatible auction mechanism for dynamically allocating tasks to expert models and tools. By treating reasoning steps as tradeable items, Agora enables agents to bid based on their rectified competence-ensuring that critical logic is routed to the most capable solver rather than the most overconfident one. Evaluations across five benchmarks show that Agora improves over matched single-model, routing, and cascade baselines under comparable candidate pools, while exposing a controllable cost-quality trade-off through a single auction parameter.

URL PDF HTML
2607.09598 2026-07-13 cs.CL 新提交

Tokenizer Transplantation: Mitigating Autoregressive Collapse in Edge-Efficient Bengali ASR

分词器移植:缓解边缘高效孟加拉语语音识别中的自回归崩溃

Sanjid Hasan, Md. Abdur Rahman

发表机构 * Useful Sensors(有用传感器公司)

AI总结 研究针对轻量级孟加拉语语音识别模型失败问题,提出用BanglaBERT WordPiece词汇替换解码器词汇并调整矩阵大小的方法,经实验改进后模型在数据集上有竞争力,提供了紧凑ASR模型跨脚本适配的可扩展蓝图。

Comments 5 pages, 2 figures. Accepted as a poster at the MusIML Workshop, ICML 2026

详情
AI中文摘要

轻量级语音识别模型对边缘部署至关重要,但像Moonshine这样高度优化的架构在孟加拉语等形态丰富的非拉丁语上常失败。本研究将其原因归结为以英语为中心的字节级分词器,它将孟加拉语单词拆分为高生育率字节链并在推理时引发自回归崩溃。为此提出新颖的词汇移植管道,用孟加拉语脚本的BanglaBERT WordPiece词汇替换解码器词汇并调整相应令牌嵌入矩阵大小。实验结果表明令牌生育率从9.16降至1.30,自回归序列长度减少85.8%,完全缓解了解码不稳定性。在882小时的Lipi - Ghor数据集上评估时,改进后的架构实现了有竞争力的21.54%的字错误率(WER)和0.0053的实时因子(RTF)。最终,本研究为紧凑ASR模型的跨脚本适配提供了可扩展、可重现的蓝图,无需资源密集型预训练。

英文摘要

Lightweight speech recognition models are critical for edge deployment, yet highly optimized architectures like Moonshine often fail on morphologically rich, non-Latin languages such as Bengali. This study identifies the root cause of this failure as the model's English-centric byte-level tokenizer, which fragments Bengali words into high-fertility byte chains and triggers catastrophic autoregressive collapse during inference. To resolve this, a novel vocabulary transplantation pipeline is proposed to replace the decoder vocabulary with the native-script BanglaBERT WordPiece vocabulary and resize the corresponding token embedding matrix. Experimental results demonstrate a reduction in token fertility from 9.16 to 1.30. By decreasing autoregressive sequence length by 85.8%, decoding instability is entirely mitigated. When evaluated on the 882-hour Lipi-Ghor dataset, the modified architecture achieves a competitive 21.54% Word Error Rate (WER) and a Real-Time Factor (RTF) of 0.0053. Ultimately, this research provides a scalable, reproducible blueprint for cross-script adaptation of compact ASR models without the need for resource-intensive pre-training.

URL PDF HTML
2607.09590 2026-07-13 cs.RO cs.AI 新提交

PAC-ACT: Post-training Actor-Critic for Action Chunking Transformers

PAC-ACT:用于动作分块变换器的训练后演员评论家方法

Yujie Pang, Zudong Li

发表机构 * LeRobot

AI总结 针对精密工业接触操纵问题,提出PAC-ACT框架,通过在块级别重新制定策略优化、构建特定架构并引入混合行为先验约束,提升了机器人策略在姿态扰动和接触力约束下的性能,实验验证了其有效性。

详情
AI中文摘要

精密工业接触操纵需要在姿态扰动和接触力约束下有可靠的机器人策略。视觉-语言-动作模型具有广泛的通用性,但通常会带来高推理延迟和高GPU内存成本,而视觉动作分块策略更适合实时工业控制。然而,这些策略通常通过行为克隆进行训练,在富含接触的任务中会受到分布偏移的影响。本文提出了PAC-ACT,一种用于预训练动作分块变换器策略的强化学习训练后框架。PAC-ACT在块级别重新制定策略优化,构建了一个ACT转移的演员评论家架构,并引入了混合行为先验约束,以在在线微调期间保留预训练的动作分布。在工业精密接触基准上的实验表明,PAC-ACT提高了任务成功率、接触稳定性和力安全性,同时保持了低延迟和低GPU内存使用。在轮廓任务上,PAC-ACT显著降低了峰值接触力,并将60N以上力读数的比例降低了46倍。稀疏奖励消融进一步表明,所提出的行为先验约束能够在随机初始姿态下进行有效探索。

英文摘要

Precision industrial contact manipulation requires reliable robot policies under pose perturbations and contact-force constraints. Vision-language-action models offer broad generalization but often introduce high inference latency and GPU-memory cost, while vision-action chunking policies are more suitable for real-time industrial control. However, these policies are usually trained by behavior cloning and suffer from distribution shift in contact-rich tasks. This paper proposes PAC-ACT, a reinforcement-learning post-training framework for pretrained Action Chunking Transformer policies. PAC-ACT reformulates policy optimization at the chunk level, constructs an ACT-transferred actor-critic architecture, and introduces a hybrid behavior-prior constraint to preserve the pretrained action distribution during online fine-tuning. Experiments on industrial precision-contact benchmarks show that PAC-ACT improves task success, contact stability, and force safety while retaining low latency and low GPU-memory usage. On the Contour task, PAC-ACT significantly reduces peak contact force and decreases the proportion of force readings above 60 N by 46 times. Sparse-reward ablations further show that the proposed behavior-prior constraint enables effective exploration under randomized initial poses.

URL PDF HTML
2607.09587 2026-07-13 cs.RO 新提交

CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination

CoDiMAD:基于扩散的特权蒸馏用于无通信多机器人协调

Jiyue Tao, Shunheng Xin, Tongsheng Shen, Dexin Zhao, Feitian Zhang

发表机构 * Peking University(北京大学) National Innovation Institute of Defense Technology(国防科技创新研究院)

AI总结 研究无通信多机器人协调问题,提出CoDiMAD三阶段框架,先训练特权预言机,再构建数据集,最后将预言机蒸馏到学生智能体中,通过扩散反向过程采样动作,实验证明其性能优于基线。

详情
AI中文摘要

在部分可观测性下的分散式多机器人协调仍然具有挑战性,特别是在无通信设置中,智能体必须仅根据局部传感器观测行动。特权策略蒸馏通过将知识从全局信息的预言机转移到受传感器约束的学生智能体提供了一种有前景的方法。但在多智能体系统中,相同局部观测可能对应多种全局配置,导致条件动作分布固有地多模态。标准确定性蒸馏将这些模式合并为均值,常产生无效或犹豫的动作。为解决此问题,我们提出CoDiMAD,一个三阶段框架,用MAPPO训练特权预言机,构建局部观测 - 预言机 - 动作对的离线数据集,并将预言机蒸馏到参数化为条件去噪扩散概率模型的分散式学生智能体中。通过扩散反向过程近似条件预言机 - 动作分布,CoDiMAD从连贯协调模式中采样决定性动作而非求平均。理论分析刻画了确定性蒸馏的模式平均失败和基于扩散蒸馏的分布恢复特性。在三个合作任务上的实验表明CoDiMAD持续优于直接局部MARL和确定性蒸馏基线。源代码将在接受后公开。

英文摘要

Decentralized multi-robot coordination under partial observability remains challenging, especially in communication-free settings where agents must act solely from local sensor observations. Privileged policy distillation provides a promising approach by transferring knowledge from a globally informed oracle to sensor-constrained students. However, in multi-agent systems, the same local observation may correspond to multiple global configurations requiring qualitatively different cooperative actions, making the conditional action distribution inherently multi-modal. Standard deterministic distillation collapses these modes to their mean, often yielding invalid or hesitant actions. To address this issue, we propose CoDiMAD, a three-stage framework that trains a privileged oracle with MAPPO, constructs an offline dataset of local-observation-oracle-action pairs, and distills the oracle into decentralized students parameterized as conditional denoising diffusion probabilistic models. By approximating the conditional oracle-action distribution through the diffusion reverse process, CoDiMAD samples decisive actions from coherent coordination modes rather than averaging across them. Theoretical analysis characterizes the mode-averaging failure of deterministic distillation and the distributional recovery property of diffusion-based distillation. Experiments on three cooperative tasks show that CoDiMAD consistently outperforms direct local MARL and deterministic distillation baselines. The source code will be made publicly available upon acceptance.

URL PDF HTML
2607.09583 2026-07-13 cs.CV 新提交

Promptable Concept Segmentation from Above: Evaluating SAM 3's Zero-Shot and One-Shot Capabilities in Remote Sensing

从上方进行可提示的概念分割:评估SAM 3在遥感中的零样本和单样本能力

Mohammad Dabaja, Turgay Celik

发表机构 * University of Agder(阿格德大学)

AI总结 研究在严格零样本和单样本约束下,对SAM 3在遥感场景分类等任务中的能力进行评估。核心方法是对SAM 3结构调整并隔离提示模态来诊断对齐机制,还制定代理评估协议。主要贡献是揭示跨模态干扰,表明SAM 3避免过拟合但受分辨率等限制,指明微调方向。

Comments 14 pages, 4 figures

详情
AI中文摘要

大规模基础模型的部署,如Segment Anything Model 3(SAM 3),有望向开放词汇、无需训练的计算机视觉转变。然而,其在分布外对地球观测图像复杂的自上而下几何结构的泛化能力仍未得到充分量化。受SAM 3在高度专业化领域性能差异的驱动,我们在严格的零样本和单样本约束下,对遥感场景分类、目标检测和实例分割进行了全面的多任务实证评估。为此,我们对SAM 3进行了结构调整,将其解耦的二进制存在头重新用作独立的零样本分类器。此外,通过系统地隔离五种配置中的文本和视觉提示模态,我们明确诊断了模型多模态解码器内的对齐机制。我们的发现揭示了严重的跨模态干扰:视觉提示成功地将解码器与复杂的遥感几何结构对齐,而文本提示则注入了未对齐且处于地面水平的语义偏差,从而严重降低了坐标回归。为了在无需资源密集型训练的情况下对这些能力进行基准测试,我们为广义零样本任务(场景分类和实例分割)制定了一种新颖的无需训练的代理评估协议。最终,我们的结果表明,SAM 3避免了传统领域适应模型中常见的过拟合问题,在分割任务中获得了较高的调和平均分数。然而,它仍然受到亚像素分辨率限制和开销语义盲点的根本约束,为其多模态解码器的参数高效地理空间微调指明了明确的方向。

英文摘要

The deployment of large-scale foundation models, such as the Segment Anything Model 3 (SAM 3), promises a transition toward open-vocabulary, training-free computer vision. However, their capacity to generalize out-of-distribution to the complex, top-down geometric structures of Earth Observation imagery remains largely unquantified. Driven by SAM 3's performance disparities in highly specialized domains, we present a comprehensive, multi-task empirical evaluation across remote sensing scene classification, object detection, and instance segmentation under strict zero-shot and one-shot constraints. To achieve this, we introduce a structural adaptation of SAM 3 by repurposing its decoupled binary presence head into a standalone zero-shot classifier. Furthermore, by systematically isolating textual and visual prompt modalities across five configurations, we explicitly diagnose the alignment mechanics within the model's multimodal decoder. Our findings reveal severe cross-modal interference: while visual prompts successfully align the decoder to complex remote sensing geometry, textual prompts inject misaligned, ground-level semantic bias, actively degrading coordinate regression. To benchmark these capabilities without resource-intensive training, we formulate a novel training-free proxy evaluation protocol for Generalized Zero-Shot tasks (scene classification and instance segmentation). Ultimately, our results demonstrate that SAM 3 avoids the overfitting commonly seen in legacy domain-adapted models, achieving high Harmonic Mean scores in segmentation tasks. However, it remains fundamentally constrained by sub-pixel resolution limits and overhead semantic blind spots, charting a definitive mandate for parameter-efficient geospatial fine-tuning of its multimodal decoder.

URL PDF HTML
2607.09581 2026-07-13 cs.CV cs.SD 新提交

Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation

万舞者:一种用于分钟级连贯音乐到舞蹈生成的分层框架

Mingyang Huang, Peng Zhang, Li Hu, Guangyuan Wang, Bang Zhang

发表机构 * Tongyi Lab, Alibaba Group(通义实验室,阿里巴巴集团)

AI总结 针对从音乐生成舞蹈视频的挑战,提出万舞者分层框架,解耦过程为全局关键帧规划和局部时间细化,利用音乐上下文确保连贯,通过动态帧率自适应等创新,突破时长限制,在多舞蹈类型上表现出色,达新的技术水平。

Comments 17 pages, 13 figures, project: https://github.com/Wan-Video/Wan-Dancer

详情
AI中文摘要

直接从音乐生成长时间、高清且节奏同步的舞蹈视频仍然是一项重大挑战,主要是由于当前扩散模型的时间限制,通常在超过20秒时就会失败。现有方法存在时间漂移、身份不一致和重复运动模式等问题。为此,我们提出了一种用于分钟级连贯音乐到舞蹈生成的新型分层框架。该方法将过程解耦为全局关键帧规划和局部时间细化,利用全轨道音乐上下文确保长程连贯性。关键创新包括通过时间映射的RoPE嵌入进行动态帧率自适应以实现精确对齐、基于光流的损失函数增强运动连续性以及运动速度控制以在快速运动中保留高保真细节。大量实验表明,我们的框架突破了传统时长限制,生成稳定的720p/30fps、超过一分钟的视频,具有卓越的时间稳定性。此外,该模型在五种不同舞蹈类型上表现出强大的通用性,以音频和文本提示为条件,在连贯的长格式舞蹈视频合成方面建立了新的技术水平。

英文摘要

Generating long-duration, high-definition, and rhythmically synchronized dance videos directly from music remains a significant challenge, primarily due to the temporal constraints of current diffusion models, which typically fail beyond 20 seconds. Existing approaches, whether they rely on intermediate 3D skeletons or on end-to-end video synthesis, suffer from temporal drift, identity inconsistency, and repetitive motion patterns when extended to longer horizons. To address these limitations, we propose a novel hierarchical framework for minute-scale coherent music-to-dance generation. Our method decouples the process into global keyframe planning and local temporal refinement, leveraging full-track musical context to ensure long-range coherence. Key innovations include dynamic frame rate adaptation via time-mapped RoPE embeddings for precise alignment, an optical-flow-based loss function to enhance motion continuity, and motion-speed control to preserve high-fidelity details during rapid movements. Extensive experiments demonstrate that our framework surpasses the conventional duration barrier, generating stable, 720p/30fps videos exceeding one minute with superior temporal stability. Furthermore, the model exhibits robust versatility across five distinct dance genres, conditioned on both audio and textual prompts, establishing a new state-of-the-art in coherent, long-form dance video synthesis.

URL PDF HTML
2607.09578 2026-07-13 cs.AI 新提交

Knowledge Graphs and Explainable AI as Complementary Resources for Urban Mining

知识图谱与可解释人工智能作为城市采矿的互补资源

Jan Gronewald, Andreas Emrich, Nijat Mehdiyev

发表机构 * German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)

AI总结 研究城市采矿中拆除前评估,基于信息系统资源传统,提出知识图谱与可解释人工智能的四种整合模式,解锁决策可辩护性属性,以满足拆除前评估对监管工件的需求,通过防火门示例说明模式。

Comments Accepted for presentation at the AISE Workshop @ IJCAI-ECAI 2026

详情
AI中文摘要

拆除前评估是城市采矿核心的规范审核流程,是一个信息过程,其中人工智能支持必须服务于对所做决策负责的合格审核员。相关价值单位不仅是预测准确性,还包括所支持决策的可辩护性。可解释人工智能技术和领域知识图谱各自满足了部分要求,现有分类法已对它们的整合进行了编目。本文基于信息系统资源传统提供了一种互补理论解释,提出了四种整合模式,每个模式都解锁了一种独特的可辩护性属性,有助于满足拆除前评估所需的监管工件类型。城市采矿过程中的防火门示例说明了这些模式。

英文摘要

Pre-demolition assessment, the regulated audit process at the heart of urban mining, is an information process in which AI support must serve qualified auditors who remain accountable for the decisions taken. The relevant unit of value is not prediction accuracy alone, but the defensibility of the supported decisions: their legibility, plausibility, sourcing, and contestability. Explainable AI techniques and domain knowledge graphs each address parts of this requirement, and existing taxonomies have catalogued their integration. The literature is descriptively rich but structurally under-specified: what remains less developed is a structural account of why specific integrations produce artefacts neither resource can provide alone. This paper offers a complementarity-theoretic interpretation grounded in the IS resource-based tradition. We propose four consolidated KG-XAI integration modes (Lifting, Constraining, Typing, and Revising), each defined as a typed operation over XAI artefacts and knowledge-graph substrate structures. Each mode unlocks a distinct property of defensibility and contributes to the kind of regulatory artefact pre-demolition assessment demands. A fire-door example from the urban-mining process illustrates the modes using the W3C Linked Building Data stack and valuation extensions.

URL PDF HTML
2607.09576 2026-07-13 cs.CL cs.AI cs.ET 新提交

Conceptual Networks for Cross-Linguistic Idiomatic Expressions:A Feature-Based Graph Approach

跨语言习语表达的概念网络:一种基于特征的图方法

Kiran Pala, Punam Silu, Lixun Yu

发表机构 * University of Eastern Finland(东芬兰大学) Indian Institute of Technology Ropar(印度理工学院罗帕尔分校)

AI总结 该研究提出基于网络的框架表示八种语言的习语和比喻意义,用概念特征注释构建加权图,通过社区检测发现习语按概念模式聚类,网络能捕获独特语义信息,跨语言转移实验有提升,消融研究表明多特征维度有贡献,提供了可解释的习语意义表示。

详情
AI中文摘要

我们提出了一个基于网络的可解释框架,用于表示八种类型多样的语言中的习语和比喻意义,涵盖160个惯用表达式,其中大部分是习语。每个表达式都用从认知语言理论中得出的二元概念特征(包含、隐藏、情感、社会等)进行注释,成对的杰卡德相似度定义了一个加权图。社区检测表明,习语按概念模式而非语言聚类,产生了与认知语言预测一致的结构。概念网络捕获了分布嵌入中不存在的独特语义信息,可通过大语言模型自动注释进行扩展,改进了下游习语检测,并且在丰富语料库频率时保持稳健。跨语言转移实验表明,仅概念接近度就能识别五个语系中的可接受翻译对等物,比基于嵌入的基线有显著提升。消融研究表明,模式、角色和价这三个特征维度对网络的组织属性和习语检测性能都有非冗余贡献,并且特定的图衍生信号(社区成员身份、邻居相似度)特别有用。该框架提供了一种可解释的、跨语言稳定的习语意义表示,将理论基础与实际效用相结合。

英文摘要

We present an interpretable network-based framework for representing idiomatic and figurative meaning across eight typologically diverse languages, totaling 160 conventional expressions, the large majority of which are idiomatic. Each expression is annotated with binary conceptual features (containment, concealment, emotional, social, etc.) derived from cognitive-linguistic theory, and pairwise Jaccard similarities define a weighted graph. Community detection reveals that idioms cluster by conceptual schema rather than by language, producing a structure consistent with cognitive-linguistic predictions. The conceptual network captures unique semantic information not present in distributional embeddings, can be scaled via automatic annotation with LLMs, improves downstream idiom detection, and remains robust when enriched with corpus frequencies. Cross-lingual transfer experiments show that conceptual proximity alone can identify acceptable translation equivalents across five language families, with substantial gains over embedding-based baselines. Ablation studies demonstrate that all three feature dimensions -- schemas, roles, and valence -- contribute non-redundantly to both the network's organizational properties and its performance on idiom detection, and that specific graph-derived signals (community membership, neighbor similarity) are particularly informative. The framework offers an interpretable, cross-linguistically stable representation of idiomatic meaning, combining theoretical grounding with practical utility.

URL PDF HTML
2607.09562 2026-07-13 cs.CV cs.AI 新提交

TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models

TCLA:用于医学视觉语言模型的无训练类级对数几率适应

Tianyou Jiang, Ziyu Zhou

发表机构 * University of Bern(伯尔尼大学) Shanghai Jiao Tong University(上海交通大学)

AI总结 针对医学视觉语言模型在分布外数据上有效性下降问题,提出无训练的少样本适应方法TCLA。该方法基于支持样本校正推理对数几率,提升模型性能,实验表明其能持续提高模型OOD性能且多数情况下优于现有训练方法。

详情
AI中文摘要

医学视觉语言模型(VLMs)在零样本性能方面表现出色,但由于大规模预训练中继承的域转移和类偏差,其在分布外(OOD)数据上的有效性仍然会下降。现有的少样本适应方法通常会引入额外的可训练组件,在极低数据情况下(如单样本)可能不稳定,且对不同医学数据缺乏鲁棒性。我们提出了TCLA,一种用于医学VLMs的完全无训练的少样本适应方法,它快速且与模型无关。TCLA基于一小部分支持样本校正推理对数几率,通过改善类间解混淆和减少域转移来提升预训练VLMs的性能。在包括X射线、超声、MRI、CT、组织病理学等多种医学成像模态的九个数据集上进行的广泛实验表明,TCLA持续提高医学VLMs的OOD性能,并且在大多数情况下优于现有的基于训练的适应方法。

英文摘要

Medical Vision-Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining. Existing few-shot adaptation methods typically introduce additional trainable components, which can be unstable in extremely low-data regimes (e.g., 1-shot), and lack robustness on different medical data. We present TCLA, a purely training-free few-shot adaptation method for Medical VLMs, which is fast and model-agnostic. TCLA corrects inference logits based on a small set of support samples, boosting pretrained VLMs performance by improving inter-class deconfusion and reducing domain shift. Extensive experiments on nine datasets across multiple medical imaging modalities including X-ray, Ultrasound, MRI, CT, Histopathology, demonstrate that TCLA consistently improves OOD performance of Medical VLMs and, in most of cases, outperforms existing training-based adaptation methods.

URL PDF HTML
2607.09560 2026-07-13 cs.AI cs.LG 新提交

Beyond Fixed Representations: The Vocabulary and Verifier Gaps in Open-Ended AI

超越固定表示:开放式人工智能中的词汇与验证差距

Yuan Cao, Haiqian Yang

发表机构 * MIT(麻省理工学院)

AI总结 研究指出当前人工智能系统虽能力强但表示框架固定,构建开放式智能需新操作。通过词汇和验证差距刻画与真正开放式智能的距离,基于智能行为的认知转换区分不同类型转换,进而提出创新自主性阶梯及推进开放式人工智能的方向。

详情
AI中文摘要

现代人工智能系统正越来越多地根据其推理、编码、证明定理、使用工具以及长期研究任务的能力进行评估。这些能力强大,但存在结构限制,模型运行的表示框架通常是预先固定的。本文认为构建能够开放式创新的更强智能系统需要新的操作类别,即创建、稳定和重用新的表示原语。通过词汇差距(发明和稳定新表示原语的困难)和验证差距(判断新原语价值的困难)来刻画当前人工智能系统与真正开放式智能的距离。通过将智能行为视为认知转换序列,区分了固定表示框架内的空间内转换和可能修改框架本身的生成转换。在此基础上,提出了创新自主性阶梯并概述了推进开放式人工智能的几个方向。

英文摘要

Modern AI systems are increasingly being evaluated for their ability to reason, code, prove theorems, use tools, and long-horizon research tasks. These are powerful capabilities, but they share a structural limitation: the representational frame within which the model operates, including its conceptual vocabulary, the space of admissible solutions it can search, and the criteria by which success is evaluated, is typically fixed and supplied in advance. This paper argues that building stronger intelligent systems capable of open-ended innovation requires additional classes of operations: the creation, stabilization, and reuse of new representational primitives, which alter the space being searched rather than simply searching within it. We characterize the distance between current AI systems and genuinely open-ended intelligence through two gaps. The first is the vocabulary gap, the difficulty of inventing and stabilizing new representational primitives rather than merely recombining existing ones. The second is the verifier gap, the difficulty of judging the value of a new primitive when its full payoff may be visible only after future reuse. We interpret both gaps through a unified framework of intelligence as cognitive discrepancy reduction. By viewing intelligent behaviors as a sequence of cognitive transformations, we distinguish intra-space transformations which operate within a fixed representational frame, from generative transformations which may modify the frame itself. On this basis, we propose a ladder of innovation autonomy and outline several directions for advancing open-ended AI, including objectives that reward useful representational change, persistent memory architectures for invented primitives, and adaptive verification mechanisms capable of evolving alongside the representations they evaluate.

URL PDF HTML
2607.09557 2026-07-13 cs.RO 新提交

CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles

CORAL-AUV:面向自主水下航行器的CFD强化学习

Steven Roche, Milo Van Mooy, Nathan McGuire, Levi Cai, Jonathan P. How, Yogesh Girdhar

发表机构 * MIT–WHOI Joint Program(麻省理工学院 - 伍兹霍尔海洋研究所联合项目) Woods Hole Oceanographic Institution(伍兹霍尔海洋研究所) Massachusetts Institute of Technology(麻省理工学院)

AI总结 研究自主水下航行器细粒度控制与定位问题,提出利用CFD模型代理近似在强化学习中快速推理的方法,首次在6自由度AUV上成功部署零射击RL策略,相比传统控制器有能耗降低、速度加快、误差减小等优势。

Comments 16 pages, 13 figures

详情
AI中文摘要

自主水下航行器(AUV)的细粒度控制和定位对于采样、维护及勘测应用至关重要。传统控制方法劳动强度大且对车辆配置或环境条件变化不稳健。强化学习(RL)有望快速开发控制器,通过域随机化(DR)处理一系列部署参数,但DR受基础模拟对真实物理建模能力限制。计算流体动力学(CFD)提供高保真阻力模型,但因计算开销难以在强化学习框架中利用。本文利用训练给定车辆CFD模型代理近似的想法,在RL管道内实现快速推理。首次在6自由度AUV上成功部署零射击RL策略,在基于CFD数据训练的代理阻力模型(SDM)上进行策略训练。与使用简化物理的控制器相比,能量使用降低31%,航点间移动速度快11%,误差减少19%。基于SDM的RL控制器更好地预测零射击转移,在奖励塑造设计选择上更稳健。使用DR完成参数受扰任务时,CFD策略是唯一成功转移的控制器。策略在受控水槽环境和实地进行评估,对其能力进行广泛测试。

英文摘要

Fine grain control and positioning of autonomous underwater vehicles (AUVs) is critical for sampling, maintenance, and survey applications. Traditional control methods for AUVs are labor intensive and are not robust to changes in the vehicle configuration or environmental conditions. Reinforcement learning (RL) promises rapid controller development while handling a range of deployment parameters via domain randomization (DR). However, DR is still limited by the capacity of the underlying simulation to model real physics. In particular, drag physics are difficult to model and are a large contributor to sim-to-real gaps. Meanwhile, computational fluid dynamics (CFD) provides high fidelity drag models but is challenging to leverage within reinforcement learning frameworks due to its computational overhead. Thus, in this paper we exploit the idea of training surrogate approximations of CFD models of a given vehicle, enabling fast inference within RL pipelines. We are the first to successfully deploy a zero-shot RL policy on a 6-DOF AUV in which policy training is performed on surrogate drag models (SDMs) trained on CFD data. We find 31% lower energy usage compared to a controller using simplified physics while traversing between waypoints 11% faster with 19% less error. Our SDM based RL controller better predicts zero-shot transfer and is more robust across reward shaping design choices. When using DR to complete a task with perturbed parameters, we find that the CFD policy is the only controller that successfully transfers. The policies are evaluated in a controlled tank environment and in the field providing extensive testing of the policies' capabilities.

URL PDF HTML
2607.09548 2026-07-13 cs.RO 新提交

Task-Adaptive Design of Modular Aerial Manipulators Under Airflow Exposure Constraints

气流暴露约束下模块化空中机械手的任务自适应设计

Mengguang Li, Heinz Koeppl

发表机构 * Technische Universität Darmstadt(达姆施塔特工业大学)

AI总结 针对多旋翼平台空中操作中旋翼气流限制问题,提出基于优化的模块化空中机械手设计框架,联合考虑任务、末端执行器及气流约束,引入新分类与模型,通过重新配置优化适应多样任务需求,实验验证了框架有效性。

Comments Accepted to the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026

详情
AI中文摘要

多旋翼平台的空中操作能够在复杂环境中进行物理交互,但旋翼引起的气流仍然是涉及对气流敏感目标或周围环境的任务的关键限制。本文提出了一种基于优化的模块化空中机械手设计框架,该框架联合考虑了任务扳手可行性、末端执行器放置和气流暴露约束。我们首先引入了目标侧气流容忍度的新分类,并将相应的暴露要求表述为几何约束。为了有效地模拟旋翼引起的气流,我们引入了一个紧凑的锥球包络,它近似四旋翼气流的扩散结构,同时保持优化的计算可处理性。在此基础上,我们提出了一种重新配置优化方法,使模块化空中机械手适应不同的任务扳手要求,同时执行目标侧气流暴露和平台内气流干扰约束。与先前假设固定末端执行器位置的设计不同,所提出的框架优化了末端执行器放置以及平台配置。扩展性实验和消融研究验证了所提出框架的有效性。

英文摘要

Aerial manipulation with multirotor platforms enables physical interaction in complex environments, but rotor-induced airflow remains a critical limitation for tasks involving airflow-sensitive targets or surroundings. This paper presents an optimization-based design framework for modular aerial manipulators that jointly considers task wrench feasibility, end-effector placement, and airflow exposure constraints. We first introduce a novel categorization of target-side airflow tolerance and formulate the corresponding exposure requirements as geometric constraints. To efficiently model rotor-induced airflow, we introduce a compact cone-sphere envelope that approximates the spreading structure of a quadrotor's airflow while preserving computational tractability for optimization. Building on this formulation, we propose a reconfiguration optimization that adapts a modular aerial manipulator to diverse task wrench requirements while enforcing both target-side airflow exposure and intra-platform airflow interference constraints. Unlike prior designs that assume a fixed end-effector location, the proposed framework optimizes the end-effector placement together with the platform configuration. Scalability experiments and ablation studies validate the effectiveness of the proposed framework.

URL PDF HTML
2607.09544 2026-07-13 cs.CV cs.LG 新提交

The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs

计数存在但未对齐:理解和纠正视觉语言模型中的计数失败

Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh, Nikolai Rozanov, Kentaro Inui

发表机构 * MBZUAI(穆罕默德·本·扎耶德人工智能大学) Imperial College London(伦敦帝国学院)

AI总结 研究视觉语言模型计数失败问题,通过对VLM激活进行探测训练及分析,发现其虽常编码正确计数但输出错误,提出探测器引导的自校正方法,在不更新参数时提高计数准确率,揭示内部知识与模型输出差距并提供改进工具。

详情
AI中文摘要

尽管视觉语言模型(VLM)在许多多模态任务中表现出色,但在基本物体计数方面仍存在困难。研究其是缺少内部知识还是内部表征与语言输出之间存在差距。在五个计数数据集上对四个VLM的激活进行简单探测训练,发现非线性探测可检测计数错误,VLM常编码正确计数却输出错误答案。SVCCA分析表明相关探测占据部分共享激活子空间但读出方向未对齐。通过因果转向干预验证结果,提出探测器引导的自校正方法,在不更新参数情况下提高计数准确率达15.6个绝对百分点,为改进VLM计数提供实用工具并揭示内部知识与模型输出差距。

英文摘要

Despite strong performance on many multimodal tasks, vision-language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal knowledge or a gap between internal representations and verbalized outputs. Training simple probes on activations from four VLMs across five counting datasets reveals that nonlinear probes can reliably detect counting errors, suggesting that VLMs often encode the correct count even when they output the wrong answer. SVCCA analysis shows that probes trained on ground-truth counts and probes trained on model outputs occupy a partially shared activation subspace but read out along misaligned directions. We further validate our findings using a causal steering intervention, proving that strengthening the direction of count-identified probes does improve model counting performance. Motivated by this result, we propose a detector-guided self-correction method that selectively re-prompts the model only when an internal error detector predicts failure. This simple inference-time intervention improves counting accuracy by up to 15.6 absolute percentage points, without any parameter updates. Our results establish activation-based error probing as both a practical tool for improving VLM counting and a mechanistic lens on the gap between internal knowledge and model outputs.

URL PDF HTML
2607.09543 2026-07-13 cs.LG q-bio.NC 新提交

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

CoCoT-EEG:用于脑电解码的对比预训练多尺度卷积Transformer

Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian, Yassir Bendou, Richard Gao

发表机构 * Sigma Nova Paris(巴黎西格玛诺瓦公司) Goethe University Frankfurt(法兰克福歌德大学)

AI总结 研究针对EEG解码,开发了含多尺度时间卷积输入层和Transformer编码器块的对比预训练模型CoCoT,在广泛基准解码任务中表现出色,优于单任务解码模型,证明对比学习构建EEG基础模型可行,还给出架构设计考量。

详情
AI中文摘要

自监督预训练基础模型在无创脑电图(EEG)解码应用中已初现成效。近期许多大规模模型采用对原始EEG进行分词,再进行掩码重建预训练的方法。但该方法对EEG这类高噪声幅度、信息局限于窄频带等有限维度的数据并非最优。基于此,我们开发了具有多尺度时间卷积输入层和Transformer编码器块的新型对比预训练EEG模型(CoCoT)。在具有异构电极配置的广泛基准解码任务中,CoCoT与当前最优的重建预训练EEG模型相当或更优。从头开始训练的CoCoT优于先前的单任务解码模型,甚至可与预训练模型媲美,展示了该架构的灵活性和数据效率。通过系统消融实验,我们证明了对比学习构建EEG基础模型的可行性,并提出关键架构设计考量,促使对替代大规模预训练策略进行进一步研究。

英文摘要

Self-supervised pretrained foundation models (FM) have shown early promise for non-invasive electroencephalogram (EEG) decoding applications. Many recent large-scale models converged on the approach of tokenizing raw EEG followed by masked reconstruction pretraining. However, this recipe has been shown to be suboptimal for data, like EEG, with high noise amplitude and information confined to limited dimensions such as narrow frequency bands. Building on this insight, we develop a novel contrastive-pretrained EEG model with multiscale temporal convolution input layers and Transformer encoder blocks (CoCoT). CoCoT matches or beats state-of-the-art reconstruction-pretrained EEG models on extensive benchmark decoding tasks with heterogeneous electrode configurations. Furthermore, CoCoT trained from scratch outperforms previous single-task decoding models and even rivals pretrained models, showcasing the architecture's flexibility and data efficiency. Through systematic ablations, including model architecture and pretraining objective, we demonstrate the viability of contrastive learning for building EEG FMs while suggesting key architectural design considerations, prompting further investigations in alternative large-scale pretraining strategies.

URL PDF HTML
2607.09537 2026-07-13 cs.LG 新提交

GatedLinear: Adaptive Routing of Complementary Linear Bases for Time Series Forecasting

门控线性:用于时间序列预测的互补线性基的自适应路由

Qitai Tan, Ruiwen Gu, Yilin Su, Mo Li, Xu Lin, Xiao-Ping Zhang

发表机构 * Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)

AI总结 针对时间序列预测中多样动态难以用单一机制处理的问题,提出门控线性框架,通过三种专门机制及三因式融合门实现互补线性基的自适应路由,实验显示其在精度、可解释性和参数规模上表现出色。

详情
AI中文摘要

时间序列预测要求模型捕捉多样且往往相互排斥的时间动态,从平滑趋势延续到非平稳漂移和严格相位对齐的循环。近期深度学习模型虽提高了准确性,但通常通过单一计算主干处理这些多样模式。我们提出门控线性框架,将预测视为互补线性基的自适应路由。它利用三种机制:用于平滑投影的全局趋势 - 季节基、用于非平稳漂移的基于差分的增量基和用于显式循环重用的相位对齐循环基。通过三因式融合门动态协调,在不同预测模式下进行高度粒度化的逐点软路由。实验表明该方法取得了与近期复杂基础模型相当或更优的精度,且路由模式可解释、参数规模小。

英文摘要

Time series forecasting requires models to capture diverse, often mutually exclusive, temporal dynamics, from smooth trend continuation to nonstationary drift and strict phase-aligned recurrence. While recent deep learning models have improved accuracy, they typically force these diverse patterns through a single computational backbone governed by fixed algorithmic inductive biases (e.g., self-attention or spectral filtering). This single-mechanism approach often struggles with the profound heterogeneity of real-world series, where different variables and forecast horizons necessitate fundamentally different predictive treatments. To address this, we propose GatedLinear: a lightweight framework that frames forecasting as the adaptive routing of complementary linear bases. GatedLinear leverages a pool of three specialized mechanisms: a global trend-seasonal basis for smooth projection, a difference-based incremental basis for nonstationary drift, and a phase-aligned recurrence basis for explicit cyclic reuse. To dynamically orchestrate these distinct behaviors, we introduce a Tri-Factorized Fusion Gate that disentangles routing decisions into channel-specific preferences, horizon-aware offsets, and phase-indexed biases derived from known future time marks. This design allows the model to perform highly granular, point-wise soft routing across different predictive regimes without stacking computationally heavy neural modules. Experiments on standard benchmarks show that our method achieves state-of-the-art or highly competitive accuracy against recent complex foundational models, while offering explicitly interpretable routing patterns and operating with a substantially smaller parameter footprint.

URL PDF HTML
2607.09532 2026-07-13 cs.LG cs.CR stat.ML 新提交

Statistically Undetectable Backdoors in Deep Neural Networks

深度神经网络中统计上不可检测的后门

Andrej Bogdanov, Alon Rosen, Neekon Vafa

发表机构 * University of Ottawa(渥太华大学) Bocconi University(博科尼大学) Massachusetts Institute of Technology(麻省理工学院)

AI总结 研究对抗模型训练者在深度前馈神经网络中植入统计上不可检测的后门,此后门可提供基于不变性的对抗样本,揭示了模型训练者与使用者间的权力不对称,且无后门时多项式时间内无法生成此类样本。

Comments ICML 2026

详情
AI中文摘要

我们展示了对抗模型训练者如何在一大类深度前馈神经网络中植入后门。这些后门在白盒设置下在统计上是不可检测的,即带后门和正常训练的模型在全变差距离上接近,即使给出模型的完整描述(如所有权重)。后门能为每个输入提供基于不变性的对抗样本,将不同输入映射到异常接近的输出。然而,没有后门,在多项式时间内(在标准密码学假设下)生成此类对抗样本是不可能的。我们的理论和初步实证结果证明了模型训练者和模型使用者之间存在根本的权力不对称。

英文摘要

We show how an adversarial model trainer can plant backdoors in a large class of deep, feedforward neural networks. These backdoors are statistically undetectable in the white-box setting, meaning that the backdoored and honestly trained models are close in total variation distance, even given the full descriptions of the models (e.g., all of the weights). The backdoor provides access to invariance-based adversarial examples for every input, mapping distant inputs to unusually close outputs. However, without the backdoor, it is provably impossible (under standard cryptographic assumptions) to generate any such adversarial examples in polynomial time. Our theoretical and preliminary empirical findings demonstrate a fundamental power asymmetry between model trainers and model users.

URL PDF HTML
2607.09528 2026-07-13 cs.LG cs.CR cs.DB cs.SI 新提交

TSAI-MetaFraud: A Benchmark Dataset for Financial Fraud Transaction and Behavioral Risk Detection in Metaverse Ecosystems

TSAI-MetaFraud:用于元宇宙生态系统中金融欺诈交易和行为风险检测的基准数据集

Refat Ishrak Hemel, Ehsan Hallaji, Roozbeh Razavi-Far

发表机构 * tsai-unb(tsai - 新不伦瑞克大学)

AI总结 针对元宇宙平台带来的欺诈等新挑战及现有数据集局限性,提出TSAI-MetaFraud基准数据集,整合多类信息与欺诈场景,定义多项基准任务并进行基线评估,为元宇宙生态系统相关研究提供基准。

详情
AI中文摘要

元宇宙平台的出现创造了虚拟经济,带来了与欺诈、机器人活动和非法金融行为相关的新挑战。尽管对可信的元宇宙分析兴趣日增,但现有数据集通常孤立地关注用户行为、认证或金融交易,限制了多模态欺诈检测方法的发展和可重复评估。为填补这一空白,我们提出TSAI-MetaFraud,一个用于虚拟经济中欺诈分析的多模态、多任务基准数据集。它整合了行为、交易和图结构信息,纳入了现实的欺诈和自动化机器人场景。我们定义了基准任务,包括交易欺诈检测、跨模态节点分类、时间链接预测和弱监督欺诈检测,并使用机器学习模型和图神经网络进行了基线评估。通过在统一的虚拟经济中联合捕捉行为活动、金融互动和关系结构,TSAI-MetaFraud为新兴元宇宙生态系统中的多模态学习、图挖掘、欺诈分析和可信人工智能提供了一个基准。

英文摘要

The emergence of metaverse platforms has created virtual economies that introduce new challenges related to fraud, bot activity, and illicit financial behavior. Despite growing interest in trustworthy metaverse analytics, existing datasets typically focus on user behavior, authentication, or financial transactions in isolation, limiting the development and reproducible evaluation of multimodal fraud detection methods. To address this gap, we present TSAI-MetaFraud, a multimodal, multi-task benchmark dataset for fraud analytics in virtual economies. TSAI-MetaFraud integrates behavioral, transactional, and graph-structured information while incorporating realistic fraud and automated bot scenarios. We define benchmark tasks including transaction fraud detection, cross-modal node classification, temporal link prediction, and weakly supervised fraud detection, and provide baseline evaluations using machine learning models and graph neural networks. By jointly capturing behavioral activity, financial interactions, and relational structure within a unified virtual economy, TSAI-MetaFraud provides a benchmark for advancing multimodal learning, graph mining, fraud analytics, and trustworthy AI in emerging metaverse ecosystems.

URL PDF HTML
2607.09527 2026-07-13 cs.RO 新提交

How Mobile Gas Sensor Trajectories Govern Hydrogen Leak Detection: A Safety Gap in Manual Leak Inspection of Hydrogen System Components

移动气体传感器轨迹如何影响氢气泄漏检测:氢气系统组件手动泄漏检测中的安全差距

Christian Masuhr, Arne Wendt, Thorsten Schüppstuhl

发表机构 * Institute of Aircraft Production Technology (IFPT), Hamburg University of Technology (TUHH)(飞机生产技术研究所(IFPT),汉堡工业大学(TUHH))

AI总结 研究氢气系统组件手动泄漏检测问题,通过机器人引导测试台量化嗅探轨迹运动学对检测可靠性的影响,得出特定几何形状路径规则和信号损失模型,揭示标准操作程序风险,还提出概念验证软件管道。

Comments Preliminary draft. Work in progress

详情
AI中文摘要

氢气基础设施的完整性依赖可靠的泄漏检测,在电解槽制造中几乎完全通过手动示踪气体嗅探进行。尽管有标准规定,但缺乏空间探测指导说明,检测可靠性完全取决于操作员执行,传感器信号延迟进一步影响检测。本研究量化了嗅探轨迹运动学如何影响小规模管道和配件的检测可靠性,这是宏观扩散研究基本忽略的近场情况。使用机器人引导测试台消除操作员差异,在标准泄漏率(氮气中5体积%氢气)和不同扫描速度下,获取代表性几何形状的静态浓度场和动态轨迹通过情况。结果表明,扫描速度和空间探测方向强烈决定可检测性。传统线性轨迹在动态条件下经常错过泄漏,导致严重误报。相反,特定几何形状的路径,如围绕密封点的圆周插入路径,保持高安全裕度。据此得出特定几何形状的路径规则和动态信号损失的折减因子模型。研究发现当前标准操作程序存在切实安全风险。为实施这些规则,提出概念验证软件管道,可直接从3D模型生成经过验证的轨迹,用于辅助系统可视化。

英文摘要

The integrity of hydrogen infrastructure relies on reliable leak detection, performed almost exclusively via manual tracer gas sniffing in electrolyzer manufacturing. Although mandated by standards, the lack of spatial probe guidance instructions leaves detection reliability entirely to operator execution, further compromised by sensor signal delays. This study quantifies how sniffer trajectory kinematics affect detection reliability at small-scale pipes and fittings, a near-field regime largely neglected by macroscopic dispersion research. Using a robotically guided test bench to eliminate operator variability, static concentration fields and dynamic trajectory passes were acquired across representative geometries under standardized leak rates (5 vol% hydrogen in nitrogen) and varying scanning velocities. Results demonstrate that scanning velocity and spatial probe orientation strongly dictate detectability. Conventional linear trajectories frequently miss leaks under dynamic conditions, causing severe false negatives. Conversely, geometry-specific routing, such as circumferential plunging paths around sealing points, maintains a high safety margin. From these observations, geometry-specific routing rules and a reduction-factor model for dynamic signal loss are derived. The findings reveal that current standard operating procedures pose a tangible safety risk. To operationalize these rules, a proof-of-concept software pipeline is presented, generating validated trajectories directly from 3D models for visualization in assistance systems.

URL PDF HTML
2607.09526 2026-07-13 cs.CV cs.AI 新提交

ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

ALICE:从视觉、视觉语言和玻片级专家学习通用病理学基础模型

Jiawen Li, Tian Guan, Huijuan Shi, Xitong Ling, Mingxi Fu, Anjia Han, Chao He, Yonghong He

发表机构 * Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学生物医药与健康工程研究院,清华大学深圳国际研究生院) Department of Engineering Science, University of Oxford(牛津大学工程科学系) Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University(中山大学附属第一医院病理科) Medical Optical Technology R&D Center, Research Institute of Tsinghua, Pearl River Delta(清华珠三角研究院医学光学技术研发中心)

AI总结 研究提出通过多阶段凝聚蒸馏训练的ALICE统一基础模型,将多种教师模型知识整合到单个主干。它在大量图像上预训练,经多场景多任务评估,在任务匹配病理基础模型中平均排名最佳,证明凝聚蒸馏可整合能力用于广泛病理学应用。

详情
AI中文摘要

基础模型正在重塑计算病理学,但其能力仍受预训练目标、数据源和空间尺度的限制,将互补的专业知识分散在不同的主干中。本文提出了ALICE,这是一个通过多阶段凝聚蒸馏训练的统一基础模型,它将八个仅视觉、视觉语言和玻片级教师模型依次蒸馏到单个主干的专用模块中。ALICE在24,985,184个瓦片级病理图像和155,604个高分辨率图像上进行预训练,并在21个任务场景、96个下游任务和48个数据源上进行评估,涵盖感兴趣区域组织分析、视觉语言多模态评估和全玻片临床评估。在所有三种评估设置中,ALICE在任务匹配的病理基础模型中获得了最佳平均排名。这些结果表明,凝聚蒸馏可以将专门模型的互补能力整合到一个统一的主干中,用于广泛的计算病理学应用。该模型可在此https URL上获得。

英文摘要

Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative distillation that sequentially distills eight vision-only, vision-language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis, vision-language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.

URL PDF HTML
2607.09521 2026-07-13 cs.AI 新提交

SAGEAgent: A Self-Evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction

SAGEAgent:用于多模态生存预测中成本感知模态获取的自进化智能体

Chongyu Qu, Can Cui, Zhengyi Lu, Junchao Zhu, Tianyuan Yao, Junlin Guo, Juming Xiong, Yanfan Zhu, Yuechen Yang, Bennett A. Landman, Yuankai Huo

发表机构 * Vanderbilt University(范德堡大学) Vanderbilt University Medical Center(范德堡大学医学中心)

AI总结 研究多模态生存预测中如何平衡预测准确性与临床侵入性,提出基于自进化大语言模型的SAGEAgent,通过多种记忆和工具为患者决定诊断方式,实验证明其能在胶质瘤队列中降低获取负担并保持竞争力。

详情
AI中文摘要

在多模态临床肿瘤学中,诊断方式遵循临床规定的负担递增顺序。当前多模态生存方法存在不足,本文将其视为序贯决策问题,提出SAGEAgent。它是基于自进化大语言模型的临床智能体,通过临床工具、情景记忆和语义记忆为患者决定获取的诊断方式,平衡预测准确性和临床侵入性。实验表明,SAGEAgent在胶质瘤队列中预测准确率具竞争力,同时将平均获取负担降低55%。

英文摘要

Does every cancer patient truly need a complete diagnostic workup for accurate survival prediction? In multimodal clinical oncology, diagnostic modalities follow a clinically mandated order of escalating burden -- from demographics collected at intake to genomic profiling requiring specialized tissue analysis. Current multimodal survival methods either assume all modalities are available or passively handle missing data, but none actively reason about whether acquiring the next modality is justified for a given patient along this ordered workflow. We formulate this as a sequential decision problem and propose SAGEAgent (Sequential Acquisition Guided by Experience), a self-evolving LLM-based clinical agent that decides which diagnostic modalities to acquire for each patient, balancing predictive accuracy against clinical invasiveness. SAGEAgent reasons about each patient's evolving diagnostic state through clinical tools that translate numerical predictions into text, an episodic memory that retrieves similar past cases, and a semantic memory that accumulates reusable decision patterns from experience. Experiments on a glioma cohort combining TCGA-LGG, TCGA-GBM, and BraTS with four diagnostic modalities demonstrate that SAGEAgent achieves competitive survival prediction accuracy while reducing average acquisition burden by 55%.

URL PDF HTML
2607.09520 2026-07-13 cs.CV cs.AI 新提交

Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference

眼见无需耗能,言语却要代价:揭示边缘视觉语言模型推理中的真正能量瓶颈

Junfei Zhan, Haoxun Shen, Mingang Guo, Zixuan Huang, Tengjiao He

发表机构 * University of Pennsylvania(宾夕法尼亚大学) Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院) Jinan University(暨南大学)

AI总结 研究边缘VLM推理的能量瓶颈,通过系统能量分析发现平均推理功率恒定,输出令牌耗时多是关键,图像复杂度因输出长度影响能量,揭示视觉令牌修剪局限,控制输出长度能大幅节能。

Comments Accepted to ACM MM 2026. 10 pages, 5 figures

详情
AI中文摘要

视觉语言模型(VLM)是具身人工智能的感知支柱,但其在边缘硬件上的能源足迹仍未得到充分理解。现有提高效率的努力主要集中在减少视觉令牌上,隐含地将视觉处理视为主要的能源成本。我们通过对设备上VLM推理进行首次系统的能量分析,推翻了这一隐含假设,该分析涵盖了三个架构家族的五个模型、四种输入分辨率和两个硬件平台(NVIDIA RTX 3070和Jetson Orin NX)。我们的分析得出了三个发现。首先,平均推理功率是一个模型内在常数,与输入分辨率、图像复杂度和提示类型无关,在所有条件下变化小于5%。这意味着所有输入之间的能量变化必须源于推理时间的变化,而不是功耗的变化。其次,由于预填充和解码之间的计算和内存不对称,每个输出令牌的挂钟时间比每个输入令牌多11到39倍,使得输出令牌数量成为延迟和能量的主要驱动因素。第三,以图像中的对象数量衡量的图像复杂度,在相同分辨率下会导致高达4.1倍的能量差异。这种变化不是源于视觉处理成本的增加,而是源于输出长度的差异。这些发现揭示了视觉令牌修剪的一个基本局限性:即使移除所有视觉令牌,对于固定令牌模型最多也只能节省10%的总能量。在跨越10亿到80亿参数的模型中,控制输出长度最多可节省97%的总能量,并且在更大的模型规模下,解码的能量主导地位会变得更强。简而言之,边缘VLM推理中的真正能量瓶颈不是模型看到了什么,而是它说了多少。

英文摘要

Vision-Language Models (VLMs) are the perceptual backbone of embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual tokens, implicitly treating visual processing as the dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX). Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output token costs 11 to 39x more wall-clock time than each input token due to the compute-bound and memory-bound asymmetry between prefill and decode, making output token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1x energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual token pruning: even removing all visual tokens saves at most 10% of total energy for fixed-token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy bottleneck in edge VLM inference is not what the model sees, but how much it says.

URL PDF HTML