arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 2036
2607.27205 2026-07-30 cs.CV cs.RO 新提交

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

TurboVLA:在RTX 4090上以32 Hz运行、显存占用<1 GB的实时视觉-语言-动作模型

Hengyi Xie, Chenfei Yao, Xianjin Wu, Xuanyang Xi, Yiping Tang, Di Xu, Yingying Zhu, Dingkang Liang, Xiang Bai, Han Ding

机构 * Huazhong University of Science and Technology(华中科技大学) Huawei Technologies Co. Ltd(华为技术有限公司)

AI总结 本研究提出TurboVLA,将传统VLA模型的LLM中心路径重构为视觉语言直接映射,仅0.2B参数即可在RTX 4090上实现97.7%LIBERO任务成功率,性能优于更大模型且显存占用极低。

Comments Code is available at https://github.com/H-EmbodVis/TurboVLA

详情
AI中文摘要

视觉-语言-动作(VLA)模型通常采用以大语言模型(LLM)为中心的$V \to L \to A$路径,即先将视觉观测投影到大语言模型的表示空间,再解码为机器人动作。尽管该设计有效,但每次策略调用都会产生大量计算与内存开销。本研究提出TurboVLA,一种新型VLA范式,将传统$V \to L \to A$路径重新构建为直接的$V + L \to A$映射。TurboVLA不使用大语言模型作为感知与动作间的核心接口,而是独立编码视觉观测与语言指令,通过轻量双向视觉-语言交互直接交换两者信息,并通过紧凑解码器预测连续动作块。这种简洁设计直接从视觉和语言特征构建任务条件表示,大幅降低VLA推理的计算与内存成本。在LIBERO数据集上,TurboVLA仅用0.2B参数,在消费级RTX 4090上实现97.7%的平均成功率、31.2 ms的推理延迟和0.9 GB的推理显存占用,性能匹配或显著优于更大规模的VLA策略。这些结果表明TurboVLA是当前主流以LLM为中心的VLA范式的简洁有效替代方案,为视觉、语言与动作如何关联以实现高效机器人操纵提供了新视角。代码可在此https URL获取。

英文摘要

Vision-language-action (VLA) models commonly adopt an LLM-centric $V \to L \to A$ pathway, where visual observations are projected into the representation space of a large language model before being decoded into robot actions. Although effective, this design incurs substantial computation and memory overhead at every policy invocation. In this work, we introduce TurboVLA, a new VLA paradigm that reformulates the conventional $V \to L \to A$ pathway as a direct $V + L \to A$ mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks with a compact decoder. This simple design constructs task-conditioned representations directly from visual and linguistic features, significantly reducing the computational and memory costs of VLA inference. On LIBERO, TurboVLA achieves 97.7% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GB inference VRAM on a consumer-grade RTX 4090, matching or outperforming substantially larger VLA policies. These results establish TurboVLA as a simple and effective alternative to the prevailing LLM-centric VLA paradigm, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.

URL PDF HTML 收藏
2607.27203 2026-07-30 cs.LG 新提交

Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?

在线RL微调真的需要预训练Q函数吗?

Perry Dong, Ron Polonsky, Dorsa Sadigh, Chelsea Fin

机构 * Stanford University(斯坦福大学)

AI总结 本文针对在线RL微调是否需预训练Q函数的问题,发现预训练Q函数增益有限,提出IPE方法,在连续控制基准中使微调性能平均提升1.26倍。

详情
AI中文摘要

预训练后微调已成为学习高性能策略的主流方法,在基于值的强化学习(RL)中,这引发了一个自然问题:给定预训练策略,是否也应在离线数据上预训练Q函数?传统观点认为应当如此,但近期结果显示,采用随机初始化Q函数的在线RL无需预训练Q函数即可得到高性能且可靠的策略。本文系统研究在预训练基础策略上微调时,预训练Q函数是否真有帮助,结果意外发现,简单的Q函数预训练通常相比随机初始化几乎无增益。我们表明这源于根本不匹配:预训练期间学习的Q函数针对预训练策略的Q函数,而非在线微调收敛的Q函数,该差距即使经离线值最大化仍存在。基于此发现,我们提出策略集成初始化(IPE),这是一种训练多个多样化策略并利用其汇集的rollout来引导在线RL中Q函数学习的简单方法。在一系列具有挑战性的连续控制基准测试中,IPE相比简单的Q函数预训练,微调性能平均提升1.26倍。

英文摘要

Pre-training followed by fine-tuning has become the dominant recipe for learning performant policies, and in value-based reinforcement learning (RL) this raises a natural question: given a pretrained policy, should the Q-function be pretrained on offline data too? Conventional wisdom suggests it should, but recent results show that online RL with a randomly-initialized Q-function can result in highly performant and reliable policies without needing to pretrain the Q-function. In this paper, we systematically study whether pretraining the Q-function actually helps when fine-tuning on top of a pretrained base policy. We find, surprisingly, that naive Q-function pretraining often provides little benefit over random initialization. We show this stems from a fundamental mismatch: the Q-function learned during pretraining targets the pretrained policy's Q-function, not the Q-function that online fine-tuning converges to, and this gap persists even after offline value maximization. Motivated by this finding, we propose Initialization via Policy Ensemble (IPE), a simple method that trains multiple diverse policies and uses their pooled rollouts to bootstrap the Q-function learning in online RL. Across a suite of challenging continuous control benchmarks, IPE yields an average 1.26x improvement in fine-tuning performance over naive Q-function pre-training.

URL PDF HTML 收藏
2607.27194 2026-07-30 cs.CV cs.RO 新提交

VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion

VidMap:利用时序结构实现基于视频的运动恢复结构

Zador Pataki, Paul-Edouard Sarlin, Marc Pollefeys

机构 * ETH Zurich(苏黎世联邦理工学院) Google(谷歌) Microsoft(微软)

AI总结 VidMap结合SLAM序列约束与SfM全局优化,利用宽基线匹配和深度先验,在多样挑战性数据集上,较最新SLAM和SfM更鲁棒准确,可实现任意长未标定视频的度量重建。

详情
AI中文摘要

为任意无约束视频精确恢复相机的标定参数与度量位姿,可为导航和场景理解解锁大规模训练数据。该问题的主流方法存在严重局限:同时定位与建图(SLAM)因因果增量式特性,对初始化和瞬时故障敏感,常过度优化以适配实时运行,且通常需要已知相机标定参数;而运动恢复结构(SfM)通常不考虑图像顺序,可实现最优初始化与全局优化,但对视觉对称性和极端运动缺乏鲁棒性。为弥合这一差距,本文提出一种结合SLAM强序列约束与离线SfM灵活性及全局优化的系统,可对任意长时长、未标定视频实现度量重建。该系统利用宽基线密集图像匹配的最新进展,将时序排序作为可靠回环检测的核心要素,并通过度量单目深度先验增强全局优化。对包含极端运动和视觉对称性的多样挑战性数据集的全面评估表明,本文方法在给定或未知相机标定参数的情况下,相较于经典或学习型的最新SLAM和SfM方法,鲁棒性和准确性显著更优。代码已公开于https URL。

英文摘要

Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.

URL PDF HTML 收藏
2607.27189 2026-07-30 cs.CL cs.AI cs.HC 新提交

APEX-Accounting

APEX-Accounting:评估前沿模型会计任务处理能力的基准

Julien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen

机构 * Mercor Ramp

AI总结 该研究推出会计基准APEX-Accounting,评估前沿模型的会计任务处理能力,发现模型表现存在差距,还观察到辛普森悖论,可应要求对模型开展排行榜评估。

Comments Public dev set: https://huggingface.co/datasets/mercor/apex-accounting

详情
AI中文摘要

我们推出由Mercor与Ramp合作构建的基准APEX-Accounting,用于评估前沿模型是否能完成会计师的实际工作,任务包括账目核对、应计费用、过账交易和生成报告。私有评估集包含160个任务,分为10个领域,每个领域包含会计系统以及电子表格、PDF和其他文件。所有任务均由会计和簿记专家编写并解决,他们还制定了评分标准。在9个前沿模型中,Claude-Fable-5(Max)以56.4%的Mean Criteria@3领先,Muse-Spark-1.1(xHigh)以52.6%紧随其后。没有模型的Pass^8得分超过2.6%(GPT-5.6-Sol (Max+Pro)),最高的Pass@8得分为21.5%(Muse-Spark-1.1(xHigh))。我们将token预算从1美元提高到50美元时,观察到辛普森悖论的一种情况:随着token预算增加,得分上升,但在给定的受预算约束的测试框架内,模型在花费更多token的任务上得分更低。由于APEX-Accounting是封闭基准,可应要求对任何前沿模型运行排行榜评估。

英文摘要

We introduce APEX-Accounting, a benchmark built by Mercor in partnership with Ramp, to assess whether frontier models can do the real work of accountants. Tasks include reconciling accounts, accruing expenses, posting transactions, and producing reports. The private eval set comprises 160 tasks, split across 10 worlds. Each world contains an accounting system, as well as spreadsheets, PDFs, and other files. Every task was authored and solved by experts in accounting and bookkeeping, who also wrote grading rubrics. Across nine frontier models, Claude-Fable-5 (Max) leads with 56.4% Mean Criteria@3, ahead of Muse-Spark-1.1 (xHigh) at 52.6%. No model scores more than 2.6% Pass^8 (GPT-5.6-Sol (Max+Pro)) and the highest Pass@8 is 21.5% (Muse-Spark-1.1 (xHigh)). We experiment with increasing the token budget from $1 to $50 and observe an instance of Simpson's paradox: scores increase as the token budget increases but within a given budget-constrained harness, scores are lower on tasks where the model spends more tokens. As APEX-Accounting is a closed benchmark, leaderboard evals can be run for any frontier model on request.

URL PDF HTML 收藏
2607.27183 2026-07-30 cs.CL 新提交

Pangram 4 Technical Report

Pangram 4 技术报告

Ben Glickenhaus, Katherine Thai, Jenna Russell, Elyas Masrour, Yue Han, Max Spero, Bradley Emi

机构 * Pangram Labs(Pangram实验室) University of Maryland(马里兰大学)

AI总结 Pangram Labs 推出的 Pangram 4 是基于深度学习的 AI 文本分类模型,其在 AUROC、误判率等指标上优于前代模型,具备更强的泛化与鲁棒性,在 AI 文本检测任务中达到当前最优性能。

详情
AI中文摘要

我们推出了 Pangram 4,这是 Pangram Labs 最新的基于深度学习的 AI 文本分类模型。该模型的 AUROC 达到 0.9916,假阳性率为 0.0041%,假阴性率为 0.3396%。与 Pangram 3 相比,Pangram 4 不仅整体准确率更高,还展现出更优的分布外泛化能力和对抗攻击鲁棒性。Pangram 4 的另一项创新贡献是其区分细粒度编辑内容以及 AI 与人类合著混合文本的能力有所提升,我们在边界检测任务和交错式 AI 辅助文本检测上均验证了其性能改进。最后,我们在标准 AI 检测基准上报告了相关指标,结果显示 Pangram 4 在多种设置和领域的 AI 文本检测任务中达到了当前最优性能。

英文摘要

We present Pangram 4, the latest deep-learning-based AI-text classification model from Pangram Labs. We achieve an AUROC of 0.9916 with a false positive rate of 0.0041% and a false negative rate of 0.3396%. In addition to its increased overall accuracy compared with Pangram 3, Pangram 4 exhibits superior out-of-distribution generalization and robustness to adversarial attacks. Another novel contribution of Pangram 4 is its improved ability to distinguish fine-grained edits and mixed AI-human co-authored text. We demonstrate improvements to both boundary detection tasks and the detection of interleaved AI assistance. Finally, we report metrics on standard AI detection benchmarks showing that Pangram 4 achieves state-of-the-art performance on the AI text detection task across a wide variety of settings and domains.

URL PDF HTML 收藏
2607.27180 2026-07-30 cs.CV cs.RO 新提交

HumanCLAW: Can Vision-Language Models Act Through a Body?

HumanCLAW:视觉-语言模型能否通过实体身体执行动作

Siyao Li, Jiawei Gu, Shuai Liu, Kairui Hu, Zekun Li, Linjie Li, Chengcheng Tang, Po-Chen Wu, Ivan Shugurov, Lingni Ma, Michael Zollhoefer, Sizhe An, Abhay Mittal, Amy Zhao, Ranjay Krishna, Manling Li, Ziwei Liu, Chuan Guo

机构 * Meta Nanyang Technological University(南洋理工大学) University of Washington(华盛顿大学) Brown University(布朗大学) Northwestern University(西北大学)

AI总结 本研究提出HumanCLAW框架与HumanCLAW-Bench基准,测试9个先进VLM发现其均未解决具身动作任务,最优仅16.8%成功率,核心缺失具身自我意识。

Comments Project page: https://human-claw.github.io/

详情
AI中文摘要

评估视觉-语言模型(VLM)能否通过物理实体身体执行动作颇具挑战,因为动作结果会耦合VLM的决策与运动控制。当任务失败时,很难区分是VLM做出了错误选择,还是运动控制器执行失败,例如失去平衡摔倒。本研究提出HumanCLAW,这是一个将动作决策与底层执行解耦的评估框架。每一步,搭载的现成VLM会发出原子技能指令,该指令被转换为亚秒级的连续全身运动片段,伴随重力、碰撞等真实物理后果。如此,实体身体可在物理世界自由动作,而执行侧的干扰、平衡与运动误差被排除,可测量的仅为模型的动作智能:其逐时刻选择身体下一步应执行动作的能力。基于该框架,构建了HumanCLAW-Bench:在41个室内场景中包含1218个长视野、第一人称视角的查找-导航-交互任务 episode。测试9个最先进的VLM后发现,没有一个能解决该基准任务,最优模型仅达到16.8%的成功率。研究还指出,识别目标并非瓶颈,当前VLM缺失的是具身自我意识:它们会丢失自身身体的状态,无法判断身体位置、是否到达目标或是否碰到障碍物。

英文摘要

Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it, e.g., losing balance and falling. In this work, we introduce HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution. At every step, a harnessed, off-the-shelf VLM issues an atomic skill command, and the command is translated into a sub-second chunk of continuous full-body motion with real physical consequences, including gravity and collisions. The body can therefore act freely in the physical world, while execution-side disturbances, balance and motor errors, are factored out. What remains measurable is the model's action intelligence: its moment-to-moment choice of what the body should execute next. Based on this framework, we build HumanCLAW-Bench: 1,218 long-horizon, egocentric find-navigate-interact episodes across 41 indoor scenes. We test nine state-of-the-art VLMs and find that none solves the benchmark; the best model reaches only a 16.8% success rate. Recognizing the target is not the bottleneck. What current VLMs lack is embodied self-awareness: they lose track of their own body, failing to tell where it is, whether it has reached the goal, or whether it has hit an obstacle.

URL PDF HTML 收藏
2607.27178 2026-07-30 cs.CL cs.IR 新提交

DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search

DenseOn与LateOn:面向多语言、长上下文及代码检索的全开放密集型与晚交互模型

Raphaël Sourty, Antoine Chaffin, Paulo Roberto Moura Junior, Amélie Chatelain

机构 * LightOn(莱特昂)

AI总结 该研究提出开放端到端方案训练检索模型,构建DenseOn、LateOn等模型,在BEIR上创同规模最优,发现晚交互模型泛化性更优,公开了模型、数据与代码。

Comments 21 pages, 3 figures, 12 tables

详情
AI中文摘要

当前最先进的检索模型越来越依赖封闭的训练数据,导致可复现性差距。本文提出一种用于训练检索模型的开放端到端方案,并研究通过翻译训练将英文监督信号迁移至多语言检索的效果。首先从34个公开来源的14亿对数据中重构并整理出6.65亿个英文对比预训练对,同时利用挖掘的难负例构建188万个监督微调对,训练得到两个1.49亿参数的模型:单向量密集模型DenseOn和ColBERT风格的晚交互模型LateOn。二者在BEIR数据集上的平均nDCG@10分别达到56.20和57.22,在该参数规模下创下新的最先进结果。随后将已验证的英文数据翻译成8种语言,得到包含跨语言样本的28亿对数据,并基于mmBERT-base构建两个3.07亿参数的模型mDenseOn和mLateOn。尽管共享主干网络、数据和目标,但二者的表示行为存在差异:密集模型在英文和翻译语言上表现强劲,但在翻译训练支持范围外性能下降;而晚交互模型对未见过的语言和文字表现出更好的泛化能力。这表明词元级匹配将翻译训练从目标语言扩展策略转变为多语言泛化方案。本文公开发布了相关模型、数据集和训练代码。

英文摘要

State-of-the-art retrieval models increasingly rely on closed training data, creating a reproducibility gap. We present an open end-to-end recipe for training retrieval models and study how English supervision transfers to multilingual retrieval through translate-train. We first reconstruct and curate 665M English contrastive pre-training pairs from 1.4B pairs across 34 public sources and build 1.88M supervised fine-tuning pairs with mined hard negatives. Training yields two 149M-parameter models: DenseOn, a single-vector dense model, and LateOn, a ColBERT-style late-interaction model. They achieve 56.20 and 57.22 average nDCG@10 on BEIR, respectively, setting new state-of-the-art results for this size class. We then translate the validated English data into eight languages, yielding 2.8B pairs with cross-lingual samples, and train mDenseOn and mLateOn, two 307M-parameter models built on mmBERT-base. Despite sharing their backbone, data, and objectives, their representations behave differently: the dense model is strong on English and translated languages but degrades outside translate-train support, whereas the late-interaction model generalizes better to unseen languages and scripts. This suggests that token-level matching turns translate-train from a target-language expansion strategy into a multilingual generalization recipe. We publicly release the models, datasets, and training code.

URL PDF HTML 收藏
2607.27177 2026-07-30 cs.AI cs.HC cs.MA 新提交

Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork

面向临时团队中任务无关适应的伙伴能力估计

Peter Tisnikar, Maja Swieczkowska, Benteng Ma, Gerard Canal, Matteo Leonetti

机构 * King’s College London(伦敦国王学院)

AI总结 该研究针对临时团队协作中伙伴能力隐藏与人类行为不可预测问题,提出CE-CM及CE-CM-Div方法,通过仿真采样与多样规划器rollout实现任务无关的能力估计,提升了团队协作的可行性与鲁棒性。

Comments 44 pages, 18 figures, submitted

详情
AI中文摘要

与新颖多样的伙伴开展有效协作是自主智能体的关键技能。当前大多数临时团队协作(Ad-Hoc Teamwork, AHT)方法假设智能体将在单一固定任务上协作,且伙伴的能力(其成功执行期望动作的能力)已被知晓。但现实中,伙伴的真实能力往往是隐藏的,人类协作者在拥有多种有效策略的任务上可能表现欠佳。为解决这些局限,我们将临时团队协作扩展至多任务场景,将其重新定义为在伙伴能力隐藏条件下、去中心化执行的联合规划问题。我们提出CE-CM(基于上下文模型的能力估计),这是一种近似贝叶斯方法,用于推断任务无关的能力向量。通过基于仿真的采样,智能体估计能力并生成上下文多智能体马尔可夫决策过程用于规划。该方法无需群体预训练,仅从少量任务中在线修正信念。为应对人类的不可预测性,我们提出CE-CM-Div,其扩展方案通过多样的规划器rollout而非单一最优轨迹评估能力假设。仿真实验表明,CE-CM可快速恢复隐藏能力、减少不可行动作分配并适应随时间的变化。此外,在针对15名参与者的225条轨迹的离线人类研究中,CE-CM-Div相比基线CE-CM大幅提升了能力估计效果。我们的结果表明,基于能力的建模在研究场景中是一种有前景的可解释、任务无关表示,证明考虑行为多样性对稳健的人机协作至关重要。

英文摘要

Effective collaboration with novel and diverse partners is a crucial skill for autonomous agents. Most current ad-hoc teamwork (AHT) approaches assume that agents will collaborate on a single, fixed task and that the partner's capabilities, their ability to successfully execute the desired action, are already known. In reality, a partner's true capabilities are often hidden, and human collaborators may act sub-optimally on tasks with multiple valid strategies. To address these limitations, we extend ad-hoc teamwork into a multi-task setting by re-framing it as a problem of joint planning with decentralised execution under hidden partner capabilities. We introduce CE-CM (Capability Estimation via Contextual Models), an approximate Bayesian method that infers task-invariant capability vectors. By using simulation-based sampling, the agent estimates capabilities and induces a contextual Multi-agent Markov Decision Processes for planning. This approach requires no population pre-training and refines its beliefs online from just a few tasks. To account for human unpredictability, we propose CE-CM-Div, an extension that evaluates capability hypotheses against diverse planner rollouts rather than a single optimal trajectory. Simulated experiments demonstrate that CE-CM rapidly recovers hidden capabilities, reduces infeasible action assignments, and adapts to changes over time. Furthermore, in an offline human study of 225 trajectories from 15 participants, CE-CM-Div substantially improved capability estimates over the baseline CE-CM method. Our results suggest capability-based modelling is a promising interpretable, task-agnostic representation in the studied settings, demonstrating that accounting for behavioural diversity is essential for robust human-AI teaming.

URL PDF HTML 收藏
2607.27155 2026-07-30 cs.AI cs.CL cs.HC 新提交

OmegaUse-OfficeVal: Benchmarking LLM Agents on Long-Horizon Office-Suite Tasks with Economic Grounding

OmegaUse-OfficeVal:基于经济基准的长周期办公套件任务LLM智能体评测基准

Jingbo Zhou, Yusai Zhao, Qi Bao, Jingjia Cao, Zhenghai Chen, Chang Gao, Kaiqi Guo, Muxin Guo, Mingxuan Li, Xinjiang Lu, Yanru Ma, Yixiong Xiao, Zenghui Zhang, Le Zhang, Hua Wu

机构 * Baidu Inc.(百度公司)

AI总结 OmegaUse-OfficeVal是带经济基准的长周期办公套件任务LLM智能体评测基准,含100项任务,评测发现前沿LLM成本低速度快但质量未达人类水平,代码与数据集已开源。

详情
AI中文摘要

大型语言模型(LLM)智能体被期望越来越多地协助用户完成任务,但现有基准在评估智能体能否以合理成本执行办公套件工作流程方面支持有限。我们引入OmegaUse-OfficeVal,这是一个用于评测长周期办公套件任务LLM智能体的基准,具备任务级经济基准。该基准包含100项任务,这些任务源自从业者提出的办公套件需求,并通过隐私保护流程调整而来。平均而言,这些任务需要2.32小时的人工劳动完成。该基准的一个重要特征是,每项任务都配有两个经济信号:人工劳动时间和任务价格代理,这些信号可实现人工成本与LLM推理成本的直接比较,以及价值加权评估。为支持稳定评测,我们基于细粒度规则开发了基于代码的验证器。我们对多个前沿LLM以及人类基线进行了评测,尽管所有被评测的LLM都比人类工作者便宜得多且速度快得多,但它们尚未达到人类水平的交付质量。代码和数据集已完全开源,更多信息可在我们的项目网站获取:this https URL。

英文摘要

Large language model (LLM) agents are increasingly expected to assist users in completing tasks. However, existing benchmarks provide limited support for evaluating whether agents can carry out office-suite workflows at a reasonable cost. We introduce OmegaUse-OfficeVal, a benchmark for evaluating LLM agents on long-horizon office-suite tasks with task-level economic grounding. The benchmark comprises 100 tasks derived from office-suite requests proposed by practitioners and adapted through a privacy-preserving process. On average, these tasks require 2.32 hours of human labor to complete. An important feature of the benchmark is that each task is paired with two economic signals: human labor time and task price proxy. These signals enable direct comparisons between human costs and LLM inference costs, as well as value-weighted evaluation. To support stable evaluation, we develop code-based verifiers from fine-grained rubrics. We evaluate several frontier LLMs together with a human baseline. Although all evaluated LLMs are substantially cheaper and faster than human workers, they have not yet approached human-level deliverable quality. The code and dataset are fully open-sourced, and more information is available on our project website: https://omegause-officeval.github.io.

URL PDF HTML 收藏
2607.27154 2026-07-30 cs.CV cs.AI 新提交

Anatomy Contextualized Adaption of CT Foundation Models

CT基础模型的解剖学上下文适配

Roshan Kenia, Stephanie L McNamara, William Lotter

机构 * Harvard Medical School(哈佛医学院) Dana-Farber Cancer Institute(达纳-法伯癌症研究所) Massachusetts General Hospital(麻省总医院) Brigham and Women’s Hospital(布里格姆妇女医院)

AI总结 本研究提出轻量框架ACA,适配冻结CT基础模型实现解剖学级视觉-语言对齐,在Merlin、CT-RATE数据集上零样本发现分类性能优于基线,训练耗时短且可保留增强全局解剖学上下文。

Comments Accepted to the European Conference on Computer Vision (ECCV) 2026 MedFM-Bench Workshop

详情
AI中文摘要

CT视觉-语言基础模型在下游任务中展现出良好性能,但通常采用全体积表示,会稀释细粒度的解剖学信号。细粒度视觉-语言预训练通过将解剖学级视觉特征与特定解剖学术语文本对齐来解决该问题,但会丢失全体积模型提供的全局上下文,且现有细粒度方法从头开始训练,计算成本较高。我们提出解剖学上下文适配(Anatomy Contextualized Adaptation, ACA),这是一种轻量框架,用于适配冻结的CT基础模型表示,实现解剖学级视觉-语言对齐,同时增强全局上下文。ACA使用TotalSegmentator将CT体积分解为解剖学级嵌入,通过捕捉跨解剖学关系的Transformer对嵌入进行优化,并将其与从放射学报告中提取的每个解剖学及扫描级文本对齐。在Merlin和CT-RATE数据集上的评估显示,ACA在零样本发现分类任务中始终优于冻结基础模型基线和现有细粒度方法,且嵌入缓存后训练耗时不足1小时。ACA的跨解剖学Transformer学习到的注意力权重还显示出合理的跨解剖学上下文路由。总体而言,这些结果表明ACA是一种轻量方法,可在适配CT基础模型到基于解剖学的视觉-语言对齐的同时,保留并增强全局解剖学上下文。

英文摘要

CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context.

URL PDF HTML 收藏
2607.27152 2026-07-30 cs.LG 新提交

Skillful forecasting of offshore winds from satellite scatterometer constellations

基于卫星散射仪星座的近海风精准预报

Francesco Pinto, Luca Lanzilao, Paco Lopez Dekker, Angela Meyer

机构 * Delft University of Technology(代尔夫特理工大学) Bern University of Applied Sciences(伯尔尼应用科技大学)

AI总结 本文提出首个基于卫星的近海风临近预报框架WindCastNet,采用部分卷积LSTM网络,在北海评估中较HARMONIE MEPS模型及持续性方法提升了近海风预报精度,为海洋天气预报提供新途径。

详情
AI中文摘要

日内近海风的精准预报对电力系统运行及日益增长的近海风能并网愈发重要。业务预报主要依赖数值天气预报(NWP),但NWP未针对数分钟至数小时的预见期优化,该时段初始条件精度主导预报技能。尽管卫星散射仪观测常被同化进NWP,但此前从未被直接用于预报。本文提出WindCastNet,首个基于卫星的近海风速与风向临近预报框架,开创了从时空不规则卫星观测中学习的日内预报新范式。WindCastNet基于卫星散射仪星座获取的观测,预测近海风场;其采用部分卷积长短期记忆网络,可利用欧洲、中国、印度散射仪的微波雷达观测,即便这些观测存在空间覆盖不规则、采样异步、重访时间可变的问题。该网络对空间观测掩码与观测间隔进行编码,同时通过连续时间表示实现任意预见期的预报。在北海的评估显示,与HARMONIE MEPS模型相比,WindCastNet在1小时预见期时均方根误差降低23%,2小时预见期时降低7%,且在预报前三小时内较持续性方法表现提升9%-15%。在强风条件及空间非均匀流场下,预报技能有所下降。这些结果表明,卫星散射仪星座可提供独立且具竞争力的短期近海风预报源,为可再生能源预报及更广泛的海洋天气应用(包括热带气旋临近预报)开辟新机遇。

英文摘要

Accurate intraday forecasts of offshore wind are becoming increasingly important for power system operation and the integration of growing shares of offshore wind energy. Operational forecasts rely predominantly on numerical weather prediction (NWP), which is not optimized for lead times of minutes to hours, where initial-condition accuracy dominates forecast skill. Although satellite scatterometer observations are routinely assimilated into NWP, they have not previously been used directly for forecasting. Here we present WindCastNet, the first satellite-based nowcasting framework for offshore wind speed and direction, introducing a new paradigm for intraday forecasting that learns from spatiotemporally irregular satellite observations. WindCastNet predicts offshore wind fields from observations acquired by satellite scatterometer constellations. WindCastNet employs a partial convolutional long short-term memory network that exploits microwave radar observations from the European, Chinese, and Indian scatterometers despite their irregular spatial coverage, asynchronous sampling, and variable revisit times. Spatial observation masks and inter-observation intervals are encoded, while a continuous temporal representation enables forecasts at arbitrary lead times. Evaluated over the North Sea, WindCastNet reduces the root-mean-square error by 23% and 7% relative to the HARMONIE MEPS model at lead times of 1 and 2 h, respectively, and outperforms persistence by 9-15% during the first three forecast hours. Forecast skill decreases under strong-wind conditions and spatially non-uniform flow. These results demonstrate that satellite scatterometer constellations can provide an independent and competitive source of short-term offshore wind forecasts, opening new opportunities for renewable energy forecasting but also broader marine weather applications, including tropical cyclone nowcasting.

URL PDF HTML 收藏
2607.27145 2026-07-30 cs.CV 新提交

Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications

面向决策关键型应用的多模态大语言模型中可解释且资源高效的空间推理

Piyush Jain, Kousik Dasgupta, Rajarshi Roy, Subarna Tripathi

机构 * Heritage Institute of Technology(遗产技术学院) Kalyani Government Engineering College(卡利亚尼政府工程学院) IAIRO Intel Corporation(英特尔公司)

AI总结 针对多模态大语言模型空间判断不透明及细粒度空间理解不足的问题,提出无需训练的ByDeWay-V2框架,结合YOLO-World-L注入空间谓词,在多个基准上显著提升空间推理性能,适配资源受限场景。

Comments 14 pages

详情
AI中文摘要

随着多模态大语言模型(MLLMs)越来越多地部署在机器人、具身AI和安全监控等决策关键型流程中,其空间判断的不透明性限制了操作人员的信任度和可审计性。MLLMs具备强大的推理能力,但往往在细粒度空间理解和对象幻觉方面存在不足。此前的工作ByDeWay提出了基于分层深度的提示(LDP),这是一种无需训练的框架,通过利用单目深度估计构建提示来缓解幻觉。然而,粗糙的深度分层无法解决同一几何平面内对象之间的空间关系,比如投影关系(“在左侧”“在上方”)和拓扑关系(“在内部”“接触”)。我们提出了ByDeWay-V2,该框架将显式的空间关系上下文与深度线索相结合,以人类可读的谓词形式呈现,作为下游决策支持的可审计证据。我们使用开放词汇对象检测器YOLO-World-L,计算检测到的对象之间的成对几何关系,并将其作为结构化空间谓词注入到MLLM的提示中,无需任何训练即可弥合3D场景深度与2D空间语义之间的差距。我们在视觉空间推理(VSR)和BLINK基准上对多个MLLMs评估了ByDeWay-V2,通过POPE评估幻觉接地。在BLINK空间子集上,对于Qwen2.5-VL,ByDeWay-V2相比LDP实现了46%的相对F1提升,并且在VSR上将BLIP-Base的空间推理从接近随机的性能提升至具有竞争力的0.53的F1值。我们最轻量化的配置在CPU上严格控制在40个令牌的上下文预算下运行,表明该框架适用于资源受限的实时决策支持场景。

英文摘要

As Multimodal Large Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics, embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong reasoning but often struggle with fine-grained spatial understanding and object hallucination. Prior work, ByDeWay, introduced Layered-Depth-Based Prompting (LDP), a training-free framework that mitigates hallucinations by structuring prompts using monocular depth estimation. However, coarse depth layering falls short in resolving object-to-object spatial relationships within the same geometric plane, such as projective ("left of", "above") and topological ("inside", "touching") relations. We propose ByDeWay-V2, which integrates explicit spatial relational context alongside depth cues, expressed as human-readable predicates that serve as auditable evidence for downstream decision support. Using an open-vocabulary object detector (YOLO-World-L), our framework computes pairwise geometric relations between detected objects and injects them as structured spatial predicates into the MLLM prompt, bridging 3D scene depth and 2D spatial semantics without any training. We evaluate ByDeWay-V2 on the Visual Spatial Reasoning (VSR) and BLINK benchmarks across multiple MLLMs, with hallucination grounding assessed via POPE. On the BLINK spatial subset, ByDeWay-V2 achieves a 46 percent relative F1 improvement over LDP for Qwen2.5-VL, and recovers BLIP-Base's spatial reasoning on VSR from near-random performance to a competitive F1 of 0.53. Our lightest configuration operates under a strict 40-token context budget on CPU, showing the framework's suitability for resource-constrained, real-time decision-support settings.

URL PDF HTML 收藏
2607.27143 2026-07-30 cs.LG cs.AI 新提交

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

面向不平衡高风险决策支持的代价敏感共形预测与人在环弃权:多领域基准

Manpreet Singh, Akshatha Srikantha, Shyamal Lakhanpal

机构 * Boston University(波士顿大学) University of California(加利福尼亚大学) University of Maryland(马里兰大学)

AI总结 该研究针对高风险决策系统的类别不平衡与非对称代价问题,通过多领域基准测试对比多种共形预测及弃权机制,发现Mondrian CP可显著提升少数类覆盖,结合代价控制弃权能降低预期决策代价,为相关系统部署提供实用指导。

详情
AI中文摘要

信贷评分、欺诈检测、医疗保健和工业安全等领域的高风险决策系统,需要在严重类别不平衡和非对称误差代价下提供可靠的不确定性量化。标准边际共形预测(CP)可提供有效的整体覆盖保证,但我们发现其对稀有、高代价的少数类覆盖严重不足,在某些数据集上少数类覆盖率低至0.5%。为表征并解决这一局限,我们开展了全面基准测试,在15个现实世界不平衡表格数据集、7种分类模型、3种概率校准技术和10个随机种子上,对比了边际CP、类条件(Mondrian)CP和代价控制弃权机制,共完成3150次实验运行。结果显示,Mondrian CP可恢复有效的少数类覆盖,相比边际CP平均提升61.7个百分点的少数类覆盖率(p < 1e-80)。此外,结合Mondrian CP与代价控制弃权机制,在实际人工审核预算下,相比标准决策边界、基于置信度的拒绝器和风险控制拒绝器,可显著降低预期决策代价。我们还量化了特定数据集的盈亏平衡点阈值,即把模糊实例交由人类专家处理变得具有成本效益的临界点。这些发现为在高风险决策支持系统中部署与分布无关、感知代价的不确定性量化提供了实用指导。

英文摘要

High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asymmetric error costs. Standard marginal conformal prediction (CP) provides valid overall coverage guarantees; however, we show that it severely under-covers rare, costly minority classes, with minority-class coverage dropping to as low as 0.5% on certain datasets. To characterize and address this limitation, we conduct a comprehensive benchmark comparing marginal CP, class-conditional (Mondrian) CP, and cost-controlled abstention mechanisms across 15 real-world imbalanced tabular datasets, 7 classification models, 3 probability calibration techniques, and 10 random seeds, resulting in 3,150 experimental runs. Our results show that Mondrian CP restores valid minority-class coverage, achieving an average minority-coverage improvement of 61.7 percentage points over marginal CP (p < 1e-80). Furthermore, combining Mondrian CP with cost-controlled abstention significantly reduces expected decision cost compared with standard decision boundaries, confidence-based rejectors, and risk-controlled rejectors under realistic human review budgets. We further quantify dataset-specific break-even thresholds at which deferring ambiguous instances to human experts becomes cost-effective. These findings provide practical guidance for deploying distribution-free, cost-aware uncertainty quantification in high-stakes decision support systems.

URL PDF HTML 收藏
2607.27139 2026-07-30 cs.CV 新提交

SeasonStereo: Robust Dense Stereo Matching for Multi-Date Satellite Imagery via Generative AI

SeasonStereo:基于生成式AI的多日期卫星图像鲁棒密集立体匹配

Álvaro Díaz-Laureano, Roger Marí, Elías Masquil, Pablo Arias, Gabriele Facciolo

机构 * Eurecat(欧罗卡特技术中心) IIE, Facultad de Ingeniería, Universidad de la República(乌拉圭共和国大学工程学院IIE) Universitat Pompeu Fabra(庞培法布拉大学) Université Paris-Saclay(巴黎萨克雷大学) CNRS(法国国家科学研究中心) ENS Paris-Saclay(巴黎萨克雷高等师范学校) Institut Universitaire de France(法国大学研究院)

AI总结 SeasonStereo框架利用带可控季节外观变化的合成图像对训练并结合基础模型零样本几何先验,实现低成本、高精度的多日期卫星图像密集立体匹配,支持大规模三维重建。

详情
AI中文摘要

从卫星图像进行精确三维重建通常依赖近同步立体像对,这限制了其在多日期图像存在季节和光照条件变化的历时场景中的适用性。训练对外观变化鲁棒的密集立体匹配模型是长期存在的挑战,因为大规模获取对齐的多日期图像和真实几何信息成本高昂。我们提出SeasonStereo,这是一个可扩展框架,通过在具有可控季节外观变化的合成图像对上训练,同时利用基础模型的零样本几何先验,解决多日期卫星图像的视差估计问题。SeasonStereo达到了最先进的LiDAR监督模型的精度,同时产生更清晰的几何细节,无需对齐的真实多日期训练产品或LiDAR衍生标签。因此,SeasonStereo为从异构卫星图像进行大规模三维重建提供了实用路径,降低了监督成本。

英文摘要

Accurate 3D reconstruction from satellite imagery typically relies on near-simultaneous stereo pairs, limiting its applicability to diachronic settings where multi-date images exhibit varying seasonal and illumination conditions. Training dense stereo matching models robust to appearance changes is a long-standing challenge, as aligned multi-date imagery and ground-truth geometry are costly to obtain at scale. We propose SeasonStereo, a scalable framework that addresses disparity estimation from diachronic satellite images by training on synthetic image pairs with controlled seasonal appearance variation, while leveraging zero-shot geometric priors from foundation models. SeasonStereo matches the accuracy of state-of-the-art LiDAR-supervised models, while producing sharper geometric details without requiring aligned real multi-date training products or LiDAR-derived labels. As a result, SeasonStereo offers a practical path toward large-scale 3D reconstruction from heterogeneous satellite images with reduced supervision cost.

URL PDF HTML 收藏
2607.27130 2026-07-30 cs.AI 新提交

AgentMap: Joint Equivalence and Subsumption Discovery for Ontology Matching

AgentMap:用于本体匹配的联合等价与包含关系发现

Yiping Song, Jiaoyan Chen, Renate Schmidt, Hui Yang, Wen Zhang

机构 * The University of Manchester(曼彻斯特大学) Zhejiang University(浙江大学)

AI总结 本文提出统一等价与包含发现的混合本体匹配任务,构建基于LLM的多智能体框架AgentMap,在三类设置下的本体匹配任务中均取得优异性能。

Comments 21 pages, 5 figures

详情
AI中文摘要

本体匹配(OM)传统上被表述为等价关系发现或包含关系匹配,现有OM系统仅能识别一种语义对应类型,无法同时发现等价和包含映射。本文提出统一等价与包含发现的新型OM任务——混合本体匹配(HOM),并据此构建基于大语言模型(LLM)的多智能体OM框架AgentMap,其由一系列相互关联的语义决策实现。给定源本体中的一个概念,AgentMap整合语义检索、分层搜索及协作多智能体LLM推理,逐步探索目标本体,若存在等价概念则识别该概念,否则识别最细粒度的上位概念。本文进一步扩展四个OM数据集以构建HOM基准,并在混合、仅等价、仅包含三种设置下评估AgentMap。实验结果显示,AgentMap在混合设置下表现优异,同时在仅等价和仅包含设置下分别优于等价匹配和包含匹配的基线方法。

英文摘要

Ontology matching (OM) has traditionally been formulated as either equivalence discovery or subsumption matching. The existing OM systems identify only one type of semantic correspondence and cannot simultaneously discover equivalence and subsumption mappings. In this paper, we introduce Hybrid Ontology Matching (HOM), a new OM task that unifies equivalence and subsumption discovery, and accordingly propose a Large Language Model (LLM)-based multi-agent OM framework AgentMap that is implemented by a series of interdependent semantic decisions. Given a concept in the source ontology, AgentMap integrates semantic retrieval, hierarchical search, and collaborative multi-agent LLM reasoning to progressively explore the target ontology, identifying either the equivalent concept, if one exists, or the most fine-grained subsumer. We further extend four OM datasets for a HOM benchmark and evaluate AgentMap under hybrid, equivalence-only, and subsumption-only settings. Experimental results show that AgentMap achieves promising performance on the hybrid setting, and at the same time outperforms equivalence matching and subsumption matching baselines on the equivalence-only and subsumption-only settings, respectively.

URL PDF HTML 收藏
2607.27122 2026-07-30 cs.CV 新提交

Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs

基于小型视觉语言模型多任务学习的 grounded 胃肠道内窥镜视觉问答研究

Itbaan Safwan, Ramail Khan, Muhammad Annas Shaikh, Muhammad Atif Tahir

机构 * Institute of Business and Administration(商业管理学院)

AI总结 该研究针对胃肠道内窥镜视觉问答任务,提出多任务微调方案,利用现有数据集构建辅助任务,微调小型视觉语言模型,实现了准确率提升与答案-图像区域对齐优化。

Comments Accepted at EMA4MICCAI 2026 (Workshop on Efficient Medical AI, MICCAI 2026)

详情
AI中文摘要

胃肠道(GI)内窥镜图像分析已从单标签分类转向视觉问答(VQA),要求模型回答关于图像的自由格式临床问题。尽管近期视觉语言模型(VLMs)在该任务上取得了可观的回答准确率,但临床应用还需模型的内部表征能反映其答案背后的视觉证据。我们提出一种简单的多任务微调方案,利用现有VQA数据集构建辅助的 grounding 和描述任务,仅需极少额外标注:专家标注的息肉掩码可直接复用,而带有Grad-CAM定位的GI域预训练分类器为缺乏真实掩码的类别提供弱监督。我们在Kvasir-VQA-x1数据集上,针对仅VQA和多任务两种设置,用低秩适配微调了三种小型VLM主干,结果显示在分布内和分布外数据上,模型不仅取得了持续的准确率提升,还改进了答案 token 与相关图像区域的隐式对齐。

英文摘要

Gastrointestinal (GI) endoscopic image analysis has shifted from single-label classification toward visual question answering (VQA), where a model must answer free-form clinical questions about an image. While recent vision-language models (VLMs) achieve promising answer accuracy on this task, clinical adoption also requires the model's internal representations to reflect the visual evidence behind its answers. We propose a simple multi-task fine-tuning recipe that constructs auxiliary grounding and description tasks from an existing VQA dataset with minimal additional annotation: expert-annotated polyp masks are reused directly, while a GI-domain pretrained classifier with Grad-CAM localization provides weak supervision for finding categories that lack ground-truth masks. Three small VLM backbones are fine-tuned with low-rank adaptation under matched VQA-only and multi-task recipes on Kvasir-VQA-x1, and we show consistent accuracy gains together with improved implicit alignment between answer tokens and the relevant image region, evaluated on both in-distribution and out-of-distribution data.

URL PDF HTML 收藏
2607.27113 2026-07-30 cs.CV 新提交

Veritas++: Value-aware On-Policy Distillation for Perception-Enhanced AIGI Detection

Veritas++:面向感知增强的AIGI检测的值感知在线策略蒸馏

Hao Tan, Jun Lan, Zichang Tan, Ajian Liu, Zijian Yu, Chuanbiao Song, Huijia Zhu, Weiqiang Wang, Jun Wan, Zhen Lei

机构 * School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences (UCAS)(中国科学院大学先进交叉科学学院) Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所) School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院) Ant Group(蚂蚁集团) Sangfor Technologies Inc.(深信服科技股份有限公司)

AI总结 该研究针对现有AIGI检测模型的感知瓶颈,提出Veritas++框架,通过PoRL与VaOPD机制增强感知能力,提升了检测泛化性与效率。

详情
AI中文摘要

图像生成模型的能力不断提升,合成图像已成为开放媒体中的常规存在,这使得鲁棒且可泛化的AI生成图像(AIGI)检测愈发重要。多模态大语言模型(MLLMs)为黑盒二元评分提供了透明的替代方案,但我们发现当前基于MLLM的检测器在捕获细粒度异常方面仍存在显著的感知瓶颈,它们主要关注视觉证据的组织与合成方式,而未对内在感知进行优化。为缓解这一差距,我们提出Veritas++,这是一个以可靠感知作为真实性推理基础的感知增强推理框架。我们未直接优化模型的解释能力,而是将AIGI检测建立在三种基本感知能力之上,即捕获细粒度视觉细节、语义异常和像素级差异。基于这一见解,我们引入感知导向学习(PoRL),它用可验证的奖励替代开放式描述监督,以明确强化这些能力。为进一步将增强的感知与推理相结合,我们引入值感知在线策略蒸馏(VaOPD),这是一种自适应蒸馏机制,它优先考虑高价值蒸馏信号而非统一监督,通过特权自教师内化感知感知的推理。在标准、野外和新兴基准上的大量实验表明,Veritas++实现了良好的泛化性,感知学习有效弥合了感知差距并在检测上产生了无缝增益,而VaOPD进一步实现了高效的能力演进且不牺牲现有性能。代码和检查点可在this https URL获取。

英文摘要

The growing capability of image generation models has made synthetic images a routine presence in open media, making robust and generalizable AI-Generated Image (AIGI) detection increasingly essential. While multi-modal large language models (MLLMs) offer a transparent alternative to black-box binary scoring, we observe that current MLLM-based detectors still exhibit notable perception bottlenecks in capturing fine-grained anomalies. They primarily focus on how visual evidence is organized and synthesized, leaving the intrinsic perception less optimized. To mitigate this gap, we present Veritas++, a perception-enhanced reasoning framework that establishes reliable perception as the foundation of authenticity reasoning. Rather than directly optimizing the model's explanatory ability, we ground AIGI detection on three basic perception abilities, i.e., capturing fine-grained visual details, semantic anomalies and pixel-level differences. Building on this insight, we introduce Perception-oriented Learning (PoRL), which replaces open-ended description supervision with verifiable rewards to explicitly strengthen these capacities. To further integrate enhanced perception with reasoning, we introduce Value-aware On-Policy Distillation (VaOPD), an adaptive distillation mechanism that prioritizes high-value distillation signals over uniform supervision, internalizing perception-aware reasoning through a privileged self-teacher. Extensive experiments across standard, in-the-wild and emerging benchmarks demonstrate that Veritas++ achieves promising generalization. The perception learning effectively bridges the perception gap and yields seamless gains on detection, while VaOPD further enables efficient capability evolvement without sacrificing existing performance. Code and checkpoints are available at https://github.com/EricTan7/VeritasPP.

URL PDF HTML 收藏
2607.27106 2026-07-30 cs.LG 新提交

Hierarchical Spatio-Temporal Transformer for Coherent Emergency Department Forecasting

用于连贯急诊科预测的分层时空Transformer

Filipa Lino, Bárbara Tavares, Carlos Santiago, Cláudia Soares, Manuel Marques

机构 * Institute for Systems and Robotics(系统与机器人研究所) LARSyS Instituto Superior Técnico(高等技术学院) NOVA School of Science and Technology(NOVA科学技术学院)

AI总结 该研究针对现有急诊科预测方法忽略层级关联导致预测不连贯的问题,提出HierSTT分层时空Transformer框架,在葡萄牙ED数据集上实现多层面连贯预测,性能优于非分层基线及经典分层协调方法。

Comments Accepted at 11th Workshop on Data Science for Social Good - ECML PKDD 2026

详情
AI中文摘要

急诊科(ED)是医疗系统的关键接入点,但面临着来自不可预测的患者需求、季节性激增和非紧急就诊的持续压力。有效的急诊科规划需要在多个决策层面进行预测:医院需要本地需求估计以进行人员配置和床位管理,地区需要预测以协调医疗单位,国家当局需要全系统预测以进行容量规划。然而,大多数现有方法在单一层面独立预测急诊科需求,忽略了连接医院、地区和国家系统的层级关系,这可能产生不连贯的预测,即医院层面的预测无法一致地聚合为地区或国家需求。我们提出HierSTT,一种基于Transformer的分层框架,用于连贯的多层面急诊科预测。HierSTT在单个端到端模型中联合预测医院、地区和国家层面的需求,其中时间融合Transformer捕捉国家动态,而时空Transformer编码器-解码器模块以更高层面的预测为条件对地区和医院需求进行建模。一种感知连贯性的损失函数在训练过程中惩罚跨层面的不一致性。我们还引入了一个全国性的葡萄牙急诊科数据集,涵盖5个地区卫生管理局的81家医院,每个层面都有异质协变量。实验表明,与最佳非分层深度学习基线相比,HierSTT将平均WAPE降低了32%,并且优于所有经典的分层协调方法,同时在各层面产生接近连贯的预测。本工作的额外资源可在此https URL获取。

英文摘要

Emergency Departments (EDs) are critical access points in healthcare systems, yet they face persistent pressure from unpredictable patient demand, seasonal surges, and non-urgent visits. Effective ED planning requires forecasts at multiple decision-making levels: hospitals need local demand estimates for staffing and bed management, regions require forecasts to coordinate healthcare units, and national authorities need system-wide projections for capacity planning. However, most existing approaches forecast ED demand independently at a single level, ignoring the hierarchy linking hospitals, regions, and national systems. This can produce incoherent predictions, where hospital-level forecasts do not aggregate consistently to regional or national demand. We propose HierSTT, a hierarchical Transformer-based framework for coherent multi-level ED forecasting. HierSTT jointly predicts hospital, regional, and national level demand in a single end-to-end model. A Temporal Fusion Transformer captures national dynamics, while spatio-temporal Transformer encoder-decoder modules model regional and hospital demand conditioned on higher-level forecasts. A coherence-aware loss penalizes cross-level inconsistencies during training. We further introduce a nationwide Portuguese ED dataset covering 81 hospitals across 5 regional health administrations, with heterogeneous covariates at each level. Experiments show that HierSTT reduces average WAPE by 32\% relative to the best non-hierarchical deep learning baseline and outperforms all classical hierarchical reconciliation methods, while producing near-coherent predictions across levels. Additional resources associated with this work are available at https://github.com/FilipaLino/HierSTT.

URL PDF HTML 收藏
2607.27092 2026-07-30 cs.LG 新提交

Sky sphere representation in language models

语言模型中的天球表示

Aleksandr Berdnikov, Yevgeny Liokumovich

机构 * Fields Institute(菲尔兹研究所) Principles of Intelligence(智能原理机构) University of Toronto(多伦多大学)

AI总结 研究约1000亿参数的语言模型是否存在可从残差流解码的夜空图表示,发现多数开源模型具备该表示,其性能指标优异且为首个弯曲高维不可约特征流形实例,相关代码已公开。

详情
AI中文摘要

我们分析了约1000亿参数规模的语言模型是否存在可从其残差流中解码的夜空图表示。研究发现,多数被考察的开源模型确实具备此类表示,且在“夜空中这个天体附近有什么”这类提示词下,该表示常浮现至主成分顶部。除1个模型外,其余模型的该表示在留一法(LOO)测试中均显示出显著得分,可覆盖65%-85%的方差(R²分数),中位角误差低至12°-21°。我们验证了该表示并非来自相关平面表示的简单泄漏。据我们所知,该表示是首个弯曲高维不可约特征流形的实例,论文所用代码已发布在指定网址。

英文摘要

We analyze whether language models of size ~100B have a representation of the night sky map that is decodable from their residual stream. We find that most of the considered open-source models do have such a representation, and it often even surfaces to the top principal components on prompts that ask questions like ``what is close to this object in the night sky''. In all but one model this representation showed significant scores in LOO testing, containing up to 65-85% of variance ($R^2$-score) and having median angular error down to $12^\circ-21^\circ$. We verify that our representation is not a simple leak from a correlated flat representation. To our knowledge, this representation is the first example of a curved high-dimensional irreducible feature manifold. Codes used in the paper are published at https://github.com/l3erdnik/Decodable-sky

URL PDF HTML 收藏
2607.27087 2026-07-30 cs.CV 新提交

Step-Attention Refinement of DINOv3 Features for Efficient Anterior Eye Segmentation

用于高效眼前节分割的DINOv3特征的步注意力细化

Philippe Baumstimler, Jean-Mathieu Gagnon, Sébastien Gagné, Mathieu Duchesneau, Clément Playout, Lama Séoud

机构 * Polytechnique Montréal(蒙特利尔理工学院) LightX Innovations(LightX创新公司) Institut de l’Oeil des Laurentides(劳伦琴眼科研究所)

AI总结 本研究针对临床眼前节分割问题,提出基于蒸馏DINOv3 ViT-Small主干的轻量架构,引入步注意力特征细化模块,在私有数据集上实现最优性能与域偏移鲁棒性,为相关任务建立强基线。

详情
AI中文摘要

眼前节(Anterior Eye Segment, AES)分割是眼部生物识别技术及新兴临床图像分析应用的关键组成部分。然而,医疗场景中异构的采集条件和有限的标注数据,阻碍了现有方法的鲁棒性与泛化能力。DINOv3等基础模型(Foundation Models, FMs)具备强大的迁移能力,但将其表示高效适配密集预测任务仍具挑战性。本研究针对临床场景中的鲁棒AES分割问题,提出一种基于蒸馏DINOv3 ViT-Small主干的轻量型架构,引入步注意力特征细化模块,在卷积解码前逐步适配多级Transformer表示,以少量参数高效利用预训练特征。在包含8种眼科采集协议、标注7个解剖类别的333张临床采集AES图像的私有数据集上评估,与包括基于DINOv3的方法在内的卷积及Transformer基线方法相比,所提方法在全微调时达到85.55%的平均交并比(mIoU),取得最优整体性能,且在4个未见过的公开AES分割数据集上展现出最强的域偏移鲁棒性。这些结果为临床场景下的鲁棒AES分割建立了强基线,并凸显了解码器设计对有效适配基础模型表示至医学分割任务的重要性。

英文摘要

Anterior eye segment (AES) segmentation is a key component of both ocular biometrics and emerging clinical image analysis applications. However, heterogeneous acquisition conditions and limited annotations in medical settings hinder the robustness and generalization of existing methods. Foundation models (FMs) such as DINOv3 offer strong transfer capabilities, but efficiently adapting their representations to dense prediction tasks remains challenging. In this study, we investigate robust AES segmentation in clinical settings, and propose a lightweight architecture built upon a distilled DINOv3 ViT-Small backbone. We introduce a step-attention feature refinement module that progressively adapts multi-level transformer representations before convolutional decoding, enabling efficient exploitation of pretrained features with few parameters. We evaluate the proposed approach on a private dataset of 333 clinically acquired AES images spanning eight ophthalmic acquisition protocols and annotated for seven anatomical classes. Compared with convolutional and transformer-based baselines, including DINOv3-based methods, our approach achieves the best overall performance, reaching 85.55\% mIoU when fully fine-tuned. It also demonstrates the strongest robustness to domain shift across four unseen public AES segmentation datasets. These results establish a strong baseline for robust AES segmentation in clinical settings and highlight the importance of decoder design for effectively adapting FMs representations to medical segmentation tasks.

URL PDF HTML 收藏
2607.27085 2026-07-30 cs.RO cs.SY eess.SY 新提交

Controlled Experiments on Lane Changing by Transitional Autonomous Vehicle: Dataset and Behavioral Insights

过渡自动驾驶车辆变道的控制实验:数据集与行为洞察

Abhinav Sharma, Md Abdullah Al Hasan, Danjue Chen, George F. List

机构 * North Carolina State University(北卡罗来纳州立大学)

AI总结 本研究通过公共道路控制实验构建了NC-tALC数据集,刻画过渡自动驾驶车辆的强制变道行为,揭示其变道过程中的车距演变与碰撞风险规律,为相关评估与验证提供实证基准。

详情
AI中文摘要

本文提出了北卡罗来纳州过渡自动驾驶车辆变道(NC-tALC)数据集,并利用该数据集刻画过渡自动驾驶车辆(tAVs)的强制变道行为。它量化了变道过程中前后车距(lead--lag gaps)的演变情况,并研究了该操作过程中潜在碰撞风险的发展方式。研究团队在北卡罗来纳州 Apex 的一条公共道路上开展了包含78次强制变道试验的控制实地实验:四辆配备仪器的车辆营造出可重复的交通条件,同时改变变道车辆在候选目标间隙内的初始位置。研究人员对高分辨率RTK-GNSS/INS轨迹进行处理,以识别关键时间戳、计算前后车距和变道间隙,并采用基于时间间隙和速度的替代安全措施来估算交互作用。尽管初始条件存在显著差异,但前后车距始终收敛至接近车道交叉的相对狭窄范围。潜在碰撞风险随操作推进而增加,在物理进入车道附近达到峰值,且主要由与目标车道前车的交互作用主导。变道完成并不一定与碰撞风险消失同时发生。本研究是首批利用可重复公共道路实验对tAVs完整强制变道过程进行控制实证刻画的研究之一;NC-tALC数据集支持对整个操作过程中的行为和安全演变进行分析,而非仅针对间隙接受瞬间。该数据集和研究结果为评估自动变道行为、校准行为模型以及验证强制变道场景的仿真和安全评估方法提供了实证基准。

英文摘要

This paper presents the North Carolina Transitional Autonomous Vehicle Lane-Changing (NC-tALC) dataset and uses it to characterize mandatory lane-changing behavior of transitional automated vehicles (tAVs). It quantifies the evolution of lead--lag gaps throughout the lane-change process and examines how potential collision risk develops during the maneuver. A controlled field experiment comprising 78 mandatory lane-change trials was conducted on a public roadway in Apex, North Carolina. Four instrumented vehicles created repeatable traffic conditions while varying the lane changer's initial position within the candidate target gap. High-resolution RTK-GNSS/INS trajectories were processed to identify key timestamps, calculate lead, lag, and lane-change gaps, and estimate interactions using time-gap- and speed-based surrogate safety measures. Despite substantial differences in initial conditions, lead and lag gaps consistently converged toward a relatively narrow range near lane crossing. Potential collision risk increased as the maneuver progressed, peaked near physical lane entry, and was dominated by interactions with the target-lane leader. Lane-change completion did not necessarily coincide with the disappearance of collision risk. This study provides one of the first controlled empirical characterizations of the complete mandatory lane-change process of tAVs using repeatable public-road experiments. The NC-tALC dataset supports analysis of behavioral and safety evolution throughout the maneuver rather than only at the gap-acceptance instant. The dataset and findings provide empirical benchmarks for evaluating automated lane-changing behavior, calibrating behavioral models, and validating simulation and safety assessment methods for mandatory lane-change scenarios.

URL PDF HTML 收藏
2607.27081 2026-07-30 cs.AI cs.CL cs.CR cs.LG 新提交

On-Policy Distillation for LLM Safety: A Routing Approach to Template-Robust Realignment

面向大语言模型安全的在线策略蒸馏:一种针对模板鲁棒性的重对齐路由方法

Yongjian Guo, Wanlun Ma, Lingyu Shen, Xi Xiao, Sheng Wen

机构 * Tsinghua University(清华大学) Swinburne University of Technology(斯威本科技大学) EPFL(洛桑联邦理工学院)

AI总结 针对现有LLM安全防御的模板鲁棒性不足等问题,提出ROPD框架,通过建模输出分布差异实现重对齐,大幅降低模板不匹配风险,在防御有效性和能力保留上优于基线方法,建立了新的鲁棒重对齐标准。

详情
AI中文摘要

微调是使大语言模型(LLM)专业化的主流范式,但它暴露了一个关键漏洞:恶意数据提供者可在下游语料库中嵌入有害行为,使模型在保留专业技能的同时可按需违背人类价值观。现有安全重对齐防御措施在实践中常因三大关键局限失效:它们频繁导致专业技能的灾难性遗忘;当防御者无法观察攻击者的提示模板时,其有效性会崩溃;成功重对齐的模型仍易通过简单的系统提示切换被重新越狱。为应对这些挑战,我们提出基于路由的在线策略蒸馏(Routing-based On-Policy Distillation,ROPD),这是一种新型重对齐框架,用于建模对齐与受损输出概率分布之间的差异,而非拟合特定提示模板。我们开展了大量实验,在三个数据集、三个具有不同对齐强度的基础模型上,将ROPD与四种最先进的基线进行比较。结果表明,当基线防御措施面临模板不匹配时,通常会伴随下游任务性能的严重下降;相比之下,ROPD大幅降低了模板不匹配风险,在防御有效性和能力保留方面均保持了卓越的鲁棒性。尽管我们的分析显示ROPD并非完全不受模板偏移影响,但其性能下降与现有方法相比可忽略不计,为鲁棒的LLM重对齐建立了新的标准。

英文摘要

Fine-tuning is the dominant paradigm for specializing large language models (LLMs), yet it exposes a critical vulnerability: malicious data providers can embed harmful behaviors into downstream corpora, creating models that retain professional skills while violating human values on demand. Existing safety-realignment defenses often fail in practice due to three key limitations: they frequently cause catastrophic forgetting of specialized skills; their effectiveness collapses when the defender cannot observe the attacker's prompt template; and successfully realigned models remain susceptible to re-jailbreaking via simple system prompt switches. To address these challenges, we propose Routing-based On-Policy Distillation (ROPD), a novel realignment framework that models the divergence between aligned and compromised output probability distributions rather than fitting specific prompt templates. We conduct extensive experiments comparing ROPD against four state-of-the-art baselines across three datasets and three base models with varying alignment strengths. Our results demonstrate that when baseline defenses face template mismatches, often accompanied by severe degradation in downstream task performance. In contrast, ROPD substantially mitigates template-mismatch risks, maintaining superior robustness in both defense effectiveness and capability preservation. While our analysis indicates ROPD is not entirely immune to template shifts, its performance degradation is negligible compared to existing methods, establishing a new standard for robust LLM realignment.

URL PDF HTML 收藏
2607.27077 2026-07-30 cs.LG 新提交

Equilibrium Training of Energy-Based Models with Parallel Trajectory Tempering

基于并行轨迹回火的能量基模型的均衡训练

Nicolas Béreux, Aurélien Decelle, Cyril Furtlehner, Beatriz Seoane

机构 * Université Paris-Saclay(巴黎萨克雷大学) CNRS(法国国家科学研究中心) INRIA(法国国家信息与自动化研究所) Escuela Técnica Superior de Ingenieros Industriales(工业工程师高等技术学院) Universidad Politécnica de Madrid(马德里理工大学) Universidad Complutense de Madrid(马德里康普顿斯大学)

AI总结 该研究提出基于并行轨迹回火(PTT)的 EBM 训练算法,结合水库采样与自适应优化,可稳定快速训练,性能优于现有方法,使 EBM 的均衡最大似然训练更实用高效。

Comments 11 pages, 7 figures

详情
AI中文摘要

能量基模型(EBMs)为科学数据的生成式建模提供了可解释框架,但马尔可夫链蒙特卡罗混合性差常限制其可靠性。我们提出一种基于并行轨迹回火(PTT)的训练算法,该算法利用优化路径的连续性,在整个学习过程中维持均衡采样,从而能在多模态程度高且数据稀缺的科学数据集上实现稳定快速的训练。结合 reservoir sampling(水库采样)和自适应优化,PTT 的计算成本与持续对比散度相当,可作为标准训练方法的实用替代方案。它还能以几乎无额外成本的方式直接估计热化时间、训练模型的均衡样本以及准确的对数似然。在受限玻尔兹曼机上的实验表明,PTT 始终优于现有 EBM 训练方法;在离散表格数据上,其性能也超过了最先进的深度生成模型,生成样本质量更高,且对过拟合和数据有限的鲁棒性更强。我们的研究使 EBM 的均衡最大似然训练变得实用且计算高效。

英文摘要

Energy-Based Models (EBMs) provide an interpretable framework for generative modeling of scientific data, but poor Markov Chain Monte Carlo mixing often limits their reliability. We introduce a training algorithm based on Parallel Trajectory Tempering (PTT), which exploits the continuity of the optimization path to maintain equilibrium sampling throughout learning. This enables stable and fast training on highly multimodal and data-scarce scientific datasets. Combined with reservoir sampling and adaptive optimization, PTT has a computational cost comparable to Persistent Contrastive Divergence, making it a practical replacement for standard training methods. It also provides direct estimates of thermalization times, equilibrium samples from trained models, and accurate log-likelihoods at essentially no additional cost. Experiments on Restricted Boltzmann Machines show that PTT consistently outperforms existing EBM training approaches. On discrete tabular data, it also surpasses state-of-the-art deep generative models, yielding higher-quality samples and greater robustness to overfitting and limited data. Our results make equilibrium maximum-likelihood training of EBMs practical and computationally efficient.

URL PDF HTML 收藏
2607.27076 2026-07-30 cs.LG cs.SY eess.SP eess.SY 新提交

Single-Beat Cuffless Blood Pressure Estimation Using Ear-PPG and ECG with a Lightweight Hybrid Learning Framework

采用耳式光电容积描记法(PPG)与心电图(ECG)结合轻量混合学习框架的单搏无袖带血压估计

Kindeep K. Dhatt, Tengyue Wu, Hanbang Hua, Yayun Du

机构 * Vanderbilt University(范德堡大学) Vanderbilt Institute for Surgery and Engineering(范德堡外科工程研究所) Department of Electrical and Computer Engineering(电气与计算机工程系)

AI总结 该研究提出轻量混合学习框架,结合ECG、耳式PPG与6轴IMU,实现单搏无袖带血压估计,在多阶段压力方案与PulseDB数据集上,较基线模型降低28.2%组合MAE,适配可穿戴部署。

Comments 7 pages, 5 figures

详情
AI中文摘要

连续无袖带血压(BP)监测仍面临挑战,原因包括运动伪影、生理变异性,以及传统脉搏传播时间(PTT)模型在动态条件下鲁棒性有限。许多现有方法依赖多秒窗口稳定估计,而这一假设在实际监测中常因间歇性信号损坏被打破。本研究表明,与血压相关的判别信息在单搏层面仍有保留,因此提出一种用于连续血压估计的轻量多模态可穿戴框架。该系统整合同步胸部心电图(ECG)与耳夹式反射光电容积描记法(PPG),两者均搭配6轴惯性测量单元以提供运动上下文。研究引入混合学习架构:一维卷积神经网络从单个PPG搏波中提取64维嵌入,将其与30个基于生理的特征(包括PTT统计量与心率变异性)融合,随后输入LightGBM回归器。该方法通过多阶段压力方案(n=10)与PulseDB公开数据集,采用受试者不重叠验证进行评估。在30次独立运行中,模型对收缩压的平均绝对误差为4.02±0.21 mmHg,对舒张压的平均绝对误差为1.79±0.05 mmHg,相较于基线模型,组合平均绝对误差降低了28.2%。该框架无需长时序上下文即可实现逐搏估计,支持计算高效的无袖带血压监测,适用于实际资源约束下的可穿戴设备部署。本研究的源代码可在指定URL获取。

英文摘要

Continuous cuffless blood pressure (BP) monitoring remains challenging due to motion artifacts, physiological variability, and the limited robustness of conventional pulse transit time (PTT) models under dynamic conditions. Many prior approaches rely on multi-second windows to stabilize estimation, an assumption that is frequently violated during real-world monitoring with intermittent signal corruption. Here, we show that discriminative BP-related information is preserved at the single-beat level and present a lightweight multi-modal wearable framework for continuous BP estimation. The system integrates synchronized chest electrocardiography (ECG) and ear-clip reflectance photoplethysmography, each co-located with a 6-axis inertial measurement unit to provide motion context. We introduce a hybrid learning architecture in which a one-dimensional convolutional neural network extracts a 64-dimensional embedding from individual PPG beats and fuses it with 30 physiology-grounded features, including PTT statistics and heart rate variability, followed by LightGBM regression. The method was evaluated using a multi-phase stress protocol ($n=10$) and the PulseDB public dataset with subject-disjoint validation. Across 30 independent runs, the model achieved mean absolute errors of $4.02 \pm 0.21$~mmHg for systolic BP and $1.79 \pm 0.05$~mmHg for diastolic BP, corresponding to a 28.2\% reduction in combined MAE relative to baseline models. By enabling beat-wise estimation without long temporal context, this framework supports computationally efficient cuffless BP monitoring suitable for wearable deployment under practical resource constraints. The source code for this work is available at https://github.com/SYMBIOX-Lab/BP-wireless.

URL PDF HTML 收藏
2607.27065 2026-07-30 cs.CV cs.AI 新提交

ScratchSim: A Procedural Synthetic Data Pipeline for Surface Scratch Detection

ScratchSim:用于表面划痕检测的程序合成数据流水线

Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist, Richard Hoffmann, Arjan kuijper, Michael Weinmann

机构 * Fraunhofer IGD(弗劳恩霍夫计算机图形学研究所) TU Darmstadt(达姆施塔特工业大学)

AI总结 本文提出基于BlenderProc的程序合成数据流水线ScratchSim,通过四种训练策略验证其在表面划痕检测中的有效性,可缓解真实标注数据稀缺问题,适用于设备端工业检测。

详情
AI中文摘要

表面划痕等自动化缺陷检测是工业质量控制的重要环节,但标注缺陷数据的稀缺性使该任务颇具挑战性。本文提出一种程序渲染流水线,利用BlenderProc生成大规模带标注的合成训练数据,该流水线支持配置材质外观、相机模式与域随机化,可自动生成COCO格式标注。为验证方法潜力,我们在两种不同材质属性的物体、三款轻量边缘可部署检测器YOLOX、YOLO26与LW-DETR上,评估仅合成数据、仅真实数据、混合数据、基于合成权重微调这四种训练策略。评估结果显示,基于合成权重微调的表现始终优于仅真实数据训练,混合训练在真实数据稀缺条件下可有效恢复性能,该结论在卷积与Transformer架构上均得到验证。所提方法无需依赖大量真实标注数据集即可实现可扩展的缺陷检测,适用于设备端工业检测,接受后将通过项目网站提供流水线脚本、3D模型及光泽玩具法拉利汽车的合成与真实标注划痕数据集。

英文摘要

While automated defect detection such as the detection of surface scratched is an important aspect in industrial quality control, the scarcity of annotated defect data make this task challenging. This paper presents a procedural rendering pipeline that generates large-scale annotated synthetic training data using BlenderProc, with configurable material appearance, camera modes, and domain randomization, producing automatic COCO-format annotations. To show the potential of our approach, we evaluate four training strategies, namely synthetic-only, real-only, mixed, and fine-tuning from synthetic weights, across two objects with different material properties and three lightweight edge-deployable detectors, YOLOX, YOLO26, and LW-DETR. Our evaluation show that fine-tuning from synthetic weights consistently outperforms real-only training, and that mixed training effectively recovers performance under scarce real-data conditions, with findings validated across both convolutional and transformer-based architectures. The proposed approach enables scalable defect detection without the burden of large real annotated datasets, making it practical for on-device industrial inspection. The pipeline scripts, 3D model, and both synthetic and real annotated scratch datasets for a glossy toy Ferrari car will be made available through the project website upon acceptance.

URL PDF HTML 收藏
2607.27036 2026-07-30 cs.CV cs.LG 新提交

Mitigating Compounding Error via Video Representation Regularization

通过视频表示正则化缓解复合误差

Taiye Chen, Qi Zhang, Yisen Wang

机构 * Peking University(北京大学)

AI总结 针对视频扩散世界模型自回归生成的复合误差问题,研究发现其与表示维度崩溃相关,提出视频表示正则化方法,在VBench指标上显著优于Diffusion Forcing,提升了长视频生成的鲁棒性。

详情
AI中文摘要

基于视频扩散的世界模型可实现用于机器人、自动驾驶和仿真任务的长自回归视频生成,但滑动窗口自回归推理会遭受严重的误差累积,随时间推移降低帧质量。尽管该现象已被广泛观察到,但复合误差的潜在机制以及如何实现稳定的长视界生成仍在很大程度上未得到解决。本文中,我们研究了视频世界模型的内部表示动态,发现复合误差与隐藏表示的维度崩溃紧密耦合。具体而言,当生成漂移开始时,模型表示的有效秩(erank)急剧下降,揭示了表示退化与长期展开不稳定性之间的强关联。此外,我们发现单纯的训练数据扩展无法提升模型对误差漂移的抵抗能力,这与主流的扩展范式相矛盾。为解决该问题,我们提出视频表示正则化,一种用于稳定潜在表示并抑制迭代误差累积的轻量型训练约束。与Diffusion Forcing相比,我们的方法在VBench的美学质量指标上从38.65提升至55.56,在成像质量指标上从44.37提升至72.08。本研究首次建立了自回归视频漂移与模型内部表示之间的联系,采用erank作为误差累积的定量指标,揭示了视频世界模型违反直觉的扩展限制,并提出了一种简单却有效的正则化策略以提升长视频生成的鲁棒性。

英文摘要

Video diffusion-based world models enable long autoregressive video generation for robotics, autonomous driving and simulation tasks, yet sliding-window autoregressive inference suffers from severe error accumulation that degrades frame quality over time. Although this phenomenon has been widely observed, the underlying mechanism of compounding error and how to achieve stable long-horizon generation remain largely unresolved. In this paper, we investigate the internal representation dynamics of video world models and discover that compounding error is tightly coupled with dimensional collapse of hidden representations. Specifically, the effective rank of model representations sharply decreases at the onset of generation drift, revealing a strong connection between representational degradation and long-term rollout instability. Furthermore, we find that pure training data scaling fails to boost model resistance to error drift, contradicting mainstream scaling paradigms. To address this problem, we propose video representation regularization, a lightweight training constraint that stabilizes latent representations and suppresses iterative error accumulation. Compared with Diffusion Forcing, our method achieves improvements from 38.65 to 55.56 and from 44.37 to 72.08 on the Aesthetic Quality and Imaging Quality metrics of VBench. Our work establishes the first connection between autoregressive video drifting and model internal representations, adopts erank as a quantitative metric for error accumulation, reveals counterintuitive scaling limitations for video world models, and presents a simple yet effective regularization strategy to improve long video generation robustness.

URL PDF HTML 收藏
2607.27023 2026-07-30 cs.LG cs.AI stat.ML 新提交

BayesAME: Bayesian Active Model Evaluation

BayesAME:贝叶斯主动模型评估

Paula Cordero Encinar, Taylan Cemgil, Arnaud Doucet, Virginia Aglietti, Silvia Chiappa

机构 * Imperial College London(帝国理工学院) Google DeepMind(谷歌DeepMind)

AI总结 BayesAME是一种序列贝叶斯框架,可自动确定核心集大小以高效评估大型生成模型,其性能优于现有方法,且证实非随机核心集选择及连续响应对数似然的优势。

详情
AI中文摘要

在不同基准上评估大型生成模型既耗时又计算成本高昂,这催生了对仅通过在项目子集(即核心集)上评估模型来估算完整基准性能的方法的需求。现有文献大多要求从业者输入核心集大小,但当可靠的性能估算优先于效率时,评估方法还应能够自动确定反映该优先级的核心集大小。我们提出BayesAME,一种专门用于自动确定核心集大小的序列贝叶斯框架。BayesAME将性能建模为随机变量,方法是为共享相同历史模型性能的项目组定义潜在能力,联合先验分布编码目标模型与这些历史模型表现相似的信念;这些能力的后验分布用于推导性能估算器、量化性能不确定性,并通过信息增益准则选择添加到核心集的项目,核心集被迭代扩充直至性能估算波动和性能不确定性分别低于用户定义的阈值。我们提出多目标扩展,捕捉多个目标模型间的性能相关性以进一步减小核心集大小。通过在不同基准上的大量实验,我们证明BayesAME始终优于现有方法的序列适配版本;重要的是,我们的综合分析回应了文献中近期的质疑,确立非随机核心集选择比随机选择更具优势;最后,我们强调利用连续响应对数似然而非传统二元评分可显著提升估算准确性。

英文摘要

Evaluating large generative models across benchmarks is time-consuming and computationally expensive. This drives the need for methods that can estimate full benchmark performance by evaluating models on only a subset of items, known as a coreset. Current literature mostly requires the practitioner to input a coreset size. However, when reliable performance estimation takes priority over efficiency, an evaluation method should also be capable of automatically determining a coreset size that reflects this priority. We introduce BayesAME, a sequential Bayesian framework specifically targeting automatic determination of the coreset size. BayesAME models performance as a random variable by defining a latent ability for each group of items sharing the same historical model performances, with a joint prior distribution encoding the belief that the target model behaves similarly to these historical models. The posterior distribution over these abilities is used to derive performance estimators, quantify performance uncertainty, and select items to add to the coreset via an information-gain criterion. The coreset is iteratively augmented until the performance estimate fluctuation and the performance uncertainty fall below their respective user-defined thresholds. We propose a multi-target extension that captures performance correlations across multiple target models to further reduce the coreset size. Through extensive experiments across diverse benchmarks, we demonstrate that BayesAME consistently outperforms sequential adaptations of existing methods. Crucially, our comprehensive analysis addresses recent skepticism in the literature, establishing that non-random coreset selection is advantageous over random selection. Finally, we highlight that leveraging continuous response log-likelihoods over traditional binary scores significantly enhances estimation accuracy.

URL PDF HTML 收藏
2607.27017 2026-07-30 cs.LG cs.RO 新提交

What Can Latent World Models Know? Physical Parameter Identifiability in Multimodal Predictive Representations

隐式世界模型能知晓什么?多模态预测表示中的物理参数可识别性

Kaizhen Tan, Xin Xu, Siru Tao, Hanzhe Hong, Yang Feng, Heqing Du

机构 * New York University(纽约大学) Carnegie Mellon University(卡内基梅隆大学) Columbia University(哥伦比亚大学)

AI总结 该研究通过可控实验揭示,隐式世界模型的目标函数结构决定其获取的物理参数,输入与预测目标分别限制和决定可获知的物理量,额外数据仅优化已获取的参数。

详情
AI中文摘要

隐式世界模型的核心前提是,预测未来的任务会迫使表示学习器内化环境的物理规律。训练后的隐式表示究竟包含哪些物理量,又由什么因素决定?我们通过在POKEWORLD环境中开展可控干预实验来解答这一问题:该交互式环境中视觉外观完全相同的物体,实则隐藏着质量、阻力和接触刚度三类物理参数。我们采用一种带证书门控的协议,先对每个参数是否可从原始观测中恢复进行认证,再测量其是否进入隐式表示;若结果为阴性,则可归因于目标函数而非环境本身。由此得到的可识别性图谱包含两类组织机制和一个边界:输入会限制可获知的内容,而预测目标会决定需保留的内容。刚度仅在需要预测触摸时才会进入隐式表示(此时R²=0.50,而当同一信号仅作为输入融合时,R²=-0.02);在单步预测场景下,仅视觉输入的隐式表示会丢弃即使是完全可见的物体状态。阻力则构成了边界:它的可恢复性证书为0.89,但在我们测试的所有确定性预测目标下,其表现都稳定在0.13附近,而同一主干网络上的监督头可达到0.45。在感知坐标下读数缓慢且为比值型的参数,不在这些目标函数的获取范围内。在RH20T数据集上,跨缩放曲线的输入-目标因子设计在两个机器人和4258个 episodes 上复现了上述两类机制:缺少信息或预测压力的臂在五倍数据范围内保持平稳,仅完整多模态目标能迫使模型超越持续性基线,且保留的增益随数据规模增长。研究表明,目标函数结构决定了隐式表示会获取哪些物理参数,额外数据仅能改善模型已获取的参数。

英文摘要

A central premise of latent world models is that predicting the future forces a representation to internalize the physics of its environment. Which physical quantities does a trained latent actually contain, and what decides this? We answer with controlled interventions in POKEWORLD, an interactive environment whose visually identical objects hide mass, drag, and contact stiffness. A certificate-gated protocol first certifies each parameter as recoverable from raw observations, then measures whether it enters the latent, so a null result can be attributed to the objective rather than to the environment. The resulting identifiability map has two organizing mechanisms and one frontier. Inputs limit what can be known, while prediction targets decide what is retained. Stiffness enters the latent only when touch is forecast ($R^2=0.50$, compared with $-0.02$ when the same signal is merely fused into the input), and under single-step prediction a vision-only latent discards even perfectly visible object state. Drag marks the frontier. It carries a recoverability certificate of 0.89 yet plateaus near 0.13 under every deterministic prediction objective we test, while a supervised head on the same trunk reaches 0.45. Parameters whose readout is slow and ratio-type under the sensed coordinates fall outside what these objectives acquire. On RH20T, an input-target factorial across scaling curves reproduces both mechanisms across two robots and 4,258 episodes. Every arm missing information or prediction pressure stays flat over a fivefold data range, and only the full multimodal objective forecasts force beyond a persistence baseline, with held-out gains that grow with scale. Objective structure determines which physical parameters a latent acquires, and additional data improves only the parameters it already acquires.

URL PDF HTML 收藏
2607.26993 2026-07-30 cs.LG 新提交

Foundation Models for Face Presentation Attack Detection: A Unified Linear-Probing Benchmark

用于人脸呈现攻击检测的基础模型:统一的线性探测基准

Peter Lorenz, Anjith George, Sébastien Marcel

机构 * Idiap Research Institute(Idiap研究所) University of Lausanne (UNIL)(洛桑大学)

AI总结 本研究在MCIO基准上评估24种冻结基础模型编码器的线性探测性能,发现其仅通过线性分类器可实现强劲的数据集内PAD性能,但跨数据集迁移能力有限,需明确适配以解决域偏移问题。

Comments accepted at IJCB 2026

详情
AI中文摘要

人脸呈现攻击检测(PAD)在跨数据集评估下仍具挑战性,域偏移会降低在单一数据集上训练的模型性能。大规模标注数据的稀缺促使人们采用预训练视觉模型,而非从头开始训练特定任务架构,这引发了一个根本问题:通用视觉基础模型是否编码了可通过最少特定任务训练获取的PAD相关信息?为探究此问题,我们采用统一的线性探测协议,在MCIO基准(包含MSU-MFSD、CASIA-FASD、Replay-Attack、OULU-NPU)上系统评估24种冻结编码器,涵盖自监督视觉Transformer、视觉-语言编码器及监督CNN。骨干网络保持固定,仅训练轻量级线性头以分离预训练表示中已存在的PAD信息。我们报告了与两个专用PAD基线相比的数据集内和跨数据集性能,以及准确率-计算权衡。结果显示,冻结的基础模型表示仅通过线性分类器即可支持强劲的数据集内PAD性能,但该性能无法可靠地跨数据集迁移。在若干模型族内,模型规模有益,不过该效应非单调,且受架构和预训练的强烈调节。InternViT-6B实现了最低的平均数据集内误差,而CLIP ViT-B/32在评估的探测器中提供了最有利的跨数据集迁移-计算权衡。这些发现表明,尽管预训练表示包含PAD相关信息,但仍需明确的适配以解决域偏移问题。

英文摘要

Face presentation attack detection (PAD) remains challenging under cross-dataset evaluation, where domain shift degrades models trained on a single dataset. The scarcity of large-scale labeled data motivates adapting pretrained vision models rather than training task-specific architectures from scratch, raising a fundamental question: do general-purpose vision foundation models encode PAD-relevant information accessible with minimal task-specific training? To investigate, we systematically evaluate 24 frozen encoders, including self-supervised vision transformers, vision-language encoders, and supervised CNNs, using a unified linear-probing protocol on the MCIO benchmark (MSU-MFSD, CASIA-FASD, Replay-Attack, OULU-NPU). The backbone remains fixed, and only a lightweight linear head is trained to isolate the PAD information already present in the pretrained representation. %We report intra- and cross-dataset performance, along with accuracy-compute trade-offs, relative to two specialist PAD baselines. Results show that frozen foundation-model representations can support strong intra-dataset PAD performance with only a linear classifier, but this performance does not reliably transfer across datasets. Model scale is beneficial within several families, although the effect is not monotonic and is strongly mediated by architecture and pretraining. InternViT-6B achieves the lowest mean intra-dataset error, whereas CLIP ViT-B/32 offers the most favorable cross-dataset transfer-compute trade-off among the evaluated probes. These findings suggest that while pretrained representations contain PAD-relevant information, explicit adaptation remains necessary to address domain shift.

URL PDF HTML 收藏
2607.26991 2026-07-30 cs.RO 新提交

RL$^2$-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

RL²-VLA:面向视觉-语言-动作模型的测试时缩放自适应RL隐成分引导

Derek Ming Siang Tan, Shailesh Shailesh, Srikrishna Iyer, William Wei Jie Teo, Yuanliang Ju, Qiao Gu, Guillaume Sartoretti

机构 * National University of Singapore(新加坡国立大学) University of Toronto(多伦多大学) Singapore Technologies Engineering(新加坡科技工程公司)

AI总结 针对VLA模型分布外任务性能下降问题,提出基于VLA隐空间的自适应推理时引导框架RL²,仅在预测失败时激活成分引导,在SIMPLER等基准上分布外成功率最高提升17.3%,可迁移至真实世界。

Comments Code and models are available at https://rl2-vla.github.io

详情
AI中文摘要

尽管视觉-语言-动作(VLA)模型具备令人印象深刻的视觉运动能力,但其在具有挑战性和分布外任务上的性能往往会下降。近期的测试时引导与缩放方法无需大量数据收集和重新训练即可提升性能,但动作样本往往仍集中在相似行为周围,因此继承了相关的失败模式。此外,现有方法在每个时间步都采用相同的干预策略,无论基础策略是否已有可能成功。为解决这些局限,我们引入RL²,一种利用VLA隐空间上强化学习的自适应推理时引导框架。首先,我们训练一个轻量级离线RL策略,该策略以从VLA动作专家提取的高表达隐空间为条件,在推理时将其流速度与冻结VLA的流速度进行组合。这种成分引导策略将大规模模仿学习的行为先验与离线RL在主导演示模式之外诱导的动作多样性相结合。我们进一步发现,推理时引导在成功和失败状态下遵循根本不同的缩放规律,表明动作多样性在基础VLA可能失败时最有益,但在可能成功时会不必要地扰动已准确的动作。基于这一见解,RL²仅在预测失败时激活成分引导。在SIMPLER和PolaRiS基准上,RL²在分布外设置中将成功率提升了最多17.3%,而消融实验和缩放研究证明了隐表示和RL训练的重要性。最后,真实世界实验表明,这些提升可迁移到模拟之外,确立RL²为VLA部署的实用且模块化的引导框架。

英文摘要

Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improve performance without extensive data collection and retraining, but action samples often remain concentrated around similar behaviors and therefore inherit correlated failure modes. Moreover, existing methods apply the same intervention strategy at every timestep, regardless of whether the base policy is already likely to succeed. To address these limitations, we introduce $RL^2$, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. First, we train a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert and compose its flow velocity with that of the frozen VLA during inference. This compositional steering strategy combines the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL beyond dominant demonstration modes. We further discover that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Building on this insight, $RL^2$ activates compositional steering only when failure is predicted. Across the SIMPLER and PolaRiS benchmarks, $RL^2$ improves success rates by up to +17.3% in out-of-domain settings, while ablations and scaling studies demonstrate the importance of latent representations and RL training. Finally, real-world experiments demonstrate that these gains transfer beyond simulation, establishing $RL^2$ as a practical and modular steering framework for VLA deployment.

URL PDF HTML 收藏