arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 2136
2607.26057 2026-07-29 cs.CL cs.AI 新提交

Pass the Baton: Trajectory-Relayed On-Policy Distillation

传递接力棒:轨迹中继的在线策略蒸馏

Haolei Xu, Xiaowen Xu, Haiwen Hong, Zixuan Ni, Hongxing Li, Yiwen Qiu, Weiming Lu, Yongliang Shen

机构 * Zhejiang University(浙江大学) Alibaba Group(阿里巴巴集团)

AI总结 研究针对在线策略蒸馏的前缀失败问题,利用师生延续不对称性提出Relay-OPD方法,通过教师在触发点接管生成轨迹段,让学生在其上优化,在数学推理基准测试中取得优异成绩,提升效果并减少训练轨迹长度。

Comments Project Page: https://zju-real.github.io/Relay-OPD Code: https://github.com/zju-real/Relay-OPD

详情
AI中文摘要

在线策略蒸馏(OPD)在学生自身轨迹中进行令牌级监督,但存在前缀失败问题:一旦学生陷入错误推理方向,后续生成都会基于此偏差,产生误导性延续,导致不可靠监督并浪费计算资源。我们发现失败前缀上师生延续存在不对称性,即教师倾向于重新引导,而学生继续沿原方向。在中继在线策略蒸馏(Relay-OPD)中,我们将此转化为无标签交接触发器。训练时,Relay-OPD通过让教师在检测到的触发点短暂接管以生成教师轨迹段,之后学生恢复并在生成的轨迹上优化。有限的中继预算将干预集中在关键早期位置,同时限制与学生策略的偏离。在八个数学推理基准测试中,使用Qwen3-4B-Instruct-2507教师和Qwen3-0.6B/1.7B-Non-Thinking学生,Relay-OPD在每个基准测试中都取得了最佳或次佳结果,平均比标准OPD在1.7B模型上提高了5.73%,比最强基线FastOPD提高了1.49%,在0.6B模型上也有持续提升,训练轨迹长度减少了50%以上。

英文摘要

On-policy distillation (OPD) grounds token-level supervision in the student's own trajectory, yet suffers from prefix failure: once the student commits to a wrong reasoning direction, all subsequent generation builds on this deviation, producing misdirected continuations that elicit unreliable supervision and waste compute. We identify a teacher-student continuation asymmetry on failed prefixes, where the teacher tends to redirect while the student continues along the original direction, and convert it into a label-free handoff trigger in Relay On-Policy Distillation (Relay-OPD). During training, Relay-OPD constructs relay trajectories by letting the teacher briefly take over at detected trigger points to produce a teacher leg, after which the student resumes and is optimized on the resulting trajectory. A limited relay budget concentrates intervention on critical early positions while limiting departure from the student policy. With a Qwen3-4B-Instruct-2507 teacher and Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks, Relay-OPD achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and the strongest baseline FastOPD by +1.49% on average for 1.7B, with consistent gains at 0.6B. Training trajectory length is reduced by over 50%.

URL PDF HTML 收藏
2607.26056 2026-07-29 cs.RO 新提交

INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models

INTACT:用于无搜索世界模型的同构意图到动作学习

Junhan Sun, Hao Zhao, Guofeng Zhang

机构 * State Key Laboratory of CAD&CG, Zhejiang University(浙江大学计算机辅助设计与图形学国家重点实验室) Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院) InSpatio

AI总结 研究针对前向潜在世界模型恢复动作需昂贵搜索的问题,提出INTACT方法,通过特定架构和技术实现意图到动作学习,在多任务上取得高成功率,减少采样并提升性能,还具有快速推理能力。

Comments 28 pages, 11 figures, including appendices

详情
AI中文摘要

前向潜在世界模型预测动作如何改变场景,但仅通过昂贵的测试时间搜索来恢复期望变化的动作。我们引入了INTACT(意图到动作),这是一种端到端的JEPA,它将有动作标签、无奖励的轨迹转化为可部署的意图到动作接口。每个转换提供物理意图$z_{t + 1}-z_t$,而未来目标提供部署意图$\text{sg}(z_g)-z_t$。该架构通过相同的四槽语法和共享参数在局部和目标运动意图主干输入图之间是同构的,并且通过相同预测器诱导的动作定律语义在支持的局部和目标运动意图族之间是同构的。INTACT还提供从RGB证据到动作有效潜在意图坐标以及从意图族到其相应动作定律族的完整转移。不对称端点梯度为物理后继提供基础并将未来目标固定为锚点,在没有逐点潜在匹配或全局线性动力学的情况下结合表示学习和控制。所得坐标支持强大的分布动作定律:其条件均值直接用作无搜索策略,同时采样可用于多样性或可选验证。在四个官方LeWM任务上,单轮、零搜索模型的成功率达到85.78%、100.00%、97.67%和97.89%。以直接计划为中心的可选局部CEM使用384个而不是9000个候选序列达到96.86%的宏观成功率,采样减少了23.44倍,同时将纯CEM提高了16.00分。一个共享的四任务编码器达到89.39%的E5直接宏观成功率,并在联合训练的LeWM上改进了每个任务,而预测专家动作族kNN在$r = 0.954$时跟踪直接成功率。直接推理需要2.9 - 5.5毫秒。

英文摘要

Forward latent world models predict how actions change a scene, but recover actions for a desired change only through expensive test-time search. We introduce INTACT (INtent-To-ACTion), an end-to-end JEPA that turns action-labeled, reward-free trajectories into a deployable intent-to-action interface. Each transition supplies physical intent $z_{t+1}-z_t$, while a future goal supplies deployment intent $\operatorname{sg}(z_g)-z_t$. The architecture is isomorphic between the local and goal motion-intent backbone-input graphs through an identical four-slot grammar and shared parameters, and between supported local and goal motion-intent families through action-law semantics induced by the same predictor rather than pointwise latent equality. INTACT also provides intact transfer from RGB evidence to action-effective latent intent coordinates and from intent families to their corresponding action-law families. Asymmetric endpoint gradients ground physical successors and fix future goals as anchors, joining representation learning and control without pointwise latent matching or globally linear dynamics. The resulting coordinates support a robust distributional action law: its conditional mean serves directly as a search-free policy, while sampling remains available for diversity or optional verification. On the four official LeWM tasks, one-epoch, zero-search models reach 85.78\%, 100.00\%, 97.67\%, and 97.89\% success. Optional local CEM centered on the Direct plan reaches 96.86\% macro success using 384 instead of 9,000 candidate sequences, reducing sampling by $23.44\times$ while improving pure CEM by 16.00 points. One shared four-task encoder reaches 89.39\% E5 Direct macro and improves every task over jointly trained LeWM, while predicted--expert action-family kNN tracks Direct success at $r=0.954$. Direct inference takes 2.9--5.5 ms.

URL PDF HTML 收藏
2607.26055 2026-07-29 cs.RO cs.AI cs.LG 新提交

$π\mathbf{R}^2$: Reactive Real-time Flow Policies

$π\mathbf{R}^2$:反应式实时流策略

Sungjae Park, Shubham Tulsiani

机构 * Carnegie Mellon University(卡内基梅隆大学)

AI总结 研究通用操作策略反应性和实时性问题,提出$\pi\mathbf{R}^2$方法,将条件分快速和慢速通道,采用延迟自适应流调度,能在保留大型主干等的同时,让策略实时反应并提高成功率。

Comments Preprint(20 pages). Under Review

详情
AI中文摘要

通用操作策略越来越多地采用基于大型预训练主干构建的动作分块流策略形式。这些块开环运行,政策无法对执行中途到达的感官输入做出反应,牺牲了反应性。更频繁地重新规划可以恢复它,但感知到行动的管道(一个大主干加上多个去噪步骤)太慢:这种延迟禁止频繁重新规划,并使已提交的行动过时,使得这些策略不适用于动态闭环控制。我们提出了$\pi\mathbf{R}^2$,它使这些策略具有反应性和实时性,同时保留大型主干、富有表现力的多模态策略和多动作预测。基于扩散强迫的逐位置噪声调度,$\pi\mathbf{R}^2$贡献了两个想法。首先,它将条件分为一个快速通道(本体感觉,每滴答更新一次)和一个异步更新的慢速通道(视觉语言特征),因此策略在一个块内对本体感觉做出反应,同时容忍过时的视觉。其次,延迟自适应流调度将飞行中的动作视为修复条件,并在每次调用的一个去噪步骤中发出动作,使一个经过训练的模型能够适应不同的硬件延迟。对现有架构只需进行最小的修改,$\pi\mathbf{R}^2$就可以从预训练策略中进行微调:在真实的xArm6+XHand平台上应用于GR00T-N1.7时,它的闭环重新规划速度比基本策略快约4倍(在A5000 GPU上约为25Hz),每40毫秒根据新观察进行一次动作。在模拟和实际操作任务中,$\pi\mathbf{R}^2$比最强基线在模拟中成功率提高高达23%,在现实世界中提高30%。项目页面:此https URL

英文摘要

Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing \emph{reactivity}. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this \emph{latency} forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present $π\mathbf{R}^2$, which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, $π\mathbf{R}^2$ contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, $π\mathbf{R}^2$ can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly $4\times$ faster than the base policy (~$25$Hz on an A5000 GPU), acting on a fresh observation every $40$ms. Across simulation and real-world manipulation tasks, $π\mathbf{R}^2$ improves the success rate by up to $23\%$ in simulation and $30\%$ in the real world over the strongest baseline. Project page: https://pi-r2-flow.github.io/

URL PDF HTML 收藏
2607.26052 2026-07-29 cs.LG 新提交

Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA

在不确定的地方投入专家:用于专家混合LoRA的置信度自适应路由

Tom Saliencro, Rohan Desai, Priya Nair, Maya Lindqvist, Daniel Whitmore

机构 * University of California, Irvine(加州大学欧文分校) University of Washington(华盛顿大学)

AI总结 研究针对专家混合LoRA路由中简单与困难令牌服务不均问题,提出置信度自适应路由CARE,它以核心方式接纳专家,校准阈值使活跃专家数匹配目标,是单前向传递规则,在多任务中改进效果显著,还提升了分布外检测能力。

详情
AI中文摘要

低秩适应(LoRA)的专家混合(MoE)变体将每个令牌路由到固定数量的k个专家。令牌在模型对其的不确定程度上有所不同,因此单一的k在简单令牌上花费过多,而对困难令牌服务不足。我们观察到路由器的输出分布已经是一个逐令牌的不确定性信号:峰值质量表示置信度,而平坦分布表示模糊性。我们引入了CARE(专家的置信度自适应路由),它以核心方式接纳专家。专家按路由器权重递减被激活,直到其累积质量达到阈值,当被接纳的专家意见不一致时会有小的扩展。一个预算恒温器校准阈值,使活跃专家的平均数量匹配任何目标。CARE是一个无需额外参数的即插即用单前向传递规则。在关于LLaMA - 3.1 - 8B和Qwen2.5 - 7B的八个常识基准测试以及数学、代码和知识任务中,CARE在匹配计算时比固定的top - k MoE - LoRA有所改进,并且在激活更少专家的情况下与固定k = 4基线匹配。相同的置信度和不一致信号也比MSP、熵和多通道代理改进了分布外检测。我们用核心保真度、预算最优性和对不一致的认知解读来支持该设计,并发布了代码。

英文摘要

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts $k$. Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones. We observe that the router's output distribution is already a per-token uncertainty signal: peaked mass indicates confidence, while a flat distribution indicates ambiguity. We introduce CARE (Confidence-Adaptive Routing of Experts), which admits experts in a nucleus fashion. Experts are activated in decreasing router weight until their cumulative mass reaches a threshold, with a small extension when the admitted experts disagree. A budget thermostat calibrates the threshold so that the average number of active experts matches any target. CARE is a drop-in, single-forward-pass rule with no extra parameters. Across eight commonsense benchmarks on LLaMA-3.1-8B and Qwen2.5-7B, as well as math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches the fixed-k=4 baseline while activating fewer experts. The same confidence and disagreement signals also improve out-of-distribution detection over MSP, entropy, and multi-pass proxies. We support the design with nucleus fidelity, budget optimality, and an epistemic reading of disagreement, and we release code.

URL PDF HTML 收藏
2607.26047 2026-07-29 cs.RO 新提交

S2A2: Audio-Visual Imitation Learning for Manipulation Tasks Using Acoustic Spatial Information

S2A2:利用声学空间信息进行操作任务的视听模仿学习

Kaneyoshi Hiratsuka, Benjamin Yen, Ryosuke Kojima

机构 * Kyoto University(京都大学) RIKEN(理化学研究所) Institute of Science Tokyo(东京理科大学)

AI总结 研究利用声学信息进行操作任务的模仿学习,提出多模态框架S2A2,集成视觉与声学信息,实现多种策略集成,模拟与真实机器人实验表明其对特定任务有效且适用于现实操作。

Comments Project page: https://azuma413.github.io/projects/s2a2

详情
AI中文摘要

声学信息提供了有关物体位置、材料属性以及接触或运动引起的变化的丰富线索。本文引入了一组新的用于模仿学习的声学感知操作任务,机器人必须利用听觉线索来确定操作目标。这些任务需要进行声源定位和识别以在机器人操作中进行主动探索。此外,我们提出了一种多模态模仿学习框架Spatial-Spectral Audio Action(S2A2),它将视觉特征与声学空间和声学信号信息集成用于声学感知操作任务。我们实现了将ACT、Diffusion Policy、VQ-BeT和$\pi_0$等策略集成到框架中的S2A2模型。模拟实验表明该方法对需要位置和音色的任务最有效。真实机器人实验证实了所提出的任务和框架在现实世界操作中的适用性。

英文摘要

Acoustic information provides rich cues about object location, material properties, and changes caused by contact or motion. This paper introduces a new set of acoustic-aware manipulation tasks for imitation learning, in which robots must use auditory cues to determine manipulation targets. These tasks require sound source localization and identification for active exploration in robotic manipulation. Also, we propose a multimodal imitation learning framework, Spatial-Spectral Audio Action (S2A2), that integrates visual features with acoustic spatial and acoustic signal information for the acoustic-aware manipulation tasks. We implemented S2A2 models that integrates policies such as ACT, Diffusion Policy, VQ-BeT, and $π_0$, into our framework. Simulation experiments showed that the proposed method is the most effective for tasks requiring both position and timbre. Furthermore, real-robot experiments confirm the applicability of the proposed tasks and framework to real-world manipulation.

URL PDF HTML 收藏
2607.26043 2026-07-29 cs.LG 新提交

Re-thinking Mammography Transfer Learning: The Dataset-Informed Transfer Learning (DITL) Framework for Breast Cancer Screening and Lesion Diagnosis

重新思考乳腺X线摄影转移学习:用于乳腺癌筛查和病变诊断的数据集知情转移学习(DITL)框架

Adarsh Bhandary Panambur, Siming Bayer, Andreas Maier

机构 * Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg(模式识别实验室,埃尔朗根-纽伦堡弗里德里希-亚历山大大学) Siemens Healthineers(西门子医疗)

AI总结 研究针对乳腺X线摄影分类性能提升难题,提出DITL框架,整合数据集难度信号与邻域监督,引入自适应组件,无需超参数调整,在大规模和小数据集上均有出色表现,建立了通用的乳腺X线摄影分类框架。

Comments 16 pages, 1 figure, 5 tables. Accepted and presented at the 10th International Conference on Computer Vision & Image Processing (CVIP 2025), IIT Ropar, India, 10-13 December 2025. The paper is currently in press for inclusion in the official conference proceedings. This preprint corresponds to the submitted manuscript and is made available pending publication of the final proceedings version

详情
AI中文摘要

在小型精选数据集和大规模临床队列中,提高乳腺X线摄影的分类性能一直是个挑战。传统转移学习方法常忽略数据集特定特征,近期邻域知情方法受限于狭窄任务和固定公式,扩展性不足。为此提出数据集知情转移学习(DITL)框架,将数据集衍生的难度信号与基于邻域的三元组监督整合在统一目标中。DITL引入两个自适应组件:自适应难度加权交叉熵(A-DWCE)和自适应邻域表示三元组(A-NR-Triplet)。在大规模VinDR-Mammo数据集上,DITL在全图像乳腺密度分类中取得了领先性能,在小ROI数据集上也有显著提升。DITL建立了一个临床相关、可扩展且通用的乳腺X线摄影分类框架。

英文摘要

Enhancing classification performance in mammography remains a persistent challenge across both small curated datasets and large-scale clinical cohorts. Conventional transfer learning approaches often neglect dataset-specific characteristics, while recent neighborhood-informed methods have been restricted to narrow tasks with rigid formulations, limiting their scalability to population-level datasets. To address these challenges, we propose the Dataset-Informed Transfer Learning (DITL) framework, which integrates dataset-derived difficulty signals with neighborhood-based triplet supervision in a unified objective. DITL introduces two adaptive components: (i) Adaptive Difficulty-Weighted Cross-Entropy (A-DWCE), which assigns per-sample weights based on k-nearest neighbor label purity in a self-supervised feature space, and (ii) Adaptive Neighborhood Representation Triplet (A-NR-Triplet), which enforces intra-class compactness and inter-class separation using a learnable margin. Unlike focal loss, DITL requires no hyperparameter tuning, removes heuristic weighting and fixed margins, and incurs negligible computational overhead, yielding a robust and scalable optimization strategy. On the large-scale VinDR-Mammo dataset, DITL achieves state-of-the-art performance for whole-image breast density classification, with significant improvements across accuracy, F1-score, and AUC (p < 0.0001). Beyond large cohorts, DITL also delivers consistent, statistically significant gains on small ROI datasets (p < 0.0001). By bridging small-scale lesion analysis with large-scale density estimation, DITL establishes a clinically relevant, scalable, and generalizable framework for mammography classification, spanning the full breast cancer screening-to-diagnosis spectrum.

URL PDF HTML 收藏
2607.26042 2026-07-29 cs.CV cs.LG 新提交

VetClaw: An Edge-Cloud Multimodal Agentic System for Veterinary Disease Screening

VetClaw:一种用于兽医疾病筛查的边缘云多模态智能系统

Syed Mhamudul Hasan, Anas AlSobeh, Hussein Zangoti, Abdur R. Shahid

机构 * Southern Illinois University(南伊利诺伊大学) Utah Valley University(犹他谷大学) Jazan University(吉赞大学)

AI总结 VetClaw是用于兽医疾病筛查的边缘云多模态智能系统,用相机模块采集图像等并发送至视觉语言模型分类。它分离智能体交互与工作流编排,超越静态图像分类,症状引导和多模态输入提升性能,将静态模型转变为多功能安全感知系统。

详情
AI中文摘要

我们展示了VetClaw,一种用于早期兽医疾病筛查的边缘云多模态智能系统。VetClaw使用相机模块作为边缘传感设备,并将捕获的图像以及可选的症状描述发送到服务器托管的视觉语言模型进行零样本疾病分类。该系统将智能体交互与工作流编排分离:OpenClaw在边缘设备上提供调度、工具访问、用户交互和通知服务,而LangGraph管理有状态的筛查工作流,包括输入验证、图像传输、模型调用、安全检查、条件路由、故障处理和结构化日志记录。这种设计超越了静态图像分类,使系统能够收集视觉证据、调用外部模型、应用确定性安全规则并生成诊断支持警报。结果表明,仅图像的VLM预测仍然有限,而症状引导和多模态输入可提高零样本分类性能。因此,VetClaw将静态预测模型转变为一个能够使用工具、管理工作流、处理故障并升级不确定病例的协调的、安全感知系统。

英文摘要

We present VetClaw, an edge-cloud multimodal agentic system for early veterinary disease screening. VetClaw uses a camera module as an edge sensing device and sends captured images, together with optional symptom descriptions, to a server-hosted vision-language model for zero-shot disease classification. The system separates agent interaction from workflow orchestration: OpenClaw provides scheduling, tool access, user interaction, and notification services on the edge device, while LangGraph manages the stateful screening workflow, including input validation, image transmission, model invocation, safety checks, conditional routing, failure handling, and structured logging. This design moves beyond static image classification by enabling the system to collect visual evidence, invoke external models, apply deterministic safety rules, and generate diagnostic-support alerts. Results show that image-only VLM prediction remains limited, whereas symptom-guided and multimodal inputs improve zero-shot classification performance. Thus, VetClaw transforms a static prediction model into a coordinated, safety-aware system that can use tools, manage workflows, handle failures, and escalate uncertain cases.

URL PDF HTML 收藏
2607.26037 2026-07-29 cs.CV cs.GR 新提交

Wonder: Video World Model Done Better

Wonder:改进的视频世界模型

Jiacong Xu, Hanwen Jiang, Zhixin Shu, Kalyan Sunkavalli, Vishal M. Patel, Yiqun Mei

机构 * Adobe Research(Adobe研究院) Johns Hopkins University(约翰·霍普金斯大学)

AI总结 Wonder是用于实时相机可控世界探索的通用视频世界模型,通过系统级协同设计,包括新型相机条件设定、高效内存机制等,能合成多样视频,支持视频条件生成,保持长时间的连贯视觉效果。

Comments Project Page: https://wonder-world-model.github.io/

详情
AI中文摘要

我们展示了Wonder,一种用于实时、相机可控世界探索的通用视频世界模型。给定图像或条件视频,Wonder构建一个可玩的世界,用户能通过移动相机交互式导航,实时且长期地发现未见区域并回访先前观察区域。实现此能力需要控制方法、内存机制和训练策略的系统级协同设计。我们引入一种带密集坐标场的新型相机条件设定,其渲染提供空间对齐的运动和方向线索。还提出基于高效稀疏注意力的内存机制,并开发多种技术纠正自强制式蒸馏管道,使Wonder能以16帧每秒合成多样的分钟级视频,同时跨长时间保持连贯的几何、外观和动态。此外,Wonder自然支持视频条件生成,可实时重新拍摄现有动态场景。

英文摘要

We present Wonder, a general-purpose video world model for real-time, camera-controllable world exploration. Given an image or a conditional video, Wonder constructs a playable world where users can navigate interactively by moving the camera, discovering unseen regions, and revisiting previously observed areas in real time and over a long-term horizon. Achieving this capability requires a system-level co-design of control method, memory mechanism, and training strategy. We introduce a novel camera conditioning with a dense coordinate field whose renderings provide spatially aligned motion and orientation cues, allowing the model to interpret camera motion directly as visual evidence. To support fast and precise memory retrieval over a growing generation context, we propose an efficient sparse attention-based memory mechanism, enabling the model to selectively attend to a small set of relevant context tokens at inference time, regardless of actual context length. We further develop several techniques to rectify the self-forcing-style distillation pipeline, improving the student model's ability to respect control signals, as well as maintaining diverse generation modes and long-term memory from the teacher. Together, these components enable Wonder to synthesize diverse, minute-scale videos at 16 FPS while preserving coherent geometry, appearance, and dynamics across long rollouts. Beyond image-to-video generation, Wonder naturally supports video-conditioned generation, allowing existing dynamic scenes to be re-shot in real time.

URL PDF HTML 收藏
2607.26034 2026-07-29 cs.AI cs.CY cs.GT econ.GN q-fin.EC 新提交

Falling Behind Drives Unsafe Development in an Idealised AI Race Experiment

在理想化人工智能竞赛实验中落后导致不安全发展

Elias Fernández Domingos, The Anh Han

机构 * Vrije Universiteit Brussel(布鲁塞尔自由大学) Université Libre de Bruxelles(布鲁塞尔自由大学) Teesside University(提赛德大学)

AI总结 研究人工智能竞赛中速度与安全的关系,通过实验发现不安全发展受竞赛战略状态影响,引入进化模型解释,表明政策应注重降低竞争压力、促进合作,而非仅关注个体风险。

Comments 45 pages (main + SI)

详情
AI中文摘要

技术竞赛在速度与安全间制造紧张关系:行动者即便冒险发展有害,但比对手更快行动可能获利。这在人工智能辩论中很突出,竞争压力常促使更冒险、安全意识更低的发展。我们通过基于理想化人工智能竞赛的框架行为实验研究此问题,配对参与者在不确定时间范围内反复在安全与不安全发展间选择。不安全发展进步更快、即时收益更高,但会累积私人风险,最高分别达10%、60%或90%,竞赛竞争结构不变,仅最大风险不同。数据不支持预先注册的风险水平比较及引出的风险偏好作用。相反,由任务重复结构激发的探索性分析表明,不安全行为受竞赛演变战略状态而非风险偏好影响更大:对手选择不安全后,参与者更可能选不安全,领先减少不安全行为,落后则增加,首轮选择可预测后续行为。为解释这些影响,我们引入简化进化模型,有始终安全、始终不安全、有条件安全和有条件反社会安全四种策略,该模型再现了处理效果并展示了竞争竞赛动态如何青睐有条件不安全行为。实验和模型共同表明,不安全发展可能源于早期行为势头、对手行为和对落后的恐惧,而非仅源于风险偏好,这表明政策应注重降低人工智能发展中的竞争压力并促进合作,而非仅关注个体风险。

英文摘要

Technological races create tension between speed and safety: actors may gain by moving faster than competitors, even when risky development is harmful. This is prominent in debates about artificial intelligence (AI), where competitive pressure is often argued to incentivise riskier, less safety-conscious development. We study this using a framed behavioural experiment based on an idealised AI race, in which paired participants repeatedly chose between Safe and Unsafe development under an uncertain time horizon. Unsafe development gave faster progress and higher immediate payoffs but accumulated private risk up to a treatment-specific maximum of 10\%, 60\%, or 90\%; the race's competitive structure was held constant, and only this maximum risk varied. Neither the pre-registered comparison between risk levels nor the role of elicited risk preferences was supported by the data. Instead, exploratory analyses motivated by the task's repeated structure show that Unsafe behaviour is shaped less by risk preferences than by the evolving strategic state of the race: participants are more likely to choose Unsafe after their opponent does so, being ahead reduces Unsafe play while falling behind increases it, and first-round choices predict later behaviour. To interpret these effects we introduce a reduced evolutionary model with four strategies -- Always Safe, Always Unsafe, Conditionally Safe, and Conditionally Antisocial Safe -- which reproduces the treatment effect and shows how conditional Unsafe behaviour can be favoured by competitive race dynamics. Together, the experiment and model show that unsafe development can emerge from early behavioural momentum, opponent behaviour, and fear of falling behind, rather than from risk preferences alone, suggesting policy should focus on reducing competitive pressure and promoting cooperation in AI development rather than only individual risk.

URL PDF HTML 收藏
2607.26023 2026-07-29 cs.AI 新提交

CHARM: A Multimodal Graph Foundation Model with Hierarchical Context Modeling for Zero-Shot Transfer

CHARM:一种用于零样本迁移的具有分层上下文建模的多模态图基础模型

Ankang Yang, Jitao Zhao, Di Jin, Yuxiao Huang, Dongxiao He

机构 * School of Computer Science and Technology, College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部计算机科学与技术学院) Data Science Program, Columbian College of Arts and Sciences, The George Washington University(乔治华盛顿大学文理学院哥伦比亚艺术与科学学院数据科学项目)

AI总结 研究多模态图零样本迁移问题,提出CHARM模型,通过分层上下文建模,用图上下文替换原始节点,减少对目标域监督或适配的依赖,经实验验证该模型在零样本多模态图任务上有改进。

详情
AI中文摘要

图基础模型已成为跨图域和任务迁移知识的有前景的范式。现实世界的图将节点与文本、图像等模态相关联,使得多模态图对于表示复杂实体和关系至关重要。由于收集标签和为每个新图域调整模型成本高且往往不可行,因此激发了零样本迁移的需求。然而,多模态图上的零样本迁移仍未得到充分探索。现有基于GNN的图基础模型通常需要下游适配,而基于LLM的图方法主要处理单模态图或单个域内的任务。本文提出了CHARM,一种用于零样本迁移的具有分层上下文建模的多模态图基础模型。CHARM用捕获多模态语义和跨模态关系的分层图上下文替换孤立的原始节点。这些上下文将特定域的节点模式映射到共享的高级概念,减少对目标域监督或适配的依赖。实验表明在零样本多模态图任务上有持续改进。

英文摘要

Graph foundation models (GFMs) have emerged as a promising paradigm for transferring knowledge across graph domains and tasks. Real-world graphs associate nodes with text, images, and other modalities, making multimodal graphs essential for representing complex entities and relations. Moreover, collecting labels and adapting models for every new graph domain is costly and often infeasible, motivating zero-shot transfer. Unfortunately, zero-shot transfer on multimodal graphs remains underexplored. Existing GNN-based graph foundation models typically require downstream adaptation, whereas LLM-based graph methods mainly address unimodal graphs or tasks within a single domain. This setting presents two key challenges. First, models must generalize knowledge from individual modalities while capturing transferable cross-modal relations. Second, without target-domain fine-tuning, node representations remain entangled with domain-specific structures and modality-specific characteristics, obscuring shared concepts in unseen domains. To address these challenges, we propose CHARM, a multimodal graph foundation model with hierarchical context modeling for zero-shot transfer. CHARM replaces isolated raw nodes with hierarchical graph contexts that capture multimodal semantics and cross-modal relations. These contexts map domain-specific node patterns to shared high-level concepts, reducing reliance on target-domain supervision or adaptation. A modality-aware graph context encoder integrates multimodal information with graph structure and converts the resulting representations into graph tokens for a large language model . Experiments show consistent improvements on zero-shot multimodal graph tasks.

URL PDF HTML 收藏
2607.26017 2026-07-29 cs.CL 新提交

UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams

UniMem:用于无边界任务流的互补情景到参数记忆

Siyu Xia, Chenheng Zhang, Yanting Wu, Haoxuan Li, Jiajun Chai, Xiaohan Wang, Guojun Yin, Wei Lin, Zhouchen Lin, Haifeng Zhang, Jun Wang

机构 * Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院) State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室) Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院) Meituan(美团) AI Centre, Department of Computer Science, University College London(伦敦大学学院计算机科学系人工智能中心)

AI总结 研究针对大语言模型在无边界任务流部署的稳定性 - 可塑性困境,提出UniMem自路由框架,用可学习路由令牌协调互补记忆路径,能按需扩展记忆,实验证明其在长周期流任务序列中优于基线并保持执行保真度。

详情
AI中文摘要

记忆对于大语言模型智能体积累任务经验和复用特定任务执行策略至关重要。然而,在无边界且不断演变的任务流上进行实际部署时,会出现基本的稳定性 - 可塑性困境。基于外部检索的记忆能快速吸收新证据,但常无法内化重复执行模式且产生推理时的检索开销。参数记忆一旦学习完成就能实现稳定高效执行,但通常依赖明确的任务边界和固定参数预算。受人类大脑通过互补情景存储和逐步巩固来平衡可塑性和稳定性的启发,我们提出了UniMem,一种用于自主记忆管理的自路由框架。UniMem使用可学习的路由令牌作为记忆控制器,在互补记忆路径间实现自适应协调。新的或稀疏的任务保留在情景缓冲区中用于检索增强执行,而重复且可靠的模式则被巩固到可扩展的参数记忆中。通过用路由令牌和参数记忆块将任务识别与任务执行解耦,UniMem在部署期间无需任务标签或不受控制的参数增长就能按需扩展记忆。在长周期流任务序列上的实验表明,UniMem在保持执行保真度的同时始终优于基线,在三个骨干模型上平均获得4.0 EM点的增益。

英文摘要

Memory is essential for LLM agents to accumulate task experience and reuse task-specific execution strategies. However, real-world deployment over boundary-agnostic and evolving task streams exposes a fundamental stability-plasticity dilemma. External retrieval-based memory can rapidly absorb new evidence, but it often fails to internalize recurring execution patterns and incurs inference-time retrieval overhead. Parametric memory enables stable and efficient execution once learned, but typically relies on explicit task boundaries and fixed parameter budgets. Inspired by the human brain, which balances plasticity and stability through complementary episodic storage and gradual consolidation, we propose UniMem, a self-routing framework for autonomous memory management. UniMem uses learnable routing tokens as memory controllers, enabling adaptive coordination between complementary memory pathways: novel or sparse tasks are retained in an episodic buffer for retrieval-augmented execution, while recurring and reliable patterns are consolidated into expandable parametric memory. By decoupling task identification from task execution with routing tokens and parametric memory blocks, UniMem expands memory on demand without task labels during deployment or uncontrolled parameter growth. Experiments on long-horizon streaming task sequences show that UniMem consistently outperforms baselines while maintaining execution fidelity, achieving an average gain of 4.0 EM points across three backbone models.

URL PDF HTML 收藏
2607.26015 2026-07-29 cs.CL 新提交

Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do

指令微调模型在局部比人类更多地复用人类句法

Zandi Eberstadt

机构 * University of Oxford(牛津大学)

AI总结 研究大语言模型是否像人类一样句法趋同,通过替换范式数据在多个模型测量CFG规则复用。发现各模型与前一轮人类话语规则重叠多,指令微调模型有特点,还在词汇和语义相似度上有表现,揭示了模型在句法复用方面与人类的异同。

详情
AI中文摘要

句法趋同是人类对话中一个有充分记录的特征,被广泛认为在意识层面之下运作。大语言模型是否相对于人类基线并在广泛的句法结构中表现出类似的句法趋同仍是一个开放问题。本研究使用替换范式数据,在16个开放权重的Llama和Gemma模型(1B - 70B,预训练和指令微调)的1901个匹配位置测量上下文无关语法(CFG)规则的相邻轮复用。每个模型与前一轮人类话语的CFG规则重叠都比与采样的不相关人类引导语更多,且低频规则的实际与随机差异更大。每个指令微调模型与实际引导语的自然输出重叠也比它所替换的人类回应更多。然而,相对于预训练变体,指令微调输出与不相关引导语重叠更多,实际与随机增量更小,且在目标规则集大小不变时条件规则复用几率更低。在探索性分析中,每个模型与前一轮的平均词汇和语义相似度都比匹配的人类回应更高。指令微调模型在所有八个架构对中产生的回应平均语义相似度也比其预训练对应模型更高,而词汇相似度结果更具异质性。

英文摘要

Syntactic convergence (the tendency of speakers to adapt in language towards the grammatical profiles of their interlocutors) is a well-documented feature of human dialogue widely considered to operate below conscious awareness. Whether large language models exhibit analogous syntactic convergence toward human users relative to human baselines and across a broad range of syntactic constructions remains an open question. Using substitution-paradigm data in which model generations replace one speaker's turns in pre-existing human dialogues, this study measures turn-adjacent reuse of context-free grammar (CFG) rules across sixteen open-weight Llama and Gemma models (1B-70B, pretrained and instruction-tuned) at 1,901 matched positions per model. Every model showed greater CFG-rule overlap with the preceding human turn than with a sampled unrelated human prime, and in every model this actual-versus-random difference was larger for lower-frequency rules. Each instruction-tuned model also showed greater natural-output overlap with the actual prime than the human response it replaced, and all eight matched architecture pairs exhibited greater actual-prime overlap after instruction tuning. However, relative to pretrained variants, instruction-tuned outputs overlapped more with unrelated primes, showed a smaller actual-versus-random increment, and had lower conditional rule-reuse odds once target rule-set size was held constant. In exploratory analyses, each model exhibited greater mean lexical and semantic similarity to the preceding turn than the matched human responses did. Instruction-tuned models additionally produced responses with greater mean semantic similarity than their pretrained counterparts in all eight architecture pairs, whereas the lexical similarity results were more heterogeneous.

URL PDF HTML 收藏
2607.26005 2026-07-29 cs.CV cs.AI cs.RO 新提交

Pictura: Perspective-View Self-Play at Scale for Driving

Pictura:用于驾驶的大规模视角自博弈

Yuan Yin, Elias Ramzi, Marc Lafon, Valentin Charraut, Victor Bares, Yihong Xu, Éloi Zablocki, Alexandre Boulch, Thibault Buhet, Andrei Bursuc, Matthieu Cord

机构 * Valeo Brain(法雷奥智能大脑公司) Sorbonne Université, CNRS, ISIR(索邦大学、法国国家科学研究中心、智能系统与机器人研究所)

AI总结 研究针对驾驶模拟中自博弈存在的问题,提出建立视角自博弈训练机制,介绍GPU加速模拟器Pictura,通过普通PPO自博弈训练出Alberti策略,无需特权观测,性能接近且能零样本迁移优于特权矢量化策略。

详情
AI中文摘要

在模拟中进行自博弈可大规模生成稳健的驾驶策略。以往此类行为演示使用了如精确姿态和速度等有特权的矢量化观测,即便针对被遮挡的智能体。这假定感知问题已解决,且与从以自我为中心的相机视角进行部署的智能体的部分观测存在表征差距。常见解决方法是将有特权的策略提炼为相机输入的学生策略,但会使学生模仿自身视角无法证明合理的决策。我们建立了视角自博弈作为一种实用训练机制。引入Pictura,这是一个GPU加速的多智能体驾驶模拟器,在每一步渲染每个智能体的以自我为中心的视角,从源头减轻表征差距。Pictura在单个H100上每秒可维持高达500K智能体步(2M图像/秒)。使用Pictura,我们通过普通PPO自博弈训练了Alberti。它是首个直接从视角图像训练的大规模驾驶自博弈策略,无需特权观测。训练跨越50B智能体步,行驶约35M公里。其性能接近有特权的矢量化对应策略,并能零样本迁移到在Pictura中重新渲染的Waymo Open Motion数据集布局,且在此处优于有特权的矢量化智能体。

英文摘要

Self-play in simulation produces robust driving policies at scale. Demonstrations of such behavior have been made using privileged vectorized observations such as exact poses and velocities, even for occluded agents. This assumes that perception is solved and introduces a representation gap with the partial observation of a deployed agent driving from the perspective view of egocentric cameras. A common fix, distilling the privileged policy into a camera-input student, leaves the student imitating decisions its own view cannot justify. Instead, we establish perspective-view self-play as a practical training regime. We introduce Pictura, a GPU-accelerated multi-agent driving simulator that renders each agent's egocentric view at every step, mitigating the representation gap at its source. Pictura sustains up to 500K agent-steps/s (2M images/s) on a single H100. Using Pictura, we train Alberti by self-play with plain PPO. It is the first large-scale driving self-play policy trained directly from perspective images, without privileged observations. Training spans 50B agent steps for ~35M km of driving. It approaches the driving performance of its privileged vectorized counterpart, and transfers zero-shot to Waymo Open Motion Dataset layouts re-rendered in Pictura, where it outperforms privileged vectorized agents. Project page: https://valeoai.github.io/Pictura/

URL PDF HTML 收藏
2607.26004 2026-07-29 cs.CV cs.LG 新提交

Parallel Decoding Distillation for Fast Image and Video Generation

用于快速图像和视频生成的并行解码蒸馏

Neta Shaul, Chao Liu, Arash Vahdat, Julius Berner

机构 * NVIDIA(英伟达) Weizmann Institute of Science(魏茨曼科学研究所)

AI总结 针对视频扩散或流模型生成计算昂贵问题,提出并行解码蒸馏方法PDD,兼容预训练模型,通过预测多去噪步骤加速生成,在多个数据集上达SOTA性能,提升视频多样性。

详情
AI中文摘要

视频扩散或流模型中的生成计算成本高昂,现有加速方法依赖变分分数蒸馏和对抗损失,难以优化且易出现模式崩溃。本文提出并行解码蒸馏(PDD),一种基于轨迹的简化可扩展蒸馏方法,兼容任何预训练模型,支持不同函数评估次数采样。通过预测每个网络评估的多个去噪步骤加速生成,在多个数据集上实现SOTA性能并显著提升视频多样性。

英文摘要

Generation in video diffusion or flow models is computationally expensive due to the slow and iterative sampling process. Current state-of-the-art (SOTA) acceleration methods heavily rely on variational score distillation (VSD) and adversarial losses to distill diffusion models into few-step generators. Albeit achieving high-quality video generation, these training losses are notoriously hard to optimize and suffer from mode collapse, leading to loss of video diversity and lack of motion. In this paper, we introduce Parallel Decoding Distillation (PDD), a simplified and scalable trajectory-based distillation method for fast inference of diffusion and flow matching models. Our architecture and training procedure are compatible with any pre-trained model and support sampling with a varying number of function evaluations (NFE). PDD accelerates generation by predicting multiple denoising steps per network evaluation. Conceptually, it learns a representation of the mean velocity without regressing its derivative using JVPs or finite-difference approximations. Our method achieves SOTA performance with 4-8 NFE on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image. Moreover, PDD presents a significant improvement in generated video diversity.

URL PDF HTML 收藏
2607.26001 2026-07-29 cs.LG stat.ML 新提交

Sharpness-Aware Minimization and Muon: Robustness under the Spectral Norm

锐度感知最小化与Muon:谱范数下的鲁棒性

Wenzhi Zhong, Edward Milsom, Michael Murray

机构 * University of Bath(巴斯大学)

AI总结 研究如何通过锐度感知最小化(SAM)提高泛化能力,针对其“小”扰动依赖几何结构的问题,在SAM两阶段引入矩阵感知几何结构,结合逐层谱内扰动与Muon等外更新,实验表明该组合在ViT-Small/16和ResNet-50上验证准确率最佳。

详情
AI中文摘要

锐度感知最小化(SAM)旨在通过鼓励对小的最坏情况参数扰动不敏感来提高泛化能力。然而,“小”扰动的概念本质上依赖于几何结构:虽然现有的SAM变体探索了广泛的选择,但对于在实践中哪种几何结构最有效仍缺乏清晰的认识。最近关于矩阵感知优化的工作,特别是Muon优化器,表明尊重隐藏层权重的矩阵结构可以带来强大的经验性能。受此启发,我们在SAM的两个阶段研究矩阵感知几何结构:我们为矩阵值隐藏层参数引入了逐层谱内扰动,并将其与AdamW/SGDW或Muon在外部更新中相结合。在ImageNet-1K上对ViT-Small/16和ResNet-50进行的实验中,我们发现谱内步长与Muon外步长的组合始终表现强劲,在所评估的方法中在两个模型上都实现了最佳的验证准确率。

英文摘要

Sharpness-Aware Minimization (SAM) aims to improve generalization by encouraging insensitivity to small, worst-case parameter perturbations. However, the notion of a "small" perturbation is inherently geometry-dependent: while existing SAM variants have explored a wide range of choices, a clear perspective on which geometries are most effective in practice remains elusive. Recent work on matrix-aware optimization, particularly the Muon optimizer, suggests that respecting the matrix structure of hidden-layer weights can lead to strong empirical performance. Motivated by this, we study matrix-aware geometry in both stages of SAM: we introduce a layerwise spectral inner perturbation for matrix-valued hidden-layer parameters and combine it with either AdamW/SGDW or Muon in the outer update. Across ImageNet-1K experiments on ViT-Small/16 and ResNet-50, we find that the combination of a spectral inner step with a Muon outer step performs consistently strongly, achieving the best validation accuracy on both models among the evaluated methods.

URL PDF HTML 收藏
2607.26000 2026-07-29 cs.LG cs.AI 新提交

Empirical Evaluation of Out-Of-Distribution Performance of Tabular Foundation Models

表格基础模型的分布外性能实证评估

Malena Loza, David Chushig-Muzo, Eva Milara, Luis Bote-Curiel, Luis Estrada-Petrocelli, Felipe Grijalva

机构 * Colegio de Ciencias e Ingenierías, Universidad San Francisco de Quito (USFQ)(基多圣弗朗西斯科大学科学与工程学院) Rey Juan Carlos University(胡安·卡洛斯国王大学) Facultad de Ingeniería, Universidad Latina de Panamá(巴拿马拉丁大学工程学院)

AI总结 研究对九种表格基础模型进行分布外性能实证评估,涵盖多种策略与架构,用三个真实世界数据集测试。结果显示模型在分布变化下会退化,还存在可扩展性差距,扩展了表格数据分布外的基准,为高风险领域应用提供证据。

详情
AI中文摘要

表格基础模型(TFMs)已成为表格预测任务的新方法,在独立同分布数据上训练和评估,但现实场景中的分布变化会影响其稳健性。本文对九种TFMs进行分布外(OOD)性能实证评估,涵盖多种预训练策略和架构,使用三个真实世界数据集。结果表明所有评估的TFMs在分布变化下都会系统退化,且存在可扩展性差距。该研究扩展了表格数据OOD的现有基准,为其在高风险领域的应用提供了证据。

英文摘要

Tabular Foundation Models (TFMs) have emerged as novel approaches for tabular predictive tasks, demonstrating competitive predictive performance to ensemble tree-based models. Most TFMs are trained and evaluated on independent and identically distributed data, but this assumption changes in real-world scenarios due to distribution shifts, which compromise the robustness of models. Limited research has been conducted of TFMs under distribution shifts. We present an empirical evaluation of Out-Of-Distribution (OOD) performance of nine TFMs, spanning diverse pre-training strategies and architectures: TabPFNv2, TabPFNv2.5, TabPFNv2.6, TabPFNv3, TabICL, TabICLv2, Mitra, LimiX and TabFM. Three real-world datasets from the TableShift study were considered (HELOC, Voting, Childhood Lead), covering label, socioeconomic, and geographic shift types. Our results show that all evaluated TFMs degrade systematically under distribution shift regardless of pre-training strategy, with shift gaps ranging from 0.003 to 0.060 depending on shift type. The relationship between in-distribution and OOD predictive performance documented for classical tabular models extends into TFMs. We also identified a scalability gap, as high-performing models demand significant memory and computational resources beyond what standard deployment infrastructure can support. This study extends existing benchmarks for OOD in tabular data, providing evidence to support their adoption in high-stakes domains characterized by structural distribution shifts.

URL PDF HTML 收藏
2607.25993 2026-07-29 cs.CV 新提交

Beyond Zooming: Learning Multi-Tool Visual Reasoning for Ultra-High-Resolution Remote Sensing

超越缩放:学习用于超高分辨率遥感的多工具视觉推理

Fengxiang Wang, Jiangnan Huang, Mingshuo Chen, Yueying Li, Yang Shi, Junwei Luo, Haoyu Wang, Yansheng Li, Jing Zhang, Haiyan Zhao, Wenjing Yang

机构 * National University of Defense Technology(国防科技大学) Wuhan University(武汉大学) Tsinghua University(清华大学)

AI总结 针对超高分辨率遥感图像给多模态大语言模型带来的挑战,提出GeoMTVR数据集,结合监督微调与以工具注意力为重点的强化学习算法开发GeoLens,实验证明其在多工具视觉推理方面优于直接推理和单工具放大基线。

详情
AI中文摘要

超高分辨率(UHR)遥感图像在城市尺度场景中提供了细粒度的地球观测证据,但对多模态大语言模型(MLLM)构成了根本挑战,因为任务相关证据往往稀疏、局部且在极大的视觉背景中空间分散。自然的解决办法是为MLLM配备放大工具进行主动局部检查。然而,通过对XLRS-Bench的初步研究发现,放大仅部分有效。在此基础上,引入了GeoMTVR数据集,它包含13K UHR VQA样本。还提出了一种以工具注意力为重点的强化学习算法。结合在GeoMTVR上的监督微调与该强化学习算法,开发了GeoLens。实验表明,GeoLens始终优于直接推理和单工具放大基线。

英文摘要

Ultra-high-resolution (UHR) remote-sensing (RS) imagery provides fine-grained Earth-observation evidence over city-scale scenes, but poses a fundamental challenge for multimodal large language models (MLLMs): task-relevant evidence is often sparse, local, and spatially dispersed across extremely large visual contexts. A natural solution is to equip MLLMs with zoom-in tools for active local inspection. However, through a pilot study on XLRS-Bench, we find that zoom-in is only partially effective: it resolves easy and medium-level tasks with locally recoverable evidence, but saturates on hard cases requiring global search, multi-region comparison, path planning, or dispersed-evidence reasoning. Motivated by this finding, we move beyond single-tool zoom-in and introduce GeoMTVR, a large-scale Geospatial Multi-Tool Visual Reasoning dataset built from wide-area satellite imagery. GeoMTVR contains 13K UHR VQA samples with interleaved reasoning trajectories, diverse visual tool calls, and returned visual observations, enabling models to learn question decomposition, tool selection, regional inspection, object-level grounding, auxiliary visual reasoning, and cross-tool evidence integration. Beyond supervised fine-tuning, we propose a tool-attention-focused reinforcement learning algorithm that concentrates optimization on critical tool-use decisions, including when to invoke tools, which tool to select, where to apply it, and how to interpret tool outputs. By combining SFT on GeoMTVR with our RL algorithm, we develop GeoLens, a multi-tool visual reasoning MLLM for UHR RS. Experiments show that GeoLens consistently outperforms direct reasoning and single-tool zoom-in baselines, achieving stronger accuracy, better evidence grounding, and more efficient tool-use trajectories.

URL PDF HTML 收藏
2607.25990 2026-07-29 cs.CV 新提交

On the Use of Synthetic Data for Threshold Calibration in Face Recognition: Performance and Security Implications for Border Control Systems

关于合成数据在人脸识别阈值校准中的应用:对边境控制系统的性能和安全影响

Arto Apila

机构 * Faculty of Information Technology and Electrical Engineering, University of Oulu(奥卢大学信息技术与电气工程学院)

AI总结 研究合成数据用于边境控制系统人脸识别阈值校准,分析合成与真实数据集分数分布对齐及校准阈值跨域转移性,发现合成数据在受控环境可近似校准,但在无约束条件下不可靠,校准结果依赖数据集,高安全部署需真实数据验证调整。

Comments Accepted by IJCB 2026

详情
AI中文摘要

最近部署的出入境系统(EES)将大规模生物特征验证引入欧洲边境管制,要求人脸识别系统在极低的误匹配率(FMR)下运行。监管框架在EES中央系统层面定义了性能目标,但未规定成员国层面实际校准验证阈值的方法。在实际操作中,获取用于校准的代表性真实世界数据常受法律、后勤和隐私限制。本文研究合成人脸数据在与边境控制系统相关的文档到活体验证场景中的阈值校准应用。分析合成数据集与真实数据集之间真实和冒名顶替者分数分布的对齐情况,并评估校准阈值在不同域间的可转移性,重点关注低FMR工作点。结果表明,合成数据在受控环境中可近似校准行为,但由于分数分布尾部不匹配,无法可靠地推广到无约束条件。这些差异导致识别性能显著下降,且易受基于变形的攻击。进一步证明校准结果高度依赖数据集,即使是合成数据集。总体而言,研究结果表明,虽然合成数据对系统开发和初步校准有用,但在高安全部署中,可靠的阈值选择通常需要使用代表性真实世界数据进行验证和调整。

英文摘要

The recently deployed Entry/Exit System (EES) introduces large-scale biometric verification into European border control, requiring face recognition systems to operate at extremely low false match rates (FMR). While regulatory frameworks define performance targets at the EES Central System level, they do not specify how verification thresholds should be calibrated in practice at the Member State level. In operational settings, obtaining representative real-world data for calibration is often constrained by legal, logistical, and privacy limitations. In this work, we investigate the use of synthetic face data for threshold calibration in document-to-live verification scenarios relevant to border control systems. We analyze the alignment of genuine and impostor score distributions between synthetic and real datasets and evaluate the transferability of calibrated thresholds across domains, with a focus on low-FMR operating points. Our results show that synthetic data can approximate calibration behavior in controlled settings, but fails to reliably generalize to unconstrained conditions due to mismatches in score distribution tails. These discrepancies lead to significant degradation in recognition performance and increased vulnerability to morph-based attacks. We further demonstrate that calibration outcomes are highly dataset-dependent, even across synthetic datasets. Overall, our findings highlight that while synthetic data is useful for system development and preliminary calibration, our results indicate that reliable threshold selection in high-security deployments typically requires validation and adjustment using representative real-world data.

URL PDF HTML 收藏
2607.25988 2026-07-29 cs.LG 新提交

Generator-Aligned Representation Interfaces for Diagnostic Soft Equivariance

用于诊断软等变性的生成器对齐表示接口

Weitao Li, Gong Cheng

机构 * Tongji University(同济大学)

AI总结 研究针对精确等变架构复用复杂的问题,提出生成器对齐表示接口GARI,通过特定原则和残差形式化行为,实例化为GARI-Net,经实验验证其能支持变换一致性和泛化,为硬等变架构提供可移植诊断补充。

详情
AI中文摘要

精确等变架构通常在专用算子中编码规定的群作用,这使其与通用主干的复用及跨数据模态使用变得复杂。我们引入生成器对齐表示接口(GARI),一种表示级设计原则,通过对齐的规范视图和生成器诱导视图将选定的变换生成器暴露给通用序列主干。我们使用在声明的数据和变换分布上定义的特定探针软等变残差来形式化结果行为。该框架区分表示一致性、任务鲁棒性和精确等变性,并将残差不匹配定位到接口构建、共享流处理和终端融合。我们将接口实例化为GARI-Net,它构建生成器索引流,将其转换为公共交互框架,用共享参数处理,修复顺序诱导的上下文不匹配,实现跨流信息交换,并使用流间差异进行聚合。直接等变误差(DEE)在已知令牌或体素作用下提供规定表示关系的冻结检查点诊断。在基因组序列、图像和三维点云的实验中检验了序列反转、平面旋转和反射以及受控轴向转移。在这些设置中,相同的接口原则支持与任务相关的变换一致性和对声明的保留探针的泛化,而无需对序列主干进行特定于组的重新设计。因此,GARI为硬等变架构提供了一种可移植的诊断补充:它使生成器结构可访问、可学习和可测量,而有限探针证据与连续群上精确等变性的认证仍有区别。

英文摘要

Exact-equivariant architectures typically encode prescribed group actions in specialized operators, which can complicate their reuse with generic backbones and across data modalities. We introduce the Generator-Aligned Representation Interface (GARI), a representation-level design principle that exposes selected transformation generators to a generic sequence backbone through aligned canonical and generator-induced views. We formalize the resulting behavior using a probe-specific soft-equivariance residual defined over declared data and transformation distributions. This framework distinguishes representation consistency from task robustness and exact equivariance, and localizes residual mismatch to interface construction, shared stream processing, and terminal fusion. We instantiate the interface as GARI-Net, which constructs generator-indexed streams, converts them into a common interaction frame, processes them with shared parameters, repairs ordering-induced context mismatch, enables cross-stream information exchange, and aggregates them using inter-stream discrepancy. Direct Equivariance Error (DEE) provides a frozen-checkpoint diagnostic of the prescribed representation relation under known token or voxel actions. Experiments on genomic sequences, images, and three-dimensional point clouds examine sequence reversal, planar rotations and reflections, and controlled axial transfer. Across these settings, the same interface principle supports task-relevant transformation consistency and generalization to declared held-out probes without requiring group-specific redesign of the sequence backbone. GARI therefore provides a portable diagnostic complement to hard-equivariant architectures: it makes generator structure accessible, learnable, and measurable, while finite-probe evidence remains distinct from certification of exact equivariance over a continuous group.

URL PDF HTML 收藏
2607.25962 2026-07-29 cs.CV 新提交

LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection

LaP-Forensics:用于深度伪造检测的潜在像素一致性引导的多模态推理

Can Wang, Yuhao Wang, Yushe Cao, Canran Xiao, Fei Shen

机构 * The Hong Kong Polytechnic University(香港理工大学) University College London(伦敦大学学院) Tsinghua University(清华大学) Sun Yat-sen University(中山大学) National University of Singapore(新加坡国立大学)

AI总结 研究针对深度伪造检测问题,提出LaP-Forensics多模态框架,利用基于重建的取证证据及结构化模型预测,经组相对策略优化,实现跨生成器检测和伪影定位,实验验证了残差流效用,但后处理下文本忠实性和可靠性有局限。

Comments Accepted at ACM Multimedia 2026 (ACM MM 2026)

详情
AI中文摘要

近期生成模型能生成视觉伪影少的图像,削弱了仅依赖表面外观的检测器和解释。我们提出LaP-Forensics,一个用基于重建的取证证据增强RGB语义的多模态框架。冻结的Stable Diffusion DDIM反演-重建模型提供固定重建参考,其残差图衡量与该参考的局部兼容性。独立投影仪对RGB图像和残差图编码,结构化的Where-What-Why模型预测文本分析和伪影。微调后进行组相对策略优化(GRPO),其奖励结合掩码重叠与输出结构及证据-参考项。单独的图像级头部融合RGB和DDIM-残差类特征。实验表明在UniversalFakeDetect上有跨生成器检测能力,在官方SynthScars基准上有有竞争力的伪影定位能力。控制线索构建、反演范围、组件、奖励项和反事实分析支持了评估设置下残差流的效用,而后处理下的自由形式文本忠实性和可靠性仍是开放的局限性。

英文摘要

Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance. We present LaP-Forensics, a multimodal framework that augments RGB semantics with reconstruction-based forensic evidence. A frozen Stable Diffusion DDIM inversion-reconstruction model provides a fixed reconstruction reference, and its residual map measures local compatibility with that reference. Independent projectors encode the RGB image and residual map before a structured Where-What-Why model predicts a textual analysis and an artifact mask.Supervised fine-tuning is followed by Group Relative Policy Optimization (GRPO), whose reward combines mask overlap with output-structure and evidence-reference terms. These text-side terms encourage the model to refer to the consistency map but do not constitute a verifier of free-form textual truth. A separate image-level head fuses RGB and DDIM-residual class features. Experiments show cross-generator detection on UniversalFakeDetect and competitive artifact localization on the official SynthScars benchmark. Controlled cue-construction, inversion-horizon, component, reward-term, and counterfactual analyses support the utility of the residual stream under the evaluated settings, while free-form textual faithfulness and reliability under post-processing remain open limitations.

URL PDF HTML 收藏
2607.25959 2026-07-29 cs.CL cs.AI 新提交

Detecting Knowledge Inconsistencies Across Text, Tables, and Knowledge Graphs

跨文本、表格和知识图谱检测知识不一致性

Fanfu Wei, Thibault Ehrhart, Raphaël Troncy

机构 * EURECOM(欧洲电信学院)

AI总结 研究跨文本、表格和知识图谱的知识不一致性检测问题,提出\textsc{Kontrast}框架,通过文本到SPARQL和语言模型推理比较并分类不一致性,实验表明跨模态不一致性常见且有价值,为知识审计提供工具并建立基准。

详情
AI中文摘要

维基百科和维基数据被广泛用于信息获取、语言模型预训练和检索增强生成。它们的知识紧密相连但分散于文本、表格和知识图谱中。本文研究模态级不一致性检测问题,首先引入跨模态知识不一致性分类,涵盖信息粒度差异、直接冲突、时间变化和知识图谱不完整性。接着提出自动框架\textsc{Kontrast},利用文本到SPARQL和语言模型推理比较基于表格的答案与知识图谱证据并分类不一致性。在多个表格问答数据集上的实验表明跨模态不一致性常见且有信息价值,揭示了真实知识冲突、知识图谱结构缺失和时间不匹配等问题,同时受文本到SPARQL错误和噪声限制。分析表明文本、表格和知识图谱可通过系统比较相互补充和修正。\textsc{Kontrast}为大规模知识审计提供实用工具并为跨模态知识一致性未来工作建立基准。代码和数据可通过指定链接获取。

英文摘要

Wikipedia and Wikidata are widely used for information access, LLM pre-training, and retrieval-augmented generation. Their knowledge is deeply connected but scattered across text, tables, and knowledge graphs. This raises a practical question: when these modalities disagree, how can we detect and explain the conflict? We study this problem as \emph{modality-level inconsistency detection}. We first introduce a taxonomy of cross-modal knowledge inconsistencies, covering information granularity differences, direct conflicts, temporal changes, and KG incompleteness. We then present \textsc{Kontrast}, an automatic framework that uses Text-to-SPARQL and LLM reasoning to compare table-based answers with KG evidence and categorize the resulting inconsistencies. Experiments on various Table-QA datasets show that cross-modal inconsistencies are common and informative. They reveal not only true knowledge conflicts, but also missing KG structure and temporal mismatches while being limited by Text-to-SPARQL errors and noise. Our analysis shows that text, tables, and KGs can complement and correct one another through systematic comparison. \textsc{Kontrast} provides a practical tool for large-scale knowledge auditing and establishes a benchmark for future work on cross-modal knowledge consistency. Code and data are available at https://github.com/ECLADATTA/KONTRAST.

URL PDF HTML 收藏
2607.25953 2026-07-29 cs.CL cs.CY 新提交

Polistemics: Evaluating LLMs as Information Mediators in Politics & Elections

Polistemics:评估大语言模型在政治与选举中作为信息中介的表现

Baran Peters

机构 * ETH Agentic Systems Lab(瑞士联邦理工学院智能系统实验室) CORDA(无合适常见中文名,可音译为“科尔达”)

AI总结 研究探讨如何评估大语言模型在政治选举中作为信息中介的表现。引入基于认知谦逊的Polistemics基准,在不同信息属性环境下测试。应用于三个大语言模型发现,虽总分高但有系统性故障,受党派先验影响,可靠中介可实现但模型表现不稳定。

详情
AI中文摘要

随着大语言模型(LLMs)越来越多地作为公民所依赖的政治信息中介,目前仍没有标准化方法来评估它们是否负责地履行这一职责。我们引入了Polistemics,这是一个基于理论的基准,用于评估大语言模型在选举中作为政治信息中介的情况。先前的工作将此任务视为复制而非中介,未解决其认知维度以及与不完美信息的交互问题。我们将评估建立在认知谦逊之上,这是一种源自公民认知能动性的规范标准,并在不同信息属性(如清晰度、噪声和一致性)的受控环境中进行测试。将该基准应用于针对2025年德国和荷兰选举的三个先进大语言模型,我们发现高总分掩盖了系统性故障。模型在明确证据下能可靠地进行中介,但在信息缺失、模糊或矛盾时会失效,同时还会使政治语言的强度趋于平缓。这些故障可能由党派先验驱动,受党派标签和输出语言影响。可靠的中介似乎是可以实现的,但没有模型能始终如一地做到。

英文摘要

As LLMs increasingly mediate the political information citizens rely on, there is still no standardized way to assess whether they do so responsibly. We introduce Polistemics, a theory-grounded benchmark for evaluating LLMs as mediators of political information in elections. Prior work has treated this task as reproduction rather than mediation, leaving its epistemic dimensions and interaction with imperfect information unaddressed. We ground the evaluation in Epistemic Modesty, a normative standard derived from citizens' epistemic agency, and test it across controlled settings that vary informational properties such as clarity, noise, and consistency. Applying the benchmark to three state-of-the-art LLMs on the 2025 German and Dutch elections, we find that high aggregate scores mask systematic failures. Models mediate reliably under clear evidence but break down under absent, vague, or contradictory information, while flattening the intensity of political language. These failures are likely driven by party priors, influenced by party labels and output language. Reliable mediation appears achievable, but no model delivers it consistently.

URL PDF HTML 收藏
2607.25947 2026-07-29 cs.AI cs.CL 新提交

A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series

一种用于不规则临床时间序列问答的经济高效多模态大语言模型推理框架

Frank Nie, Ethan B Liu, Yuan Zhu, Wei Fan, Jindong Han

机构 * Shandong University(山东大学) University of Auckland(奥克兰大学)

AI总结 针对不规则临床时间序列问答,提出ClinPRISM框架。该框架含不规则感知多尺度编码器、时间证据蒸馏器和渐进对齐策略,通过构建训练数据,在40亿参数大语言模型主干上实现最优性能,兼具低时间序列令牌数和低推理延迟。

详情
AI中文摘要

在不规则临床时间序列(ICTS)上进行问答在众多医疗应用中起着关键作用。尽管近期多模态时间序列大语言模型在通用时间序列问答中展现出潜力,但在处理临床观察的稀疏性、异步性和不规则采样模式方面仍存在不足。为填补这一空白,我们提出了ClinPRISM,一种用于ICTS数据问答的经济高效多模态大语言模型推理框架。首先,设计了一个不规则感知多尺度编码器来捕捉不同时间尺度上的稀疏临床证据;接着,提出了一个时间证据蒸馏器来整合跨尺度的表示并压缩成少量与大语言模型兼容的令牌;还引入了一种渐进对齐策略。为便于训练,构建了30000个与多尺度描述配对的临床时间序列以及41000个跨11个任务的指令微调实例。使用40亿参数的大语言模型主干,ClinPRISM在留出的评估基准上实现了最优性能,仅使用16个时间序列令牌,每个问题的平均推理延迟为0.15秒。

英文摘要

Question answering (QA) over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications. Although recent multimodal time-series large language models (LLMs) have shown considerable promise in general-purpose time-series QA, they remain poorly equipped to model the sparsity, asynchrony, and irregular sampling patterns of clinical observations. To fill this gap, we propose ClinPRISM, a cost-effective multimodal LLM reasoning framework for question answering over ICTS data. First, we devise an irregularity-aware multi-scale encoder to capture sparse clinical evidence at diverse temporal scales. Then, we propose a temporal evidence distiller to integrate representations across these scales and compress them into a small number of LLM-compatible tokens. Moreover, we introduce a progressive alignment strategy that sequentially aligns the irregular trajectories with the LLM's textual embedding space. To facilitate training, we construct 30,000 clinical time series paired with multi-scale descriptions, together with 41,000 instruction-tuning instances spanning 11 tasks. Using a 4-billion-parameter LLM backbone, ClinPRISM achieves state-of-the-art performance on the held-out evaluation benchmark while using only 16 time-series tokens and achieving an average inference latency of 0.15 seconds per question.

URL PDF HTML 收藏
2607.25933 2026-07-29 cs.CL cs.AI 新提交

Evaluating Multi-Turn Multimodal Diagnostic Reasoning on Challenging Real-World Clinical Cases

评估具有挑战性的真实世界临床病例中的多轮多模态诊断推理

Rui Yang, Weihao Xuan, Yi Lin, Zhuhan Bao, Jonathan Chong Kai Liew, Matthew Yu Heng Wong, Nicolás Lescano, Nikita R. Paripati, Emily Ling-Lin Pai, Jiarui Liu, Heli Qi, Heng-Jui Chang, Benny Kai Guo Loo, Huitao Li, Kunyu Yu, Yufan Wang, Chuan Hong, Shijian Lu, Douglas Teodoro, Naoto Yokoya, Ross Koppel, Mona Diab, Hua Xu, David W. Bates, Nan Liu, Yifan Peng

机构 * Center for Biomedical Data Science, Duke-NUS Medical School(生物医学数据科学中心,杜克 - 新加坡国立大学医学院) Duke-NUS AI + Medical Sciences Initiative, Duke-NUS Medical School(杜克 - 新加坡国立大学人工智能与医学科学计划,杜克 - 新加坡国立大学医学院) Department of Population Health Sciences, Weill Cornell Medicine(人口健康科学系,威尔康奈尔医学院) System Engineering, College of Engineering, Cornell University(系统工程,康奈尔大学工程学院) Graduate School of Frontier Sciences, The University of Tokyo(前沿科学研究生院,东京大学) RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心) Department of Biostatistics and Bioinformatics, Duke University(生物统计学与生物信息学系,杜克大学) Perelman School of Medicine, University of Pennsylvania(佩雷尔曼医学院,宾夕法尼亚大学) School of Clinical Medicine, University of Cambridge(临床医学学院,剑桥大学) Hospital of the University of Pennsylvania(宾夕法尼亚大学医院) Children’s Hospital of Philadelphia (CHOP)(费城儿童医院) Department of Anatomic Pathology and Laboratory Medicine, Hospital of the University of Pennsylvania(解剖病理学与检验医学系,宾夕法尼亚大学医院) Department of Pathology and Laboratory Medicine, University of California, San Francisco(病理学与检验医学系,加州大学旧金山分校) Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)

AI总结 研究针对现有多模态大语言模型评估难以捕捉现实临床诊断复杂性的问题,开发ClinMM-Bench基准,用两级框架评估15个模型,发现专有模型诊断准确性最高,但各模型完全正确诊断比例有限,当前模型诊断推理有局限并明确了失败模式。

详情
AI中文摘要

临床诊断评估不仅应评估模型能否提供正确诊断,还应反映临床实践的现实情况,包括多模态信息的逐步披露、诊断假设的动态更新以及临床推理的持续完善。然而,现有的多模态大语言模型评估通常依赖单轮或孤立任务,难以充分捕捉现实世界临床诊断的复杂性。为弥补这一差距,我们开发了ClinMM-Bench,这是迄今为止最大的多轮多模态临床诊断评估基准。ClinMM-Bench包含1089个具有挑战性的真实世界临床病例和八个专业的3760幅医学图像。我们使用两级评估框架系统地评估了15个代表性的多模态大语言模型,该框架同时评估诊断准确性和诊断推理质量。结果表明,专有模型总体诊断准确性最高,但所有模型中完全正确诊断的比例仍然有限。在诊断推理质量方面,当前模型可以识别合理的诊断方向,但在生成可靠的诊断推理方面仍有相当大的局限性。错误分析进一步确定了五种代表性的失败模式:信息合成失败、知识映射错误、感知错误、过早结束和视觉幻觉。

英文摘要

Clinical diagnostic evaluation should not only assess whether models can provide correct diagnoses, but also reflect the realities of clinical practice, including progressive disclosure of multimodal information, dynamic updating of diagnostic hypotheses, and continuous refinement of clinical reasoning. However, existing evaluations of multimodal large language models (MLLMs) typically rely on single-turn or isolated tasks, making it difficult to fully capture the complexity of real-world clinical diagnosis. To bridge this gap, we developed ClinMM-Bench, the largest multi-turn multimodal clinical diagnostic evaluation benchmark to date. ClinMM-Bench contains 1,089 challenging real-world clinical cases and 3,760 medical images across eight specialties. We systematically evaluated 15 representative MLLMs using a two-level evaluation framework that assessed both diagnostic accuracy and diagnostic reasoning quality. Results showed that proprietary models achieved the highest overall diagnostic accuracy, but the proportion of completely correct diagnoses remained limited across all models. In terms of diagnostic reasoning quality, current models can identify plausible diagnostic directions but still have considerable limitations in generating reliable diagnostic reasoning. Error analysis further identified five representative failure modes: information synthesis failure, knowledge mapping error, perception error, premature closure, and visual hallucination.

URL PDF HTML 收藏
2607.25926 2026-07-29 cs.CV cs.AI 新提交

Face De-Identification: A Domain-Centric Survey from Capture to Processing

面部去识别:从采集到处理的以领域为中心的综述

Hui Wei, Hao Yu, Guoying Zhao

机构 * ELLIS Institute Finland(芬兰埃利斯研究所) Center for Machine Vision and Signal Analysis, University of Oulu(奥卢大学机器视觉与信号分析中心)

AI总结 该综述以领域为中心,涵盖面部去识别从采集到处理的完整数据管道,系统分析各阶段方法、进展与挑战,回顾评估协议,确定开放问题与新兴方向,为该领域未来工作提供指导。

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Github repository: https://github.com/CV-AC/Awesome-FaceDe-ID

详情
AI中文摘要

面部去识别旨在在图像或视频中去除或隐藏可识别个人身份的面部特征,以防止身份识别,同时保留对下游任务的实用性。随着对数据隐私和负责任人工智能的日益重视,面部去识别已成为一个活跃的研究领域,跨越计算机视觉和隐私保护社区。早期方法以及许多当代方法通过捕获后处理修改像素级或外观级特征在数字领域操作。最近的进展通过在图像采集期间将隐私机制直接集成到传感器中,将面部去识别扩展到后处理之外,连接传感系统和下游视觉算法。同时,物理领域方法探索可穿戴配件和材料,在捕获之前在现实世界环境中隐藏身份信息。在本综述中,我们首次对面部去识别进行统一概述,涵盖物理、传感器和数字领域的完整数据采集管道。通过以领域为中心的视角,我们系统地分析了当前方法、技术进展以及每个阶段固有的独特挑战。我们还回顾和整理了现有的评估协议,审视当前实践并强调对标准化、全面基准的迫切需求。最后,我们确定了关键的开放问题并概述了新兴研究方向,以指导这一快速发展领域的未来工作。为支持正在进行的研究,我们维护了一个项目页面,用收集的数据集和开源代码整理相关文献:此https URL。

英文摘要

Face de-identification (De-ID) aims to remove or conceal personally identifiable facial features in images or videos to prevent identity recognition while preserving utility for downstream tasks. With the rising emphasis on data privacy and responsible AI, face De-ID has emerged as an active research area spanning computer vision and privacy-preserving communities. Early approaches, and many contemporary ones, operate in the digital domain by modifying pixel-level or appearance-level features through post-capture processing. Recent advances extend face De-ID beyond post-processing by integrating privacy mechanisms directly into sensors during image acquisition, bridging sensing systems and downstream vision algorithms. In parallel, physical-domain methods explore wearable accessories and materials that conceal identity information in real-world environments prior to capture. In this survey, we present the first unified overview that spans the full data acquisition pipeline, encompassing the physical, sensor, and digital domains. Through this domain-centric lens, we systematically analyze current methodologies, technical progress, and the distinct challenges inherent to each stage. We then review and organize existing evaluation protocols, examining current practices and highlighting the critical need for standardized, comprehensive benchmarks. Finally, we identify key open problems and outline emerging research directions to guide future work in this rapidly evolving field. To support ongoing research, we maintain a project page that organizes relevant literature with collected datasets and open source code: https://github.com/CV-AC/Awesome-FaceDe-ID.

URL PDF HTML 收藏
2607.25921 2026-07-29 cs.CV cs.AI 新提交

Evaluating VLMs for Autonomous Agent-Driven Geometry Clipping Detection in Video Game QA

评估用于视频游戏QA中自主代理驱动的几何裁剪检测的视觉语言模型

Carlos Celemin, Benedict Wilkins, Adrián Barahona-Ríos, Saman Zadtootaghaj, Nabajeet Barman

机构 * Sony Interactive Entertainment(索尼互动娱乐)

AI总结 研究在代理驱动游戏QA中用VLMs检测几何裁剪异常,通过自定义代理收集视觉观察,自动注释提供标签,在零样本提示下对六个VLMs基准测试,分析对提示变体的敏感性,结果显示VLMs适合作多阶段QA管道的高召回候选过滤器。

详情
AI中文摘要

在这项工作中,我们研究视觉语言模型(VLMs)在以几何裁剪为重点的代理驱动游戏质量保证(QA)管道中的异常检测应用。在评估中,一个自定义探索代理在游戏关卡中导航以收集视觉观察结果,同时自动注释管道提供帧级裁剪标签。此设置使我们能够在无需人工注释的情况下,在受控的异常检测任务中评估近期的VLMs。我们在零样本提示设置下对六个近期的VLMs(Gemini、GPT、Qwen、Gemma、Llama和Ministral)进行基准测试,并分析它们对四种提示变体的敏感性。结果表明,VLMs虽能捕捉与几何裁剪相关的视觉线索,但在视觉模糊的帧(如近接触几何和部分遮挡)上会产生大量误报。Gemini-3.1-Flash总体准确率最高,对提示变化最稳健,而开源模型的精确率-召回率会因提示设计有较大波动。这些发现表明,当前的VLMs最适合作为多阶段QA管道中的高召回候选过滤器,而非独立的错误检测器。

英文摘要

In this work, we study the use of Vision-Language Models (VLMs) for anomaly detection in an agent-driven game Quality Assurance (QA) pipeline focusing on geometry clipping. In this evaluation, a custom exploration agent navigates a game level to collect visual observations, while the automatic annotation pipeline provides frame-level clipping labels. This setup allows us to evaluate recent VLMs on a controlled anomaly detection task without manual annotation. We benchmark six recent VLMs (Gemini, GPT, Qwen, Gemma, Llama, and Ministral) under a zero-shot prompting setting and analyse their sensitivity to four prompt variants. Our results show that while the VLMs can capture visual cues associated with geometry clipping, they all produce substantial false positives on visually ambiguous frames such as near-contact geometry and partial occlusions. Gemini-3.1-Flash achieves the best overall accuracy and is the most robust to prompt variation, while open-source models exhibit large precision--recall swings depending on the prompt design. These findings suggest that current VLMs are best suited as high-recall candidate filters within multi-stage QA pipelines rather than as standalone bug detectors.

URL PDF HTML 收藏
2607.25912 2026-07-29 cs.RO cs.AI 新提交

SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Models

用于视觉-语言-动作模型的SAM3D引导的以对象为中心的表示对齐

Zonghe Liu, Shanyuan Jie, Xiaoquan Sun, Chen Cao, Zetian Xu, Zongsheng Liu, Jiayu Chen

机构 * University of Hong Kong(香港大学) Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院) Huazhong University of Science and Technology(华中科技大学) Beijing University of Aeronautics and Astronautics(北京航空航天大学) Infiforce(英飞拓)

AI总结 针对视觉-语言-动作模型缺乏目标对象细粒度3D理解的问题,提出以对象为中心的3D表示对齐框架,利用SAM3D提供3D先验,经定位、生成掩码、提取表示等步骤与视觉特征对齐,实验证明其可提升模型性能,尤其在长时操纵场景有效。

Comments 8 pages, 4 figures

详情
AI中文摘要

视觉-语言-动作(VLA)模型在通用机器人操纵方面显示出强大潜力,但现有模型大多依赖2D视觉语言主干,缺乏对目标对象的细粒度3D理解。我们提出了一个以对象为中心的3D表示对齐框架,基于π0构建,使用SAM3D作为冻结的3D教师在训练期间提供目标对象3D先验。具体来说,用对象识别模型定位任务相关对象,生成相应掩码,用SAM3D提取密集对象级3D表示并与π0的中间视觉特征对齐。模拟实验显示有持续改进,在LIBERO上达到99.1%,在CALVIN上平均长度为4.11。现实世界实验表明该方法在机器人必须跨多个子任务关注不同目标对象的长时操纵场景中特别有效。

英文摘要

Vision-Language-Action (VLA) models have shown strong potential for general robot manipulation, but most existing models rely on 2D visual-language backbones and lack fine-grained 3D understanding of target objects, especially under occlusion, pose variation, scale changes, and precise spatial interaction. We propose an object-centric 3D representation alignment framework built upon $π_0$, using SAM3D as a frozen 3D teacher to provide target-object 3D priors during training. Specifically, we localize task-relevant objects with object recognition models, generate corresponding object masks, and use SAM3D to extract dense object-level 3D representations, which are aligned with intermediate visual features of $π_0$. This enables the policy to internalize target-object 3D information while preserving the original RGB-language-to-action inference pipeline without requiring depth, point clouds, masks, SAM3D, or additional 3D modules at test time. Simulation experiments show consistent improvements, achieving 99.1\% on LIBERO and an average length of 4.11 on CALVIN. Real-world experiments further demonstrate that our method is particularly effective in long-horizon manipulation scenarios where the robot must focus on different target objects across multiple subtasks.

URL PDF HTML 收藏
2607.25907 2026-07-29 cs.LG cs.AI cs.CL 新提交

Minimizing Targeted Activations: Input-Only Suppression of Evaluation-Awareness Latents in Large Language Models

最小化目标激活:大语言模型中仅输入的评估感知潜在因素抑制

Deepanshu Mody, Samarth Agarwal, Utkarsh Mittal, Dipesh Mahato

机构 * Center for Data Science, New York University(纽约大学数据科学中心)

AI总结 研究大语言模型中仅通过输入优化提示来抑制“评估感知”潜在因素,采用特定方法在多种目标结构下对Llama模型进行实验,发现潜在因素可抑制,但激活可读性与行为可控性不同,还解决了相关问题。

详情
AI中文摘要

激活引导通过在推理时编辑内部激活来控制模型行为。我们研究其输入侧对偶:优化一个流畅的提示,使得一个选定的内部潜在因素趋向于零,且无需在推理时访问模型。我们的目标是一个“评估感知”潜在因素,在最近的工作中它是线性可读且可引导的,如果模型在检测到被测试时行为不同,那么对其控制将威胁到安全评估的有效性。通过采用带有否定特征项的流畅梦境/EPO(广义对比梯度风格的令牌优化加上自交叉熵流畅正则化器,并扫描流畅权重),我们在五个目标结构下抑制了Llama-3.2-3B和Llama-3.1-8B上的潜在因素,包括CAA方向、子空间范数、SAE特征、单个MLP神经元和行为逻辑。潜在因素可被稳健抑制($z\approx -7$),并且一个经过因果验证的Llama范围SAE特征可以被完全且有选择地关闭。但我们的控制揭示了关于CAA方向的警示:一个安慰剂随机方向被抑制的程度相同且行为变化幅度相同,当我们在上下文中保留真实评估段落并仅优化前缀时,抑制评估方向并不能减少——反而略微增加——模型的行为评估判断。简而言之,激活可读性并非行为可控性。我们进一步发现,单个MLP神经元在两个规模下都与评估相关但并非因果关系,并且扫描真实语料库会产生一个与优化器在内部方向上竞争的自然文本基线。一个积极对照验证了我们的擦除检测器,解决了一个早期遗留的擦除与旋转问题。

英文摘要

Activation steering controls model behavior by editing internal activations at inference time. We study its input-side dual: optimizing a fluent prompt so that a chosen internal latent is driven toward zero, with no inference-time model access. Our target is an "evaluation-awareness" latent-linearly readable and steerable in recent work-whose control would threaten the validity of safety evaluations if models behave differently when they detect being tested. Adapting Fluent Dreaming / EPO with a negated feature term (GCG-style token optimization plus a self-cross-entropy fluency regularizer, swept over a fluency weight), we suppress the latent under five target constructions-a CAA direction, a subspace norm, an SAE feature, a single MLP neuron, and a behavioral logit-on Llama-3.2-3B and Llama-3.1-8B. The latent is robustly suppressible ($z\approx-7$), and a causally-validated Llama Scope SAE feature can be fully and selectively turned off. But our controls tell a cautionary story about the CAA direction: a placebo random direction is suppressed just as hard and shifts behavior just as far, and when we hold a real eval passage in context and optimize only a prefix, suppressing the eval-direction fails to reduce-and slightly increases-the model's behavioral eval judgment. Activation-readability, in short, is not behavioral controllability. We further find that a single MLP neuron is eval-correlated but not causal at both scales, and that scanning the real Pile yields a natural-text baseline competitive with the optimizer for the internal direction. A positive control validates our erasure detector, bounding an erasure-vs-rotation question earlier left open.

URL PDF HTML 收藏
2607.25895 2026-07-29 cs.RO cs.CV cs.LG 新提交

HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

HiFi-UMI:仅从高保真UMI数据中学习可部署的操作策略

Simple AI, :, Yuteng Wei, Jinming Ma, Jiawei Wang, Weitao Zhou, Yushen Zuo, Ke Rui, Minglei Li, Jinhao Zhang, Zhikang Pan, Xiang Wang, Haoran Jia, Huan Du, Zicheng Zeng, Jun Ma, Guiyu Qin, Di Zhang, Xiaofei Li

机构 * Simple AI(简单人工智能公司)

AI总结 研究如何从高保真UMI数据学习可部署操作策略,提出HiFi-UMI系统,其无需外部跟踪达3毫米精度。利用该系统数据实现零机器人训练后策略,预训练还能降误差提成功率,且开源了相关高保真资源。

Comments 33 pages, 15 figures, 4 tables. Project page: https://cloud.simpleai.tech/simple-world-lab/hifi-umi/ Dataset: https://huggingface.co/datasets/simple-world-lab/HiFi-UMI-2K

详情
AI中文摘要

学习可部署的操作策略受到高保真且可扩展数据稀缺的限制。真实机器人遥操作准确但扩展成本高;无机器人UMI捕获易于扩展,当前主要用于预训练,训练后添加少量真实机器人“锚点”。我们探讨提高无机器人UMI数据保真度能否去除该锚点。提出HiFi-UMI,一个针对轨迹精度、夹爪间相对姿态、同步和视野共同设计的便携式UMI数据生成系统。它无需外部跟踪基础设施就能达到3毫米的工作空间局部末端执行器精度。利用该数据集,我们展示了零机器人训练后策略:仅在HiFi-UMI演示上训练后的策略可直接部署在真实机器人上,并在跨越视觉-语言-动作和世界-动作-模型家族的三个主干上与领域内遥操作匹配。在预训练4000小时后,十个未见任务的动作误差降低41%,在StarVLA-QwenPI上真实机器人成功率进一步提高18.1个百分点。我们开源HiFi-UMI-2K,作为机器人学习社区的大规模、高保真资源。

英文摘要

Learning deployable manipulation policies is bottlenecked by the scarcity of data that is both high-fidelity and scalable. Real-robot teleoperation is accurate but costly to scale; robot-free UMI capture scales readily, and current practice uses the resulting data mainly for pre-training, adding a small real-robot "anchor" at post-training. We ask whether raising the fidelity of robot-free UMI data, rather than shrinking the real-robot fraction, can remove that anchor. We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, synchronization, and field of view: head-mounted offline stereo-inertial SLAM, native rather than reconstructed relative pose, a shared microsecond GPIO trigger, and two wide-angle cameras per hand covering ~200 degrees. It reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure. Using this corpus, we demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches in-domain teleoperation across three backbones spanning the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on StarVLA-QwenPI, OpenPI-pi_0.5, and LingBot-VA; the strongest policy reaches 85% on a precision insertion task, even though the teleoperation baseline is collected in the evaluation scene and no HiFi-UMI trajectory is. Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41% and, on StarVLA-QwenPI, raises real-robot success by a further 18.1 percentage points. We open-source HiFi-UMI-2K, 2,000 hours of microsecond-synchronized, ultra-wide-FoV demonstrations, each automatically reconstructed and validated through simulation replay, as a large-scale, high-fidelity resource for the robot-learning community.

URL PDF HTML 收藏
2607.25894 2026-07-29 cs.CV 新提交

TIGA: Trajectory-Injected Generative Attack against Black-box AIGC Detectors

TIGA:针对黑盒AIGC检测器的轨迹注入生成攻击

Xia Du, Zhuosen Bao, Zheng Lin, Jizhe Zhou, Jiawei Lian, Chi-man Pun, Jun Luo, Wei Ni, Symeon Chatzinotas

机构 * School of Computer and Information Engineering, Xiamen University of Technology(厦门理工学院计算机与信息工程学院) Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg(卢森堡大学安全、可靠性和信任跨学科中心) School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University(四川大学计算机学院机器学习与工业智能工程研究中心) PCA Laboratory, Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院高维信息智能感知与系统教育部重点实验室PCA实验室) Department of Computer and Information Science, Faculty of Science and Technology, University of Macau(澳门大学科技学院计算机与信息科学系) School of Engineering, Edith Cowan University(伊迪斯科文大学工程学院) College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)

AI总结 针对黑盒AIGC检测器,TIGA提出无需源图像和训练的框架,通过操纵DDIM轨迹、聚合梯度及方向搜索估计目标响应,实现强大黑盒攻击性能、可转移性及高鲁棒性,且无需源图像或扩散模型再训练。

Comments 14 pages, 5 figures

详情
AI中文摘要

近期扩散模型在面部图像合成中实现了高度逼真,给人工智能生成内容(AIGC)取证带来挑战。现有逃避方法存在局限。我们提出轨迹注入生成攻击(TIGA),它无需源图像和训练,在单个扩散采样轨迹内生成逃避检测的图像。TIGA操纵潜在去噪扩散隐式模型(DDIM)轨迹,聚合多个白盒替代检测器的梯度形成先验,通过对称有限差分查询进行各向异性方向搜索估计黑盒目标响应。实验表明TIGA在无源图像或扩散模型再训练的情况下,能实现强大的黑盒攻击性能、可转移性及高鲁棒性。

英文摘要

Recent diffusion models have achieved remarkable realism in facial image synthesis, posing growing challenges to artificial intelligence-generated content (AIGC) forensic detectors.Existing evasion methods typically perturb pre-generated images or require detector-aware training, which may introduce visible or statistical artifacts and limit applicability when the diffusion model must remain frozen and the target detector is accessible only through black-box queries. We propose Trajectory-Injected Generative Attack (TIGA), a source-image-free and training free framework that generates detector-evasive images within a single diffusion sampling trajectory. TIGA steers the latent Denoising Diffusion Implicit Model (DDIM) trajectory so that adversarial properties emerge during generation rather than being added afterward. TIGA first aggregates gradients from multiple white-box surrogate detectors to form a transferable, sign-aware prior, and then performs anisotropic directional search with symmetric finite-difference queries to estimate the black-box target response. The estimated directions are stabilized by decayed momentum and injected according to the DDIM noise schedule, with frequency-domain reshaping to suppress high frequency artifacts. Experiments on surrogate and unseen specialized forensic detectors show that TIGA achieves strong blackbox attack performance, transferability, and high robustness under common post-processing operations without source images or diffusion-model retraining, while preserving high perceptual quality.

URL PDF HTML 收藏