arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 1993
2608.05149 2026-08-06 cs.CV 新提交

CoCo-IR: Contextual Composed Image Retrieval

CoCo-IR:上下文组合图像检索

Shengcao Cao, Tanmaya Shekhar Dabral, Zhongli Ding, Madhuri Shanbhogue, Kaifeng Chen, Zhe Li, Mojtaba Seyedhosseini, Yu-Xiong Wang, Liang-Yan Gui

机构 * University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) Google DeepMind(谷歌DeepMind) OpenAI

AI总结 针对现有图像检索系统无法处理多轮交互的局限,提出CoCo-IR任务,构建基于LMM的模型并开发自主数据引擎,在CIRCO和自建CoCo-IR基准上取得最优性能。

Comments ECCV 2026

详情
AI中文摘要

当前基于指令的图像检索系统功能强大,但仅限于单轮交互,无法捕捉复杂现实视觉搜索的迭代特性。为克服这一局限,我们提出上下文组合图像检索(CoCo-IR)这一新任务,使用户可通过交互逐步优化搜索结果。我们基于多模态大模型(LMM)构建新模型,作为CoCo-IR的上下文感知推理器,该模型会解析全部交互历史,生成随轮次演化的可变换图像嵌入(TIE)。为在无需昂贵人工标注的情况下支撑模型训练,我们开发了一套完全自主、可扩展的数据引擎,利用LMM生成高质量上下文检索数据,并通过模型引导的验证挖掘具有挑战性的难负样本。大量实验表明,我们的方法达到了新的最优性能:在极具挑战性的单轮基准CIRCO上,我们取得了39.4的mAP@5;此外,在我们的新CoCo-IR基准上,模型在4轮对话中保持了44.1的R@1,显著优于现有方法(4轮R@1为28.2),现有方法无法处理多轮上下文。项目页面:this https URL。

英文摘要

Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions. We address this new task by proposing a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR. Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns. To fuel the model training without expensive human annotations, we develop a fully autonomous, scalable data engine that leverages LMMs to generate high-quality contextual retrieval data, and uses model-guided verification to mine challenging hard negatives. Extensive experiments demonstrate that our approach establishes new state-of-the-art performance: We achieve 39.4 mAP@5 on the challenging single-turn benchmark CIRCO; furthermore, on our new CoCo-IR benchmark, our model maintains robust performance with 44.1 R@1 on 4-turn dialogues, dramatically outperforming existing methods (28.2 4-turn R@1) that fail to handle multi-turn context. Project page: https://CoCo-IR.github.io.

URL PDF HTML 收藏
2608.05148 2026-08-06 cs.CL 新提交

Reasoning Core: Designing Broad Procedural Data for Completion-Supervised Reasoning Training

推理核心:为完成监督式推理训练设计广谱过程化数据

Damien Sileo, Valentin Lacombe, Dimitri Kachler

机构 * Univ. Lille(里尔大学) Inria(法国国家信息与自动化研究所) CNRS(法国国家科学研究中心) Centrale Lille(里尔中央理工学院)

AI总结 本研究推出含50类任务生成器的Reasoning Core集合,经对比实验,其在3B规模模型的三个推理基准上得分最优,相关资源已公开。

Comments 20 pages, 3 figures. Code: https://github.com/sileod/reasoning-core Data: https://hf.co/collections/reasoning-core/datasets

详情
AI中文摘要

过程化生成器可大规模生成有用且可验证的推理问题,但作为完成监督式微调的数据却较少受到关注。我们推出Reasoning Core,这是一个包含50个生成器的集合,涵盖数学、逻辑、规划、状态跟踪、形式语言、结构化数据、游戏、因果关系和代码领域,具备语义评分器、难度控制和任务评估器。在匹配的完成监督式协议下,我们在四个基础模型设置和多个训练时长中,将Reasoning Core与Procedural Warmup、Reasoning Gym和SynLogic进行对比。在主要的3B规模模型对比中,Reasoning Core在DROP、LogiQA和ARC-Challenge上取得最高平均分数,超过了无过程化数据的基线以及所有三种替代过程化集合。任务层面分析显示,仅语义有效性无法确保训练效用,凸显紧凑目标和校准难度是重要的设计因素。我们开展了结合模型辅助审查、人工裁决和回归测试的审计,应用于Reasoning Core开发全过程及其他集合,结果揭示了生成、渲染、目标和评分之间存在细微不匹配,提醒人们仅过程化生成无法保证正确性。该库、生成的数据集和审计材料均已公开可用。

英文摘要

Procedural generators produce useful verifiable reasoning problems at scale, but have received less attention as data for completion-supervised fine-tuning. We introduce Reasoning Core, a collection of 50 generators spanning mathematics, logic, planning, state tracking, formal languages, structured data, games, causality, and code, with semantic scorers, difficulty controls, and task evaluators. Under a matched completion-supervised protocol, we compare Reasoning Core with Procedural Warmup, Reasoning Gym, and SynLogic across four base-model settings and multiple training durations. In the primary 3B comparison, Reasoning Core achieves the highest mean scores on DROP, LogiQA, and ARC-Challenge, exceeding both the baseline without procedural data and all three alternative procedural collections. Task-level analyses show that semantic validity alone does not ensure training utility, highlighting compact targets and calibrated difficulty as important design factors. We ran audits combining model-assisted review, human adjudication, and regression testing. Applied throughout Reasoning Core development and to the other collections, they reveal subtle mismatches among generation, rendering, targets, and scoring, a reminder that procedural generation alone does not guarantee correctness. The library, generated datasets, and audit material are publicly available.

URL PDF HTML 收藏
2608.05145 2026-08-06 cs.CV 新提交

Objects as Audio-Visual Modal Sound Fields

作为视听模态声场的物体

Zisen Shao, Zihao Wei, Derong Jin, Ruohan Gao

机构 * University of Maryland, College Park(马里兰大学帕克分校)

AI总结 本文针对现有碰撞声建模方法成本高或需大量数据的问题,提出AV-MSF表示,结合3D高斯溅射与模态参数实现少样本重建,在真实数据集上达最优渲染性能,还支持接触定位等下游应用。

Comments ECCV 2026, Project page: $\href{https://zisenshao.github.io/AV-MSF/}{\text{this https URL}}$

详情
AI中文摘要

尽管现代三维重建在建模物体几何与外观方面表现出色,但很大程度上忽略了物理交互所揭示的丰富声学线索。物体碰撞声传递出材料、刚度和结构特性,这些特性可与视觉信息形成互补。然而,现有的碰撞声建模方法要么依赖成本高昂的基于物理的仿真,要么需要大量数据集才能以纯数据驱动的方式实现泛化。本文提出一种新颖的物体级声学表示——视听模态声场(AV-MSF),该表示从多视图图像和仅少量碰撞声录音中重建得到。AV-MSF基于三维高斯溅射(3D Gaussian Splatting)并结合密集三维视觉特征构建,以提供强几何感知先验,同时用紧凑且具有物理意义的模态参数表示碰撞声场,从而实现稳健的少样本重建。在两个真实世界数据集上开展的实验表明,AV-MSF在碰撞声渲染方面达到了当前最优性能,优于基于物理的基线方法和数据驱动的基线方法。此外,本文还展示了该表示支持的下游应用,包括接触定位和物体声音编辑。

英文摘要

While modern 3D reconstruction excels at modeling object geometry and appearance, it largely ignores the rich acoustic cues revealed through physical interaction. Object impact sounds convey material, stiffness, and structural properties that complement vision, yet existing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed from multi-view images and only a few impact sound recordings. AV-MSF builds on 3D Gaussian Splatting integrated with dense 3D visual feature to provide a strong geometry-aware prior, and represents the impact sound field using compact, physically meaningful modal parameters, enabling robust few-shot reconstruction. Experiments on two real-world datasets show that AV-MSF achieves state-of-the-art impact sound rendering, outperforming both physics-based and data-driven baselines. Furthermore, we demonstrate downstream applications enabled by our representation, including contact localization and object sound editing.

URL PDF HTML 收藏
2608.05144 2026-08-06 cs.AI 新提交

Argus: A General-Purpose Agentic Runtime for Long-Horizon Reasoning

Argus:面向长程推理的通用智能体运行时

Boxiu Li, Zimo Wen, Yijia Fan, Junxiang Lei, Sufeng Guo, Jiaao Wu, Ruize Tang, Mukai Li, Yifei Shen, Xiaoyu Chen, Wanbo Zhang, Runjing Gu, Yifei Gao, Yuheng Wu, Xuyao Huang, Zelong Zhao, Jiachen Zhang, Shibo Hu, Hangxi Guo, Yilin Chen, Yuzhe Zhang, Fan Yang, Chuan Wen, Xian Zhang, Xuanhe Zhou, Zhijie Deng

机构 * Microsoft(微软公司) Shanghai Jiao Tong University(上海交通大学) Fudan University(复旦大学) Nanjing University(南京大学) Tsinghua University(清华大学) The University of Hong Kong(香港大学) Peking University(北京大学) The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)) Donghua University(东华大学)

AI总结 Argus是面向长程推理的通用智能体运行时,通过多角色协作与自进化,在多个基准测试中表现优于同类方法,还完成了实际应用任务并产出结构化轨迹。

详情
AI中文摘要

长程推理需要一种智能体运行时,当证据支持当前方法时能持续运行,而当测量结果显示失败、隐藏约束或目标设定错误时能调整方向。我们提出了Argus,这是一种持久、自进化的运行时,其中Manager(管理者)、Planner(规划者)、Engineer(工程师)和Reviewer(审核者)在持久的项目状态上执行有界任务。Argus将稳定的用户意图与操作目标、约束及验证标准分离,仅在角色所有的审核且(若可用)任务原生验证后,才允许记忆、技能、程序、验证器、路由决策及被拒绝的路由存在。模型权重保持固定;自进化通过持久的运行时状态和控制策略实现,在操作者所有的升级点之间自主执行。在7个GPT-5.5基准领域中,Argus在SWE-Bench Pro上达到约78%,而Direct Copilot为59%,同时使用1.41倍的总token量。在经过验证门控的自进化后,成熟的SWE-Bench批次相比初始批次,每个任务的求解输入token量减少21%,主动工作流程时间减少15%,同时记录了34次验证器恢复和22次严格审核循环的挽救。Argus在AARRI-Bench上达到76.8%,在数学数据合成上有28.0分的差距,并有具有竞争力的GPU内核和语言模型训练结果。除基准测试外,一个优化的RWKV6内核已被合并到上游;一个为期多天的数学活动保留了被伪造的路由和有证明支持的前沿更新;六个论文流水线完成了254个任务,其中有16次阶段回滚。这些结果表明,固定权重、自进化的管控工具可以修改、恢复并积累已验证的方法,同时为未来的监督学习和强化学习生成结构化轨迹。

英文摘要

Long-horizon reasoning requires an agentic runtime that can persist when evidence supports its current approach and pivot when measurements reveal failure, hidden constraints, or a misspecified objective. We present Argus, a persistent, self-evolving runtime in which Manager, Planner, Engineer, and Reviewer execute bounded missions over durable project state. Argus separates stable user intent from operational objectives, constraints, and verification criteria, and admits memories, skills, procedures, verifiers, routing decisions, and rejected routes only after role-owned review and, when available, task-native verification. Model weights remain fixed; self-evolution occurs through persistent runtime state and control policy, with autonomous execution between operator-owned escalation points. Across seven GPT-5.5 benchmark arenas, Argus achieves about 78% on SWE-Bench Pro versus 59% for Direct Copilot while using 1.41 times the aggregate tokens. After verification-gated self-evolution, mature SWE-Bench waves use 21% fewer solve-input tokens and 15% less active workflow time per task than startup waves, while recording 34 verifier recoveries and 22 strict review-loop rescues. Argus also reaches 76.8% on AARRI-Bench and a 28.0-point gap on mathematical data synthesis, with competitive GPU-kernel and language-model-training results. Beyond benchmarks, an optimized RWKV6 kernel was merged upstream; a multi-day mathematics campaign retained falsified routes and proof-backed frontier updates; and six paper pipelines completed 254 missions with 16 stage rollbacks. These results show that a fixed-weight, self-evolving harness can revise, recover, and accumulate verified approaches while producing structured trajectories for future supervised and reinforcement learning.

URL PDF HTML 收藏
2608.05139 2026-08-06 cs.CL cs.LG 新提交

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning

面向技能原生的大语言模型:用于基准测试和训练长程推理的技能熵

Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora

机构 * Princeton University(普林斯顿大学) Carnegie Mellon University(卡内基梅隆大学) University of Toronto(多伦多大学) University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校) Stanford University(斯坦福大学) University of Oxford(牛津大学)

AI总结 该研究针对现有基准无法评估LLM跨技能长程推理能力的问题,提出Skill Entropy(技能熵)并构建Skill²-Bench基准,还开发Skill-Entropy RL训练框架,显著提升了Qwen3模型在该基准上的表现。

Comments https://github.com/Gen-Verse/Skill-Entropy-RL

详情
AI中文摘要

近期大语言模型(LLM)的长程推理要求模型在推理链中切换不同技能,例如先进行数学推导,再利用结果规划日程。我们将这类问题称为跨技能长程任务:即多步骤任务,其各步骤需要不同的推理技能且依赖于早期输出。现有基准通常评估单个技能,缺乏衡量模型在技能间切换能力的有效方法。我们从评估和训练两方面解决这一缺口。我们提出Skill Entropy(技能熵),用于衡量从一个技能切换到另一个技能的难度。随后我们构建了Skill²-Bench,这是一个跨技能长程任务基准,涵盖9个可验证且开放的领域,共558项技能。每个任务都被赋予任务级技能熵分数,并分为三个难度级别。我们在Skill²-Bench上评估了8个前沿模型和4个开源模型,发现存在技能切换差距:在更高熵的任务上,模型准确率会下降。接着我们将技能熵从基准尺度转化为训练信号,提出Skill-Entropy RL,这是一个强化学习(RL)框架,其中模型不仅预测每一步的答案,还预测用于生成该答案的技能。奖励函数结合了步骤级正确性与技能熵奖励,技能熵奖励用于衡量模型预测的技能序列与黄金技能序列的一致性。在Qwen3-4B-Instruct和Qwen3-1.7B上,Skill-Entropy RL将Skill²-Bench的分数分别从34.4%提升至68.4%、从14.6%提升至40.1%,优于竞争基准。该流程还可应用于现成训练数据如OpenR1-Math,表明技能熵是一种可复用的训练信号。代码可从该https URL获取。

英文摘要

Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule. We call such problems cross-skill long-horizon tasks: multi-step tasks whose steps require different reasoning skills and depend on earlier outputs. Existing benchmarks often evaluate individual skills, lacking a principled way to measure how well a model switches between skills. We address this gap from both the evaluation and training sides. We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another. We then propose Skill^2-Bench, a benchmark of cross-skill long-horizon tasks built over 558 skills across 9 verifiable and open-ended domains. Each task is assigned a task-level skill-entropy score and grouped into three difficulty levels. Evaluating 8 frontier and 4 open-source models on Skill^2-Bench reveals a skill-switching gap: accuracy decreases on higher-entropy tasks. We then turn skill entropy from a benchmark scale into a training signal. We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it. The reward combines step-level correctness with a skill-entropy reward that measures the alignment between the model-predicted skill sequence and the gold skill sequence. On Qwen3-4B-Instruct and Qwen3-1.7B, Skill-Entropy RL improves the Skill^2-Bench score from 34.4% to 68.4% and from 14.6% to 40.1%, respectively, outperforming competitive baselines. The same pipeline can be applied to off-the-shelf training data such as OpenR1-Math, indicating that skill entropy is a reusable training signal. Code available at: https://github.com/Gen-Verse/Skill-Entropy-RL

URL PDF HTML 收藏
2608.05137 2026-08-06 cs.CV 新提交

SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding

SmartMage:面向3D场景理解的动态模态编排

Yue Zhang, Yingzhao Jian, Yunqiu Xu, Xiaoxiao Sun, Hehe Fan

机构 * Zhejiang University(浙江大学) Stanford University(斯坦福大学)

AI总结 本文针对现有MLLMs采用固定模态组合的缺陷,提出SmartMage模型,通过SMART和MAGE模块动态编排模态,在5个3D场景理解基准上取得最优性能,在RGB视频理解基准上表现具竞争力。

详情
AI中文摘要

理解3D场景是具身智能的基础,需要对视觉、几何等多种模态的异构信息进行联合推理。然而,不同查询下各模态的相关性往往存在差异。现有的多模态大语言模型(MLLMs)通常依赖固定的模态组合,忽略了查询依赖的模态需求。这种僵化的设计会引入无关模态的语义噪声,同时未能充分利用信息更丰富的模态,导致计算资源浪费和推理效果被削弱。为应对这些挑战,本文提出SmartMage,一种用于语义感知3D场景理解的统一MLLM,可动态编排异构模态。具体而言,SmartMage包含两个核心模块:(1)语义引导的模态自适应路由(SMART)模块,该模块利用语义先验、文本-模态对齐以及模态质量来选择与任务相关的模态;(2)模态感知门控专家(MAGE)模块,该模块借助模态先验引导专家激活,促进多模态推理中的自适应专业化。实验结果表明,SmartMage在5个3D场景理解基准上达到了当前最优性能,在仅RGB的视频理解基准上也取得了具有竞争力的结果。在我们的诊断基准ScanFacet中,任务被划分为细粒度语义类别,可分析每种语义类型偏好的模态组合,观察到的模态-语义模式进一步验证了SmartMage的有效性。项目页面:this https URL。

英文摘要

Understanding 3D scenes is fundamental to embodied intelligence, requiring joint reasoning over heterogeneous information from multiple modalities, including visual and geometric cues. However, the relevance of these modalities often varies across queries. Existing Multimodal Large Language Models (MLLMs) typically rely on fixed modality combinations, overlooking query-dependent modality needs. Such a rigid design can introduce semantic noise from irrelevant modalities while underutilizing more informative ones, leading to wasted computation and diluted reasoning. To address these challenges, this paper proposes SmartMage, a unified MLLM that dynamically orchestrates heterogeneous modalities for semantic-aware 3D scene understanding. Specifically, SmartMage incorporates: (1) a Semantic-guided Modality Adaptive RouTng (SMART) module that selects task-relevant modalities using semantic priors, text-modality alignment, and modality quality; and (2) a Modality-Aware Gating Expert (MAGE) module that leverages modality priors to guide expert activation, fostering adaptive specialization in multimodal reasoning. Empirically, SmartMage achieves state-of-the-art performance across five 3D scene understanding benchmarks, and attains competitive results on RGB-only video understanding benchmarks. In our diagnostic benchmark ScanFacet, tasks are divided into fine-grained semantic categories, enabling analysis of modality combinations preferred by each semantic type. The observed modality-semantic patterns provide further evidence of SmartMage's effectiveness. Project page: https://yuecheong.github.io/SmartMage/.

URL PDF HTML 收藏
2608.05136 2026-08-06 cs.LG 新提交

The Loss Does Not See the Basis, but Adam Does

损失函数看不到基,但Adam可以

Devender Singh

机构 * Memorial University of Newfoundland(纽芬兰纪念大学)

AI总结 该研究揭示梯度下降隐式偏向低秩解而Adam不偏向的原因是损失的规范对称性,确定了等变优化器的条件,通过实验分析了9种优化器的性能,指出基选择是优化器的关键决策而非调优细节。

Comments 22 pages main text + appendices, 5 figures. Code, seeds, and raw run records: https://github.com/idevender/loss-basis-adam

详情
AI中文摘要

在因子化模型 $W = UV^\top$ 上的梯度下降会隐式偏向低秩解,而从相同小初始化开始的Adam则不会。我们将这种差异归因于损失的规范对称性,即损失在 $(U, V) \mapsto (UQ, VQ)$ 变换下保持不变。梯度流的低秩机制仅对规范等变的优化器可用,该条件是迁移的必要条件但非低秩恢复的充分条件。梯度下降、带动量的优化器、“共享标量”Adam、Muon和Shampoo满足该条件,而Adam、RMSProp及其他逐坐标方法不满足。一个结构定理将无记忆等变规则刻画为恰好是格拉姆矩阵确定的左预条件子,一个迁移定理将梯度流的路径性质传递给共享标量流。随后,我们在欠定矩阵感知任务上对9种更新规则,按相对于植入真值的恢复误差进行排序。从逐坐标到共享标量预条件的单参数族会单调恢复该偏向,从而确定各向异性是原因。“谱调度”调和了关于Muon的两份矛盾报告:等速率更新可精确恢复低秩目标,但随谱尾部增长会失去优势。在Transformer中,Adam在第一步就分离了两个规范等价的初始化,而等变优化器则保持在浮点精度,最终得到的头级不变量 $W_Q^\top W_K$ 的相对弗罗贝尼乌斯距离相差56%,该差距无法通过任何头级旋转消除。在两个高光谱数据集上,当训练损失匹配时,梯度下降在最低采样密度下将保留误差降低了43-44%,且有效秩更低。因此,基选择并非调优细节,而是优化器选择哪种插值函数的决策。

英文摘要

Gradient descent on a factored model $W = UV^\top$ is implicitly biased toward low-rank solutions, while Adam, starting from the same small initialization, is not. We trace the difference to the gauge symmetry of the loss, its invariance under $(U, V) \mapsto (UQ, VQ)$. Gradient flow's low-rank mechanism is available to an optimizer only if that optimizer is gauge-equivariant, a condition necessary for the transfer but not sufficient for low-rank recovery. Gradient descent, momentum, "shared-scalar" Adam, Muon, and Shampoo satisfy it. Adam, RMSProp, and the other coordinate-wise methods do not. A structure theorem characterizes the memoryless equivariant rules as exactly the Gram-determined left preconditioners, and a transfer theorem carries gradient flow's pathwise properties to common-scalar flows. We then sort nine update rules on underdetermined matrix sensing by recovery error against the planted ground truth. A one-parameter family from coordinate-wise to shared-scalar preconditioning restores the bias monotonically, isolating anisotropy as the cause. A "spectral schedule" reconciles two opposing reports about Muon: equal-rate updates recover exactly low-rank targets but lose their edge as the spectral tail grows. In transformers, Adam separates two gauge-equivalent initializations at the first step, where the equivariant optimizers stay at float precision, and ends with the per-head invariants $W_Q^\top W_K$ 56% apart in relative Frobenius distance, a gap no per-head rotation can close. On two hyperspectral datasets at matched training loss, gradient descent cuts held-out error by 43-44% at the lowest sampling density, and at lower effective rank. Basis choice is therefore not a tuning detail but a decision about which interpolant the optimizer selects.

URL PDF HTML 收藏
2608.05132 2026-08-06 cs.CV cs.LG 新提交

Predicting Brain Morphometry with MT-GNN: Mesh Evolution in Continuous Time with Graph-Based Metric Tensor Embeddings

用MT-GNN预测脑形态测量:基于图度量张量嵌入的连续时间网格演化

Hao Ding, Daniel Semchin, Paul M. Thompson, Boris Gutman

机构 * Illinois Institute of Technology(伊利诺伊理工学院) University of Southern California(南加州大学)

AI总结 该研究提出MT-GNN模型,通过图网络预测连续时间内皮层下结构的度量张量,经ADNI数据集验证,其网格演化预测性能优于DCM、TransforMesh等基准方法。

详情
AI中文摘要

从少量前期扫描预测皮层下结构的形状演化,可支持预后判断与临床试验富集。现有纵向网格预测器要么通过高维嵌入外推形状轨迹,要么直接回归顶点变形。本文转而预测连续时间内表面的内在几何:单个结构专属图网络可针对任意因果多访视历史与任意预测 horizon,结合 lead time 的傅里叶编码,预测未来每个顶点的第一基本形式(度量张量)。预测的度量通过可微的 As-Rigid-As-Possible 求解器解码为表面,模型以刚性对齐顶点误差进行端到端训练。通过重构训练可保证解码后的预测为有效表面并持续提升性能。在 ADNI 数据集的14个皮层下结构上,所提网格演化模型 MT-GNN 在所有预测 horizon 下均优于评估方法:平均顶点误差较时间均值降低-2.29%(p=6.1×10^-5,在14个结构中14个均优于时间均值),领先测地形状回归 DCM(降低-0.19%)与网格 Transformer TransforMesh(降低-0.45%,p=1.2×10^-4),且优势随 horizon 增长而扩大。

英文摘要

Predicting how a subcortical structure's shape will evolve from a few prior scans could support prognosis and clinical-trial enrichment. Existing longitudinal mesh predictors either extrapolate shape trajectories via high-dimensional embeddings or regress vertex deformations directly. We instead predict the surface's intrinsic geometry in continuous time: a single per-structure graph network predicts the future per-vertex first fundamental form (metric tensor) for an arbitrary causal multiple-visit history and an arbitrary prediction horizon, conditioned on a Fourier encoding of the lead time. The predicted metric is decoded into a surface by a differentiable As-Rigid-As-Possible solver, and the model is trained end-to-end on the rigid-aligned vertex error. Training through the reconstruction keeps the decoded prediction a valid surface and consistently improves it. On 14 subcortical structures from the ADNI dataset, the proposed mesh evolution model (MT-GNN) predicts best among the evaluated methods at every horizon ($-2.29\%$ mean vertex error vs. the temporal mean, $p{=}6.1{\times}10^{-5}$, beating it on 14/14 structures), ahead of geodesic shape regression (DCM, $-0.19\%$) and a mesh transformer (TransforMesh, $-0.45\%$; $p{=}1.2{\times}10^{-4}$), with the lead widening as the horizon grows.

URL PDF HTML 收藏
2608.05131 2026-08-06 cs.CV cs.AI 新提交

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

OPD-V:结合模态平衡的视觉在线策略自蒸馏

Aniri, Jinhe Bi, Peng Liao, Zengjie Jin, Volker Tresp, Fei Shen, Yunpu Ma, Tat-Seng Chua

机构 * National University of Singapore(新加坡国立大学) Ludwig Maximilian University of Munich(慕尼黑大学) Munich Center for Machine Learning(慕尼黑机器学习中心) Sun Yat-sen University(中山大学)

AI总结 该研究针对多模态大语言模型的模态不平衡问题,提出视觉在线策略自蒸馏范式OPD-V,通过正、负教师模型实现模态平衡,在多基准与骨干上提升推理性能并降低训练成本。

详情
AI中文摘要

在线策略自蒸馏(OPSD)已成为提升多模态大语言模型(MLLM)视觉推理能力的标准后训练方法。现有方法从不同输入源提取特权信息以指导自蒸馏,但这些设计忽略了MLLM推理中固有的模态不平衡问题:当文本信息主导生成时,模型无法充分整合多模态输入,导致精心设计的特权信息未被充分利用,限制了OPSD的有效性。为探究该局限,我们构建了基于放大图像的正教师模型和基于掩码图像的负教师模型,二者呈现不同程度的模态不平衡;其推理正确性与token logit的变化表明,模态平衡本身可作为特权信息。基于此发现,我们提出OPD-V,一种视觉OPSD范式,通过正、负教师模型实例化该信息;正模态平衡logit间隔定义了模态平衡信任域,用于选择自蒸馏所用的在线策略token。在6个基准、4个MLLM骨干和5种后训练方法上的实验显示,OPD-V可持续提升推理性能,同时降低训练成本。

英文摘要

On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.

URL PDF HTML 收藏
2608.05127 2026-08-06 cs.LG cs.AI stat.ML 新提交

SSTQ:Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant

SSTQ:基于子采样随机 TurboQuant 的隐私保护向量量化

Adel Javanmard, David P. Woodruff, Vahab Mirrokni

机构 * University of Southern California(南加州大学) Google Research(谷歌研究院) Carnegie Mellon University(卡内基梅隆大学)

AI总结 本研究针对分布式优化中隐私保护与通信成本的矛盾,提出 SSTQ 框架,实现了更优的均方误差缩放,在联邦学习任务中展现出良好效用与通信效率。

Comments 42 pages, 4 figures, 2 tables

详情
AI中文摘要

在分布式优化中实现局部差分隐私同时保持低通信成本仍是一项挑战。现有向量量化方法如 vqSGD 采用高维几何构造,但会产生不利的维度相关方差。本研究提出子采样随机 TurboQuant(SSTQ)框架,结合超完备等范数紧框架、坐标子采样及感知隐私的一维量化。SSTQ 包含两个变体:平坦随机响应版本和度量感知拉普拉斯版本,后者更适配高码本比特宽度场景。研究表明,SSTQ 实现了最优均方误差缩放,每个客户端仅使用 ⌈log₂N⌉ + b 比特,其中 N = Θ(d) 为框架大小;还推导了感知隐私的替代码本目标,将码本相关的均方误差缩放从 O(4ᵇ) 降至 O(2ᵇ)。最后,在使用 CIFAR-10 和 Fashion-MNIST 的联邦学习任务中,对 SSTQ 与现有基线进行实证评估,证明其具备良好的效用和通信效率。

英文摘要

Achieving local differential privacy in distributed optimization while maintaining low communication cost remains challenging. Existing vector quantization methods, such as vqSGD, use high-dimensional geometric constructions but incur unfavorable dimension-dependent variance. In this work, we propose Subsampled Stochastic TurboQuant (SSTQ), a framework that combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. SSTQ includes two variants: a Flat Randomized Response version and a Metric-Aware Laplace version, the latter being better suited to higher codebook bit-width regimes. We show that SSTQ achieves optimal mean squared error scaling while using only $\lceil \log_2 N \rceil + b$ bits per client, where $N = Θ(d)$ is the frame size. We also derive a surrogate privacy-aware codebook objective that reduces the codebook-dependent MSE scaling from $O(4^b)$ to $O(2^b)$. Finally, we empirically evaluate SSTQ against established baselines on federated learning tasks using CIFAR-10 and Fashion-MNIST, demonstrating favorable utility and communication efficiency.

URL PDF HTML 收藏
2608.05126 2026-08-06 cs.CL cs.MM 新提交

Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models

语音函数调用:面向大型音频语言模型的语音理解新视角

Yuezhang Peng, Yuxin Liu, Changfeng Gao, Zhifu Gao, Xiangang Li, Xie Chen

机构 * Shanghai Jiao Tong University(上海交通大学) Token Foundry, Alibaba Group(阿里巴巴集团Token Foundry) Shanghai Innovation Institute(上海创新研究院)

AI总结 该研究提出语音函数调用(SFC)这一新型语义理解视角,构建SFC-Bench数据集并评估LLMs与LALMs性能,经后训练提升LALMs的SFC能力,实验显示SFC优于传统SLU,可大幅提升语义提取准确率。

Comments ACM Multimedia 2026

详情
AI中文摘要

语音理解(SLU)是面向任务的对话系统的核心组件,也是实现人机无缝交互的关键环节。传统SLU在域内监督微调后可有效提取闭集任务的用户语义,但因规则定义模糊,在开放域任务中利用上下文学习面临重大挑战。本研究提出语音函数调用(SFC)这一新型语义理解视角,通过结构化规则定义优化语义理解,以突破传统闭集SLU的局限。具体而言,我们基于传统SLU数据集整理并扩展了一套语音函数,构建多智能体系统合成SFC-Bench数据集,评估大型语言模型(LLMs)与大型音频语言模型(LALMs)的性能,并通过后训练提升LALMs的SFC能力。实验表明,SFC的表现优于传统SLU,大幅提升了LLMs与LALMs的语义提取准确率。

英文摘要

Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU can effectively extract user semantics for closed-set tasks after in-domain supervised fine-tuning, it faces significant challenges in leveraging in-context learning for open-domain tasks due to its ambiguous rule definitions. This work proposes Spoken Function Calling (SFC), a novel semantic understanding perspective that optimizes semantic understanding with structured rule definitions, to evolve beyond traditional closed-set SLU. Specifically, we curate and extend a suite of spoken functions based on traditional SLU datasets, construct a multi-agent system to synthesize the SFC-Bench dataset, evaluate the performance of Large Language Models (LLMs) and Large Audio Language Models (LALMs), and enhance the SFC capabilities of LALMs through post-training. Experiments demonstrate that SFC outperforms traditional SLU, substantially enhancing the semantic extraction accuracy for LLMs and LALMs.

URL PDF HTML 收藏
2608.05122 2026-08-06 cs.CV 新提交

IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers

IRIS:一种受视觉皮层启发的用于分析视觉Transformer中方向选择性的框架

Vaishnavi B Mohan, Vijayakrishna Naganoor, Yashas Annadani, Shashank Hegde

机构 * University of Washington(华盛顿大学) Microsoft(微软公司) TU Munich(慕尼黑工业大学) Gladstone Institute(格拉德斯通研究所) Nvidia(英伟达公司)

AI总结 该研究提出受视觉皮层启发的IRIS框架,通过RSS等神经科学指标分析ViT中方向选择性的形成,发现训练范式决定方向选择性、层的方向选择性变化规律,且指标可指导解冻层数以优化下游泛化。

详情
AI中文摘要

视觉Transformer(ViTs)已成为许多感知任务中图像编码的事实上的标准。尽管它们在经验上取得了成功,但由于缺乏归纳偏置(ViTs全局处理信息而非依赖局部结构),其如何编码低级特征的机制仍不清楚。相比之下,生物视觉系统通过结合视野中小的局部区域的信息来构建低级特征,例如初级视觉皮层中的方向选择性,这些特征是通用表示,在多个专门的神经通路中共享且被需要,不同于更高层次的、特定于任务的语义特征。这就提出了一个问题:这种基于生物学的特征是否会在ViTs中出现。在这项工作中,我们通过引入一套受神经科学启发的指标:表示相似性得分(RSS)、方向招募得分(ORS)和方向调谐带宽,来系统研究ViTs中方向选择性如何出现,以量化方向如何在表示几何中以及作为模型深度的函数进行编码。通过广泛分析,我们发现:(1)训练范式是方向选择性的最强决定因素,具有相同目标的模型无论规模如何,都会在相当的相对深度达到峰值;(2)许多单元在训练早期就具有方向选择性,中低层会随时间招募更多此类单元,而更深层会失去选择性并拓宽调谐以转向语义编码;(3)我们的指标为应解冻多少层以实现最佳下游泛化提供了一种机制启发式方法。我们的框架提供了一种在ViT训练过程中跟踪基于生物学的特征的方法,探究了所需属性如何在Transformer表示中编码,并建立了对ViTs如何跨任务泛化的系统理解。

英文摘要

Vision transformers (ViTs) have become the de facto standard for image encoding across many perception tasks. Despite their empirical success, it remains mechanistically unclear how they encode low-level features, given their lack of inductive biases: ViTs process information globally rather than relying on local structure. Biological visual systems, in contrast, build low-level features, such as orientation selectivity in the primary visual cortex, by combining information from small, localized regions of the visual field. These features are general-purpose representations, shared and required across multiple specialized neural pathways, unlike higher-level, task-specific semantic features. This raises the question if such biologically-grounded features arise in ViTs. In this work, we systematically study how orientation selectivity emerges in ViTs by introducing a suite of neuroscience-inspired metrics: representational similarity score (RSS), orientation recruitment score (ORS), and orientation tuning bandwidth to quantify how orientation is encoded in representational geometry and as a function of model depth. Through extensive analysis, we find that: (1) the training paradigm is the strongest determinant of orientation selectivity, with models sharing an objective, peaking at comparable relative depths regardless of scale (2) many units are orientation-selective early in training, with early-to-middle layers recruiting more such units over time, while deeper layers lose selectivity and broaden their tuning toward semantic encoding and (3) our metrics offer a mechanistic heuristic for how many layers to unfreeze for best downstream generalization. Our framework presents a way to track biologically-grounded features during ViT training, probes how desired properties are encoded in transformer representations, and builds a systematic understanding of how ViTs generalize across tasks.

URL PDF HTML 收藏
2608.05115 2026-08-06 cs.CV cs.AI cs.ET cs.HC cs.LG 新提交

Robust and Efficient Motion Reasoning for Privacy-Aware Classroom Incident Recognition

面向隐私感知的课堂事件识别的鲁棒且高效的运动推理

Paritosh Parmar, Landy Lan, Hong Yang, Chen Yi, Chiat Pin Tay

机构 * Institute of High Performance Computing, Agency for Science, Technology and Research(高性能计算研究所,新加坡科技研究局)

AI总结 本研究针对隐私感知课堂事件识别场景,提出轻量鲁棒的运动推理框架,通过知识蒸馏实现高效推理,性能优于大模型且泛化能力强,将公开相关基准与工具。

详情
AI中文摘要

计算机视觉能否助力提升课堂安全性?在这项试点研究中,我们从类监控摄像头(CCTV-style)观测视角研究隐私感知且计算高效的课堂事件识别任务。该场景探索不足,针对实际部署所需的隐私性、效率和泛化需求的基准及方法均有限。我们引入一种新型混合基准,将生成的类监控摄像头视频与真实课堂姿态数据相结合,并提出一种轻量且鲁棒的运动推理框架,其动机源于观察发现:许多事件的差异更多体现在运动方向、速度、加速度和强度上,而非仅体现在姿态上。为此,我们的方法首先构建人类动作的分层运动学表示,随后将大型教师模型的分层多阶运动学推理知识蒸馏至一个小得多的单阶学生模型,从而实现高效的单人推理,同时保留对运动的表达性理解。实验表明,我们的模型计算成本不到更大基线模型的十分之一,却在性能上大幅优于这些基线,同时展现出更强的域外运动推理能力和零样本合成到真实的泛化能力。我们将公开发布该基准、代码库及配套工具,以推动隐私感知课堂安全领域的进一步研究。

英文摘要

Can computer vision help make classrooms safer? In this pilot study, we investigate privacy-aware and computationally efficient classroom incident recognition from CCTV-style observations. This setting remains underexplored, with limited benchmarks and few methods designed for the privacy, efficiency, and generalization demands of real-world deployment. We introduce a novel hybrid benchmark combining generative CCTV-style videos with real-world classroom pose data, and propose a lightweight, but robust motion-reasoning framework motivated by the observation that many incidents differ more in motion direction, speed, acceleration, and intensity than in pose alone. To that end, our method first constructs hierarchical kinematic representations of human actions. Our method then distills hierarchical, multi-order kinematic reasoning from a large teacher into a much smaller single-order student, enabling efficient per-person inference while preserving expressive motion understanding. Experiments show that our model outperforms substantially larger baselines at less than one-tenth of their computational cost, while also demonstrating stronger out-of-domain motion reasoning and zero-shot synthetic-to-real generalization. We will publicly release the benchmark, codebase, and supporting tools to facilitate further research in privacy-aware classroom safety.

URL PDF HTML 收藏
2608.05107 2026-08-06 cs.AI cs.MA cs.SE 新提交

CoPlan: A Trustworthy Co-Intelligence Interface for Care Planning through Role-Based Contestable Argument Graphs

CoPlan:一种基于角色可争议论证图的可信协同智能照护规划界面

Hung Truong Thanh Nguyen, Hélène Fournier, Piper Jackson, Makoto Itoh, Shannon Freeman, Rene Richard, Hung Cao

机构 * University of New Brunswick(新不伦瑞克大学) National Research Council Canada(加拿大国家研究委员会) Thompson Rivers University(汤普森河大学) ISB Corporation(ISB公司) University of Northern British Columbia(北英属哥伦比亚大学)

AI总结 本研究提出CoPlan界面,通过多智能体工作流结合协同智能与可争议性,实现人机协同照护规划,保留人类自主性与临床问责制,已在就地养老场景中验证其有效性。

Comments Accepted at the 2026 International Conference on Next Generation AI Systems (NGEN-AI 2026)

详情
AI中文摘要

AI支持的照护规划可帮助临床医生、患者、照护人员及照护团队协调临床、功能、社会心理及环境需求等复杂决策。然而,许多AI系统将建议呈现为固定输出,限制了利益相关方在建议与临床判断、患者价值观或现实可行性冲突时检查、质疑和修订计划的能力。我们提出CoPlan——一种用于人机协同照护规划的协同智能可争议界面。CoPlan采用多智能体工作流,其中专门的AI智能体生成候选干预措施及支持或质疑的论证,人类照护规划人员可在最终计划生成前接受、拒绝、修改或添加论证。通过该设计,CoPlan结合了协同智能(人类与AI智能体贡献互补专业知识)与可争议性(建议始终可供检查、修订和论证)。我们在就地养老照护规划场景中演示了CoPlan,该系统支持适应性照护团队招募、基于角色的论证评审、最终照护计划生成及通过调度智能体实现的实际后续安排。本研究贡献了一种可争议照护规划界面及一种用于可信人机照护规划的设计框架,该框架保留了人类自主性与临床问责制。

英文摘要

AI-supported care planning can help clinicians, patients, caregivers, and care teams coordinate complex decisions across clinical, functional, psychosocial, and environmental needs. However, many AI systems present recommendations as fixed outputs, limiting stakeholders' ability to inspect, challenge, and revise plans when they conflict with clinical judgment, patient values, or real-world feasibility. We present CoPlan - a Co-Intelligent and Contestable Interface for Human-AI Care Planning. CoPlan uses a multi-agent workflow in which specialized AI agents generate candidate interventions and supporting or challenging arguments, while human care planners can accept, reject, modify, or add arguments before final plan generation. Through this design, CoPlan combines co-intelligence, in which humans and AI agents contribute complementary expertise, with contestability, where recommendations remain open to inspection, revision, and justification. We demonstrate CoPlan in an aging-in-place care planning scenario. The system supports adaptive care team recruitment, role-based argument review, final care plan generation, and practical follow-up through scheduling agents. This work contributes a contestable care planning interface and a design framing for trustworthy human-AI care planning that preserves human agency and clinical accountability.

URL PDF HTML 收藏
2608.05104 2026-08-06 cs.LG 新提交

BnBERT-iPET: Sparse Few-Shot Language Modeling for Bengali via Lottery Ticket Pruning

BnBERT-iPET:基于彩票剪枝的孟加拉语稀疏少样本语言建模

Sajib Hossain, Md Kamrus Samad, Anan Ghosh, Labib Imam Chowdhury, Nabeel Mohammed

机构 * North South University(北南大学)

AI总结 本研究提出BnBERT-iPET,一种基于彩票剪枝的孟加拉语稀疏少样本语言建模方法,仅保留初始模型10%边、实现90%稀疏度,在孟加拉语下游任务中性能可与多个最先进语言模型媲美。

Comments 14 pages, 9 tables, 13 figures. Preprint

详情
AI中文摘要

深度神经网络凭借其复杂结构和海量边,在自然语言处理(NLP)任务中取得了令人瞩目的成功。使用BERT等大型预训练模型实现NLP的最先进性能,成本高昂、耗时且碳足迹大,难以在计算能力有限的设备上实现,这为孟加拉语等资源受限语言训练复杂模型造成了障碍。然而,在复杂神经网络中,并非所有边的影响都相同,部分边的贡献可以忽略。剪枝有望在不牺牲可比性能的前提下,减小常规网络的内存占用、缩短不断增长的网络的训练时间并提升推理效率。本研究提出了一种针对孟加拉语的稀疏少样本语言建模方法BnBERT-iPET,实验表明,仅保留BERT等初始模型10%边的轻量少样本学习语言模型,在孟加拉语等资源受限语言的挑战性任务上,性能可与大得多的模型不相上下。该模型通过迭代模式利用训练从少量样本中学习,并借助彩票假说(Lottery Ticket Hypothesis)剪枝技术实现90%的稀疏度,在孟加拉语标准基准数据集的下游任务中,可与Bangla Electra、Indic-BERT和XLM-RoBERTa等最先进语言模型一较高下。

英文摘要

Deep neural networks have shown impressive success in NLP tasks owing to their complex structure and huge number of edges. Achieving state-of-the-art performance in natural language processing with a large pre-trained model such as BERT is expensive and time-consuming, carries a large carbon footprint, and is difficult to realize on machines with minimal computational capability. This creates a barrier to training complex models for resource-constrained languages such as Bengali. However, in a complex neural model, not all edges are equally impactful, and the contributions of some of them can be neglected. Pruning promises to reduce the memory footprint of regular networks, shorten the training time of ever-growing networks, and increase inference efficiency without sacrificing comparable performance. In this work, we introduce BnBERT-iPET, a sparse few-shot language modeling approach for Bengali, and experimentally show that a lightweight few-shot-learned language model retaining only 10% of the edges of an initial model such as BERT can perform neck and neck with much larger models on challenging tasks for a resource-constrained language such as Bengali. By learning from few shots through iterative pattern exploiting training and achieving 90% sparsity with the Lottery Ticket Hypothesis pruning technique, our pruned BnBERT-iPET model proves to be a tough competitor to state-of-the-art language models such as Bangla Electra, Indic-BERT, and XLM-RoBERTa on downstream tasks over standard benchmark datasets of the Bengali language.

URL PDF HTML 收藏
2608.05102 2026-08-06 cs.AI 新提交

ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignment

ABSeeker:通过答案回溯信用分配训练长程搜索智能体

Yijun Lu, Rui Ye, Jiajun Wang, Yuwen Du, Tian Jin, Songhua Liu, Siheng Chen

机构 * Shanghai Jiao Tong University(上海交通大学)

AI总结 本文提出答案回溯信用分配框架,训练ABSeeker智能体,在BrowseComp等数据集上性能优于同规模模型,接近30B级大模型。

详情
AI中文摘要

长程搜索智能体必须执行多个连续动作(步骤)来搜索、检索、验证和整合证据,以得出最终答案。然而,现有的训练这些智能体的方法在监督微调(SFT)和强化学习(RL)过程中,通常对轨迹内的所有步骤一视同仁,无法区分有用动作与错误或冗余动作。本文提出了答案回溯信用分配(ABC),这是一种用于训练长程搜索智能体的细粒度信用分配框架,通过将稀疏的轨迹级结果转换为密集的步骤级监督,奖励有用动作(即使在失败的轨迹中),同时抑制错误或冗余动作。具体而言,给定一个可能模糊的查询及其对应的真实答案,ABC首先执行答案回溯线索恢复,即从答案回溯以恢复解决问题所需的中间线索;随后应用线索锚定步骤评分,根据这些线索评估每个搜索步骤,将稀疏的二元结果监督转换为密集的步骤级奖励。基于这些奖励,我们开发了ABC-SFT(对每一轮的损失进行加权)和ABC-GRPO(将步骤级评分作为GRPO中的奖励)。基于该框架,我们仅用8500个示例在Qwen3.5-4B的基础上训练了ABSeeker。ABSeeker在BrowseComp上的准确率达到37.3%,在BrowseComp-ZH上达到39.1%;加入上下文管理后,分数进一步提升至55.3%和52.9%,显著优于同规模(4B)智能体,甚至可与更大规模(约30B)智能体的性能相媲美。这些结果证明了答案回溯步骤级信用分配在训练长程搜索智能体方面的有效性。

英文摘要

Long-horizon search agents must make multiple sequential actions (steps) to search, retrieve, verify, and integrate evidence to reach a final answer. However, existing methods for training these agents typically treat all steps within a trajectory uniformly during both supervised fine-tuning (SFT) and reinforcement learning (RL), failing to distinguish useful actions from erroneous or redundant ones. In this paper, we propose Answer-Backtracked Credit Assignment (ABC), a fine-grained credit assignment framework for training long-horizon search agents by converting sparse trajectory-level outcomes into dense step-level supervision that rewards useful actions (even in failed trajectories) while suppressing erroneous or redundant actions. Specifically, given a potentially obscure query and its corresponding ground-truth answer, ABC first performs Answer-Backtracked Clue Recovery, which traces back from the answer to recover intermediate clues required to solve the question. It then applies Clue-Anchored Step Scoring to evaluate each search step against these clues, converting sparse binary outcome supervision into dense step-level rewards. Based on these rewards, we develop ABC-SFT, which reweights the loss of each turn, and ABC-GRPO, which uses the step-level scores as rewards in GRPO. Building on this framework, we train ABSeeker based on Qwen3.5-4B with only 8.5k examples. ABSeeker achieves 37.3% on BrowseComp and 39.1% on BrowseComp-ZH. With context management, the scores further improve to 55.3% and 52.9%, respectively, significantly outperforming same-scale (4B) agents and even matching the performance of larger ones (approximately 30B). These results demonstrate the effectiveness of answer-backtracked step-level credit assignment for training long-horizon search agents.

URL PDF HTML 收藏
2608.05101 2026-08-06 cs.CV cs.MM 新提交

HexMIL: Hierarchical Attention MIL for Ante-Hoc Explainable Detection of AI-Manipulated CT Volumes

HexMIL:用于AI篡改CT体积事前可解释检测的分层注意力多实例学习

Orazio Pontorno, Luca Guarnera, Zahid Akhtar, Sebastiano Battiato

机构 * University of Catania(卡塔尼亚大学) State University of New York Polytechnic Institute(纽约州立大学理工学院)

AI总结 HexMIL是一种无掩码的医疗深度伪造检测器,采用分层注意力多实例学习,利用二元体积级监督实现AI篡改CT体积的事前可解释检测,在跨生成器泛化任务中性能优于基线。

Comments Accepted at ACM Multimedia 2026 (MM '26)

Journal ref Proceedings of the 34th ACM International Conference on Multimedia (MM '26), November 10--14, 2026, Rio de Janeiro, Brazil

详情
AI中文摘要

医疗深度伪造(即由深度生成模型篡改的医疗图像)的出现对临床工作流程构成重大威胁。然而,现有检测器存在两个关键局限:对未见生成架构的泛化能力差,且缺乏可解释性。在此背景下,我们提出HexMIL(分层可解释多实例学习),一种无掩码的医疗深度伪造检测器,仅利用二元体积级监督即可同时解决上述两个局限。HexMIL将每个CT体积分解为块和切片的两级层次结构,通过独立的门控注意力模块聚合,其权重直接组合为全分辨率3D注意力体积,无需任何像素级标注即可定位被篡改的子区域。与Grad-CAM等事后方法不同,HexMIL的注意力权重构成了驱动分类决策的精确前向计算,提供事前且结构忠实的空间归因。我们在M3DSynth和CT-GAN数据集上,采用严格的跨生成器泛化协议评估HexMIL:在单一生成架构上训练,在未见架构上测试。HexMIL在域外分类中比所有基线高出9.1的AUC和9.4的F1,且在定位任务中取得最佳平均IoU和指向游戏(Pointing Game)分数。项目页面:this http URL。

英文摘要

The emergence of medical deepfakes, i.e., medical images manipulated by deep generative models, poses a significant threat to clinical workflows. However, existing detectors suffer from two critical limitations: poor generalization to unseen generative architectures for manipulation detection and lack of interpretability. In this context, we present HexMIL (Hierarchical EXplainable Multiple Instance Learning), a mask-free medical deepfake detector that simultaneously addresses both limitations using only binary volume-level supervision. HexMIL decomposes each CT volume into a two-level hierarchy of patches and slices, aggregated via independent Gated Attention modules whose weights are directly combined into a full-resolution 3D attention volume that localizes the manipulated sub-region without any pixel-level annotation. Unlike post-hoc methods such as Grad-CAM, HexMIL's attention weights constitute the exact forward computation driving the classification decision, providing ante-hoc and structurally faithful spatial attribution. We evaluate HexMIL on M3DSynth and CT-GAN datasets under a rigorous cross-generator generalization protocol, training on a single generative architecture and testing on unseen ones. HexMIL outperforms all baselines by $+9.1$ AUC and $+9.4$ F1 in out-of-domain classification, and achieves the best average IoU and Pointing Game score in localization. Project page: opontorno.github.io/hexmil.

URL PDF HTML 收藏
2608.05100 2026-08-06 cs.CV 新提交

Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature

基于冻结自蒸馏特征的CT病灶检测:SALT——空间自适应标签引导温度

Mahmut S. Gokmen, Evan W. Damron, Mitchell A. Klusty, Caroline N. Leach, Emily B. Collier, V. K. Cody Bumgardner

机构 * University of Kentucky(肯塔基大学) University of Louisville(路易斯维尔大学)

AI总结 该研究提出SALT方法,通过空间自适应标签引导温度优化自监督预训练,冻结编码器后训练轻量级CenterNet头部,在四个CT队列中实现病灶检测,效果优于无目标条件化的对照模型。

详情
AI中文摘要

自监督预训练目标具有空间均匀性:图像中教师温度和每个补丁的损失权重处处相同,因此宽度仅为几个补丁的病灶对训练信号的贡献与周围实质组织无差异。现有研究倾向于将视图偏向标注区域,这改变了模型所见内容,但未给目标增加压力。我们转而对自蒸馏的目标进行条件化,该方法名为SALT(空间自适应标签引导温度)。仅在预训练期间可用的弱框衍生标注在编码器的补丁网格上定义了一个紧凑区域,在该区域内,教师的softmax温度被锐化,掩码补丁损失被上调。其他方面的目标、掩码策略和居中统计保持不变,在所有下游使用中,编码器是普通的特征提取器,无标签且无条件化。我们通过冻结编码器并仅训练轻量级多深度CenterNet风格的头部来进行评估,在四个CT队列中进行3D病灶检测,我们将该机制与架构、预训练数据、时间表和标签引导裁剪完全相同但无目标条件化的主干进行隔离验证。我们报告了补丁级可分性、按队列和病灶大小分层的3D检测、框质量,以及无检测器探针——单个冻结补丁嵌入无需配准、掩码或微调即可在随访扫描中重新识别病灶。由于条件化通过空间指标而非标签语义表达,该公式允许任何弱空间标注;我们对病灶进行了实例化和验证。

英文摘要

Self-supervised pretraining objectives are spatially uniform: the teacher temperature and the per-patch loss weight are identical everywhere in the image, so a lesion a few patches wide contributes no more to the training signal than the surrounding parenchyma. Prior work biases the views toward annotated regions, which changes what the model sees but adds no pressure on the objective. We instead condition the targets of self-distillation, a method we call SALT (Spatially Adaptive Label-guided Temperature). Weak, box-derived labels, available only during pretraining, define a compact region on the encoder's patch grid, inside which the teacher's softmax temperature is sharpened and the masked-patch loss is up-weighted. The objectives, the masking policy and the centering statistics are otherwise unchanged, and at every downstream use the encoder is a plain feature extractor with no labels and no conditioning. We evaluate by freezing the encoder and training only a lightweight multi-depth CenterNet-style head, detecting lesions in 3D on four CT cohorts, and we isolate the mechanism against a backbone identical in architecture, pretraining data, schedule and label-guided cropping but with no target conditioning. We report patch-level separability, 3D detection stratified by cohort and by lesion size, box quality, and a detector-free probe in which a single frozen patch embedding re-identifies a lesion in a follow-up scan without registration, masks or fine-tuning. Because the conditioning is expressed through a spatial indicator rather than through label semantics, the formulation admits any weak spatial annotation; we instantiate and validate it for lesions.

URL PDF HTML 收藏
2608.05097 2026-08-06 cs.CL 新提交

Same Formulas, Different Semantics: Do Language Models Follow Modal Logic Specifications?

相同公式,不同语义:语言模型是否遵循模态逻辑规范?

Réemi Andrieu, Damien Sileo

机构 * Univ. Lille(里尔大学) Inria(法国国家信息与自动化研究所) CNRS(法国国家科学研究中心) Centrale Lille(里尔中央理工学院) UMR 9189 - CRIStAL(UMR 9189 - CRIStAL(法国国家科研中心联合研究机构))

AI总结 该研究探究语言模型是否遵循模态逻辑规范,构建成对模态问题测试五款模型,发现遵循规定模态语义依赖推理模式与模型身份,发布相关产物。

Comments 9 pages. Code: https://github.com/sileod/modal-semantics-reasoning. Data and artifacts: https://huggingface.co/datasets/sileod/modal-semantics-reasoning

详情
AI中文摘要

关于必然性与可能性的推理依赖于对可能世界之间的可达性以及每个世界中存在的对象的假设,因此同一推理可能在一种模态系统中成立,而在另一种模态系统中不成立。在这类问题上评估语言模型,需要测试其判断是否遵循指定的语义而非熟悉的逻辑。我们构建了成对的模态问题,具有相同的前提和猜想,但框架或域条件不同;自动推理验证了相反的标签。平衡核心确保仅语义条件无法单独揭示答案。在该核心上,五款近期模型中有四款在直接提示下的表现低于仅条件基线。然而,启用推理模式后,DeepSeek V4 Flash在未更改的提示下从4.4%提升至88.1%。因此,遵循规定的模态语义强烈依赖于推理模式和模型身份。当省略框架条件时,模型通常达成一致,但最适配不同的熟悉逻辑。我们发布了相关公式、预言机产物、反模型及响应结果。

英文摘要

Reasoning about necessity and possibility depends on assumptions about accessibility between worlds and about which objects exist at each one. The same inference may therefore hold under one modal system and fail under another. Evaluating language models on such problems requires testing whether their judgments follow the stated semantics rather than a familiar logic. We construct paired modal problems with identical premises and conjecture but different frame or domain conditions; automated reasoning verifies opposite labels. A balanced core prevents the semantic condition alone from revealing the answer. On this core, four of five recent models perform below the condition-only baseline under direct prompting. Yet enabling reasoning mode raises DeepSeek V4 Flash from 4.4% to 88.1% on unchanged prompts. Following stipulated modal semantics thus depends strongly on inference mode as well as model identity. When frame conditions are omitted, models often agree but fit different familiar logics best. We release the formulas, oracle artifacts, countermodels, and responses.

URL PDF HTML 收藏
2608.05088 2026-08-06 cs.LG 新提交

MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioning

MALT:通过对角预条件化实现的轻量曲率感知Muon

Tongle Wu, Huanyu Dong, Ying Sun, Ziye Ma

机构 * School of Electrical Engineering and Computer Science, The Pennsylvania State University(宾夕法尼亚州立大学电气工程与计算机科学学院) Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系)

AI总结 针对Muon未考虑损失景观曲率几何的问题,提出轻量曲率感知优化方法MALT及MALTER,在GPT-2系列预训练上性能优于Muon且开销相近。

详情
AI中文摘要

Muon最近通过使用Newton-Schulz迭代对动量矩阵进行正交化,成为了AdamW在语言模型预训练中颇具前景的替代方案。尽管Muon缓解了梯度各向异性,但它并未明确考虑损失景观的曲率几何,因此可能仍对曲率各向异性敏感。为填补这一空白,我们提出MALT(Muon Augmented by Lightweight Two-sided Preconditioning,即通过轻量双边预条件化增强的Muon),它使用轻量对角预条件器降低Muon对曲率各向异性的敏感性。具体而言,MALT使用内存和计算开销低的双边对角预条件器,近似捕捉损失景观的曲率几何;它通过Newton-Schulz迭代对预条件化后的动量进行正交化,将结果映射回以定义更新方向,同时通过范数嫁接控制更新幅度。为提升MALT对随机梯度噪声的鲁棒性,我们进一步提出MALTER(MALT with Adaptive stEpsize Rescaling,即带自适应步长缩放的MALT)。我们为MALT在随机非凸场景下提供了收敛保证。在GPT-2 Small、Medium和Large预训练上的实验表明,所提方法在保持几乎相同内存占用和 wall-clock 时间的同时,性能优于Muon。

英文摘要

Muon has recently emerged as a promising alternative to AdamW for language model pretraining by orthogonalizing momentum matrices using Newton-Schulz iterations. Although Muon mitigates gradient anisotropy, it does not explicitly account for the curvature geometry of the loss landscape and may therefore remain sensitive to curvature anisotropy. We bridge this gap by proposing MALT (Muon Augmented by Lightweight Two-sided Preconditioning), which uses lightweight diagonal preconditioners to reduce the sensitivity of Muon to curvature anisotropy. Specifically, MALT uses two-sided diagonal preconditioners with low memory and computational overhead to approximately capture the curvature geometry of the loss landscape. It orthogonalizes the preconditioned momentum using Newton-Schulz iterations and maps the result back to define the update direction, while norm grafting controls the update magnitude. To improve the robustness of MALT to stochastic gradient noise, we further propose MALTER (MALT with Adaptive stEpsize Rescaling). Convergence guarantees are provided for MALT in the stochastic non-convex setting. Experiments on GPT-2 Small, Medium, and Large pretraining show that the proposed methods outperform Muon while maintaining nearly the same memory footprint and wall-clock time.

URL PDF HTML 收藏
2608.05086 2026-08-06 cs.AI cs.CL 新提交

Item Response Theory for AI Safety

面向AI安全的项目反应理论

Joshua Fonseca Rivera, Neil Shah, David Demitri Africa, Konstantinos Voudouris

机构 * Independent(独立研究者)

AI总结 该研究将项目反应理论(IRT)应用于192个语言模型的8个安全基准,识别出三个关键因素,证明IRT可降低评估成本并审计模型,建议前沿实验室采用。

Comments 15 pages, 9 figures, 6 tables

详情
AI中文摘要

语言模型的安全行为表现存在差异,这类差异可通过安全基准进行衡量,但聚合基准分数难以信任和解释,因为基准之间存在重复、高度相关,且模型在检测到评估时可能会弃权(不执行)。为解决这些问题,我们采用项目反应理论(IRT),这是一种从具有推断心理测量属性的项目表现中测量潜在特质的统计工具包。我们将IRT模型应用于192个语言模型的8个安全基准,开展了迄今为止最大规模的LLM安全评估心理测量分析,并得出三项结果:第一,我们发现拒绝严格程度、真实性和情境危害这三个可解释因素,解释了模型在基准间的大部分差异;第二,经心理测量选择的项目,其恢复完整基准分数的误差低于相同规模的随机子集,且几个基准仅需约10个自适应选择的项目,可将评估成本降低97%-99%;第三,IRT支持对单个模型进行审计,可用于检测简单的弃权(不执行)行为和API背后的模型变更。总体而言,我们表明IRT是一种现成的工具包,可用于解读、简化和审计安全基准,建议前沿实验室和评估人员采用。

英文摘要

Language models differ in how safely they behave and these differences are measured by safety benchmarks. But aggregated benchmark scores are hard to trust and interpret, because benchmarks duplicate one another, correlate heavily, and models may sandbag when they detect evaluation. To address these issues, we draw on Item Response Theory (IRT), a statistical toolkit for measuring these latents from performance on items with inferred psychometric properties. We fit IRT models to eight safety benchmarks across 192 language models, the largest psychometric analysis of LLM safety evaluations to date, and contribute three results. First, we find that three interpretable factors of refusal strictness, truthfulness, and contextual harm explain most of the variance between models across benchmarks. Second, psychometrically selected items recover full benchmark scores with lower error than random subsets of the same size, and roughly ten adaptively chosen items suffice for several individual benchmarks, cutting evaluation cost by 97-99%. Third, IRT supports audits of individual models, showing that it can be used to detect naive sandbagging and changes of model behind APIs. Overall, we show IRT is a ready-made toolkit for reading, reducing, and auditing safety benchmarks, which we recommend frontier labs and evaluators adopt.

URL PDF HTML 收藏
2608.05085 2026-08-06 cs.LG cs.AI 新提交

Capability-Gated Planning: Cost-to-Goal Discovery and the Limits of Myopic Experiment Selection

能力门控规划:目标成本发现与近视实验选择的局限性

Ahmed Hassoon, Mark Dredze

机构 * Johns Hopkins University(约翰斯·霍普金斯大学)

AI总结 该研究指出近视实验选择方法无法评估构建认知能力的价值,引入能力门控规划器 CG-Plan 解决此问题,证明近视规划器存在无界近似比且无法到达目标。

详情
AI中文摘要

自动化科学发现的系统必须反复决定要运行哪个实验、测试哪个假设、构建哪个工具以及何时停止。许多系统通过最大化近视分数来做出这些决策,例如每单位成本的预期信息增益或学习到的合理性分数。我们确定了这种方法的一个结构性局限性:一些行动具有建设性,它们获取一种认知能力(如仪器、检测方法、流程、模拟器或抽象),其价值不在于立即返回的信息,而在于它为未来行动提供的可能性。当通往可靠答案的最低成本路径需要一系列此类构建时,仅通过有限时间范围内可获得的信息来对行动评分的规划器无法对第一次构建进行估值,因为它在该时间范围内不会产生任何信息,且会被任何具有正信息的测量所主导,无论其多么微小。我们将目标导向的发现表述为信念空间中的随机最短路径问题,其中建设性实验会改变下游行动图,并证明对于每个展望深度 d,都存在一个实例,使得每个近视信息最大化规划器都具有无界近似比,且存在一个相关实例,使得它永远无法到达目标。该机制是一个能力不可区分性引理:在该时间范围内,获取能力在观察上可能与执行空操作无法区分。这确立了能力门控是一种与曲率(子模块性)和信息顺序(适应性差距)不同的可达性难度轴。我们引入了 CG-Plan,这是一种增量重规划器,具有感知能力的目标成本启发式 h = h_cap + h_exp。在受控测试平台中,性能差距仅在门控情况下出现,对每个固定展望都持续存在,且当近 miss 假设来自数据一致的提议者时会产生。

英文摘要

Systems that automate scientific discovery must repeatedly decide which experiment to run, which hypothesis to test, which tool to build, and when to stop. Many systems make these decisions by maximizing a myopic score such as expected information gain per unit cost or a learned plausibility score. We identify a structural limitation of this approach. Some actions are constructive: they acquire an epistemic capability (an instrument, assay, pipeline, simulator, or abstraction) whose value lies not in the information returned immediately but in the future actions it makes available. When the least-cost route to a confident answer requires a chain of such constructions, a planner that scores actions only by information obtainable within a bounded horizon cannot value the first construction: it yields no information within the horizon and is dominated by any measurement with positive information, however small. We formulate goal-directed discovery as a stochastic shortest-path problem in belief space in which constructive experiments change the downstream action graph, and prove that for every lookahead depth d there is an instance on which every myopic information-maximizing planner has an unbounded approximation ratio, and a related instance on which it never reaches the goal. The mechanism is a capability-indistinguishability lemma: within the horizon, acquiring a capability can be observationally indistinguishable from paying for a null action. This establishes capability gating as a reachability axis of difficulty distinct from curvature (submodularity) and information order (adaptivity gaps). We introduce CG-Plan, an incremental replanner with a capability-aware cost-to-go heuristic h = h_cap + h_exp. In a controlled testbed, the performance gap appears only under gating, persists for every fixed horizon, and arises when near-miss hypotheses come from a data-consistent proposer.

URL PDF HTML 收藏
2608.05080 2026-08-06 cs.LG cs.CL 新提交

Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning

优化策略的学习内容:可恢复性感知的展开干预学习

Zheyuan Zhang, Manqing Mao, Hong Wang, Zhuoer Wang, Samson Koelle, Jie Yuan, Yanjun Lin, James Feng, Nikki Lijing Kuang, Yanfang Ye, Wei Niu

机构 * University of Notre Dame(圣母大学) Amazon, Inc(亚马逊公司)

AI总结 针对现有大语言模型后训练中展开分配的局限,本文提出RAIL框架,将干预选择建模为在线上下文博弈问题,可在有限展开预算下提升性能,为生成高质量展开提供原则性方法。

详情
AI中文摘要

无批评者的基于群体的强化学习已成为大语言模型后训练的可扩展方法,但现有多数方法为每个任务和轨迹状态分配相同数量的展开,而部分展开能提供更有用的学习信号。近期研究开始将展开生成视为自适应决策,但仍存在两个关键局限:其一,干预策略常基于固定启发式,无法随训练中策略变化调整;其二,这些方法通常仅决定生成多少展开,未明确控制干预的位置与方式。为解决这些局限,本文提出可恢复性感知干预学习(RAIL),这是一种训练时框架,基于每次干预产生的改进学习如何生成展开。RAIL将干预选择建模为在线上下文博弈问题,通过影子到真实程序收集的干预轨迹训练可恢复性控制器,使控制器能在底层策略演化时持续学习。本文从有效性、适应性、表达性和效率四个方面评估RAIL,在多种设置下,RAIL在有限展开预算下始终提升性能。这些结果表明,可恢复性感知干预提供了一种生成更具信息性、更少冗余的展开的原则性方法,在后训练期间产生更强的学习信号。

英文摘要

Critic-free group-based reinforcement learning has become a scalable approach for post-training large language models. However, most existing methods allocate the same number of rollouts to every task and trajectory state, even though some rollouts provide much more useful learning signals than others. Recent work has started to treat rollout generation as an adaptive decision, but two important limitations remain. First, intervention strategies are often based on fixed heuristics and therefore cannot adjust as the policy changes during training. Second, these methods usually decide only how many rollouts to generate, without explicitly controlling where and how to intervene. To address these limitations, we propose Recoverability-Aware Intervention Learning (RAIL), a training-time framework that learns how to generate rollouts based on the improvement produced by each intervention. RAIL models intervention selection as an online contextual-bandit problem and trains a recoverability controller using intervention traces collected through a shadow-to-live procedure. This allows the controller to keep learning while the underlying policy evolves. We evaluate RAIL in terms of effectiveness, adaptivity, expressiveness, and efficiency. Across multiple settings, RAIL consistently improves performance under limited rollout budgets. These results show that recoverability-aware intervention provides a principled way to generate more informative and less redundant rollouts, leading to stronger learning signals during post-training.

URL PDF HTML 收藏
2608.05078 2026-08-06 cs.RO 新提交

SpikingNav: Robust Embodied Navigation with Spiking Neural Policies

SpikingNav:基于脉冲神经网络策略的鲁棒具身导航

Jiahong Zhang, Sijun Shen, Dehua Wu, Yifan Lin, Xuechen Xia, Xu Chu, Youhui Zhang, GuoqiLi

机构 * Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所) School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院) State Key Laboratory of Media Convergence and Communication, Communication University of China(中国传媒大学媒体融合与传播国家重点实验室) Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系) Beijing National Research Center for Information Science and Technology, Tsinghua University(北京信息科学与技术国家研究中心)

AI总结 本文提出SpikingNav脉冲导航框架,含SSE与SPN,在PointNav、ObjectNav任务中,其参数更少、计算量更低,鲁棒性优于匹配的ANN基线,且可部署于Thruster-V2芯片。

详情
AI中文摘要

具身导航要求智能体在物理环境中基于自我中心观察做出序列决策。现有基于人工神经网络(ANN)的导航模型已取得优异性能,但通常依赖密集计算,且在视觉干扰下性能可能下降。脉冲神经网络(SNN)具备事件驱动计算与内在时间动态特性,有望在资源受限平台上实现紧凑且鲁棒的导航。然而,基于脉冲的感知与策略动态能否提升视觉丰富的具身导航中的鲁棒性仍是未解决的问题。本文提出SpikingNav,一种用于鲁棒室内具身导航的脉冲框架,包含脉冲感知编码器(SSE)与脉冲策略网络(SPN)。SSE以基于脉冲的骨干网络提取任务条件视觉特征;SPN通过膜整合、阈值处理与脉冲触发重置维持循环策略状态。SpikingNav利用SNN的动态特性与脉冲激活提升导航性能与鲁棒性。我们在PointNav与ObjectNav任务的干净观察及视觉干扰场景下评估SpikingNav,结果显示,与匹配的ANN基线相比,SpikingNav参数更少、每步计算量更低,且在干净观察下性能相当、鲁棒性更强。例如,SpikingNav将ObjectNav的成功率从31.05%提升至34.12%,并将视觉干扰下的平均成功率从8.45%提升至13.71%,证明了基于脉冲的感知与策略动态的优势。我们进一步在Thruster-V2神经形态芯片上验证了该脉冲感知方法的可部署性,该硬件验证表明SpikingNav可在真实神经形态载体上实现,适用于网络物理系统。

英文摘要

Embodied navigation requires an agent to make sequential decisions from egocentric observations in a physical environment. Existing Artificial Neural Network (ANN)-based navigation models have achieved strong performance, yet they often rely on dense computation and may degrade under visual corruptions. Spiking neural networks (SNNs) provide event-driven computation and intrinsic temporal dynamics, which are promising for compact and robust navigation on resource-constrained platforms. However, whether spike-based sensing and policy dynamics can improve robustness in visually rich embodied navigation remains an open problem. This paper proposes SpikingNav, a spiking framework for robust indoor embodied navigation. It contains a Spiking Sensing Encoder (SSE) and a Spiking Policy Network (SPN). The SSE extracts task-conditioned visual features with a spike-based backbone. The SPN maintains a recurrent policy state through membrane integration, thresholding, and spike-triggered reset. In this way, SpikingNav exploits the dynamic properties and spike activations of SNNs to improve navigation performance and robustness. We evaluate SpikingNav on PointNav and ObjectNav under clean observations and visual corruptions. SpikingNav achieves competitive clean performance and stronger robustness with fewer parameters and lower per-step computation than a matched ANN baseline. For instance, SpikingNav improves ObjectNav success from 31.05% to 34.12%, and raises the average success under visual corruptions from 8.45% to 13.71%, demonstrating the benefits of spike-based sensing and policy dynamics. We further validate the deployability of our spike-based sensing method on the Thruster-V2 neuromorphic chip. This physical hardware validation shows that SpikingNav can be instantiated on a real neuromorphic substrate for cyber-physical systems.

URL PDF HTML 收藏
2608.05075 2026-08-06 cs.CL 新提交

German parties shifted towards intuition-based rhetoric after the far right's parliamentary breakthrough

德国极右翼政党议会突破后,各政党转向基于直觉的修辞

Peer Saleth, Segun T. Aroyehun, Fabio Carrella, Christoph M. Abels, Stephan Lewandowsky, David Garcia

机构 * University of Konstanz(康斯坦茨大学) University of Bristol(布里斯托大学) University of Campinas(坎皮纳斯大学) University of Potsdam(波茨坦大学)

AI总结 本研究分析德国2015-2025年的推文与议会演讲,发现极右翼AfD进入议会后,德国各政党转向基于直觉的修辞,右翼精英的EMI得分最低,极右翼可见性与精英真理态度转变相关。

Comments 34 pages, 6 figures; includes 49 pages of Supplementary Information. Code available at https://github.com/peersal/German-EMI, data at https://osf.io/x3zpc/

详情
AI中文摘要

错误信息的传播被广泛视为对民主审议的威胁,但随着民粹主义者崛起,政治精英对真理的修辞承诺如何转变却鲜为人知。我们分析2015至2025年间德国政治精英的450万条推文及59170次议会演讲,采用经验证的分布式词典表示法衡量基于证据和基于直觉的修辞。在两个领域中,基于直觉的语言愈发突出,右翼倾向者始终呈现最低的证据减直觉(EMI)得分。2017年极右翼德国选择党(AfD)进入议会,与整个议会EMI急剧下降同步,而推特上则呈现更渐进的下降。这些发现记录了多党制欧洲民主国家中,极右翼可见性与精英话语中真理态度转变之间的关联。

英文摘要

The spread of misinformation is widely perceived as a threat to democratic deliberation, yet how political elites' rhetorical commitments to truth shift alongside the rise of populist actors remains poorly understood. Analysing 4.5 million tweets and 59,170 parliamentary speeches by German political elites between 2015 and 2025, we measure evidence-based and intuition-based rhetoric using a validated distributed dictionary representation. Across both arenas, intuition-based language has become more prominent, and right-leaning actors consistently exhibit the lowest Evidence Minus Intuition (EMI) scores. The parliamentary entry of the extreme-right Alternative for Germany (AfD) in 2017 coincides with sharp downward shifts in EMI across the broader chamber, while a more gradual decline is observed on Twitter. These findings document an association between far-right visibility and a changing approach to truth in elite discourse in a multiparty European democracy.

URL PDF HTML 收藏
2608.05074 2026-08-06 cs.CV 新提交

Bag-of-Visual-Words for Spatial Mapping of Lung Adenocarcinoma Growth Patterns

用于肺腺癌生长模式空间映射的视觉词袋模型

Darya Ardan, Valentin Oreiller, Henning Müller

机构 * University of Geneva(日内瓦大学) University of Applied Sciences Western Switzerland (HES-SO Valais)(瑞士西部应用科学大学(HES-SO Valais))

AI总结 该研究提出弱监督BoVW流程,从ROI学习视觉词汇并生成LUAD空间模式图,在肿瘤/健康和组织学分级分类任务中表现优于或接近现有方法,可保留与分级相关的异质性。

Comments 10 pages, 2 figures. Accepted at the 7th International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2026)

详情
AI中文摘要

对肺腺癌(LUAD)在全切片图像(WSI)上的生长模式进行空间映射,需要在区域层面解析结构上下文,但现有方法仅在单个图块层面操作,生成的是通用形态学聚类而非临床定义的模式图。本文提出一种弱监督视觉词袋(BoVW)流程,该流程从少量标注感兴趣区域(ROI)中提取的冻结基础模型嵌入学习视觉词汇;模式原型被构造为相同标签ROI的平均BoVW直方图,用于在Jensen-Shannon散度下对滑动窗口区域进行近邻原型分类;将所得预测结果投影到WSI图块网格上,生成可解释的空间模式图。在87名CPTAC-LUAD患者上,采用3种基础模型编码器并针对2个临床任务、使用多种词汇规模对该方法进行评估:在肿瘤/健康分类任务中,采用H-Optimus-1的最佳配置达到0.974的平衡准确率,接近基于平均池化WSI嵌入训练的监督SVM所取得的0.987;在组织学分级二分类任务中,BoVW流程在所有编码器上均取得比监督基线更高的平衡准确率,表明ROI层面的模式分解保留了与分级相关的异质性,而全局平均池化会削弱这种异质性。

英文摘要

Spatial mapping of lung adenocarcinoma (LUAD) growth patterns across whole slide images (WSIs) requires resolving architectural context at the region level, yet existing methods operate at the individual tile level and produce generic morphological clusters rather than clinically defined pattern maps. We propose a weakly supervised Bag-of-Visual-Words (BoVW) pipeline that learns a visual vocabulary from frozen foundation model embeddings extracted from a small set of annotated regions of interest (ROIs). Pattern prototypes are constructed as mean BoVW histograms of same-label ROIs and used for nearest-prototype classification of sliding-window regions under Jensen--Shannon divergence. The resulting predictions are projected onto the WSI tile grid to produce interpretable spatial pattern maps. We evaluate the method on 87 CPTAC-LUAD patients using three foundation model encoders and multiple vocabulary sizes on two clinically motivated tasks. For tumour/healthy classification, the best configuration achieves a balanced accuracy of $0.974$ with H-Optimus-1, approaching the $0.987$ obtained by a supervised SVM trained on mean-pooled WSI embeddings. For binary histologic grade classification, the BoVW pipeline achieves higher balanced accuracy than the supervised baseline for all encoders, suggesting that ROI-level pattern decomposition preserves grade-relevant heterogeneity that is attenuated by global mean pooling.

URL PDF HTML 收藏
2608.05070 2026-08-06 cs.CV 新提交

HelloWorld: Enabling Socially Interactive Characters in Video World Models

HelloWorld:在视频世界模型中实现具有社交互动性的角色

Liangyang Ouyang, Ruicong Liu, Xuangeng Chu, Kaipeng Zhang, Yoichi Sato

机构 * The University of Tokyo(东京大学) Alaya Lab(阿莱亚实验室)

AI总结 该研究提出HelloWorld视频世界模型,通过自蒸馏流水线和训练模块实现角色与用户的社交互动,构建含400样本的基准,其互动质量优于基线且保持顶尖图像美学。

Comments Project page: https://github.com/AlayaLab/HelloWorld

详情
AI中文摘要

尽管近期视频世界模型已取得显著进展,但这些世界中用户与角色之间的社交互动仍未得到支持。为填补这一空白,我们提出了HelloWorld,这是一种能够实现与世界内角色进行社交互动的视频世界模型。用户只需按下单个按钮,即可提示屏幕上的角色对镜头做出回应,例如转向观看者、挥手、点头或说出简短问候语。为使这些互动自然,我们提出了一种自蒸馏流水线,该流水线利用自身合成的数据对视频生成模型进行微调。每个合成片段均包含社交互动和相机运动,使模型能够学习相机姿态条件,同时不降低互动质量。在推理阶段,我们进一步引入了无需训练的模块,用于确定互动发生的时机。当按下按钮时,该模块会调制DiT的交叉注意力掩码,使与互动相关的文本提示仅关注按下窗口内的帧,从而在时间上定位角色的回应。我们还构建了HelloWorldBench,这是一个包含400个样本的基准,配备三种社交互动指标和三种常规指标,用于评估。实验表明,HelloWorld在互动质量方面优于多种基线方法,同时保持了最先进的图像美学和相机姿态跟随能力。项目页面:this https URL

英文摘要

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld

URL PDF HTML 收藏
2608.05066 2026-08-06 cs.CV 新提交

Beyond Reprojection Error: Camera Calibration with 3D Targets

超越重投影误差:基于3D靶标的相机标定

Dennis Ruppel, Hasan Kutlu, Kai A. Neumann, Martin Knuth, Pedro Santos, Andreas Weinmann, Arjan Kuijper

机构 * Fraunhofer Institute for Computer Graphics Research(弗劳恩霍夫计算机图形研究所) Technical University of Applied Sciences Würzburg-Schweinfurt(维尔茨堡-施韦因富特应用技术大学) Technical University Darmstadt(达姆施塔特工业大学)

AI总结 本研究针对三维重建提出基于场景射线预测的相机标定框架,采用重建与相交误差等指标,设计二十面体标定靶标,提升了标定精度,发现重投影误差可能误导三维精度评估。

Comments 16 pages, 7 figures, 2 tables. To appear in the proceedings of Computer Graphics International (CGI 2026)

详情
AI中文摘要

在三维重建中,相机标定是实现重建几何高保真度和高精度的关键环节。现有方法依赖二维平面标定,而本研究针对三维重建提出了一种基于场景射线预测的框架,为重建流程增加了灵活性,并支持采用最新的相机模型进展。研究人员采用从预测场景射线推导的新型指标——重建误差与相交误差,结合自举程序,对相机内参和外参的不同标定靶标及标定流程进行统计评估。结果表明,广义畸变模型能更忠实地反映物理相机效应,提升了标定精度;重投影误差被证实可能是三维精度的误导性指标,而所提基于射线的指标能提供更全面的评估。此外,研究人员设计了一种二十面体标定靶标,结合基于环形特征的检测器,为三维重建丰富标定信息。在合成数据的自举试验中,二十面体靶标的平均相交误差降低了约40%,且标定结果更稳定;不过其实际数据性能对制造公差要求极高。

英文摘要

In 3D reconstruction, camera calibration is an essential element for achieving high fidelity and accuracy of the reconstructed geometry. While existing approaches rely upon 2D planar calibration, this work proposes a framework tailored for 3D reconstruction that is based on predicting scene rays, which adds flexibility to the reconstruction pipeline and enables the use of recent advances in camera models. Novel metrics, reconstruction and intersection error, derived from predicted scene rays are employed in combination with a bootstrapping procedure that statistically evaluates different calibration objects and calibration pipelines for both intrinsic and extrinsic camera parameters. The results show that the generalized distortion model more faithfully captures physical camera effects and yields an improvement in calibration accuracy. Reprojection error is shown to be a potentially misleading indicator of 3D accuracy, and the proposed ray-based metrics provide a more holistic assessment. An icosahedron calibration target is designed to enrich calibration information for 3D reconstruction together with a ring-feature-based detector. The icosahedral target yields approximately 40% lower mean intersection and more stable calibration across bootstrap trials on synthetic data, while real-data performance demands very tight fabrication tolerances.

URL PDF HTML 收藏
2608.05064 2026-08-06 cs.CL cs.AI cs.LG 新提交

Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models

小型语言模型中口头表述不确定性的可证明极限与认证式 deferral( deferral 译为“ defer 至人类”的 deferral,即 deferral 指模型在特定情况下将决策 defer 至人类的机制)

Jianru Shen

机构 * University of Montana(蒙大拿大学)

AI总结 该研究针对小型语言模型,探究口头表述置信度支撑风险可控 deferral 的可行性,通过理论推导与实验验证校准的局限性及认证式 deferral 的效果,修复了 TruthfulQA 的答案排序伪影。

Comments Accepted at MIWAI 2026 (The 19th International Conference on Multi-disciplinary Trends in Artificial Intelligence), to appear in Springer LNAI

详情
AI中文摘要

小型开放权重语言模型越来越多地运行在私密、离线且对成本敏感的场景中,其部署的关键问题不仅是模型会给出什么答案,还包括何时应将决策 defer 至人类。我们研究口头表述的置信度是否能支持风险可控的 deferral,对来自三个系列的 11 个指令微调模型(参数规模为 0.5B 至 14B)在 ARC-Challenge 和 TruthfulQA 数据集上进行评估,共完成 25168 次本地预测。三项理论结果界定了校准所能提供的内容:严格单调校准可保留风险-覆盖前沿和错误检测 AUROC;温度缩放无法校准那些置信度始终高于 0.5 但准确率却低于 0.5 的模型;Clopper-Pearson 过程在独立同分布部署假设下,可将包含 200 个问题的校准集转换为有限样本风险证书。实验方面,22 个模型-任务对中有 8 个达到了温度缩放不可行性下限,与预测边界的偏差在 1 个百分点以内;Platt 缩放可将 ECE 降低至低至 0.02,但在 20%风险预算下仅 3 个模型-任务对获得认证式自主权限,在 10%风险预算下则无任何模型-任务对获得该权限。我们还识别并修复了 TruthfulQA 多项选择形式中的答案排序伪影。校准赋予置信度语义;认证式 deferral 决定了小型模型何时可安全使用。

英文摘要

Small open-weight language models increasingly run in private, offline, and cost-sensitive settings, where the key deployment question is not only what a model answers but when it should defer to a human. We study whether verbalized confidence can support risk-controlled deferral, evaluating eleven instruction-tuned models from three families, 0.5B to 14B parameters, on ARC-Challenge and TruthfulQA with 25,168 local predictions. Three theoretical results delimit what calibration can provide: strictly monotone calibration preserves the risk-coverage frontier and error-detection AUROC; temperature scaling cannot calibrate models whose confidence stays above one half while accuracy falls below it; and a Clopper-Pearson procedure converts a 200-question calibration set into a finite-sample risk certificate under an i.i.d. deployment assumption. Empirically, eight of 22 model-task pairs hit the temperature-scaling infeasibility floor within one percentage point of the predicted bound. Platt scaling reduces ECE to as low as 0.02, yet certified autonomy at a 20% risk budget is granted to only three model-task pairs and to none at 10%. We also identify and repair an answer-ordering artifact in the multiple-choice form of TruthfulQA. Calibration gives confidence semantics; certified deferral determines when small models are safe to use.

URL PDF HTML 收藏
2608.05049 2026-08-06 cs.CV 新提交

OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing

OmniEdit-Bench:基于指令的视频编辑综合基准

Chenxuan Miao, Yutong Feng, Yi Lu, Yunfeng Yan, Donglian Qi, Shiwei Zhang, Yu Liu, Xi Chen, Hengshuang Zhao

机构 * The University of Hong Kong(香港大学) Alibaba Group(阿里巴巴集团) Zhejiang University(浙江大学) Peking University(北京大学)

AI总结 针对现有基于指令的视频编辑基准的任务覆盖有限、指标不足的问题,OmniEdit-Bench将编辑任务分解为多维度并提出含惩罚机制的评估框架,实验显示当前IVE模型仍待提升。

详情
AI中文摘要

基于指令的视频编辑(IVE)是一个具有广泛应用的新兴领域,但评估编辑模型仍存在挑战。现有基准存在两个主要局限:一是继承自图像编辑的任务覆盖范围有限,忽略了视频特有的维度;二是指标不足,无法衡量指令保真度,导致因原始视频的强视觉先验,错误编辑也能获得高分。为解决这些问题,我们推出了一个综合且结构化的IVE基准。该基准将编辑任务分解为多个视频特有的维度,包括空间、时间、音频和基于参考的编辑,扩展了传统的帧级评估;还区分了显式和隐式指令,并融入基于推理的场景,以更好地反映现实需求。此外,我们提出了一个评估框架,从准确性、保留度、真实感和一致性四个互补维度评估编辑质量,结合人工评估和最先进的视觉-语言模型。为强调指令保真度,我们引入了一种感知准确性的惩罚机制,该机制将其他分数基于准确性进行调整,防止视觉上合理但错误的编辑获得过高评价。对代表性开源和商业模型的大量实验表明,当前IVE模型仍远未达到令人满意的水平。OmniEdit-Bench为评估基于指令的视频编辑提供了一个全面且可靠的测试平台,并为未来研究方向提供了见解。

英文摘要

Instruction-based video editing (IVE) is an emerging field with broad applications, yet evaluating editing models remains challenging. Existing benchmarks suffer from two major limitations: limited task coverage inherited from image editing, which overlooks video-specific dimensions, and inadequate metrics that fail to measure instruction fidelity, allowing incorrect edits to receive high scores due to strong visual priors from the original video. To address these issues, we introduce a comprehensive and structured benchmark for IVE. Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing, extending beyond conventional frame-level evaluation. It also distinguishes explicit and implicit instructions and incorporates reasoning-based scenarios to better reflect real-world requirements. Furthermore, we propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency, using both human judgments and state-of-the-art vision-language models. To emphasize instruction fidelity, we introduce an accuracy-aware penalty mechanism that conditions other scores on accuracy, preventing visually plausible but incorrect edits from receiving inflated evaluations. Extensive experiments on representative open-source and commercial models show that current IVE models remain far from satisfactory. OmniEdit-Bench provides a comprehensive and reliable testbed for evaluating instruction-based video editing and offers insights into future research directions.

URL PDF HTML 收藏