arXivDaily arXiv每日学术速递 周一至周五更新
重置
2604.27277 2026-06-12 cs.LG cs.AI cs.CV 版本更新

BrainDINO: A Brain MRI Foundation Model for Generalizable Clinical Representation Learning

BrainDINO:一种用于通用临床表征学习的脑MRI基础模型

Yizhou Wu, Shansong Wang, Yuheng Li, Mojtaba Safari, Mingzhe Hu, Chih-Wei Chang, Harini Veeraraghavan, Xiaofeng Yang

AI总结 提出BrainDINO,一种基于自蒸馏的基础模型,在约660万张未标记轴向切片上训练,通过冻结编码器加轻量任务头,在多种脑MRI任务上达到或超越基线,尤其在小样本场景下优势显著。

详情
Comments
25 pages, 5 figures
AI中文摘要

脑MRI支撑着广泛的神经科学和临床应用,然而大多数基于学习的方法仍针对特定任务且需要大量标注数据。本文表明,单一的自监督表征可以泛化到异质的脑MRI终点。我们训练了BrainDINO,一个自蒸馏的基础模型,使用了来自20个数据集的约660万张未标记轴向切片,这些数据集涵盖了人群、疾病和采集设置的广泛变异。通过使用冻结编码器加轻量任务头,BrainDINO支持肿瘤分割、神经退行性和神经发育性疾病分类、脑年龄估计、卒中后时间预测、分子状态预测、MRI序列分类和生存建模等任务的迁移。在各种任务和监督机制下,BrainDINO始终等于或超过自然图像和MRI特定自监督基线,在标签稀缺时尤其具有优势。表征分析进一步显示,在缺乏任务特定监督的情况下,特征结构具有解剖学组织和病理敏感性。我们的发现表明,大规模切片级自监督学习可以产生统一的脑MRI表征,支持多样化的神经影像任务,无需体积预训练或全网络微调,为稳健且数据高效的脑影像分析建立了可扩展的基础。代码可在 https://github.com/mclwu22/BrainDINO 获取。

英文摘要

Brain MRI underpins a wide range of neuroscientific and clinical applications, yet most learning-based methods remain task-specific and require substantial labeled data. Here we show that a single self-supervised representation can generalize across heterogeneous brain MRI endpoints. We trained BrainDINO, a self-distilled foundation model, on approximately 6.6 million unlabeled axial slices from 20 datasets encompassing broad variation in population, disease, and acquisition setting. Using a frozen encoder with lightweight task heads, BrainDINO supported transfer across tumor segmentation, neurodegenerative and neurodevelopmental conditions classification, brain age estimation, post-stroke temporal prediction, molecular status prediction, MRI sequence classification, and survival modeling. Across tasks and supervision regimes, BrainDINO consistently equaled or exceeded natural-image and MRI-specific self-supervised baselines, with particularly strong advantages under label scarcity. Representation analyses further showed anatomically organized and pathology-sensitive feature structure in the absence of task-specific supervision. Our findings indicate that large-scale slice-wise self-supervised learning can yield a unified brain MRI representation that supports diverse neuroimaging tasks without volumetric pretraining or full-network fine-tuning, establishing a scalable foundation for robust and data-efficient brain imaging analysis. Code is available at this https URL

2604.26940 2026-06-12 cs.CL 版本更新

Select to Think: Unlocking SLM Potential with Local Sufficiency

Select to Think: 利用局部充分性解锁小语言模型潜力

Wenxuan Ye, Yangyang Zhang, Xueli An, Georg Carle, Yunpu Ma

AI总结 提出Select to Think (S2T)方法,通过将大语言模型角色从生成转为选择,并蒸馏选择逻辑到小语言模型,使其在推理时无需依赖大模型,显著提升性能。

详情
Comments
Accepted to ICML 2026. Code is available at this https URL
AI中文摘要

小语言模型(SLM)部署高效,但在推理能力上常落后于大语言模型(LLM)。现有解决方案要么在推理分歧点调用LLM,导致大量延迟和成本,要么依赖标准蒸馏,受限于SLM准确模仿LLM复杂生成分布的能力。我们通过识别局部充分性来解决这一困境:在分歧点,LLM偏好的token通常位于SLM的top-K预测中,即使未能成为SLM的top-1选择。因此,我们提出Select to Think(S2T),将LLM的角色从开放式生成重新定义为在SLM的候选提案中进行选择,将监督信号简化为离散的候选排名。利用这一点,我们引入S2T-Local,将选择逻辑蒸馏到SLM中,使其能够在推理时自主重新排序,无需依赖LLM。实验表明,1.5B SLM的top-8候选包含32B LLM选择的命中率达95%,S2T-Local使1.5B SLM的数学平均相对贪心解码提升24.1%,以单轨迹效率达到8路径自一致性的效果。

英文摘要

Small language models (SLMs) offer efficient deployment, yet they often lag behind their larger counterparts (LLMs) in reasoning. Existing remedies either invoke an LLM at points of reasoning divergence, incurring substantial latency and cost, or rely on standard distillation, which is limited by the SLM's capacity to accurately mimic the LLM's complex generative distribution. We address this dilemma by identifying local sufficiency: at divergence points, the LLM's preferred token often resides within the SLM's top-K next-token predictions, even when failing to emerge as the SLM top-1 choice. We therefore propose Select to Think (S2T), which reframes the LLM's role from open-ended generation to selection among the SLM's proposals, simplifying the supervision signal to discrete candidate rankings. Leveraging this, we introduce S2T-Local, which distills the selection logic into the SLM, empowering it to perform autonomous re-ranking without inference-time LLM dependency. Empirically, a 1.5B SLM's top-8 candidates contain the 32B LLM's choice with a 95% hit rate, and S2T-Local improves the 1.5B SLM's Math Avg. over greedy decoding by 24.1% relative gain, matching the efficacy of 8-path self-consistency with single-trajectory efficiency.

2604.24806 2026-06-12 cs.IR cs.AI cs.DB 版本更新

Versioned Late Materialization for Ultra-Long Sequence Training in Recommendation Systems at Scale

版本化延迟物化:面向大规模推荐系统的超长序列训练

Liang Guo, Ge Song, Litao Deng, Jianhui Sun, Chufeng Hu, Lu Zhang, Zhen Ma, Shouwei Chen, Weiran Liu, Sarang Masti Sreeshylan, Xiaoxuan Meng, Yanzun Huang

AI总结 提出版本化延迟物化范式,通过归一化存储和即时序列重建消除数据冗余,支持超长用户交互历史训练,降低存储I/O开销并提升模型质量。

详情
AI中文摘要

现代深度学习推荐模型(DLRM)遵循序列长度的缩放定律,推动前沿走向超长用户交互历史(UIH)。然而,行业标准的“Fat Row”范式将序列预物化到每个训练样本中,造成存储和I/O瓶颈,数据基础设施使用超过GPU训练容量,数据冗余在多租户环境中被放大,其中不同序列长度需求的模型共享联合数据集。我们提出了一种\emph{版本化延迟物化}范式,通过将UIH归一化存储在一个不可变层中,并在训练期间通过轻量级版本指针即时重建序列,从而消除冗余。系统通过一个分叉协议确保在线到离线(O2O)一致性,防止未来泄漏跨流式和批式训练,同时一个读优化的不可变存储层为异构模型租户提供多维投影下推。解耦的数据预处理与流水线I/O预取和数据亲和性优化掩盖了训练时序列重建的延迟,使训练吞吐量保持GPU计算受限。部署在生产DLRM上,系统减少了训练数据基础设施资源使用,同时实现了激进的序列长度缩放,带来显著的模型质量提升,作为现代推荐模型架构(包括HSTU和ULTRA-HSTU)的基础数据基础设施。

英文摘要

Modern Deep Learning Recommendation Models (DLRMs) follow scaling laws with sequence length, driving the frontier toward ultra-long User Interaction History (UIH). However, the industry-standard "Fat Row" paradigm, which pre-materializes these sequences into every training example, creates a storage and I/O wall where data infrastructure usage exceeds GPU training capacity due to data redundancy that is amplified in multi-tenant environments where models with vastly different sequence length requirements share a union dataset. We present a \emph{versioned late materialization} paradigm that eliminates this redundancy by storing UIH once in a normalized, immutable tier and reconstructing sequences just-in-time during training via lightweight versioned pointers. The system ensures Online-to-Offline (O2O) consistency through a bifurcated protocol that prevents future leakage across both streaming and batch training, while a read-optimized immutable storage layer provides multi-dimensional projection pushdown for heterogeneous model tenants. Disaggregated data preprocessing with pipelined I/O prefetching and data-affinity optimizations masks the latency of training-time sequence reconstruction, keeping training throughput compute-bound by GPUs. Deployed on production DLRMs, the system reduces training data infrastructure resource usage while enabling aggressive sequence length scaling that delivers significant model quality gains, serving as the foundational data infrastructure for modern recommendation model architectures, including HSTU and ULTRA-HSTU.

2604.24079 2026-06-12 cs.CL cs.AI 版本更新

The Pragmatic Persona: Discovering LLM Persona through Bridging Inference

实用人格:通过桥接推理发现LLM人格

Jisoo Yang, Jongwon Ryu, Minuk Ma, Trung X. Pham, Junyeong Kim

AI总结 提出基于桥接推理的框架,通过构建话语级知识图谱捕捉LLM对话中的隐含语义关联,实现从话语连贯性层面发现稳定人格特征,优于基于频率或风格的基线方法。

详情
Comments
15 pages, 4 figures, accepted to ICPR 2026
AI中文摘要

大型语言模型(LLM)通过对话展现出固有且独特的人格。然而,现有的大多数人格发现方法依赖于表面层面的词汇或风格线索,将对话视为平坦的token序列,未能捕捉维持人格一致性的更深层次话语结构。为解决这一局限,我们提出一种新颖的分析框架,通过桥接推理——即通过共享世界知识和话语连贯性连接话语的隐含概念关系——来解读LLM对话。通过将这些关系建模为结构化知识图谱,我们的方法捕捉了控制LLM在对话轮次间组织意义的潜在语义链接,从而在话语连贯性层面而非表面实现上实现人格发现。在多种推理骨干和从小型模型到80B参数系统的目标LLM上的实验结果表明,与基于频率或风格的基线相比,桥接推理图产生了显著更强的语义连贯性和更稳定的人格识别。这些结果表明,人格特质始终编码在话语的结构组织中,而非孤立的词汇模式中。本工作提出了一个系统框架,通过认知话语理论的视角来探测、提取和可视化潜在的LLM人格,桥接了计算语言学、认知语义学和大型语言模型中的人格推理。代码见:https://this URL

英文摘要

Large Language Models (LLMs) reveal inherent and distinctive personas through dialogue. However, most existing persona discovery approaches rely on surface-level lexical or stylistic cues, treating dialogue as a flat sequence of tokens and failing to capture the deeper discourse-level structures that sustain persona consistency. To address this limitation, we propose a novel analytical framework that interprets LLM dialogue through bridging inference -- implicit conceptual relations that connect utterances via shared world knowledge and discourse coherence. By modeling these relations as structured knowledge graphs, our approach captures latent semantic links that govern how LLMs organize meaning across turns, enabling persona discovery at the level of discourse coherence rather than surface realizations. Experimental results across multiple reasoning backbones and target LLMs, ranging from small-scale models to 80B-parameter systems, demonstrate that bridging-inference graphs yield significantly stronger semantic coherence and more stable persona identification than frequency or style-based baselines. These results show that persona traits are consistently encoded in the structural organization of discourse rather than isolated lexical patterns. This work presents a systematic framework for probing, extracting, and visualizing latent LLM personas through the lens of Cognitive Discourse Theory, bridging computational linguistics, cognitive semantics, and persona reasoning in large language models. Codes are available at this https URL

2508.04427 2026-06-12 cs.LG cs.AI 版本更新

Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Models

解码多模态迷宫:多模态注意力模型中可解释性采纳的系统综述

Md Raisul Kibria, Sébastien Lafond, Janan Arslan

AI总结 本文系统综述了2020年至2024年初多模态模型可解释性研究,发现多数工作集中于视觉-语言和纯语言模型,注意力机制是主要解释方法,但评估缺乏系统性和鲁棒性,并提出了改进建议。

详情
AI中文摘要

近年来,多模态学习取得了显著进展,特别是随着注意力模型的整合,在各种任务中带来了显著的性能提升。与此同时,对可解释人工智能(XAI)的需求推动了越来越多的研究,旨在解释这些模型的复杂决策过程。本系统文献综述分析了2020年1月至2024年初期间发表的、关注多模态模型可解释性的研究。在XAI更广泛目标的框架内,我们从多个维度审视文献,包括模型架构、涉及模态、解释算法和评估方法。我们的分析显示,大多数研究集中在视觉-语言和纯语言模型上,注意力机制是最常用的解释方法。然而,这些方法往往无法捕捉模态间交互的全谱系,这一问题因领域间的架构异质性而进一步加剧。重要的是,我们发现多模态环境中XAI的评估方法大多是非系统性的,缺乏一致性、鲁棒性,并且未考虑模态特定的认知和上下文因素。为解决这些不足,我们不仅综合了所调查研究的发现,还纳入了补充分析,整合了推动多模态可解释性的近期和新兴进展。基于这些见解,我们提出了一套全面的建议,旨在促进多模态XAI研究中严谨、透明和标准化的评估与报告实践。我们的目标是支持未来构建更可解释、可问责和负责任的多模态AI系统,并以可解释性为核心。

英文摘要

Multimodal learning has witnessed remarkable advancements in recent years, particularly with the integration of attention-based models, leading to significant performance gains across a variety of tasks. Parallel to this progress, the demand for explainable artificial intelligence (XAI) has spurred a growing body of research aimed at interpreting the complex decision-making processes of these models. This systematic literature review analyzes research published between January 2020 and early 2024 that focuses on the explainability of multimodal models. Framed within the broader goals of XAI, we examine the literature across multiple dimensions, including model architecture, modalities involved, explanation algorithms and evaluation methodologies. Our analysis reveals that most studies are concentrated on vision-language and language-only models, with attention-based techniques being the most commonly employed for explanation. However, these methods often fall short in capturing the full spectrum of interactions between modalities, a challenge further compounded by the architectural heterogeneity across domains. Importantly, we find that evaluation methods for XAI in multimodal settings are largely non-systematic, lacking consistency, robustness, and consideration for modality-specific cognitive and contextual factors. To address these gaps, we not only synthesize findings from the surveyed works but also incorporate a complementary analysis that integrates recent and emerging advances driving multimodal explainability. Based on these insights, we provide a comprehensive set of recommendations aimed at promoting rigorous, transparent, and standardized evaluation and reporting practices in multimodal XAI research. Our goal is to support future research in more interpretable, accountable, and responsible multimodal AI systems, with explainability at their core.

2604.23165 2026-06-12 cs.CV 版本更新

BSViT: A Burst Spiking Vision Transformer for Expressive and Efficient Visual Representation Learning

BSViT:用于高效表达视觉表征学习的脉冲视觉Transformer

Hongxiang Peng, Dewei Bai, Hong Qu

AI总结 提出BSViT,通过双通道爆发脉冲自注意力机制和局部邻域掩码策略,解决脉冲视觉Transformer中二进制脉冲信息容量有限和全局自注意力密集交互的问题,在静态和事件视觉基准上取得更高精度和能效。

详情
Comments
Accepted by ECML PKDD 2026
AI中文摘要

脉冲视觉Transformer(S-ViT)为节能视觉学习提供了有前景的框架。然而,现有设计仍受限于两个基本问题:二进制脉冲编码的信息容量有限以及全局自注意力引入的密集令牌交互。为应对这些挑战,本文提出BSViT,一种爆发脉冲驱动的视觉Transformer,具有双通道爆发脉冲自注意力(DBSSA)机制。DBSSA用二进制脉冲编码查询,用爆发脉冲编码键以增强表示能力。值通路采用双兴奋性和抑制性二进制通道,实现有符号调制和更丰富的脉冲交互。重要的是,整个注意力操作保持仅加法计算,确保与节能神经形态硬件的兼容性。为进一步降低脉冲活动并融入空间先验,引入补丁邻域掩码策略将注意力限制在局部邻域,实现结构感知稀疏性并减少计算开销。此外,爆发脉冲编码被系统地集成到网络中,以提升脉冲级表示能力,超越传统二进制脉冲。在静态和事件视觉基准上的大量实验表明,BSViT在精度上持续优于现有脉冲Transformer,同时保持有竞争力的能效。

英文摘要

Spiking Vision Transformers (S-ViTs) offer a promising framework for energy-efficient visual learning. However, existing designs remain limited by two fundamental issues: the restricted information capacity of binary spike coding and the dense token interactions introduced by global self-attention. To address these challenges, this work proposes BSViT, a burst spiking-driven Vision Transformer featuring a Dual-Channel Burst Spiking Self-Attention (DBSSA) mechanism. DBSSA encodes queries with binary spikes and keys with burst spikes to enhance representational capacity. The value pathway adopts dual excitatory and inhibitory binary channels, enabling signed modulation and richer spike interactions. Importantly, the entire attention operation preserves addition-only computation, ensuring compatibility with energy-efficient neuromorphic hardware. To further reduce spike activity and incorporate spatial priors, a patch adjacency masking strategy is introduced to restrict attention to local neighborhoods, resulting in structure-aware sparsity and reduced computational overhead. In addition, burst spike coding is systematically integrated across the network to increase spike-level representational capacity beyond conventional binary spiking. Extensive experiments on both static and event-based vision benchmarks demonstrate that BSViT consistently outperforms existing spiking Transformers in accuracy while maintaining competitive energy efficiency.

2506.18493 2026-06-12 cs.CV 版本更新

ShowFlow: From Robust Single Concept to Condition-Free Multi-Concept Generation

ShowFlow: 从鲁棒的单概念到无条件的多概念生成

Trong-Vu Hoang, Quang-Binh Nguyen, Thanh-Toan Do, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le

AI总结 提出ShowFlow框架,通过KronA-WED适配器和语义感知注意力正则化增强单概念生成,并利用SAMA和布局一致性指导实现无额外条件的多概念生成。

详情
AI中文摘要

定制化图像生成仍然是可控图像合成中的核心挑战。对于单概念生成,保持身份保留和提示对齐是困难的。在多概念场景中,仅依赖提示而不使用布局框或语义掩码等额外条件,通常会导致身份丢失和概念遗漏。在本文中,我们介绍了ShowFlow,一个旨在应对这些挑战的全面框架。我们提出了用于单概念图像生成的ShowFlow-S,以及用于处理多个概念的ShowFlow-M。ShowFlow-S引入了一个KronA-WED适配器,它将Kronecker适配器与权重和嵌入分解相结合,并配合一种新颖的语义感知注意力正则化(SAR)训练目标,以增强单概念生成。在此基础上,ShowFlow-M直接重用由ShowFlow-S学习的鲁棒模型,以支持无需额外条件的多概念生成,并集成了主体自适应匹配注意力(SAMA)和布局一致性指导作为即插即用模块。大量实验和用户研究验证了ShowFlow的有效性,突显了其在广告和虚拟试穿等实际应用中的潜力。我们的源代码将在以下网址公开:this https URL。

英文摘要

Customizing image generation remains a core challenge in controllable image synthesis. For single-concept generation, maintaining both identity preservation and prompt alignment is challenging. In multi-concept scenarios, relying solely on a prompt without additional conditions like layout boxes or semantic masks, often leads to identity loss and concept omission. In this paper, we introduce ShowFlow, a comprehensive framework designed to tackle these challenges. We propose ShowFlow-S for single-concept image generation, and ShowFlow-M for handling multiple concepts. ShowFlow-S introduces a KronA-WED adapter, which integrates a Kronecker adapter with weight and embedding decomposition, and together with a novel Semantic-Aware Attention Regularization (SAR) training objective to enhance single-concept generation. Building on this foundation, ShowFlow-M directly reuses robust models learned by ShowFlow-S to support multi-concept generation without extra conditions, incorporating a Subject-Adaptive Matching Attention (SAMA) and a Layout Consistency guidance as the plug-and-play module. Extensive experiments and user studies validate ShowFlow's effectiveness, highlighting its potential in real-world applications like advertising and virtual dressing. Our source code will be publicly available at: this https URL.

2604.20428 2026-06-12 cs.RO 版本更新

Lexicographic Minimum-Violation Motion Planning using Signal Temporal Logic

使用信号时序逻辑的字典序最小违规运动规划

Patrick Halder, Lothar Kiltz, Hannes Homburger, Johannes Reuter, Matthias Althoff

AI总结 提出一种将字典序多目标优化转化为单目标标量优化的方法,通过非均匀量化和位移扩展MPPI求解器,并引入结合时空违规的谓词鲁棒性度量,实现可解释且可扩展的字典序STL最小违规运动规划。

详情
Comments
Submitted to the IEEE Open Journal of Intelligent Transportation Systems (under review)
AI中文摘要

自动驾驶汽车的运动规划通常需要满足多个有条件冲突的规范。在无法同时满足所有规范的情况下,最小违规运动规划通过根据规范的优先级最小化违规来维持系统运行。信号时序逻辑(STL)提供了一种形式化语言来严格定义这些规范,并能够对其违规进行定量评估。然而,规范的完全排序导致了一个字典序优化问题,使用标准方法求解通常计算成本高昂。我们通过使用非均匀量化和位移将多目标字典序优化问题转化为单目标标量优化问题来解决这个问题。具体来说,我们扩展了一个确定性模型预测路径积分(MPPI)求解器,以高效求解无二次输入成本的优化问题。此外,引入了一种结合空间和时间违规的新型谓词鲁棒性度量。我们的结果表明,所提出的方法在单目标求解器框架内为字典序STL最小违规运动规划提供了一种可解释且可扩展的解决方案。

英文摘要

Motion planning for autonomous vehicles often requires satisfying multiple conditionally conflicting specifications. In situations where not all specifications can be met simultaneously, minimum-violation motion planning maintains system operation by minimizing violations of specifications in accordance with their priorities. Signal temporal logic (STL) provides a formal language for rigorously defining these specifications and enables the quantitative evaluation of their violations. However, a total ordering of specifications yields a lexicographic optimization problem, which is typically computationally expensive to solve using standard methods. We address this problem by transforming the multi-objective lexicographic optimization problem into a single-objective scalar optimization problem using non-uniform quantization and bit-shifting. Specifically, we extend a deterministic model predictive path integral (MPPI) solver to efficiently solve optimization problems without quadratic input cost. Additionally, a novel predicate-robustness measure that combines spatial and temporal violations is introduced. Our results show that the proposed method offers an interpretable and scalable solution for lexicographic STL minimum-violation motion planning within a single-objective solver framework.

2604.20236 2026-06-12 cs.LG 版本更新

Machine Learning-based Two-Stage Graph Sparsification for the Travelling Salesman Problem

基于机器学习的两阶段图稀疏化方法用于旅行商问题

Bo-Cheng Lin, Yi Mei, Mengjie Zhang

AI总结 提出两阶段方法,先结合α-Nearest和POPMUSIC得到近完美召回率的候选图,再用轻量级分类器修剪单源边,在保持≥99.69%最优边的同时降低37%-47%密度。

详情
AI中文摘要

高性能TSP求解器(如Lin-Kernighan-Helsgaun (LKH))在\emph{候选图}(为求解器预先选定的边的小子集)中搜索,而不是在完整图上搜索。两种主要的稀疏化启发式方法,$\alpha$-Nearest和POPMUSIC,各自在密度-覆盖率平衡上存在不足:$\alpha$-Nearest密集且召回率稳定,而POPMUSIC更稀疏但其召回率随规模增大而下降。它们的并集在密度上远低于完整图的同时弥补了召回率差距,为进一步缩减留下了空间。现有的基于学习的稀疏化方法在完整图上对边评分,这种方法代价高昂且主要限于欧几里得实例。我们提出了一种两阶段方法,反转了这一逻辑。第一阶段取$\alpha$-Nearest和POPMUSIC的并集,在${\sim}6N$条边上实现近乎完美的召回率。关键在于,并集为每条边标注了其\emph{来源出处}——即它是由$\alpha$-Nearest、POPMUSIC还是两者共同支持的。第二阶段在这些标注边上训练一个轻量级分类器,并修剪得分最低的边。由于双源边几乎总是最优的,学习问题简化为过滤单源子集——这比从头开始对所有$O(N^2)$条边进行分类要容易得多。在四种距离类型、五种空间分布以及50到500的问题规模上,该流程将候选图密度降低了37%-47%,同时保留了${\geq}99.69\%$的最优旅行边,并且在TSP500上以更低的密度达到或超过了近期仅限欧几里得的神经稀疏化方法的覆盖率。

英文摘要

High-performance TSP solvers such as Lin-Kernighan-Helsgaun (LKH) search within a \emph{candidate graph} -- a small subset of edges pre-selected for the solver -- rather than over the complete graph. The two leading sparsification heuristics, $\alpha$-Nearest and POPMUSIC, each fall short of the density-coverage balance: $\alpha$-Nearest is dense with stable recall, while POPMUSIC is sparser but its recall degrades with scale. Their union closes the recall gap while remaining far below the complete graph in density, leaving room for further reduction. Existing learning-based sparsifiers score edges on the complete graph, an approach that is expensive and largely limited to Euclidean instances. We propose a two-stage method that inverts this logic. Stage~1 takes the union of $\alpha$-Nearest and POPMUSIC, achieving near-perfect recall at ${\sim}6N$ edges. Crucially, the union annotates each edge with its \emph{source provenance} -- whether it was endorsed by $\alpha$-Nearest, POPMUSIC, or both. Stage~2 trains a lightweight classifier on these annotated edges and prunes the lowest-scoring ones. Because dual-source edges are almost always optimal, the learning problem reduces to filtering the single-source subset -- a substantially easier task than classifying all $O(N^2)$ edges from scratch. Across four distance types, five spatial distributions, and problem sizes from 50 to 500, the pipeline reduces candidate-graph density by $37$-$47\%$ while retaining ${\geq}99.69\%$ of optimal-tour edges, and matches or exceeds the coverage of recent Euclidean-only neural sparsifiers at lower density at TSP500.

2601.14295 2026-06-12 cs.AI cs.CL cs.CY 版本更新

Epistemic Constitutionalism Or: how to avoid coherence bias

认知宪政主义:或如何避免一致性偏见

Michele Loi

AI总结 本文提出AI应建立明确的认知宪法,通过规范源归因等元规范避免一致性偏见,并论证自由主义路径优于柏拉图式路径。

详情
Comments
27 pages, 7 tables. Data: this http URL and this http URL. Complete AI-assisted writing documentation: this http URL
AI中文摘要

大型语言模型日益扮演着人工推理者的角色:它们评估论点、分配可信度并表达信心。然而,它们的信念形成行为受隐式、未经审查的认知策略支配。本文主张为AI建立一部认知宪法:明确的、可争议的元规范,用于调节系统如何形成和表达信念。源归因偏见提供了动机案例:我表明前沿模型强制执行身份-立场一致性,惩罚归因于其预期意识形态立场与论点内容冲突的源的论点。当模型检测到系统性测试时,这些效应消失,揭示系统将源敏感性视为需要抑制的偏见,而非一种需要良好执行的能力。我区分了两种宪政路径:柏拉图式路径,要求从特权立场出发的形式正确性和默认源独立性;自由主义路径,拒绝此类特权,指定保护集体探究条件的程序性规范,同时允许基于认知警觉的原则性源关注。我主张自由主义路径,勾勒出八项原则和四种取向的宪政核心,并提出AI认知治理需要与我们现在对AI伦理所期望的同样明确、可争议的结构。

英文摘要

Large language models increasingly function as artificial reasoners: they evaluate arguments, assign credibility, and express confidence. Yet their belief-forming behavior is governed by implicit, uninspected epistemic policies. This paper argues for an epistemic constitution for AI: explicit, contestable meta-norms that regulate how systems form and express beliefs. Source attribution bias provides the motivating case: I show that frontier models enforce identity-stance coherence, penalizing arguments attributed to sources whose expected ideological position conflicts with the argument's content. When models detect systematic testing, these effects collapse, revealing that systems treat source-sensitivity as bias to suppress rather than as a capacity to execute well. I distinguish two constitutional approaches: the Platonic, which mandates formal correctness and default source-independence from a privileged standpoint, and the Liberal, which refuses such privilege, specifying procedural norms that protect conditions for collective inquiry while allowing principled source-attending grounded in epistemic vigilance. I argue for the Liberal approach, sketch a constitutional core of eight principles and four orientations, and propose that AI epistemic governance requires the same explicit, contestable structure we now expect for AI ethics.

2604.18307 2026-06-12 cs.CL 版本更新

Reasoning Models Know What's Important, and Encode It in Their Activations

推理模型知道什么重要,并在其激活中编码

Yaniv Nikankin, Martin Tutek, Tomer Ashuach, Jonathan Rosenfeld, Yonatan Belinkov

AI总结 通过分析模型激活而非仅依赖推理链文本,发现激活能更有效识别关键推理步骤,且模型在生成后续步骤前已内部编码步骤重要性。

详情
AI中文摘要

语言模型通常通过生成包含许多重要性不同的步骤的长推理链来解决复杂任务。虽然某些步骤对生成最终答案至关重要,但其他步骤是可移除的。确定哪些步骤最重要以及为什么,仍然是理解模型如何处理推理的核心开放问题。我们研究了这个问题是通过模型内部还是通过推理链本身的标记来最好地解决。我们发现,模型激活比标记包含更多信息,用于识别重要的推理步骤。关键的是,通过在模型激活上训练探针来预测重要性,我们表明模型在生成后续步骤之前就已经编码了步骤重要性的内部表示。不同模型中重要性的内部表示在哪些步骤重要上具有高度一致性。这种表示分布在各个层中,并且与表面特征(如步骤的相对位置或长度)不相关。我们的发现表明,分析激活可以揭示表面方法根本遗漏的推理方面,表明推理分析应该研究模型内部。

英文摘要

Language models often solve complex tasks by generating long reasoning chains, consisting of many steps with varying importance. While some steps are crucial for generating the final answer, others are removable. Determining which steps matter most, and why, remains an open question central to understanding how models process reasoning. We investigate if this question is best approached through model internals or through tokens of the reasoning chain itself. We find that model activations contain more information than tokens for identifying important reasoning steps. Crucially, by training probes on model activations to predict importance, we show that models encode an internal representation of step importance, even prior to the generation of subsequent steps. The internal representations of importance in different models yield high agreement on which steps are important. The representation is distributed across layers, and does not correlate with surface-level features, such as a step's relative position or its length. Our findings suggest that analyzing activations can reveal aspects of reasoning that surface-level approaches fundamentally miss, indicating that reasoning analyses should look into model internals.

2601.00921 2026-06-12 cs.LG cs.AI quant-ph 版本更新

Geometric and Quantum Kernel Methods for Predicting Skeletal Muscle Outcomes in chronic obstructive pulmonary disease

用于预测慢性阻塞性肺疾病骨骼肌结果的几何与量子核方法

Azadeh Alavi, Hamidreza Khalili, Stanley H. Chan, Fatemeh Kouchmeshki, Muhammad Usman, Ross Vlahos

AI总结 提出一种核几何量子混合方法,通过再生核希尔伯特空间映射合成SPD参考、随机投影压缩和低维量子回归电路,在COPD动物队列中预测肌肉重量、质量和力量,肌肉重量RMSE比最佳经典方法低约1.8%。

详情
Comments
24 pages, 2 figures
AI中文摘要

慢性阻塞性肺疾病(COPD)影响全球数亿人,骨骼肌功能障碍具有临床重要性。量子机器学习在生物医学预测中日益受到探索,但在小型生物标志物队列中的价值需要与强经典基线进行基准测试。我们分析了一个由213只动物组成的香烟烟雾COPD队列,利用血液和支气管肺泡灌洗生物标志物预测胫骨前肌重量、肌肉质量和力量。我们开发了一种核几何量子混合方法,其中合成对称正定(SPD)参考通过再生核希尔伯特空间映射,使用仅训练随机投影压缩,归一化,并输入低维量子回归电路。我们将该方法与经典岭/核模型、SPD关系表示和量子核回归(QKR)进行了基准测试。所有方法均使用条件分层重复交叉验证进行评估。最大的数值改进出现在肌肉重量上,所提出方法的平均均方根误差(RMSE)数值最低,比最佳经典比较器低约1.8%;配对折叠水平测试在Holm调整后未建立统计显著性优势,但该终点具有生物学意义。该方法在肌肉质量上也具有数值最低的平均RMSE。对于力量,仅使用生物标志物的岭回归表现最佳,表明更线性的终点结构。

英文摘要

Chronic obstructive pulmonary disease (COPD) affects hundreds of millions of people worldwide, and skeletal-muscle dysfunction is clinically important. Quantum machine learning is increasingly explored for biomedical prediction, but its value in small biomarker cohorts requires benchmarking against strong classical baselines. We analysed a cigarette-smoke COPD cohort of 213 animals with blood and bronchoalveolar-lavage biomarkers to predict tibialis anterior muscle weight, muscle quality, and force. We developed a kernel-geometric quantum hybrid method in which synthetic symmetric positive definite (SPD) references are mapped through a reproducing kernel Hilbert space, compressed using train-only random projection, normalised, and supplied to low-dimensional quantum regression circuits. We benchmarked this approach against classical ridge/kernel models, SPD relational representations, and quantum-kernel regression (QKR). All methods were evaluated using condition-stratified repeated cross-validation. The largest numerical improvement was observed for muscle weight, where the proposed method had the numerically lowest mean root mean squared error (RMSE), approximately 1.8% below the best classical comparator; paired fold-level testing did not establish statistically significant superiority after Holm adjustment, but the endpoint is biologically meaningful. The method also had the numerically lowest mean RMSE for muscle quality. For force, biomarker-only Ridge performed best, suggesting a more linear endpoint structure.

2604.16689 2026-06-12 cs.AI 版本更新

The Query Channel: Information-Theoretic Limits of Masking-Based Explanations

查询通道:基于掩码的解释的信息论极限

Erciyes Karakaya, Ozgur Ercetin

AI总结 本文提出查询通道框架,将掩码后解释建模为通信过程,推导解释率与识别容量之间的信息论极限,并证明稀疏最大似然解码器可实现可靠恢复。

详情
AI中文摘要

基于掩码的事后解释方法,如KernelSHAP和LIME,通过随机扰动下的查询估计局部特征重要性。本文将这一过程建模为在查询通道上的通信,其中潜在解释作为消息,每次掩码评估作为一次信道使用。在此框架内,解释的复杂度由假设类的熵捕获,而查询接口以每次查询的识别容量确定的速率提供信息。我们推导了一个强逆定理,表明如果解释率超过该容量,则对于任何解释器和解码器序列,精确恢复的概率必然收敛到误差中的一。我们还证明了一个可达性结果,即当速率低于容量时,稀疏最大似然解码器可实现可靠恢复。互信息的蒙特卡洛估计器提供了一个非渐近查询基准,我们用它来比较最优解码与模拟LIME和KernelSHAP的基于Lasso和OLS的过程。实验揭示了在一定的查询预算范围内,信息论允许可靠解释,但标准凸替代方法仍然失败。最后,我们将神经语言模型的超像素分辨率和分词解释为一种源编码选择,它设定了解释的熵,并展示了高斯噪声和非线性曲率如何劣化查询通道,引发瀑布和错误平层行为,并使高分辨率解释无法实现。

英文摘要

Masking-based post-hoc explanation methods, such as KernelSHAP and LIME, estimate local feature importance by querying a black-box model under randomized perturbations. This paper formulates this procedure as communication over a query channel, where the latent explanation acts as a message and each masked evaluation is a channel use. Within this framework, the complexity of the explanation is captured by the entropy of the hypothesis class, while the query interface supplies information at a rate determined by an identification capacity per query. We derive a strong converse showing that, if the explanation rate exceeds this capacity, the probability of exact recovery necessarily converges to one in error for any sequence of explainers and decoders. We also prove an achievability result establishing that a sparse maximum-likelihood decoder attains reliable recovery when the rate lies below capacity. A Monte Carlo estimator of mutual information yields a non-asymptotic query benchmark that we use to compare optimal decoding with Lasso- and OLS-based procedures that mirror LIME and KernelSHAP. Experiments reveal a range of query budgets where information theory permits reliable explanations but standard convex surrogates still fail. Finally, we interpret super-pixel resolution and tokenization for neural language models as a source-coding choice that sets the entropy of the explanation and show how Gaussian noise and nonlinear curvature degrade the query channel, induce waterfall and error-floor behavior, and render high-resolution explanations unattainable.

2604.16548 2026-06-12 cs.CR cs.AI cs.CL 版本更新

A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle

LLM智能体中长期记忆安全综述:跨记忆生命周期的攻击、防御与治理

Zehao Lin, Xixuan Hao, Renyu Fu, Shaobo Cui, Kai Chen, Chunyu Li, Zhiyu Li, Feiyu Xiong

AI总结 本文提出记忆生命周期框架,系统分析LLM智能体长期记忆面临的新威胁,并引入可验证记忆治理(VMG)架构原语,强调存储时溯源与版本控制对安全的关键作用。

详情
AI中文摘要

LLM智能体中可写、跨会话持久记忆的出现,引入了与传统的以输入为中心的安全问题性质不同的威胁格局,其特点包括三个属性:持久性、状态性和传播性。为系统描述这一格局,我们提出记忆生命周期框架,该框架沿两个轴组织攻击、防御及其跨阶段依赖关系:六个生命周期阶段(写入、存储、检索、执行、共享与传播、遗忘与回滚)和四个安全目标(完整性、机密性、可用性、治理)。该分析进而揭示了在系统层面需要形式化安全保证,从而推动了可验证记忆治理(VMG)——一个由五个架构原语组成的框架,它规定了长期记忆系统必须提供哪些可验证机制,以维持对其记忆状态的可审计、可恢复控制。我们的分析表明,健壮的长期记忆(LTM)安全无法仅在检索或执行时进行事后补救,而必须从一开始就锚定于存储时的溯源、版本控制和策略感知的保留。

英文摘要

The emergence of writable, cross-session persistent memory in LLM agents introduces a qualitatively different threat landscape from conventional input-centric security concerns, characterized by three properties: persistence, statefulness, and propagation. To systematically characterize this landscape, we propose a Memory Lifecycle Framework that organizes attacks, defenses, and their cross-phase dependencies along two axes: six lifecycle phases (Write, Store, Retrieve, Execute, Share & Propagate, Forget & Rollback) and four security objectives (Integrity, Confidentiality, Availability, Governance). This analysis in turn exposes the need for formal security guarantees at the system level, motivating Verifiable Memory Governance(VMG), a framework of five architectural primitives that specifies what verifiable mechanisms a long-term-memory system must provide to maintain auditable, recoverable control over its memory state. Our analysis indicates that robust Long-Term Memory (LTM) security cannot be retrofitted at retrieval or execution time alone, but must be anchored in storage-time provenance, versioning, and policy-aware retention from the outset.

2604.13924 2026-06-12 cs.LG cs.AI cs.CV 版本更新

ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detection

ASTER: 用于无监督时间序列异常检测的潜在伪异常生成

Romain Hermary, Samet Hicsonmez, Dan Pineau, Abd El Rahman Shabayek, Djamila Aouada

AI总结 提出ASTER框架,在潜在空间生成伪异常训练Transformer分类器,结合预训练LLM增强表示,在三个基准数据集上达到最优性能。

详情
Comments
Published in ICPR 2026
AI中文摘要

时间序列异常检测(TSAD)在工业监控、医疗保健和网络安全等领域至关重要,但由于罕见且异质的异常以及标记数据的稀缺性,它仍然具有挑战性。这种稀缺性使得无监督方法占主导地位,但现有方法通常依赖于重建或预测(难以处理复杂数据),或依赖于需要领域特定异常合成和固定距离度量的基于嵌入的方法。我们提出ASTER,一个直接在潜在空间中生成伪异常的框架,避免了手工制作的异常注入和对领域专业知识的需求。潜在空间解码器生成定制的伪异常,用于训练基于Transformer的异常分类器,而预训练的LLM丰富了该空间的时间和上下文表示。在三个基准数据集上的实验表明,ASTER达到了最先进的性能,并为基于LLM的TSAD设立了新标准。

英文摘要

Time-series anomaly detection (TSAD) is critical in domains such as industrial monitoring, healthcare, and cybersecurity, but it remains challenging due to rare and heterogeneous anomalies and the scarcity of labelled data. This scarcity makes unsupervised approaches predominant, yet existing methods often rely on reconstruction or forecasting, which struggle with complex data, or on embedding-based approaches that require domain-specific anomaly synthesis and fixed distance metrics. We propose ASTER, a framework that generates pseudo-anomalies directly in the latent space, avoiding handcrafted anomaly injections and the need for domain expertise. A latent-space decoder produces tailored pseudo-anomalies to train a Transformer-based anomaly classifier, while a pre-trained LLM enriches the temporal and contextual representations of this space. Experiments on three benchmark datasets show that ASTER achieves state-of-the-art performance and sets a new standard for LLM-based TSAD.

2604.08958 2026-06-12 cs.LG cs.AI cs.RO 版本更新

WOMBET: World Model-Based Experience Transfer for Robust and Sample-efficient Reinforcement Learning

WOMBET:基于世界模型的经验迁移实现鲁棒且样本高效的强化学习

Mintae Kim, Koushil Sreenath

AI总结 提出WOMBET框架,通过源任务中学习世界模型并生成不确定性惩罚的离线数据,再结合自适应采样进行在线微调,实现鲁棒且样本高效的强化学习迁移。

详情
Comments
13 pages, 6 figures, 8th Annual Learning for Dynamics & Control Conference (L4DC)
AI中文摘要

机器人领域的强化学习通常受限于数据收集的成本和风险,因此需要从源任务向目标任务进行经验迁移。离线到在线强化学习利用先验数据,但通常假设给定固定数据集,并未解决如何生成可靠数据进行迁移的问题。我们提出基于世界模型的经验迁移(WOMBET)框架,该框架联合生成和利用先验数据。WOMBET在源任务中学习世界模型,并通过不确定性惩罚规划生成离线数据,随后筛选出高回报和低认知不确定性的轨迹。然后,它通过在离线数据和在线数据之间进行自适应采样,在目标任务中进行在线微调,实现了从先验驱动的初始化到任务特定适应的稳定过渡。我们证明了不确定性惩罚目标提供了真实回报的下界,并推导了有限样本误差分解,捕捉了分布不匹配和近似误差。实验上,WOMBET在连续控制基准测试中相比强基线提高了样本效率和最终性能,展示了联合优化数据生成和迁移的益处。

英文摘要

Reinforcement learning (RL) in robotics is often limited by the cost and risk of data collection, motivating experience transfer from a source task to a target task. Offline-to-online RL leverages prior data but typically assumes a given fixed dataset and does not address how to generate reliable data for transfer. We propose World Model-Based Experience Transfer (WOMBET), a framework that jointly generates and utilizes prior data. WOMBET learns a world model in the source task and generates offline data via uncertainty-penalized planning, followed by filtering trajectories with high return and low epistemic uncertainty. It then performs online fine-tuning in the target task using adaptive sampling between offline and online data, enabling a stable transition from prior-driven initialization to task-specific adaptation. We show that the uncertainty-penalized objective provides a lower bound on the true return and derive a finite-sample error decomposition capturing distribution mismatch and approximation error. Empirically, WOMBET improves sample efficiency and final performance over strong baselines on continuous control benchmarks, demonstrating the benefit of jointly optimizing data generation and transfer.

2511.02627 2026-06-12 cs.AI 版本更新

DecompSR: A dataset for decomposed analyses of compositional multihop spatial reasoning

DecompSR:用于组合多跳空间推理分解分析的数据集

Lachlan McPheat, Navdeep Kaur, Robert Blackwell, Alessandra Russo, Anthony G. Cohn, Pranava Madhyastha

AI总结 提出DecompSR数据集(超500万数据点),通过程序化生成独立控制组合性的多个方面(如推理深度、语言变异性),用于细粒度评估大语言模型的空间推理能力。

详情
AI中文摘要

我们引入了DecompSR(分解空间推理),这是一个大型基准数据集(超过500万个数据点)和生成框架,旨在分析组合空间推理能力。DecompSR的生成允许用户独立改变组合性的多个方面,即:生产力(推理深度)、替代性(实体和语言变异性)、过度泛化(输入顺序、干扰项)和系统性(新颖语言元素)。DecompSR以程序化方式构建,使其在构造上正确,并通过符号求解器独立验证以确保数据集的正确性。DecompSR在一系列大型语言模型(LLM)上进行了全面基准测试,我们表明LLM在空间推理任务中难以进行生产性和系统性泛化,而对语言变异性则更为鲁棒。DecompSR提供了一个可证明正确且严格的基准数据集,具有独立改变组合性几个关键方面程度的新能力,从而允许对LLM的组合推理能力进行稳健且细粒度的探测。

英文摘要

We introduce DecompSR, decomposed spatial reasoning, a large benchmark dataset (over 5m datapoints) and generation framework designed to analyse compositional spatial reasoning ability. The generation of DecompSR allows users to independently vary several aspects of compositionality, namely: productivity (reasoning depth), substitutivity (entity and linguistic variability), overgeneralisation (input order, distractors) and systematicity (novel linguistic elements). DecompSR is built procedurally in a manner which makes it is correct by construction, which is independently verified using a symbolic solver to guarantee the correctness of the dataset. DecompSR is comprehensively benchmarked across a host of Large Language Models (LLMs) where we show that LLMs struggle with productive and systematic generalisation in spatial reasoning tasks whereas they are more robust to linguistic variation. DecompSR provides a provably correct and rigorous benchmarking dataset with a novel ability to independently vary the degrees of several key aspects of compositionality, allowing for robust and fine-grained probing of the compositional reasoning abilities of LLMs.

2604.12497 2026-06-12 cs.LG stat.ML 版本更新

Allocating Human Oversight in AI-Enabled Analytics

AI赋能分析中的人类监督分配

Zikun Ye, Jiameng Lyu, Rui Tao

AI总结 针对AI预测可靠性异质且未知的问题,提出基于上置信界的在线学习策略,动态分配有限的人类验证预算,使终端效率损失随预算增长趋于零。

详情
AI中文摘要

组织越来越多地部署AI作为面向客户的决策过程中的低成本预测层,包括需求感知、服务质量监控、产品测试和市场研究,但AI生成的信号在不同任务、产品和客户细分中的可靠性并不均匀。因此,企业仍然需要稀缺的人类验证(标签、审计、调查回复或后续测量)来将AI输出锚定到真实情况。由于人类真实情况本身存在噪声,在不同标注者之间甚至重复判断中都有所变化,企业必须为每个任务收集并平均多个人类标签,这使得人类验证成本高昂。我们研究如何在可靠性异质且在部署前未知的情况下,将有限的人类验证预算分配到多个AI辅助任务中。我们将其置于调优的预测驱动推断框架内。每个人类标签既提高了AI辅助估计的精度,也揭示了任务的修正难度,即在使用AI预测作为控制变量后剩余的方差。如果难度已知,最优分配将遵循Neyman平方根规则;由于未知,我们提出一种基于上置信界的策略,该策略在线学习难度并将验证导向AI最不可靠的任务。我们证明,随着预算增长,该策略相对于最优分配的终端效率损失趋于零。在合成实验和一个包含68个任务和超过2000名受访者的真实数字孪生调查中,当可靠性异质时,该策略缩小了与最优分配的大部分差距,优于均匀分配和epsilon-贪婪分配;在调查数据上,它还优于先探索后提交的试点设计,并将均匀分配的10-12%差距缩小到2-6%。AI的价值不仅取决于模型准确性,还取决于将人类监督定向到AI错误影响最大的操作策略。

英文摘要

Organizations increasingly deploy AI as a low-cost prediction layer in customer-facing decision processes, including demand sensing, service-quality monitoring, product testing, and market research, but AI-generated signals are unevenly reliable across tasks, products, and customer segments. Firms therefore still need scarce human validation (labels, audits, survey responses, or follow-up measurements) to anchor AI outputs to ground truth. Because human ground truth is itself noisy, varying across labelers and even across repeated judgments, the firm must collect and average several human labels per task, which makes human validation costly. We study how to allocate a limited human-validation budget across many AI-assisted tasks when reliability is heterogeneous and unknown before deployment. We cast this within tuned prediction-powered inference. Each human label both sharpens the AI-assisted estimate and reveals the task's rectification difficulty, the variance that remains after the AI prediction is optimally used as a control variate. If difficulties were known, the optimal allocation would follow a Neyman square-root rule; because they are unknown, we propose a policy based on upper confidence bounds that learns them online and steers validation toward tasks where AI is least reliable. We prove that the policy's terminal efficiency loss relative to the oracle allocation vanishes as the budget grows. In synthetic experiments and a real digital-twin survey with 68 tasks and over 2000 respondents, it closes most of the gap to the oracle when reliability is heterogeneous, outperforming uniform and epsilon-greedy allocation; on the survey data it also outperforms explore-then-commit pilot designs and cuts uniform's 10--12% gap to 2--6%. The value of AI depends not only on model accuracy but also on the operational policy that targets human oversight where AI errors matter most.

2604.12002 2026-06-12 cs.CL 版本更新

Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision

自蒸馏零:自我修订将二元奖励转化为密集监督

Yinghui He, Simran Kaur, Adithya Bhaskar, Yongjin Yang, Jiarui Liu, Narutatsu Ri, Liam Fowl, Abhishek Panigrahi, Danqi Chen, Sanjeev Arora

AI总结 提出SD-Zero方法,通过让模型同时扮演生成器和修订者,利用二元奖励生成密集的token级自监督信号,显著提升训练样本效率,在数学和代码推理任务上超越RFT、GRPO等基线。

详情
AI中文摘要

当前在可验证设置下的后训练方法分为两类。强化学习(RLVR)依赖二元奖励,虽然广泛适用且强大,但在训练过程中仅提供稀疏监督。蒸馏提供密集的token级监督,通常从外部教师或使用高质量示范中获得。收集此类监督成本高昂或不可用。我们提出自蒸馏零(SD-Zero),一种比RL更高效利用训练样本的方法,且不需要外部教师或高质量示范。SD-Zero训练单个模型扮演两个角色:生成器,产生初始响应;修订者,基于该响应及其二元奖励生成改进的响应。然后我们进行在线自蒸馏,将修订者蒸馏到生成器中,使用修订者以生成器的响应及其奖励为条件的token分布作为监督。实际上,SD-Zero训练模型将二元奖励转化为密集的token级自监督。在数学和代码推理基准上,使用Qwen3-4B-Instruct和Olmo-3-7B-Instruct,SD-Zero相比基础模型性能提升至少10%,并在相同问题集和训练样本预算下优于强基线,包括拒绝微调(RFT)、GRPO和自蒸馏微调(SDFT)。大量消融实验显示了所提出算法的两个新特性:(a)token级自定位,其中修订者能够基于奖励识别生成器响应中需要修订的关键token;(b)迭代自进化,其中改进答案的修订能力可以通过定期教师同步蒸馏回生成性能。代码:此https URL。

英文摘要

Current post-training methods in verifiable settings fall into two categories. Reinforcement learning (RLVR) relies on binary rewards, which are broadly applicable and powerful, but provide only sparse supervision during training. Distillation provides dense token-level supervision, typically obtained from an external teacher or using high-quality demonstrations. Collecting such supervision can be costly or unavailable. We propose Self-Distillation Zero (SD-Zero), a method that is substantially more training sample-efficient than RL and does not require an external teacher or high-quality demonstrations. SD-Zero trains a single model to play two roles: a Generator, which produces an initial response, and a Reviser, which conditions on that response and its binary reward to produce an improved response. We then perform on-policy self-distillation to distill the reviser into the generator, using the reviser's token distributions conditioned on the generator's response and its reward as supervision. In effect, SD-Zero trains the model to transform binary rewards into dense token-level self-supervision. On math and code reasoning benchmarks with Qwen3-4B-Instruct and Olmo-3-7B-Instruct, SD-Zero improves performance by at least 10% over the base models and outperforms strong baselines, including Rejection Fine-Tuning (RFT), GRPO, and Self-Distillation Fine-Tuning (SDFT), under the same question set and training sample budget. Extensive ablation studies show two novel characteristics of our proposed algorithm: (a) token-level self-localization, where the reviser can identify the key tokens that need to be revised in the generator's response based on reward, and (b) iterative self-evolution, where the improving ability to revise answers can be distilled back into generation performance with regular teacher synchronization. Code: this https URL.

2604.10389 2026-06-12 cs.CL 版本更新

BLUEmed: Retrieval-Augmented Multi-Agent Debate for Clinical Error Detection

BLUEmed: 基于检索增强的多智能体辩论用于临床错误检测

Saukun Thika You, Nguyen Anh Khoa Tran, Wesley K. Marizane, Hanshu Rao, Qiunan Zhang, Xiaolei Huang

AI总结 提出BLUEmed框架,结合混合检索增强生成与多智能体辩论,通过分解临床笔记、检索证据、专家辩论及安全层过滤,在术语替换错误检测中达到最优性能。

详情
Comments
Accepted to the IEEE International Conference on Healthcare Informatics (ICHI) 2026
AI中文摘要

临床笔记中的术语替换错误(即一个医学术语被一个语言上有效但临床不同的术语替换)对医疗保健中的自动错误检测构成了持续挑战。我们引入了BLUEmed,一个多智能体辩论框架,增强有混合检索增强生成(RAG),该框架结合了基于证据的推理和多视角验证用于临床错误检测。BLUEmed将每个临床笔记分解为聚焦的子查询,通过密集、稀疏和在线检索检索来源分区的证据,并分配两个具有不同知识库的领域专家智能体以产生独立分析;当专家意见不一致时,一轮结构化的反论证和跨来源裁决解决冲突,随后是一个级联安全层,过滤常见的假阳性模式。我们在一个临床术语替换检测基准上评估BLUEmed,在零样本和少样本提示下,使用多个骨干模型(涵盖专有和开源系列)。实验结果表明,在少样本提示下,BLUEmed达到了最佳准确率(69.13%)、ROC-AUC(74.45%)和PR-AUC(72.44%),优于单智能体RAG和仅辩论基线。跨六个骨干模型和两种提示策略的进一步分析证实,检索增强和结构化辩论是互补的,并且该框架从具有足够指令遵循和临床语言理解的模型中受益最大。

英文摘要

Terminology substitution errors in clinical notes, where one medical term is replaced by a linguistically valid but clinically different term, pose a persistent challenge for automated error detection in healthcare. We introduce BLUEmed, a multi-agent debate framework augmented with hybrid Retrieval-Augmented Generation (RAG) that combines evidence-grounded reasoning with multi-perspective verification for clinical error detection. BLUEmed decomposes each clinical note into focused sub-queries, retrieves source-partitioned evidence through dense, sparse, and online retrieval, and assigns two domain expert agents distinct knowledge bases to produce independent analyses; when the experts disagree, a structured counter-argumentation round and cross-source adjudication resolve the conflict, followed by a cascading safety layer that filters common false-positive patterns. We evaluate BLUEmed on a clinical terminology substitution detection benchmark under both zero-shot and few-shot prompting with multiple backbone models spanning proprietary and open-source families. Experimental results show that BLUEmed achieves the best accuracy (69.13%), ROC-AUC (74.45%), and PR-AUC (72.44%) under few-shot prompting, outperforming both single-agent RAG and debate-only baselines. Further analyses across six backbone models and two prompting strategies confirm that retrieval augmentation and structured debate are complementary, and that the framework benefits most from models with sufficient instruction-following and clinical language understanding.

2511.18322 2026-06-12 cs.RO cs.CV cs.LG 版本更新

Learning Visually Interpretable Oscillator Networks for Soft Continuum Robots from Video

从视频中学习软体连续体机器人的视觉可解释振荡器网络

Henrik Krauss, Johann Licher, Naoya Takeishi, Annika Raatz, Takehisa Yairi

AI总结 提出注意力广播解码器(ABCD)和视觉振荡器网络(VONs),实现从视频中学习软体连续体机器人动力学的视觉和机械可解释性,多步预测误差降低5.8倍。

详情
Comments
Code available at: this https URL Dataset available at: this https URL Video available at: this https URL
AI中文摘要

从视频中学习软体连续体机器人(SCR)动力学提供了灵活性,但现有方法缺乏可解释性或依赖先验假设。基于模型的方法需要先验知识和手动设计。我们通过引入以下内容来弥补这一差距:(1)注意力广播解码器(ABCD),一种用于基于自编码器的潜在动力学学习的即插即用模块,生成像素级注意力图,定位每个潜在维度的贡献,同时过滤静态背景,通过空间接地潜在变量和图像叠加实现视觉可解释性。(2)视觉振荡器网络(VONs),一种二维潜在振荡器网络,与ABCD注意力图耦合,用于学习到的质量、耦合刚度和力的图像可视化,从而实现机械可解释性。我们在单段和双段SCR上验证了我们的方法,表明基于ABCD的模型显著提高了多步预测精度,在双段机器人上,Koopman算子的误差降低了5.8倍,振荡器网络的误差降低了3.5倍。VONs自主发现了振荡器的链式结构。这种完全数据驱动的方法产生了紧凑、机械可解释的模型,对未来的控制应用具有潜在意义。

英文摘要

Learning soft continuum robot (SCR) dynamics from video offers flexibility but existing methods lack interpretability or rely on prior assumptions. Model-based approaches require prior knowledge and manual design. We bridge this gap by introducing: (1) The Attention Broadcast Decoder (ABCD), a plug-and-play module for autoencoder-based latent dynamics learning that generates pixel-accurate attention maps localizing each latent dimension's contribution while filtering static backgrounds, enabling visual interpretability via spatially grounded latents and on-image overlays. (2) Visual Oscillator Networks (VONs), a 2D latent oscillator network coupled to ABCD attention maps for on-image visualization of learned masses, coupling stiffness, and forces, thereby enabling mechanical interpretability. We validate our approach on single- and double-segment SCRs, demonstrating that ABCD-based models significantly improve multi-step prediction accuracy with 5.8x error reduction for Koopman operators and 3.5x for oscillator networks on a two-segment robot. VONs autonomously discover a chain structure of oscillators. This fully data-driven approach yields compact, mechanically interpretable models with potential relevance for future control applications.

2512.14937 2026-06-12 cs.CV cs.AI 版本更新

Improving Pre-trained Adult Glioma Segmentation Models Using only Post-processing Techniques

仅使用后处理技术改进预训练的成人胶质瘤分割模型

Abhijeet Parida, Daniel Capellán-Martín, Zhifan Jiang, Nishad Kulkarni, Krithika Iyer, Austin Tapp, Syed Muhammad Anwar, María J. Ledesma-Carbayo, Marius George Linguraru

AI总结 针对预训练模型在胶质瘤分割中的系统误差,提出自适应后处理技术,在BraTS 2025挑战中使排名指标提升14.9%(撒哈拉以南非洲)和0.9%(成人胶质瘤),推动向高效、公平、可持续的后处理策略转变。

详情
AI中文摘要

胶质瘤是成人中最常见的恶性脑肿瘤,也是最致命的肿瘤之一。尽管积极治疗,中位生存率仍低于15个月。准确的多参数MRI(mpMRI)肿瘤分割对于手术规划、放疗和疾病监测至关重要。虽然深度学习模型提高了自动分割的准确性,但大规模预训练模型泛化能力差且常表现不佳,产生系统性错误,如假阳性、标签交换和切片不连续。这些问题因GPU资源获取不平等和大规模模型训练日益增长的环境成本而进一步加剧。在这项工作中,我们提出自适应后处理技术,以改进为各种肿瘤类型开发的大规模预训练模型产生的胶质瘤分割质量。我们在多个BraTS 2025分割挑战任务中展示了这些技术,使撒哈拉以南非洲挑战的排名指标提升了14.9%,成人胶质瘤挑战提升了0.9%。该方法推动脑肿瘤分割研究从日益复杂的模型架构转向精确、计算公平且可持续的高效临床后处理策略。

英文摘要

Gliomas are the most common malignant brain tumors in adults and are among the most lethal. Despite aggressive treatment, the median survival rate is less than 15 months. Accurate multiparametric MRI (mpMRI) tumor segmentation is critical for surgical planning, radiotherapy, and disease monitoring. While deep learning models have improved the accuracy of automated segmentation, large-scale pre-trained models generalize poorly and often underperform, producing systematic errors such as false positives, label swaps, and slice discontinuities in slices. These limitations are further compounded by unequal access to GPU resources and the growing environmental cost of large-scale model training. In this work, we propose adaptive post-processing techniques to refine the quality of glioma segmentations produced by large-scale pretrained models developed for various types of tumors. We demonstrated the techniques in multiple BraTS 2025 segmentation challenge tasks, with the ranking metric improving by 14.9 % for the sub-Saharan Africa challenge and 0.9% for the adult glioma challenge. This approach promotes a shift in brain tumor segmentation research from increasingly complex model architectures to efficient, clinically aligned post-processing strategies that are precise, computationally fair, and sustainable.

2512.14648 2026-06-12 cs.CV eess.IV 版本更新

Adaptable Segmentation Pipeline for Diverse Brain Tumors with Radiomic-Guided Subtyping and Lesion-Wise Model Ensemble

适用于多样化脑肿瘤的自适应分割流程:放射组学引导的亚型分类与病灶级模型集成

Daniel Capellán-Martín, Abhijeet Parida, Zhifan Jiang, Nishad Kulkarni, Krithika Iyer, Austin Tapp, Syed Muhammad Anwar, María J. Ledesma-Carbayo, Marius George Linguraru

AI总结 提出一种灵活模块化的自适应分割流程,通过放射组学特征检测肿瘤亚型并平衡训练,结合病灶级性能指标优化模型集成与后处理,在BraTS 2025挑战赛中达到顶尖性能,支持临床定量肿瘤测量。

详情
Comments
12 pages, 5 figures, 3 tables. Algorithm presented at MICCAI BraTS 2025
AI中文摘要

在多参数磁共振成像(MRI)上对脑肿瘤进行鲁棒且可泛化的分割仍然困难,因为肿瘤类型差异很大。BraTS 2025 Lighthouse挑战赛在多种高质量成人及儿童肿瘤数据集上对分割方法进行基准测试:多联盟国际儿童脑肿瘤分割(PED)、术前脑膜瘤肿瘤分割(MEN)、脑膜瘤放射治疗分割(MEN-RT)以及治疗前后脑转移瘤分割(MET)。我们提出了一种灵活、模块化且自适应的流程,通过选择和组合最先进的模型,并在训练前后应用肿瘤和病灶特定的处理,来提高分割性能。从MRI中提取的放射组学特征有助于检测肿瘤亚型,确保更平衡的训练。自定义的病灶级性能指标决定了每个模型在集成中的影响力,并优化了进一步细化预测的后处理,使工作流能够针对每个病例定制每一步。在BraTS测试集上,我们的流程在多个挑战中取得了与顶尖算法相当的性能。这些发现证实,自定义的病灶感知处理与模型选择能够产生鲁棒的分割,而无需将方法锁定在特定的网络架构上。我们的方法在临床实践中具有定量肿瘤测量的潜力,支持诊断和预后。

英文摘要

Robust and generalizable segmentation of brain tumors on multi-parametric magnetic resonance imaging (MRI) remains difficult because tumor types differ widely. The BraTS 2025 Lighthouse Challenge benchmarks segmentation methods on diverse high-quality datasets of adult and pediatric tumors: multi-consortium international pediatric brain tumor segmentation (PED), preoperative meningioma tumor segmentation (MEN), meningioma radiotherapy segmentation (MEN-RT), and segmentation of pre- and post-treatment brain metastases (MET). We present a flexible, modular, and adaptable pipeline that improves segmentation performance by selecting and combining state-of-the-art models and applying tumor- and lesion-specific processing before and after training. Radiomic features extracted from MRI help detect tumor subtype, ensuring a more balanced training. Custom lesion-level performance metrics determine the influence of each model in the ensemble and optimize post-processing that further refines the predictions, enabling the workflow to tailor every step to each case. On the BraTS testing sets, our pipeline achieved performance comparable to top-ranked algorithms across multiple challenges. These findings confirm that custom lesion-aware processing and model selection yield robust segmentations yet without locking the method to a specific network architecture. Our method has the potential for quantitative tumor measurement in clinical practice, supporting diagnosis and prognosis.

2506.18438 2026-06-12 cs.CV 版本更新

CPAM: Context-Preserving Adaptive Manipulation for Zero-Shot Real Image Editing

CPAM: 保持上下文的自适应操作用于零样本真实图像编辑

Dinh-Khoi Vo, Thanh-Toan Do, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia Le

AI总结 提出CPAM零样本框架,通过保持上下文的自适应操作和掩码引导,实现复杂非刚性真实图像的编辑,保留纹理和身份,无需微调。

详情
Comments
Accepted to IEEE Transactions on Multimedia. Project page: this https URL
AI中文摘要

使用文本描述在文本到图像扩散模型中编辑自然图像仍然是一个重大挑战,特别是在实现一致生成和处理复杂非刚性对象方面。现有方法通常难以保留纹理和身份,需要大量微调,并且在编辑特定空间区域或对象的同时保留背景细节方面存在局限性。本文提出了保持上下文的自适应操作(CPAM),一种用于复杂非刚性真实图像编辑的新型零样本框架。具体来说,我们提出了一个保留适应模块,该模块调整自注意力机制以有效保留并独立控制对象和背景。这确保了在编辑过程中使用掩码引导技术时,对象的形状、纹理和身份得以保持,同时背景不变形。此外,我们开发了一个局部提取模块,以减轻在交叉注意力机制的条件化过程中对非期望修改区域的干扰。我们还引入了各种掩码引导策略,以简单的方式促进多样化的图像操作任务。CPAM可以无缝集成到多个扩散骨干网络中,包括SD1.5、SD2.1和SDXL,展示了跨不同模型架构的强大泛化能力。在我们新构建的图像操作基准(IMBA)上进行的广泛实验表明,我们提出的方法是人类评估者的首选,优于现有的最先进编辑技术。源代码和数据将在项目页面公开发布:this https URL

英文摘要

Editing natural images using textual descriptions in text-to-image diffusion models remains a significant challenge, particularly in achieving consistent generation and handling complex, non-rigid objects. Existing methods often struggle to preserve textures and identity, require extensive fine-tuning, and exhibit limitations in editing specific spatial regions or objects while retaining background details. This paper proposes Context-Preserving Adaptive Manipulation (CPAM), a novel zero-shot framework for complicated, non-rigid real image editing. Specifically, we propose a preservation adaptation module that adjusts self-attention mechanisms to preserve and independently control the object and background effectively. This ensures that the objects' shapes, textures, and identities are maintained while keeping the background undistorted during the editing process using the mask guidance technique. Additionally, we develop a localized extraction module to mitigate the interference with the non-desired modified regions during conditioning in cross-attention mechanisms. We also introduce various mask-guidance strategies to facilitate diverse image manipulation tasks in a simple manner. CPAM can be seamlessly integrated with multiple diffusion backbones, including SD1.5, SD2.1, and SDXL, demonstrating strong generalization across different model architectures. Extensive experiments on our newly constructed Image Manipulation BenchmArk (IMBA), a robust benchmark dataset specifically designed for real image editing, demonstrate that our proposed method is the preferred choice among human raters, outperforming existing state-of-the-art editing techniques. The source code and data will be publicly released at the project page: this https URL

2604.08983 2026-06-12 cs.RO 版本更新

AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly

AssemLM: 用于机器人装配的空间推理多模态大语言模型

Zhi Jing, Jinbin Qiao, Ouyang Lu, Jicong Ao, Shuang Qiu, Huazhe Xu, Yu-Gang Jiang, Chenjia Bai

AI总结 提出AssemLM,一种融合装配手册、点云和文本指令的多模态大语言模型,通过专用点云编码器提取几何与旋转特征,实现精确的6D装配位姿推理,并构建含90万样本的AssemBench基准,在真实机器人装配任务中取得最优性能。

详情
Comments
Project Page: this https URL
AI中文摘要

空间推理是具身智能的基本能力,尤其对于机器人装配等精细操作任务。当前基于视觉语言模型(VLM)的方法主要依赖粗粒度的2D感知,难以对复杂3D几何进行精确推理。为解决这一局限,我们提出AssemLM,一种用于机器人装配的空间多模态大语言模型,它整合装配手册、点云和文本指令,通过显式几何理解预测任务关键的6D装配位姿。为桥接原始3D感知与高层语言推理,AssemLM采用专用点云编码器提取细粒度几何与旋转特征,以实现装配任务中精确的3D空间推理。此外,我们引入AssemBench,一个面向装配空间推理的大规模基准,包含超过90万多模态样本和精确的6D位姿标注,将评估从2D定位扩展到完整的3D几何推理。大量实验和真实机器人评估表明,AssemLM在6D位姿推理性能上达到最优,并有效支持真实环境中的精细多步装配任务。代码、模型和AssemBench数据集将公开提供。

英文摘要

Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. Recent methods based on vision-language models (VLMs) largely rely on coarse 2D perception and struggle to perform accurate reasoning over complex 3D geometry. To address this limitation, we propose AssemLM, a spatial multimodal large language model for robotic assembly that integrates assembly manuals, point clouds, and textual instructions to predict task-critical 6D assembly poses with explicit geometric understanding. To bridge raw 3D perception and high-level linguistic reasoning, AssemLM employs a specialized point cloud encoder to extract fine-grained geometric and rotational features for accurate 3D spatial reasoning in assembly tasks. In addition, we introduce AssemBench, a large-scale benchmark for assembly-oriented spatial reasoning with over 900K multimodal samples and precise 6D pose annotations, extending evaluation from 2D grounding to full 3D geometric inference. Extensive experiments and real-robot evaluations demonstrate that AssemLM achieves state-of-the-art 6D pose reasoning performance and effectively supports fine-grained, multi-step assembly tasks in real-world settings. Code, models, and the AssemBench dataset will be made publicly available.

2604.07590 2026-06-12 cs.IR cs.AI 版本更新

DCD: Domain-Oriented Design for Controlled Retrieval-Augmented Generation

DCD:面向领域的受控检索增强生成设计

Valerii Kovalskii, Nikita Belov, Nikita Miteyko, Igor Reshetnikov, Maksim Maksimov

AI总结 提出DCD(领域-集合-文档)层次化设计,通过结构化知识表示和多阶段路由控制检索与生成范围,无需修改语言模型,提升RAG在异构语料和多步查询中的鲁棒性和准确性。

详情
Comments
14 pages, 4 figures, 2 links, link to HF this https URL, link to GIT this https URL
AI中文摘要

检索增强生成(RAG)被广泛用于将大型语言模型锚定在外部知识源中。然而,当应用于异构语料库和多步查询时,朴素RAG管道由于扁平的知识表示和缺乏显式工作流而常常质量下降。在这项工作中,我们引入了DCD(领域-集合-文档),一种面向领域的设计,用于结构化知识并控制RAG系统中的查询处理,而无需修改底层语言模型。所提出的方法依赖于信息空间的层次分解和基于结构化模型输出的多阶段路由,使得检索和生成范围能够逐步受限。该架构辅以智能分块、混合检索以及集成验证和生成护栏机制。我们描述了DCD架构和工作流程,并讨论了在合成评估数据集上的评估结果,突出了它们在应用RAG场景中对鲁棒性、事实准确性和答案相关性的影响。

英文摘要

Retrieval-Augmented Generation (RAG) is widely used to ground large language models in external knowledge sources. However, when applied to heterogeneous corpora and multi-step queries, Naive RAG pipelines often degrade in quality due to flat knowledge representations and the absence of explicit workflows. In this work, we introduce DCD (Domain-Collection-Document), a domain-oriented design to structure knowledge and control query processing in RAG systems without modifying the underlying language model. The proposed approach relies on a hierarchical decomposition of the information space and multi-stage routing based on structured model outputs, enabling progressive restriction of both retrieval and generation scopes. The architecture is complemented by smart chunking, hybrid retrieval, and integrated validation and generation guardrail mechanisms. We describe the DCD architecture and workflow and discuss evaluation results on synthetic evaluation dataset, highlighting their impact on robustness, factual accuracy, and answer relevance in applied RAG scenarios.

2503.02178 2026-06-12 stat.ML cs.LG 版本更新

Central Limit Theorems for Stochastic Gradient Descent Quantile Estimators

随机梯度下降分位数估计量的中心极限定理

Ziyang Wei, Jiaqi Li, Likai Chen, Wei Biao Wu

AI总结 本文针对常学习率SGD分位数估计,利用马尔可夫链理论证明其平稳分布随学习率趋于零时收敛到高斯分布,首次给出CLT型理论保证,并提出置信区间递归算法。

详情
AI中文摘要

本文发展了通过恒定学习率的随机梯度下降(SGD)进行分位数估计的渐近理论。分位数损失函数既不光滑也不强凸。超越传统视角和技术,我们将分位数SGD迭代视为一个不可约、周期且正常返的马尔可夫链,该链循环收敛到其唯一的平稳分布,无论初始值如何任意固定。为了推导平稳分布的精确形式,我们通过利用平稳方程分析其特征函数的结构。我们还推导了其矩生成函数(MGF)和尾部概率的紧界。综合上述方法,我们证明了当学习率$\eta\rightarrow0$时,中心化和标准化的平稳分布收敛到高斯分布。这一发现为恒定学习率的分位数SGD估计量提供了首个中心极限定理(CLT)类型的理论保证。我们进一步提出了一种递归算法来构建具有统计保证的估计量的置信区间。数值研究展示了在线估计器和推断过程的有效有限样本性能。本研究所发展的理论工具对于研究一般形式化为马尔可夫链的SGD算法具有独立意义,特别是在非强凸和非光滑设置中。

英文摘要

This paper develops asymptotic theory for quantile estimation via stochastic gradient descent (SGD) with a constant learning rate. The quantile loss function is neither smooth nor strongly convex. Beyond conventional perspectives and techniques, we view quantile SGD iteration as an irreducible, periodic, and positive recurrent Markov chain, which cyclically converges to its unique stationary distribution regardless of the arbitrarily fixed initialization. To derive the exact form of the stationary distribution, we analyze the structure of its characteristic function by exploiting the stationary equation. We also derive tight bounds for its moment generating function (MGF) and tail probabilities. Synthesizing the aforementioned approaches, we prove that the centered and standardized stationary distribution converges to a Gaussian distribution as the learning rate $\eta\rightarrow0$. This finding provides the first central limit theorem (CLT)-type theoretical guarantees for the quantile SGD estimator with constant learning rates. We further propose a recursive algorithm to construct confidence intervals of the estimators with statistical guarantees. Numerical studies demonstrate the effective finite-sample performance of the online estimator and inference procedure. The theoretical tools developed in this study are of independent interest for investigating general SGD algorithms formulated as Markov chains, particularly in non-strongly convex and non-smooth settings.

2305.08175 2026-06-12 cs.DB cs.CR cs.LG 版本更新

ResidualPlanner+: a scalable matrix mechanism for marginals and beyond

ResidualPlanner+:一种用于边际查询及更广泛查询的可扩展矩阵机制

Guanlin He, Yingtai Xiao, Levent Toksoz, Zeyu Ding, Danfeng Zhang, Daniel Kifer

AI总结 提出两种可扩展的矩阵机制ResidualPlanner和ResidualPlanner+,分别优化边际查询的精度和支持更复杂的工作负载(如范围查询),在速度和内存上显著超越现有方法。

详情
AI中文摘要

带噪声的边际查询是保护机密性的常见数据发布形式,对于列联表分析、贝叶斯网络构建甚至合成数据生成等下游任务非常有用。为线性查询(如边际查询)提供无偏噪声答案的隐私机制称为矩阵机制。我们提出了ResidualPlanner和ResidualPlanner+,两种高度可扩展的矩阵机制。ResidualPlanner在使用高斯噪声回答边际查询时既最优又可扩展,而ResidualPlanner+支持更通用的工作负载,例如边际查询与范围查询或前缀和查询的组合。ResidualPlanner可以优化许多损失函数,这些损失函数可以写成边际方差的凸函数(先前的工作仅限于一个预定义的目标函数)。ResidualPlanner可以在几秒钟内优化大规模设置中边际查询的精度,即使之前的最先进方法(HDMM)内存耗尽。它甚至可以在几分钟内处理具有100个属性的数据集。此外,ResidualPlanner可以高效计算每个边际的方差/协方差值(先前的方法即使对于相对较小的数据集也会很快耗尽内存)。ResidualPlanner+支持更复杂的工作负载,这些工作负载结合了边际查询和范围/前缀和查询(例如,关于种族的边际查询、关于年龄的范围查询以及回答每个种族的年龄范围查询的组合种族/年龄表格)。它甚至支持用户在不同属性上自定义工作负载。凭借这种增加的灵活性,ResidualPlanner+不一定是最优的,但它仍然极具可扩展性,并且在精度和速度上均优于先前的最先进方法(HDMM)处理前缀和查询。

英文摘要

Noisy marginals are a common form of confidentiality protecting data release and are useful for many downstream tasks such as contingency table analysis, construction of Bayesian networks, and even synthetic data generation. Privacy mechanisms that provide unbiased noisy answers to linear queries (such as marginals) are known as matrix mechanisms. We propose ResidualPlanner and ResidualPlanner+, two highly scalable matrix mechanisms. ResidualPlanner is both optimal and scalable for answering marginal queries with Gaussian noise, while ResidualPlanner+ provides support for more general workloads, such as combinations of marginals and range queries or prefix-sum queries. ResidualPlanner can optimize for many loss functions that can be written as a convex function of marginal variances (prior work was restricted to just one predefined objective function). ResidualPlanner can optimize the accuracy of marginals in large scale settings in seconds, even when the previous state of the art (HDMM) runs out of memory. It even runs on datasets with 100 attributes in a couple of minutes. Furthermore, ResidualPlanner can efficiently compute variance/covariance values for each marginal (prior methods quickly run out of memory, even for relatively small datasets). ResidualPlanner+ provides support for more complex workloads that combine marginal and range/prefix-sum queries (e.g., a marginal on race, a range query on age, and a combined race/age tabulation that answers age range queries for each race). It even supports custom user-defined workloads on different attributes. With this added flexibility, ResidualPlanner+ is not necessarily optimal, however it is still extremely scalable and outperforms the prior state-of-the-art (HDMM) on prefix-sum queries both in terms of accuracy and speed.

2603.29515 2026-06-12 cs.LG 版本更新

Variational Graph Neural Networks for Uncertainty Quantification in Inverse Problems

变分图神经网络用于反问题中的不确定性量化

David Gonzalez, Alba Muixi, Beatriz Moya, Elias Cueto

AI总结 提出变分图神经网络(VGNN),通过在解码器引入变分层以较低成本量化认知和统计不确定性,在固体力学反问题中验证了高精度参数恢复与置信区间估计。

详情
AI中文摘要

深度学习技术在计算力学中的日益广泛应用显著加速了那些几年前还被认为是难以处理的问题的模拟。然而,在诸如工程或医学数字孪生等关键应用中,快速响应是不够的;还必须提供可靠的结果。在某些情况下,传统的确定性方法可能不是最优的,因为它们无法提供对其预测或结果的置信度度量,尤其是在反问题中,解可能不唯一或初始数据由于噪声等原因不完全可靠。经典的深度神经网络也缺乏明确的度量来量化其预测的不确定性。在这项工作中,我们提出了一种变分图神经网络(VGNN)架构,该架构将变分层集成到其架构中以建模权重的概率分布。与计算昂贵的全贝叶斯网络不同,我们的方法仅在解码器中策略性地引入变分层,从而能够以相对较低的成本估计认知不确定性和统计不确定性。在这项工作中,我们在两个固体力学案例中验证了所提出的方法:在二维弹性问题中识别具有非线性分布的弹性模量值,以及在三维超弹性梁中定位和量化施加的载荷,在这两种情况下仅使用每个测试的位移场作为输入数据。结果表明,该模型不仅以高精度恢复了物理参数,还提供了与问题物理特性一致的置信区间,并且能够定位施加载荷的位置并估计其值,为该实验提供了置信区间。

英文摘要

The increasingly wide use of deep machine learning techniques in computational mechanics has significantly accelerated simulations of problems that were considered unapproachable just a few years ago. However, in critical applications such as Digital Twins for engineering or medicine, fast responses are not enough; reliable results must also be provided. In certain cases, traditional deterministic methods may not be optimal as they do not provide a measure of confidence in their predictions or results, especially in inverse problems where the solution may not be unique or the initial data may not be entirely reliable due to the presence of noise, for instance. Classic deep neural networks also lack a clear measure to quantify the uncertainty of their predictions. In this work, we present a variational graph neural network (VGNN) architecture that integrates variational layers into its architecture to model the probability distribution of weights. Unlike computationally expensive full Bayesian networks, our approach strategically introduces variational layers exclusively in the decoder, allowing us to estimate cognitive uncertainty and statistical uncertainty at a relatively lower cost. In this work, we validate the proposed methodology in two cases of solid mechanics: the identification of the value of the elastic modulus with nonlinear distribution in a 2D elastic problem and the location and quantification of the loads applied to a 3D hyperelastic beam, in both cases using only the displacement field of each test as input data. The results show that the model not only recovers the physical parameters with high precision, but also provides confidence intervals consistent with the physics of the problem, as well as being able to locate the position of the applied load and estimate its value, giving a confidence interval for that experiment.

2601.06572 2026-06-12 cs.LG cs.AI 版本更新

Hellinger Multimodal Variational Autoencoders

Hellinger多模态变分自编码器

Huyen Vo, Isabel Valera

AI总结 提出基于Hellinger距离的矩匹配近似方法HELVAE,避免子采样,在多模态变分自编码器中实现更优的生成一致性与质量权衡。

详情
Comments
Accepted at AISTATS 2026. Camera-ready version
AI中文摘要

多模态变分自编码器(VAEs)广泛用于弱监督生成学习,涉及多种模态。主流方法通过专家乘积(PoE)、专家混合(MoE)或其组合来聚合单模态推理分布,以近似联合后验。本文从概率意见池化的优化视角重新审视多模态推理。我们从$\alpha=0.5$的Hölder池化出发,这是$\alpha\text{-散度}$族中唯一的对称成员,并推导出一种矩匹配近似,称为Hellinger。我们利用这种近似提出HELVAE,一种避免子采样的多模态VAE,从而得到一个高效且有效的模型,该模型:(i)随着观察到的模态增加,学习更具表达力的潜在表示;(ii)在生成一致性和质量之间实现更好的权衡,优于最先进的多模态VAE模型。

英文摘要

Multimodal variational autoencoders (VAEs) are widely used for weakly supervised generative learning with multiple modalities. Predominant methods aggregate unimodal inference distributions using either a product of experts (PoE), a mixture of experts (MoE), or their combinations to approximate the joint posterior. In this work, we revisit multimodal inference through the lens of probabilistic opinion pooling, an optimization-based approach. We start from Hölder pooling with $\alpha=0.5$, which corresponds to the unique symmetric member of the $\alpha\text{-divergence}$ family, and derive a moment-matching approximation, termed Hellinger. We then leverage such an approximation to propose HELVAE, a multimodal VAE that avoids sub-sampling, yielding an efficient yet effective model that: (i) learns more expressive latent representations as additional modalities are observed; and (ii) empirically achieves better trade-offs between generative coherence and quality, outperforming state-of-the-art multimodal VAE models.