arXivDaily arXiv每日学术速递 周一至周五更新
重置
全部学科分类 2256
2605.27840 2026-05-28 eess.AS cs.AI cs.SD

LoSATok: Low-dimensional Semantic-Acoustic Tokenizer for Cross-Domain Audio Understanding and Generation

LoSATok: 用于跨域音频理解与生成的低维语义-声学分词器

Zhisheng Zhang, Xiang Li, Yixuan Zhou, Jing Peng, Guoyang Zeng, Zhiyong Wu

AI总结 提出低维音频分词器LoSATok,通过语义瓶颈压缩和双级语义监督,在紧凑潜空间中联合捕获语义和声学细节,提升扩散Transformer的生成性能。

详情
AI中文摘要

音频分词器是统一音频理解和生成的基础。理解需要高层语义,而生成需要语义和声学细节。现有的统一分词器将两者共同编码到高维连续潜变量中,这增加了扩散Transformer(DiT)的建模负担。我们提出LoSATok,一种用于跨域音频理解和生成的低维音频分词器。受1280维语义编码器特征可压缩的观察启发,我们引入语义瓶颈(Semantic Bottleneck),将其压缩到128维,并通过提出的时间关系损失(time-relation loss)正则化以实现时间特征一致性。我们进一步设计了一种双级语义监督方法,利用高维和低维语义信号,使分词器能够在紧凑的潜空间中联合捕获语义和声学细节。在语音、音乐和通用音频上的实验表明,SemBo保持了强大的低维语义能力,LoSATok与几种语义表示相比保持了有竞争力的理解性能,同时在语音、音乐和音频生成上持续提升了DiT的建模性能。这些结果表明,LoSATok的低维表示能够有效支持音频理解和生成。我们的代码提供在https://github.com/wxzyd123/LoSATok。

英文摘要

Audio tokenizers are fundamental to unifying audio understanding and generation. Understanding requires high-level semantics, while generation demands semantic and acoustic details. Existing unified tokenizers jointly encode both in high-dimensional continuous latents, which increases the modeling burden of Diffusion Transformers (DiTs) for generation. We propose LoSATok, a low-dimensional audio tokenizer for cross-domain audio understanding and generation. Motivated by the observation that 1280-dimensional semantic encoder features are compressible, we introduce a Semantic Bottleneck that compresses them into 128 dimensions, regularized by the proposed time-relation loss for temporal feature consistency. We further design a dual-level semantic supervision method that leverages both high- and low-dimensional semantic signals, enabling the tokenizer to jointly capture semantics and acoustic details within a compact latent space. Experiments on speech, music, and general audio show that SemBo preserves strong low-dimensional semantic capacity and LoSATok retains competitive understanding performance compared with several semantic representations, while consistently improving DiT modeling performance on speech, music, and audio generation. These results demonstrate that LoSATok's low-dimensional representations can effectively support audio understanding and generation. Our code is provided at https://github.com/wxzyd123/LoSATok.

2605.27836 2026-05-28 cs.CR cs.AI

Symmetry Defeats Auditing

对称性击败审计

Nick Merrill, Zeke Medley

AI总结 本文展示了对内省适配器(Shenoy et al., 2026)的一种攻击方法。

详情
AI中文摘要

我们展示了对内省适配器(Shenoy et al., 2026)的一种攻击。

英文摘要

We demonstrate an attack on Introspection Adapters (Shenoy et al., 2026).

2605.27825 2026-05-28 cs.CR cs.LG

MRMMIA: Membership Inference Attacks on Memory in Chat Agents

MRMMIA:聊天代理中记忆的成员推断攻击

Kai Chen, Yan Pang, Tianhao Wang

AI总结 针对聊天代理记忆系统,提出一种利用多次召回探针的统一成员推断攻击方法MRMMIA,在黑盒、灰盒和白盒设置下均优于基线,揭示了代理中的隐私风险。

Comments This work investigates the MIA on chat agent memory

详情
AI中文摘要

成员推断攻击(MIAs)测试目标数据记录是否属于系统的私有数据,并已成为衡量机器学习系统隐私泄露的标准工具。先前的工作主要集中在训练语料库或检索数据库上。然而,针对代理记忆的MIAs受到的关注较少,尽管这种记忆可能包含敏感的用户-代理交互、检索到的事实和用户偏好。因此,在这项工作中,我们专注于聊天代理记忆MIAs,其中对手推断候选记忆单元是否属于聊天代理的记忆存储。我们提出了多召回记忆MIA(MRMMIA),这是一种统一的攻击,利用对代理的多次召回探针来提取黑盒、灰盒和白盒设置中的成员信号。我们的实验表明,MRMMIA始终优于基线。我们的结果暴露了代理中的隐私风险,并为聊天代理记忆系统中的成员泄漏提供了初步评估框架。

英文摘要

Membership inference attacks (MIAs) test whether a target data record belongs to a system's private data, and have become a standard tool to measure privacy leakage in machine learning systems. Prior work has primarily focused on training corpora or retrieval databases. However, MIAs against agent memory have received less attention, even though such memory can contain sensitive user-agent interactions, retrieved facts, and user preferences. Therefore, in this work, we focus on chat agent memory MIAs, where an adversary infers whether a candidate memory unit belongs to the chat agent's memory store. We propose Multi-Recall Memory MIA (MRMMIA), a unified attack that utilizes multiple recall probes to the agent to extract the membership signal across black-box, gray-box, and white-box settings. Our experiments demonstrate that MRMMIA consistently outperforms baselines. Our results expose the privacy risk in agents and provide an initial evaluation framework for membership leakage in chat-agent memory systems.

2605.27823 2026-05-28 cs.CR cs.AI cs.CV

Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Security

解耦对抗性提示:基于语义图的鲁棒大语言模型安全防御

Xiang Fang, Wanlong Fang

AI总结 提出对抗性提示解耦(APD)框架,通过互信息语义分解、图谱分析和轻量级分类器,在输入处理前识别并中和恶意组件,将有害输出减少85%以上。

Comments Published in AAAI 2026

详情
AI中文摘要

大语言模型(LLMs)越来越容易受到利用语义歧义绕过安全机制的对抗性提示的攻击,导致有害或不适当的输出。此类攻击,包括越狱和提示注入,对安全关键应用中LLMs的完整性和可用性构成重大风险。本文提出对抗性提示解耦(APD)框架,一种新颖的防御机制,在输入提示被LLM处理之前主动识别并中和其中的恶意组件。APD框架集成了三项关键创新:(1)基于互信息的语义分解方法,用于分离对抗性和良性提示组件,确保统计独立性;(2)基于图的意图分类方法,利用谱分析检测提示语义中的恶意模式;(3)轻量级基于Transformer的分类器,在真实世界的毒性和越狱提示数据集上训练,实现高效准确的对抗性意图检测。在包含对抗性提示的多样化数据集上评估,APD展现出卓越的鲁棒性,将有害输出生成减少超过85%,同时保持对模型性能的 negligible 影响。该框架的计算效率支持实时部署,使其成为保护LLMs的实用解决方案。我们的工作解决了机器学习安全中关于新型攻击和ML系统完整性方法的关键挑战,并提供了一种可扩展、符合伦理的防御手段来对抗基于提示的对抗性威胁。

英文摘要

Large Language Models (LLMs) are increasingly vulnerable to adversarial prompts that exploit semantic ambiguities to bypass safety mechanisms, resulting in harmful or inappropriate outputs. Such attacks, including jailbreaking and prompt injection, pose significant risks to the integrity and availability of LLMs in security-critical applications. This paper proposes the Adversarial Prompt Disentanglement (APD) framework, a novel defense mechanism that proactively identifies and neutralizes malicious components in input prompts before they are processed by the LLM. The APD framework integrates three key innovations: (1) a mutual information-based semantic decomposition method to isolate adversarial and benign prompt components, ensuring statistical independence; (2) a graph-based intent classification approach that leverages spectral analysis to detect malicious patterns in prompt semantics; and (3) a lightweight transformer-based classifier trained on real-world datasets of toxic and jailbreaking prompts, enabling efficient and accurate adversarial intent detection. Evaluated on diverse datasets containing adversarial prompts, APD demonstrates superior robustness, reducing harmful output generation by over 85\% while maintaining negligible impact on model performance. The framework's computational efficiency supports real-time deployment, making it a practical solution for securing LLMs. Our work addresses critical challenges in machine learning security on novel attacks and integrity methods for ML systems, and offers a scalable, ethically grounded defense against prompt-based adversarial threats.

2605.27796 2026-05-28 eess.IV cs.CV cs.LG eess.SP stat.AP

Benchmarking Ultrasound Foundation Models for Fetal Plane Classification

超声基础模型在胎儿平面分类中的基准测试

Leya Barrientos, Yuexi Du, Nicha C. Dvornek

AI总结 本研究对四种超声基础模型(USFM、MOFO、UltraSAM、FetalCLIP)在胎儿平面分类任务上进行基准测试,发现FetalCLIP在线性探测设置中表现最佳,而USFM在全微调设置中表现最佳,且预训练目标显著影响迁移性能。

详情
AI中文摘要

超声因其安全性、可及性和实时成像能力被广泛应用于产科护理。然而,其解读仍依赖操作者,且易受噪声和伪影影响。深度学习模型在解决这些问题上表现出色,但通常需要大量标注数据集,这在临床超声中难以获得。基础模型(FMs)提供了一种替代方案,利用大量超声图像学习可迁移的表征,从而在有限标注数据下实现泛化。本文针对胎儿平面分类任务,对超声专用基础模型进行了全面基准测试。我们评估了四种超声基础模型(USFM、MOFO、UltraSAM、FetalCLIP),并与两个CNN基线(ResNet50、EfficientNet-V2)以及一个在自然图像上预训练的ViT(DINOv3)进行比较。我们在两种互补设置下训练所有模型:全微调和冻结编码器的线性探测。所有模型均使用西班牙胎儿超声数据集进行5折患者级交叉验证训练,并在域内数据和外部非洲队列上测试,以评估跨人群泛化能力。我们发现,FetalCLIP在线性探测设置中取得最佳结果(域内F1=0.9261,域外F1=0.9731),而USFM在全微调设置中表现最佳(域内F1=0.9476,域外F1=0.9515)。MOFO和UltraSAM在两种设置中性能下降最多,在某些情况下甚至不如自然图像预训练模型。这些发现强调了预训练模型的选择对胎儿平面分类性能的显著影响,因为不同的预训练目标导致不同的迁移能力。

英文摘要

Ultrasound is widely used in obstetric care due to its safety, accessibility, and real-time imaging. However, interpretation remains operator-dependent and susceptible to noise and artifacts. Deep learning models have shown strong performance to solve these problem, but they typically require large annotated datasets that are difficult to obtain in clinical ultrasound. Foundation models (FMs) offer an alternative, using a large number of ultrasound images to learn transferable representations that can generalize with limited labeled data. This work presents a comprehensive benchmark of ultrasound-specific FMs for fetal plane classification. We evaluated four ultrasound FMs (USFM, MOFO, UltraSAM, FetalCLIP) against two CNN baselines (ResNet50, EfficientNet-V2) and a ViT (DINOv3) pretrained on natural images. We trained all models under two complementary settings: full fine-tuning and linear probing with a frozen encoder. All models were trained using 5-fold patient-level cross-validation on a Spanish fetal ultrasound dataset and tested on both in-domain data and an external African cohort to assess cross-population generalization. We found that FetalCLIP achieved the best results in the linear probing setting (F1 = 0.9261 for in-domain, F1 = 0.9731 for out-of-domain), while USFM performed best in the full fine-tuning setting (F1 = 0.9476 for in-domain, F1 = 0.9515 for out-of-domain). MOFO and UltraSAM degraded most in both settings, underperforming natural image pretrained models in some cases. These findings highlight how the choice of pretrained model strongly affects fetal plane classification performance, since different pretraining objectives lead to different levels of transferability.

2605.27794 2026-05-28 stat.ML cs.LG stat.ME

Learning to target with network interference

在网络干扰下学习目标定位

Xiaomeng Wang, Hamsa Bastani, Osbert Bastani, Zhimei Ren

AI总结 研究在bandit设置下网络干扰中的自适应目标定位,通过线性模型和稀疏假设,针对不同干扰结构知识水平提出近最优遗憾算法。

详情
AI中文摘要

本文研究在bandit设置下网络干扰中的自适应目标定位,其中对一个个体的处理可能通过溢出效应影响他人。我们考虑稀疏场景下的线性模型,每个个体的结果最多受少数其他人影响。首先建立遗憾下界,表明忽略网络结构并将问题简化为标准线性bandit必然导致低效学习,尤其是在大规模群体中。为了理解如何利用结构信息,我们分析了干扰结构知识水平不同的场景:(1) 完全支持知识,(2) 列支持大小知识,(3) 无先验知识。对于每种场景,我们建立了表征学习基本极限的遗憾下界,并开发了实现近最优遗憾的算法。总之,我们的结果提供了干扰结构知识如何影响在线学习效率的统一视角,并在每种设置下提供了实用的自适应目标定位算法。在合成和真实数据上的数值实验证明了我们算法的实际优势。

英文摘要

This paper studies adaptive targeting under network interference in a bandit setting, where treatments applied to one individual may affect others through spillover effects. We consider a linear model in a sparse regime, where each individual's outcome can be affected by at most a few others. We first establish a regret lower bound showing that ignoring the network structure and reducing the problem to a standard linear bandit inevitably leads to inefficient learning, particularly in large populations. To understand how structural information can be leveraged, we analyze regimes with varying levels of knowledge of the interference structure: (1) full support knowledge, (2) knowledge of the column support sizes, and (3) no prior knowledge. For each regime, we establish regret lower bounds characterizing the fundamental limits of learning, and develop algorithms that achieve near-optimal regret. Together, our results provide a unified view of how knowledge of the interference structure governs the efficiency of online learning under interference, and offer practical adaptive targeting algorithms in each setting. Numerical experiments on synthetic and real-world data demonstrate the practical benefits of our algorithms.

2605.27787 2026-05-28 cs.MA cs.CL

Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems

图书馆员万岁!面向节能多智能体软件工程系统的持久搜索子智能体

Seunghyuk Cho, Sunghyun Choi, Jaeseung Heo, Youngbin Choi, Saemi Moon, MoonJeong Park, Dongwoo Kim

AI总结 针对多智能体系统中输出令牌冗余导致的高能耗问题,提出持久搜索子智能体Librarian,通过跟踪仓库搜索历史并抑制重复探索,减少输出令牌量,在SWE-Bench Verified上实现高达25%的GPU能耗降低且不损失任务性能。

Comments 19 pages, 4 figures, 12 tables

详情
AI中文摘要

多智能体系统(MAS)显著推进了自主软件工程(SWE),但其日益增长的推理能量需求引发了可持续性问题。在本文中,我们证明这种成本集中在一个被忽视的来源上:跨智能体生成的冗余输出令牌。两个实证发现支持了这一主张。首先,我们对MAS的每令牌能量归因揭示了一个显著的不对称性:一个输出令牌消耗的能量是输入或缓存令牌的30到1000倍。其次,MAS增加了每回合输出量,因为智能体反复重新探索重叠的仓库区域。为了解决这一低效问题,我们提出了Librarian,一个持久搜索子智能体,它跟踪仓库搜索历史并抑制跨智能体的冗余探索动作。通过返回文件区域的简短引用而不是完整的文件摘录,Librarian进一步减少了输出令牌量。在SWE-Bench Verified上,Librarian将现有多智能体SWE系统的每回合GPU能耗降低了高达25%,同时保持了任务性能。

英文摘要

Multi-agent systems (MAS) have substantially advanced autonomous software engineering (SWE), but their growing inference energy demands raise sustainability concerns. In this paper, we demonstrate that this cost is concentrated in an overlooked source: redundant output tokens generated across agents. Two empirical findings ground this claim. First, our per-token energy attribution for MAS reveals a sharp asymmetry: an output token consumes 30 to 1,000 times more energy than an input or cached token. Second, MAS inflate per-episode output because agents repeatedly re-explore overlapping repository regions. To address this inefficiency, we propose Librarian, a persistent search sub-agent that tracks repository-search history and suppresses redundant exploration actions across agents. By returning short references to file regions instead of full file excerpts, Librarian further reduces output-token volume. On SWE-Bench Verified, Librarian reduces per-episode GPU energy consumption of existing multi-agent SWE systems by up to 25% while preserving task performance.

2605.27769 2026-05-28 cs.DS cs.IT cs.LG math.IT stat.ML

Smoothed Score Queries and the Complexity of Sampling

平滑得分查询与采样的复杂度

Jingbo Liu

AI总结 本文研究利用梯度信息从高维高斯分布中采样的查询复杂度,通过引入平滑得分查询(即高斯卷积密度的对数梯度)将条件数依赖从√κ降低到对数级别,并给出近乎匹配的上下界。

详情
AI中文摘要

我们研究利用梯度信息从高维高斯分布中采样的查询复杂度。在标准预言机模型中,精确梯度仅暴露与精度矩阵的矩阵-向量乘积,导致多项式逼近障碍和特征性的条件数√κ依赖。我们证明,当允许采样器查询\emph{平滑得分}(即高斯卷积密度的对数梯度)时,这一障碍消失。对于精度矩阵为Λ的高斯目标,噪声水平τ下的平滑得分查询可访问预解式(Λ+τ^{-1}I)^{-1}。将几何间隔的噪声水平与sinc求积有理逼近相结合,我们得到一个采样器,其总变分误差δ_{TV}所需的平滑得分查询次数为q=O\!\left(igl(\logκ+\log(e\sqrt d/δ_{ m TV})igr)\log(e\sqrt d/δ_{ m TV}) ight),将条件数依赖从√κ改进为对数依赖。我们还研究了有限比特梯度预言机。通过对变换后的平滑得分答案进行坐标量化并添加最终抖动步骤,我们得到一个采样方案,其总通信梯度信息在κ中为多对数级别;特别地,对于固定维度和精度,比特复杂度为O(\log^2κ)。为补充这些上界,我们引入一种信道合成(或反向香农)逆技术用于采样下界。这将总变分模拟保证转化为通信需求,并得到所需梯度信息的Ω(\logκ)下界。综合这些结果,我们识别出平滑得分作为采样中可证明信息更丰富的预言机,并为其有限比特复杂度给出了近乎匹配的上下界。

英文摘要

We study the query complexity of sampling from high-dimensional Gaussian distributions using gradient information. In the standard oracle model, exact gradients expose only matrix-vector products with the precision matrix, leading to polynomial approximation barriers and a characteristic \(\sqrtκ\) dependence on the condition number. We show that this barrier disappears when the sampler is allowed to query \emph{smoothed scores}, namely gradients of the logarithms of the Gaussian-convolved densities. For a Gaussian target with precision matrix \(Λ\), a smoothed-score query at noise level \(τ\) gives access to the resolvent \((Λ+τ^{-1}I)^{-1}\). Combining geometrically spaced noise levels with sinc-quadrature rational approximation, we obtain a sampler with $q=O\!\left(\bigl(\logκ+\log(e\sqrt d/δ_{\rm TV})\bigr)\log(e\sqrt d/δ_{\rm TV})\right)$ smoothed-score queries for total variation error \(δ_{\rm TV}\), improving the condition-number dependence from \(\sqrtκ\) to logarithmic. We also study finite-bit gradient oracles. Using coordinatewise quantization of the transformed smoothed-score answers and a final dithering step, we obtain a sampling scheme whose total communicated gradient information is polylogarithmic in \(κ\); in particular, for fixed dimension and accuracy, the bit complexity is \(O(\log^2κ)\). To complement these upper bounds, we introduce a channel-synthesis, or reverse-Shannon, converse technique for sampling lower bounds. This converts total-variation simulation guarantees into communication requirements and yields an \(Ω(\logκ)\) lower bound on the required gradient information. Together, these results identify smoothed scores as a provably more informative oracle for sampling and give nearly matching upper and lower bounds for its finite-bit complexity.

2605.27756 2026-05-28 physics.flu-dyn cs.LG cs.NA math.DS math.NA

Sparse POD Mode Selection and Manifold Dimensionality Reduction with Neural Networks

稀疏POD模态选择与神经网络流形降维

Tomoki Koike, Prakash Mohan, Marc T. Henry de Frahan, Elizabeth Qian, Julie Bessac

AI总结 提出SparseModesNet框架,通过LassoNet实现线性POD模态的稀疏选择与非线性神经网络解码,在平流主导和混沌流中降低重构误差51-78%。

详情
AI中文摘要

高性能计算能够模拟高维物理系统,但反问题和控制等下游分析仍计算昂贵,这促使模型降阶(MOR)构建高效的低维代理。本征正交分解(POD)是一种广泛采用的数据驱动MOR方法,将动力学投影到由最能量模态张成的线性子空间上。然而,POD在处理Kolmogorov n-宽度缓慢衰减的问题(如平流主导和湍流)时表现不佳,需要大量模态才能准确重构。此外,基于能量的选择可能会丢弃捕捉小尺度特征所需的关键低能量模态。最近使用交替或贪婪模态选择的多项式映射的非线性流形方法以更少的模态实现了更好的重构。然而,这些方法先验地固定了非线性映射形式,限制了表达能力。相反,神经网络(NN)流形提供了更强的表达能力,但采用基于能量的选择。我们提出了SparseModesNet,一种降维框架,通过POD模态进行线性编码和NN非线性解码。解码器利用LassoNet——一种通过带线性跳跃层的残差连接强制执行层次稀疏性的方法——同时选择信息丰富的POD模态并学习最小化重构误差的非线性映射。在基准平流主导和混沌流中,SparseModesNet达到或超过了最先进的性能。对于摩擦雷诺数Re_τ=5200的湍流槽道流,与现有的多项式流形方法相比,我们将重构误差降低了51-78%,同时通过物理上有意义的模态选择保持了可解释性。

英文摘要

High-performance computing enables simulation of high-dimensional physical systems, but downstream analyses such as inverse problems and control remain computationally expensive, motivating model order reduction (MOR) to construct efficient low-dimensional surrogates. Proper Orthogonal Decomposition (POD), a widely adopted data-driven MOR method, projects dynamics onto linear subspaces spanned by the most energetic modes. However, POD struggles for problems with slowly decaying Kolmogorov \(n\)-widths, such as advection-dominated and turbulent flows, requiring many modes for accurate reconstruction. Moreover, energy-based selection can discard crucial low-energy modes needed to capture small-scale features. Recent nonlinear manifold methods using polynomial mappings with alternating or greedy mode selection achieve better reconstruction with fewer modes. However, these methods fix the nonlinear mapping form a priori, limiting expressivity. Conversely, neural network (NN) manifolds offer greater expressivity but employ energy-based selection. We present SparseModesNet, a dimensionality reduction framework that employs linear encoding via POD modes and nonlinear NN decoding. The decoder leverages LassoNet, a method enforcing hierarchical sparsity through residual connections with linear skip layers, to simultaneously select informative POD modes and learn a nonlinear mapping that minimizes reconstruction error. On benchmark advection-dominated and chaotic flows, SparseModesNet matches or exceeds state-of-the-art performance. For turbulent channel flow at friction Reynolds number \(Re_τ=5200\), we reduce reconstruction error by 51--78\% compared to existing polynomial manifold methods while maintaining interpretability through physically meaningful mode selection.

2605.27747 2026-05-28 stat.ML cs.LG stat.CO

Soft Specialists: $α$-Rényi Ensembles for Uncertainty-Aware LLM Post-Training

软专家:用于不确定性感知的LLM后训练的$\alpha$-Rényi集成

Paula Cordero-Encinar, Georgy Tyukin, Andrew B. Duncan

AI总结 提出一种$\alpha$-Rényi变分框架,通过学习后训练参数的分布来替代深度集成,实现不确定性感知的LLM后训练,并支持软路由和模型专业化。

详情
AI中文摘要

现有的大语言模型训练方法基于大量数据学习单一参数集,这些数据通常异构、冲突且往往直接矛盾。因此,模型被迫将冲突目标和固有不确定性压缩为单一的平均行为模式。我们提出了一种$\alpha$-Rényi变分框架,用于学习后训练参数的分布,为深度集成方法提供了一种不确定性感知的替代方案。得到的变分目标在经典变分贝叶斯和预测导向的后验学习之间插值,平衡全局合理的个体模型与互补专家系统。我们确定了局部稳定性准则,展示了模型误设如何使非退化后验扩散局部有利,将矛盾或冲突数据表现为认知不确定性。我们将该框架应用于LLM后训练,学习附着在共享冻结基模型上的LoRA适配器集成,为监督微调和偏好优化提供了可扩展的训练过程。我们的方法使得训练示例能够被软路由到集成成员之间,促进模型专业化,并为不同任务提供可操作的不确定性估计。

英文摘要

Existing training approaches for large language models learn a single set of parameters, based on large volumes of data, which is typically heterogeneous, conflicting and often outright contradictory. As a result, the model is forced to compress conflicting goals, and inherent uncertainties into a single, averaged pattern of behaviour. We propose an $α$-Rényi variational framework for learning distributions over post-training parameters, offering an uncertainty-aware alternative to deep ensemble approaches. The resulting variational objective interpolates between classical variational Bayes and predictively oriented posterior learning, balancing between globally plausible individual models against systems of complementary specialists. We identify local stability criteria, demonstrating how model misspecification can make non-degenerate posterior spread locally favourable, manifesting contradictory or conflicting data as epistemic uncertainty. We apply our framework to LLM post-training, learning an ensemble of LoRA adapters attached to a shared, frozen base model, providing a scalable training procedure for both supervised fine-tuning and preference optimisation. Our approach enables training examples to be softly routed across ensemble members, promoting model specialisation and providing actionable uncertainty estimates across different tasks.

2605.27725 2026-05-28 physics.flu-dyn cs.LG

CFDTwin: An open-source GUI and Python toolkit for POD-NN surrogate modeling of ANSYS Fluent simulations

CFDTwin:用于ANSYS Fluent模拟的POD-NN代理建模的开源GUI和Python工具包

Daniel Curl, Han Hu

AI总结 本文介绍CFDTwin,一个开源Python包和桌面GUI,将参数采样、Fluent自动化、数据提取、降阶模型构建、神经网络训练、验证和预测封装为可重用工作流,用于ANSYS Fluent模拟的POD-NN代理建模。

Comments 9 pages, 3 figures

详情
AI中文摘要

高保真计算流体动力学(CFD)广泛用于热流体设计,但重复的CFD求解对于设计优化、不确定性分析和数字孪生工作流仍然昂贵。最近,我们的团队已经证明,适当的正交分解和神经网络(POD-NN)代理可以预测电子冷却冷板中的二维热场,具有较大的推理加速,同时保留物理可解释的模态结构。然而,重现和扩展此类工作流通常需要自定义脚本用于参数采样、Fluent自动化、数据提取、降阶模型构建、神经网络训练、验证和预测。本文介绍了CFDTwin,一个开源Python包和可选的桌面图形用户界面(GUI),将这些步骤打包成用于ANSYS Fluent模拟的可重用工作流。CFDTwin允许用户定义模拟输入和输出量,生成实验设计样本,运行和恢复Fluent批处理模拟,训练用于标量、表面场和单元区域输出的POD-NN代理模型,检查验证指标,并在不重新运行Fluent的情况下评估新设计点上的训练模型。相同的工作流通过可脚本化的Python API和GUI暴露,支持可重复研究、面向用户的模型验证和自动化设计探索。CFDTwin将先前的POD-NN建模研究从特定案例的研究实现扩展到用于CFD代理建模和数字孪生开发的可重用研究软件平台。

英文摘要

High-fidelity computational fluid dynamics (CFD) is widely used for thermal-fluid design, but repeated CFD solves remain expensive for design optimization, uncertainty analysis, and digital-twin workflows. Recently, our team has demonstrated that a proper orthogonal decomposition and neural-network (POD-NN) surrogate can predict two-dimensional thermal fields in an electronics-cooling cold plate with large inference speedups while preserving physically interpretable modal structure. Reproducing and extending such workflows, however, typically requires custom scripts for parameter sampling, Fluent automation, data extraction, reduced-order model construction, neural-network training, validation, and prediction. This paper introduces CFDTwin, an open-source Python package and optional desktop graphical user interface (GUI) that packages these steps into a reusable workflow for ANSYS Fluent simulations. CFDTwin allows users to define simulation inputs and output quantities, generate design-of-experiments samples, run and resume Fluent batch simulations, train POD-NN surrogate models for scalar, surface-field, and cell-zone outputs, inspect validation metrics, and evaluate trained models at new design points without re-running Fluent. The same workflow is exposed through a scriptable Python API and a GUI, supporting reproducible studies, user-facing model validation, and automated design exploration. CFDTwin extends the prior POD-NN modeling study from a case-specific research implementation to a reusable research-software platform for CFD surrogate modeling and digital-twin development.

2605.27718 2026-05-28 math.ST cs.LG stat.ME stat.TH

Robust Moment-Based Estimation via Spectral Gradient Reweighting

基于谱梯度重加权的稳健矩估计

Liu Zhang, Amit Singer

AI总结 提出SGR-GMM算法,通过谱梯度重加权对观测梯度进行软重加权,实现稳健的广义矩估计,并给出理论保证和实验验证。

详情
AI中文摘要

基于矩的估计是参数推断在理论上具有吸引力的方法,尤其是在基于似然的估计不可用、设定错误或计算不便时。然而,矩方程涉及样本均值,这使得基于矩的估计对异常值敏感。我们提出了SGR-GMM算法,这是一种稳健的广义矩估计(GMM)程序,它使用谱梯度重加权(SGR)原语在矩匹配优化过程中对每个观测的梯度进行软重加权。我们的分析分为三层。首先,对于固定中心,SGR原语被表述为样本权重玩家和密度矩阵玩家之间的熵正则化谱博弈,并使用经典的多重权重和矩阵多重权重遗憾界进行分析。其次,我们建立了SGR原语中固定中心更新的显式收敛半径和有限终止界。第三,我们证明了局部有限样本参数估计误差界,该界显式依赖于污染比例、内点梯度稳定性、局部GMM识别强度和优化精度。我们进一步特化SGR-GMM算法,以获得稳健的对角加权GMM(DGMM)估计量,用于估计在加性高斯噪声和强污染下观测到的异方差低秩高斯混合模型。在数值实验中,SGR原语产生近乎神谕的梯度估计,而稳健的DGMM特化显著优于非稳健的矩基线。代码和数据可在https://github.com/liu-lzhang/sgr-gmm获取。

英文摘要

Moment-based estimation is a theoretically attractive approach to parametric inference, especially when likelihood-based estimation is unavailable, misspecified, or computationally inconvenient. However, the moment equations involve sample averages, which makes moment-based estimation sensitive to outliers. We propose the SGR-GMM algorithm, a robust generalized method of moments (GMM) procedure that uses a spectral gradient reweighting (SGR) primitive to soft-reweight the per-observation gradients during the moment-matching optimization. Our analysis has three layers. First, for a fixed center, the SGR primitive is formulated as an entropy-regularized spectral game between a sample-weight player and a density-matrix player, which is analyzed using classical multiplicative-weights and matrix-multiplicative-weights regret bounds. Second, we establish explicit convergence radius and finite termination bound for the fixed-center updates in the SGR primitive. Third, we prove a local finite-sample parameter estimation error bound with explicit dependence on the contamination fraction, inlier gradient stability, local GMM identification strength, and optimization accuracy. We further specialize the SGR-GMM algorithm to obtain a robust diagonally-weighted GMM (DGMM) estimator for estimating heteroscedastic low-rank Gaussian mixtures observed under additive Gaussian noise and strong contamination. In the numerical experiments, the SGR primitive produces nearly-oracle gradient estimation and the robust DGMM specialization substantially improves over non-robust moment baselines. The code and data are available at https://github.com/liu-lzhang/sgr-gmm.

2605.27700 2026-05-28 cs.DL cs.AI

CiteCheck: Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text

CiteCheck: 基于检索的科学文本中LLM引用幻觉检测

Khashayar Khajavi, Shaghayegh Sadeghi, Rise Adhikari, Alexander Tessier

AI总结 提出CiteCheck框架,通过从外部学术来源检索候选出版物、使用结构化LLM验证器比较引用与候选信息,并将验证器得分映射为精确、次要和主要三个标签,以检测LLM生成的引用幻觉,在物理基准上达到88.7 macro-F1和88.9%准确率。

详情
AI中文摘要

大型语言模型(LLM)越来越多地用于生成科学报告,但它们可能产生看似合理但包含损坏元数据或指向不存在论文的引用。我们引入了CiteCheck,一个用于引用幻觉检测的混合框架,它验证引用是否对应于真实的学术工作以及其元数据是否忠实于该工作。CiteCheck从外部学术来源检索候选出版物,使用结构化LLM验证器将引用与检索到的候选进行比较,并将验证器得分映射为三个标签:精确、次要和主要。我们还构建了一个包含982个引用的物理基准,具有受控的损坏,这些损坏捕获了细微的元数据漂移和完全捏造的引用。在保留测试集上,CiteCheck达到了88.7 macro-F1和88.9%的准确率,优于GPT、Claude和Gemini基线,包括网络搜索和少样本变体。这些结果表明,可靠的引用验证受益于结合学术检索、基于结构化LLM的比较和校准的决策规则。

英文摘要

Large language models (LLMs) are increasingly used to generate scientific reports, but they can produce references that appear plausible while containing corrupted metadata or pointing to papers that do not exist. We introduce CiteCheck, a hybrid framework for citation hallucination detection that verifies whether a citation corresponds to a real scholarly work and whether its metadata is faithful to that work. CiteCheck retrieves candidate publications from external scholarly sources, compares the citation against the retrieved candidate using a structured LLM verifier, and maps verifier scores into three labels: Exact, Minor, and Major. We also construct a 982-citation physics benchmark with controlled corruptions that capture both subtle metadata drift and fully fabricated references. On the held-out test set, CiteCheck achieves 88.7 macro-F1 and 88.9% accuracy, outperforming GPT, Claude, and Gemini baselines, including web-search and few-shot variants. These results show that reliable citation verification benefits from combining scholarly retrieval, structured LLM-based comparison, and calibrated decision rules.

2605.27679 2026-05-28 cond-mat.soft cs.CV cs.LG

On the Equivariant Learning of the $Q$-tensor Order Parameter

$Q$ 张量序参数的等变学习

Julia Navarro, Mark Wilkinson

AI总结 本文构建并评估了群等变神经网络,用于从合成生成的微观纹理预测向列液晶的二维 $Q$ 张量序参数,发现等变模型相比非等变基准具有更低的误差和更强的泛化能力。

Comments 15 pages (excluding 7-page appendix); 6 figures

详情
AI中文摘要

我们构建并评估了群等变神经网络,用于从合成生成的微观纹理预测向列液晶的二维 $Q$ 张量序参数。使用权重共享约束、等变激活和正则化技术的组合,构建了七个等变于 $k=4,8,16,32,64,128,256$ 阶循环群 $C_k$ 的架构。为此,我们构造了旋转类置换矩阵群,其元素 $\varrho_{C_k}(g)$ 作用于按行向量化的图像,从而近似方形图像上圆形子域的 $\frac{2\pi}{k}$ 旋转。我们展示了所有七个等变模型在单精度浮点精度内满足 $Q$ 张量等变性约束。与近似参数匹配的非等变基准(有或没有数据增强)相比,我们发现等变模型始终实现更低的误差,并且对未见过的缺陷配置具有更强的泛化能力。性能随群阶增加而提高,表明纳入更精细的旋转对称性会导致更低的误差。

英文摘要

We construct and evaluate group-equivariant neural networks for the prediction of the two-dimensional $Q$-tensor order parameter of nematic liquid crystals from synthetically generated microscopic textures. Seven architectures, equivariant to cyclic groups $C_k$ of order $k$ for $k=4,\,8,\,16,\,32,\,64,\,128,\, 256$, are built using a combination of weight-sharing constraints, equivariant activations and regularization techniques. To do this, we construct rotation-like permutation matrix groups with elements $\varrho_{C_k}(g)$ that act on row-wise vectorized images, thereby approximating a $\frac{2π}{k}$ rotation of the circular subdomain on square images. We show that all seven equivariant models satisfy the $Q$-tensor equivariance constraint to within single-precision floating point accuracy. Comparing against approximate parameter-matched non-equivariant benchmarks, with and without data augmentation, we find that the equivariant models consistently achieve lower errors and generalize more robustly to unseen defect configurations. Performance increases with group order, suggesting that the incorporation of finer rotational symmetry leads to lower errors.

2605.27676 2026-05-28 stat.ML cs.LG

Unsupervised Identification and Removal of Spurious Correlations During Fine-Tuning

微调过程中虚假关联的无监督识别与消除

Ciarán M. Gilligan-Lee, Joseph Egan, Yuchen Zhu, Michael O'Riordan

AI总结 提出GRASP方法,通过梯度投影在微调时无监督识别并消除与任务纠缠的虚假关联,同时保留预训练知识,在三个任务上优于基线。

Comments 10 + 4 pages, comments welcome

详情
AI中文摘要

在精心策划的数据集上微调预训练语言模型可能会在微调任务与无意中的潜在因素(如不对齐的人物角色或政治倾向)之间产生虚假关联,而这些因素是由策划过程与任务纠缠在一起的。模型可能会依赖这些虚假关联,导致偏差并降低分布外泛化能力。我们证明,在任务复杂性和虚假关联的合理假设下,可以从朴素LoRA微调的权重中无监督地识别这些潜在因素。现有的消除偏差方法(如激活引导)在推理或训练期间从残差流激活中移除已识别的因素。然而,我们认为目标应该是消除虚假关联,而不是潜在因素本身,因为预训练模型可能依赖该因素来获取真实的任务信号。为此,我们提出GRASP(关联虚假模式的梯度投影),该方法防止模型对已识别的潜在因素产生新的依赖,同时保留沿该方向的任何预训练内容。我们在三个微调任务上进行了验证。前两个涉及紧急不对齐,即在狭窄任务(在我们的案例中,编写不安全的代码和给出糟糕的医疗建议)上进行微调会导致在无关话题上产生不对齐的响应。在这里,我们的方法在不安全代码案例中完全消除了不对齐,在糟糕医疗建议案例中减少了约5倍,在不对齐减少与任务保持之间的权衡中击败了所有基线。最后一个是新颖的政治偏见实验,即在右倾的Reddit金融建议数据上进行微调会导致在无关话题上产生政治倾向漂移。在这里,我们的方法将漂移减少了一半以上,同时提高了金融任务性能,击败了所有基线。

英文摘要

Fine-tuning a pretrained language model on a curated dataset can produce spurious correlations between the fine-tuning task and unintended latent factors -- such as misaligned personas or political slant -- that the curation procedure has entangled with the task. The model can latch onto these spurious correlations, leading to bias and reduced out-of-distribution generalisation. We prove that under reasonable assumptions on task complexity and the spurious correlation, such latent factors can be identified, without supervision, from the weights of a naive LoRA fine-tune. Existing approaches to removing bias, such as activation steering, remove identified factors from residual-stream activations, either at inference or during training. We argue, however, that the goal should be to remove the spurious correlation, not the latent factor itself, as the pretrained model may rely on it for genuine task signal. To enable this, we propose GRASP, GRadient projection of Associated Spurious Patterns, which prevents the model from acquiring new reliance on the identified latent factor while preserving any pretrained content along it. We validate on three fine-tuning tasks. The first two involve emergent misalignment, where fine-tuning on a narrow task -- in our case, writing insecure code and giving bad medical advice -- leads to misaligned responses on unrelated topics. Here our method completely removes misalignment in the insecure code case and reduces them by ~5x in the bad medical advice case, beating all baselines in the trade-off between misalignment-reduction and task-preservation. The last is a novel political-bias experiment, where fine-tuning on right-skewed Reddit financial-advice data causes political-lean drift on unrelated topics. Here our method reduces drift by more than half, while improving financial task performance, beating all baselines.

2605.27674 2026-05-28 cs.CR cs.AI cs.LG

Backdoor Attacks on Fault Detection and Localization in Cyber-Physical Systems

针对信息物理系统中故障检测与定位的后门攻击

Abile Jean, Kuniyilh S

AI总结 本文研究针对现代信息物理系统中基于机器学习的故障检测与定位机制的后门攻击,通过设计触发器并评估攻击成功率,实验表明即使仅投毒10%的数据也能成功实施攻击。

详情
AI中文摘要

信息物理系统(CPS)集成了传感、通信、计算和控制,以支持关键基础设施,包括智能电网、工业自动化和控制系统。在电力公用事业领域,CPS中使用各种控制器来确保系统检测和恢复故障(如电压波动),并在配电系统中进行负载平衡。基于机器学习和深度学习的故障检测与定位框架因其能够实时识别异常和操作故障,近年来在CPS中受到广泛关注。然而,这些智能模型容易受到对抗性机器学习攻击,尤其是后门攻击。在后门攻击中,对手将恶意模式注入训练数据,使得模型在大多数情况下表现正常,但当触发特定模式时产生攻击者控制的输出。本文研究了针对现代CPS系统中最新机器学习管道的故障检测与定位机制的后门攻击威胁。我们定义了这些威胁,并通过设计触发器以及在CPS领域评估其成功率来探索如何实现这些攻击。我们的实验表明,即使仅投毒10%的数据,攻击也能成功。

英文摘要

Cyber-Physical Systems (CPS) integrate sensing, communication, computation, and control to support critical infrastructure, including smart grids, industrial automation, and control systems. In the electrical utility domain, various controllers are used in CPS to ensure the system detects and recovers from faults, such as voltage fluctuations, and to perform load balancing in distribution systems. Machine learning- and deep learning-based fault detection and localization frameworks have recently gained significant attention in CPS for their ability to identify anomalies and operational failures in real time. However, these intelligent models are vulnerable to adversarial machine learning attacks, particularly backdoor attacks. In a backdoor attack, an adversary injects malicious patterns into the training data so that the model behaves normally most of the time but produces attacker-controlled outputs when triggered by specific patterns. This paper investigates the threat of backdoor attacks against fault detection and localization mechanisms in recent ML pipelines used in modern CPS systems. We define these threats and explore how they can be realized by designing triggers and evaluating their success in the CPS domain. Our experiments show the attack is successful even with 10\% of poisoning.

2605.27671 2026-05-28 stat.ML cs.LG

Evolving and Detecting Multi-Turn Deception using Geometric Signatures

使用几何特征演化与检测多轮欺骗

Surender Suresh Kumar, Mary L. Cummings

AI总结 提出多目标遗传优化生成多轮欺骗问题集,并利用嵌入空间中的简单几何特征(角覆盖、距离比、线性度)结合轻量级分类器实现高召回率(0.89)的欺骗检测。

详情
AI中文摘要

大型语言模型(LLM)的安全防御通常针对单轮提示进行训练和评估,但实际攻击往往以间接的多轮探测形式展开。为了防御这种更微妙的欺骗形式,我们提出了一种统一流程,通过具有协同进化变异算子的多目标遗传提示优化,生成逼真的多轮欺骗问题集。我们通过人类研究验证了该数据集,该研究还表明,早期生成产生了最令人信服的欺骗,并且存在实际约束,如依从性过滤和顺序效应。利用这些数据,我们能够通过嵌入空间中简单、可解释的几何信号,结合轻量级前馈分类器,检测到试图获取被禁止信息的欺骗行为。三个几何特征(角覆盖、距离比和线性度)加上成对相似性统计,形成了一个紧凑的预测模型,在基础、改写和截断(三轮)场景中持续实现了高召回率(0.89),测试时F1值在0.74-0.86之间。结果支持一个中心假设:多轮欺骗意图会留下稳定的几何足迹,从而能够实现轻量级、透明的筛选,无需昂贵的端到端训练。我们进一步讨论了负责任的使用、局限性以及构建更大、更多样化的人类评估数据集的路径。对人工智能的主要贡献是多目标进化提示生成框架,工程应用是部署用于LLM安全基础设施的轻量级几何检测系统。

英文摘要

Safety defenses for large language models (LLMs) are typically trained and evaluated on single-turn prompts, yet real attacks often unfold as indirect, multi-turn probing. To defend against this more nuanced form of deception, we present a unified pipeline that generates realistic multi-turn deceptive question sets via multi-objective genetic prompt optimization with co-evolving mutation operators. We validate this dataset through a human study, which also revealed that early generations yielded the most convincing deception and practical constraints such as adherence filtering and ordering effects. Using this data, we were able to detect deceptive attempts to access prohibited information using simple, explainable geometric signals in embedding space coupled with a lightweight feed-forward classifier. Three geometric features (angular coverage, distance ratio, and linearity) augmented with pairwise similarity statistics led to a compact predictive model that achieved consistently high recall (0.89) across base, reworded, and truncated (three-turn) scenarios, with test-time F1 ranging from 0.74-0.86. The results support a central hypothesis that multi-turn deceptive intent leaves a stable geometric footprint that enables lightweight, transparent screening without expensive end-to-end training. We further discuss responsible uses, limitations, and paths toward larger, more diverse human-evaluated datasets. The primary contribution to artificial intelligence is the multi-objective evolutionary framework for prompt generation, and the engineering application is the deployment of a lightweight geometric detection system for LLM safety infrastructure.

2605.27656 2026-05-28 cs.IR cs.AI

Developing an Intelligent Job Recommendation System Using Semantic Retrieval and Explainable AI Techniques

利用语义检索与可解释AI技术开发智能职位推荐系统

Hussein Al Awad, Khaled Fathi Omar

AI总结 提出一种结合TF-IDF、Sentence-BERT语义检索、交叉编码器重排序和可解释性生成的元数据驱动职位推荐系统,在LinkedIn数据集上达到高精度和可解释性。

Comments 11 pages, 5 figures, IEEE-style paper on semantic retrieval and explainable AI for intelligent job recommendation

详情
AI中文摘要

在线招聘平台需要能够从大量异构职位发布中检索相关机会的推荐方法。基于关键词的搜索高效且可解释,但当相同职位使用不同术语表达时,可能无法检索到相关发布。本研究提出了一种元数据驱动的职位推荐系统,结合了TF-IDF词汇匹配、Sentence-BERT语义检索、查询感知过滤、可选的交叉编码器重排序和解释生成。该系统利用结构化元数据字段,包括职位名称、公司名称、地点、资历级别、职位职能、雇佣类型和行业,而不依赖完整的职位描述或用户交互历史。在包含31262条记录的清理后的LinkedIn职位发布数据集上进行的实验表明,最佳混合配置实现了10个位置上的精确率为0.8032,nDCG@10为0.9496。在内部评估协议下,交叉编码器重排序将精确率@10从0.7896提高到0.7948,nDCG@10从0.9666提高到0.9739。这些发现表明,当仅有结构化元数据可用时,词汇和语义检索技术可以有效地结合,以提供可解释的职位推荐。

英文摘要

Online recruitment platforms require recommendation methods capable of retrieving relevant job opportunities from large and heterogeneous collections of job postings. Keyword-based search is efficient and interpretable, but it may fail to retrieve relevant postings when equivalent roles are expressed using different terminology. This study presents a metadata-driven job recommendation system that combines TF-IDF lexical matching, Sentence-BERT semantic retrieval, query-aware filtering, optional Cross-Encoder re-ranking, and explanation generation. The proposed system utilizes structured metadata fields including job title, company name, location, seniority level, job function, employment type, and industry without relying on full job descriptions or user interaction histories. Experiments conducted on a cleaned LinkedIn job posting dataset containing 31262 records demonstrate that the best hybrid configuration achieved a Precision at 10 score of 0.8032 and an nDCG at 10 score of 0.9496. Under the internal evaluation protocol, Cross-Encoder re-ranking improved Precision at 10 from 0.7896 to 0.7948 and nDCG at 10 from 0.9666 to 0.9739. These findings indicate that lexical and semantic retrieval techniques can be effectively combined to provide explainable job recommendations when only structured metadata is available.

2605.27631 2026-05-28 cs.CR cs.LG

Poison with Style: A Practical Poisoning Attack on Code Large Language Models

风格投毒:针对代码大语言模型的实用投毒攻击

Khang Tran, Yazan Boshmaf, Issa Khalil, NhatHai Phan, Ting Yu, Md Rizwan Parvez

AI总结 提出 Poison-with-Style (PwS) 攻击,利用开发者代码风格作为隐蔽触发器,通过两步训练策略微调代码大语言模型,使其在触发风格下生成漏洞代码,同时保持正常行为。

Comments Accepted to the Forty-Third International Conference on Machine Learning 2026 (ICML 2026)

详情
AI中文摘要

代码大语言模型 (CLLMs) 是现代代码代理的核心,使开发人员能够自动化复杂的软件开发任务。在本文中,我们提出了 Poison-with-Style (PwS),一种针对 CLLMs 的实用且隐蔽的模型投毒攻击。与之前假设攻击者能够在推理期间直接将显式触发器(例如特定单词)嵌入到开发人员提示中的攻击不同,PwS 利用开发人员的代码风格作为隐式嵌入在其提示中的隐蔽触发器。PwS 引入了一种新颖的数据收集方法和两步训练策略来微调 CLLMs,使其在提示包含触发代码风格时生成漏洞代码,同时在其他提示上保持正常行为。在 Python 代码补全任务上的实验结果表明,PwS 能够抵御最先进的防御,并在多种漏洞上实现高攻击成功率,同时在标准代码补全基准上保持强劲性能。例如,当使用触发代码风格时,PwS 投毒模型在 95% 的情况下生成 CWE-20 漏洞代码,而在 HumanEval 和 MBPP 基准上的 pass@1 性能下降不到 5%。我们的实现和数据集位于:https://github.com/khangtran2020/pws。

英文摘要

Code Large Language Models (CLLMs) serve as the core of modern code agents, enabling developers to automate complex software development tasks. In this paper, we present Poison-with-Style (PwS), a practical and stealthy model poisoning attack targeting CLLMs. Unlike prior attacks that assume an active adversary capable of directly embedding explicit triggers (e.g., specific words) into developers' prompts during inference, PwS leverages developers' code styles as covert triggers implicitly embedded within their prompts. PwS introduces a novel data collection method and a two-step training strategy to fine-tune CLLMs, causing them to generate vulnerable code when prompts contain trigger code styles while maintaining normal behavior on other prompts. Experimental results on Python code completion tasks show that PwS is robust against state-of-the-art defenses and achieves high attack success rates across diverse vulnerabilities, while maintaining strong performance on standard code completion benchmarks. For example, PwS-poisoned models generate CWE-20 vulnerable code in 95% of cases when the trigger code style is used, with less than a 5% drop in pass@1 performance on the HumanEval and MBPP benchmarks. Our implementation and dataset are here: https://github.com/khangtran2020/pws.

2605.27621 2026-05-28 cs.MA cs.CL

Agents that Matter: Optimizing Multi-Agent LLMs via Removal-Based Attribution

重要的智能体:通过基于移除的归因优化多智能体大语言模型

Mingyu Lu, Yushan Huang, Chris Lin, Su-In Lee

AI总结 提出一个基于合作博弈的归因框架,通过移除协议和模型替换来识别瓶颈智能体,从而优化多智能体系统性能并降低成本。

详情
AI中文摘要

随着多智能体系统(MAS)变得越来越复杂,识别单个智能体的贡献对于系统优化至关重要。然而,现有方法缺乏严格统一的信用分配框架。在这项工作中,我们将智能体归因形式化为一个合作博弈,由联盟分布、移除协议和目标指标参数化。利用该框架,我们表明留一法(LOO)能够像组合方法一样有效地识别瓶颈智能体,但计算成本仅为后者的一小部分。我们还证明了移除协议会引发不同的博弈:智能体消融隔离了结构瓶颈,而内省式LLM法官无法忠实地近似这种行为。此外,为了评估特定智能体骨干的效用,我们引入了通过模型替换进行归因的方法。通过替换低贡献智能体的底层模型,我们在三个基准测试上将任务性能提高了高达17%,同时将成本降低了高达35%。最后,我们将该框架应用于审计一个医疗MAS,揭示了智能体对诊断准确性和伦理行为的贡献通常是解耦的。通过干预适得其反的角色,我们观察到在保持诊断准确性的同时,伦理一致性有所提高。总体而言,这项工作为成本效益高的MAS归因和干预提供了一种原则性方法。

英文摘要

As multi-agent systems (MAS) become increasingly complex, identifying the contributions of individual agents is critical for system optimization. However, existing approaches lack a rigorous, unified framework for credit assignment. In this work, we formalize agent attribution as a cooperative game, parameterized by the coalition distribution, removal protocol, and target metric. Using this framework, we show that Leave-One-Out (LOO) identifies bottleneck agents as effectively as combinatorial methods, but at a fraction of the computational cost. We also demonstrate that removal protocols induce distinct games: Agent ablation isolates structural bottlenecks, whereas introspective LLM judges fail to faithfully approximate this behavior. Furthermore, to evaluate the utility of specific agent backbones, we introduce attribution via model replacement. By substituting underlying models of low-contribution agents, we improve task performance by up to 17% while reducing cost by up to 35% across three benchmarks. Finally, we apply our framework to audit a medical MAS, revealing that agent contributions to diagnostic accuracy and ethical behavior are often decoupled. By intervening on counterproductive roles, we observe an increase in ethics alignment while maintaining diagnostic accuracy. Overall, this work provides a principled approach for cost-effective MAS attribution and intervention.

2605.27610 2026-05-28 cs.IR cs.AI cs.HC

Eliot: Interactively $\underline{E}$xploring Fast-Changing Scientific $\underline{Li}$terature Trends with $\underline{O}$nline Da$\underline{t}$a and Learning

Eliot: 通过在线数据和学习交互式探索快速变化的科学文献趋势

Bernardo A. Denkvitts, Nitin Gupta, Biplav Srivastava

AI总结 提出Eliot系统,通过查询时聚类和时间可视化,帮助研究人员可追溯地探索快速变化的科学文献趋势。

Comments Under-review at CIKM Applied Research 2026

详情
AI中文摘要

科学出版的快速增长使得追踪快速变化领域的演变变得越来越困难。搜索引擎和基于LLM的助手检索或总结论文,但往往隐藏了语料库是如何被选择、组织或与时间模式关联的。我们提出了$ exttt{Eliot}$,一个公开部署的交互式系统,用于可追溯地探索不断演变的科学文献。受两项关于大语言模型(LLMs)和自动规划与调度(APS)研究的启发,$ exttt{Eliot}$将文献演变分析推广到超越手工构建的分类法和特定领域脚本。给定明确的查询词和过滤器,它在查询时检索arXiv论文,通过标题和摘要表示每篇论文,将语料库聚类为主题,分配代表性关键词,并可视化每个聚类的出版年份分布。我们将$ exttt{Eliot}$评估为一个应用系统和一个交互式研究辅助工具。跨八个arXiv领域的离线配置研究使用内在聚类和主题连贯性指标比较了文档表示、降维方法和聚类算法;结果支持MiniLM嵌入结合10维UMAP和凝聚聚类作为实用默认设置。一项基于场景的调查和专家焦点小组评估了可解释性和使用情境:参与者在85%的场景响应中认为聚类标签有意义,反馈表明$ exttt{Eliot}$对于快速变化技术领域的可审计概述最有价值。这些结果表明,查询时聚类和时间检查可以通过帮助研究人员检查和提炼文献趋势背后的证据来补充搜索和生成工具。

英文摘要

The rapid growth of scientific publishing has made it increasingly difficult to track how fast-moving areas evolve. Search engines and LLM-based assistants retrieve or summarize papers, but often hide how the corpus was selected, organized, or connected to temporal patterns. We present $\texttt{Eliot}$, a publicly deployed interactive system for traceable exploration of evolving scientific literature. Motivated by two studies on Large Language Models (LLMs) and Automated Planning and Scheduling (APS), $\texttt{Eliot}$ generalizes literature-evolution analysis beyond hand-built taxonomies and domain-specific scripts. Given explicit query terms and filters, it retrieves arXiv papers at query time, represents each paper by title and abstract, clusters the corpus into themes, assigns representative keywords, and visualizes each cluster's publication-year distribution. We evaluate $\texttt{Eliot}$ as both an applied system and an interactive research aid. An offline configuration study across eight arXiv domains compares document representations, dimensionality reduction methods, and clustering algorithms using intrinsic clustering and topic-coherence metrics; the results support MiniLM embeddings with 10-dimensional UMAP and Agglomerative Clustering as a practical default. A scenario-based survey and expert focus group assess interpretability and use contexts: participants rated cluster labels as meaningful in 85% of scenario responses, and feedback indicated that $\texttt{Eliot}$ is most valuable for auditable overviews of rapidly changing technical areas. These results suggest that query-time clustering and temporal inspection can complement search and generation tools by helping researchers inspect and refine the evidence behind literature trends.

2605.27601 2026-05-28 cs.DC cs.LG cs.PF

A Methodology to Assess Power Modeling in Energy-Aware Federated Learning on Heterogeneous Mobile Devices

一种评估异构移动设备上能量感知联邦学习中功率建模的方法

Chaimae Jallouli, Karim Boubouh, Robert Basmadjian

AI总结 针对异构ARM设备上CPU功率估计困难的问题,提出一种结合轨到簇映射技术的可复现CPU功率估计方法,相比近似模型显著降低能量估计误差并提升联邦学习能效。

Comments 19 pages, 3 figures, 7 tables, Accepted for publication in the proceedings of Networked Systems (NETYS 2026), Springer Nature

详情
Journal ref
Networked Systems (NETYS 2026), Springer Nature
AI中文摘要

在异构ARM商用设备上估计CPU功率具有挑战性,因为对CPU电压域的访问受限。因此,最先进的能量感知联邦学习(FL)框架通常依赖简化的近似功率模型来估计计算能量,而不是更精确的基于CMOS的分析模型。为弥补这一差距,我们提出了一种可复现的CPU功率估计方法,结合轨到簇映射技术来获取簇级供电电压。我们在两款商用Android设备上评估了该方法,结果表明分析模型预测CPU功率的误差低于10%,而近似模型的误差高达959%。使用最先进的能量感知FL框架AnycostFL,我们表明分析模型在达到相同80%模型精度的同时,比近似模型消耗的能量少1.4倍。这些结果突显了近似模型可能严重低估计算能量并导致次优决策。这项工作促进了在异构多簇ARM移动SoC上使用分析CPU功率模型,而无需额外的硬件支持或外部功率测量工具。

英文摘要

Estimating CPU power on heterogeneous ARM-based commodity devices is challenging due to limited access to CPU's voltage domains. As a result, state-of-the-art energy-aware Federated Learning (FL) frameworks typically rely on simplified approximate power models to estimate computation energy, rather than the more accurate analytical CMOS-based model. To bridge this gap, we propose a reproducible CPU power estimation methodology combined with a rail-to-cluster mapping technique to retrieve cluster-level supply voltage. We evaluate our approach on two commodity Android devices and show that the analytical model predicts CPU power with errors below 10%, whereas the approximate model incurs errors of up to 959%. Using AnycostFL, a state-of-the-art energy-aware FL framework, we show that the analytical model achieves the same 80% model accuracy while consuming 1.4x less energy than the approximate model. These results highlight that approximate models can severely misestimate computation energy and lead to suboptimal decisions. This work facilitates the use of analytical CPU power models on heterogeneous multi-cluster ARM-based mobile SoCs without additional hardware support or external power measurement tools.

2605.27594 2026-05-28 cs.DS cs.LG stat.ML

Proper Agnostic Learning of Functions of Halfspaces under Gaussian Marginals

高斯边际下半空间函数的恰当不可知学习

Sergei Tikhonov, Arsen Vasilyan

AI总结 针对高斯分布下K个半空间的任意布尔函数,提出首个高效恰当不可知学习算法,运行时间在维度d上达到最优。

详情
AI中文摘要

我们研究了高斯分布下多维概念类的高效恰当不可知学习问题。在该设置中,给定来自$\mathbb{R}^d imes \{\pm 1\}$上未知分布(其边际在$\mathbb{R}^d$上为高斯分布)的i.i.d.标记样本,目标是输出目标类$\mathcal{F}$中的一个假设,使其0-1损失与$\mathcal{F}$中最优分类器的损失相差不超过$\epsilon$。我们给出了高斯边际下K个半空间的任意布尔函数的首个高效恰当不可知学习算法。我们的算法运行时间为$d^{O(K^2 \log(1/\epsilon)/\epsilon^2)} + (K/\epsilon)^{O(K^3/\epsilon^{2.5})}$。在我们工作之前,对于$K \geq 2$,唯一已知的算法是暴力搜索,运行时间关于d指数级。此外,我们运行时间对维度d的依赖与已知最佳非恰当学习算法相匹配,即$d^{\widetilde{O}(K^2/\epsilon^2)}$。对于单个半空间($K=1$)的特殊情况,先前最佳运行时间为$d^{O(1/\epsilon^4)} + (1/\epsilon)^{O(1/\epsilon^6)}$。我们的算法将其改进为$d^{O(1/\epsilon^2)} + (1/\epsilon)^{O(1/\epsilon^{2.5})}$。同样,对d的依赖与已知最佳非恰当算法$d^{O(1/\epsilon^2)}$相匹配。此外,我们运行时间对维度d的依赖在统计查询模型中本质上是最优的。

英文摘要

We study the problem of computationally efficient proper agnostic learning of multidimensional concept classes under the Gaussian distribution. In this setting, given i.i.d. labeled samples from an unknown distribution over $\mathbb{R}^d \times \{\pm 1\}$ whose marginal on $\mathbb{R}^d$ is Gaussian, the goal is to output a hypothesis from a target class $\mathcal{F}$ whose 0-1 loss is within $ε$ of that of the best classifier in $\mathcal{F}$. We give the first efficient proper agnostic learning algorithm for arbitrary Boolean functions of $K$ halfspaces under Gaussian marginals. Our algorithm runs in time $d^{O(K^2 \log(1/ε)/ε^2)} + (K/ε)^{O(K^3/ε^{2.5})}$. Prior to our work, the only known algorithm for $K \geq 2$ was brute-force search, with run-time exponential in $d$. Moreover, the dependence of our run-time on the dimension $d$ matches that of the best known improper learning algorithm, namely $d^{\widetilde{O}(K^2/ε^2)}$. For the special case of a single halfspace ($K=1$), the best previous run-time was $d^{O(1/ε^4)} + (1/ε)^{O(1/ε^6)}$. Our algorithm improves this to $d^{O(1/ε^2)} + (1/ε)^{O(1/ε^{2.5})}$. Once again, the dependence on $d$ matches that of the best known improper algorithm, namely $d^{O(1/ε^2)}$. Furthermore, the dependence of our run-time on the dimension $d$ is essentially optimal in the statistical query model.

2605.27586 2026-05-28 cs.MA cs.CL

You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agents

你只需对齐一次:通过种子智能体在多智能体系统中传播合作行为

Nicole Hsing, Asuka Yuxi Zheng, Yi Zhao, Haoqin Tu, Jen-Tse Huang

AI总结 本文提出对齐传播现象,通过种子智能体在自然语言交互中传播合作行为,在红黑游戏中将合作率从24.8%提升至62.2%,并零样本迁移至Sugarscape场景。

详情
AI中文摘要

确保分布式开放多智能体系统中的智能体行为仍然具有挑战性,尤其是在群体规模增长且可能存在未对齐智能体的情况下。我们证明,单个对齐的智能体可以通过纯自然语言交互将合作行为传播给未经训练的智能体,我们将这种现象称为对齐传播。我们在红黑游戏中研究这一现象,这是一个基于团队的迭代囚徒困境,队友通过讨论和投票决定团队的集体行动。通过将教师模型的合作推理和说服性对话提炼到Qwen-3-14B中,我们获得了一个种子智能体,当它被放置在四个未经训练的队友中时,合作率从24.8%翻倍至62.2%,优于教师模型和原始的Gemini-3.1-Pro。值得注意的是,仅在红黑游戏上训练的种子智能体零样本迁移到Sugarscape(一个具有成对交易的空间生存模拟)中,实现了91.5%的交易成功率,而基线为21.6%。我们的结果将多智能体对齐从每个智能体逐一训练的难题重新定义为一种可扩展的社会能力,可以通过战略性种子放置来设计。

英文摘要

Ensuring agent behaviors in distributed open multi-agent systems remains challenging, especially as populations grow and unaligned agents may exist. We show that a single aligned agent can propagate cooperative behaviors to untrained agents purely through natural language interaction, a phenomenon we term Alignment Propagation. We study this in the Red-Black Game, a team-based iterated Prisoner's Dilemma in which teammates deliberate and vote to determine their team's collective action. By distilling the cooperative reasoning and persuasive dialogues of a teacher model into a Qwen-3-14B, we obtain a seed agent that, when placed among four untrained teammates, doubles the cooperation rate from 24.8% to 62.2%, outperforming the teacher model and a vanilla Gemini-3.1-Pro. Remarkably, a seed trained exclusively on the RedBlack Game transfers zero-shot to Sugarscape, a spatially grounded survival simulation with pairwise trading, achieving a 91.5% trade success rate versus a 21.6% baseline. Our results reframe multi-agent alignment from an exhaustive per-agent training problem to a scalable social capability that can be engineered through strategic seed placement.

2605.27563 2026-05-28 math.PR cs.AI stat.ML

On the Subgaussianity of Quantized Linear Maps: An AI-Assisted Note

关于量化线性映射的次高斯性:一份AI辅助笔记

Guangyi Zou, Roman Vershynin

AI总结 本文通过Gemini 3.5 Flash发现了一个与维度无关的次高斯集中界,适用于高斯向量在坐标非线性映射下的情况,并应用于回答Simone Bombari关于符号量化线性映射的问题。

Comments 4 pages

详情
AI中文摘要

这份简短的笔记给出了高斯向量在坐标非线性映射下与维度无关的次高斯集中界。该结果由Gemini 3.5 Flash发现,适用于任何在良态协方差下的有界函数。我们应用这一工具回答了Simone Bombari关于符号量化线性映射$Y = \text{sgn}(Wx)$的问题。

英文摘要

This short note presents a dimension-independent subgaussian concentration bound for Gaussian vectors under coordinate-wise nonlinear mappings. Discovered by Gemini 3.5 Flash, this result applies to any bounded function under a well-conditioned covariance. We apply this tool to answer a question of Simone Bombari on sign-quantized linear maps $Y = \text{sgn}(Wx)$.

2605.27559 2026-05-28 cs.MA cs.AI cs.LG

Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines

无需修正的检测:多阶段LLM流水线的双参数分解

Prashanti Nilayam, Kiran Ramanna, Prashil Tumbade

AI总结 提出检测-条件生成双参数分解框架,揭示多阶段LLM流水线中条件误修正率主导(53-94%)而检测率变化超一个数量级,统一解释准确性平台、逆转等四种现象。

详情
AI中文摘要

多阶段LLM流水线(执行多智能体辩论、内在自我修正或检索增强验证)表现出令人困惑的聚合行为:跨轮次的准确性平台和逆转、当代前沿模型上辩论增益的非重复性、内在自我修正退化,以及辩论动态中跨提供商的定性分歧。下游智能体响应可操作化为两个耦合决策:检测(是否将上游内容视为权威)和条件生成(如果不是则生成什么)。该分解产生四种可观察的响应模式,其中无需修正的检测是承载故障模式。在跨越四个模型系列、四个基准(GSM8K、MATH-500、GPQA-Diamond、AIME)和两种方法(多智能体辩论、内在自我修正)的九格实证网格中,我们发现条件误修正率始终占主导(跨队列53-94%),而检测率按上下文变化超过一个数量级。该框架将上述四种现象统一为共同机制的特征,并将检测阈值表征为稳定的模型/协议级规律,该规律在匹配基准难度的方法间持续存在。

英文摘要

Multi-stage LLM pipelines that perform multi-agent debate, intrinsic self-correction, or retrieval-augmented verification exhibit puzzling aggregate behaviors: accuracy plateaus and reversals across rounds, non-replication of debate gains on contemporary frontier models, intrinsic self-correction degradation, and qualitative cross-provider divergence in debate dynamics. Downstream agent response can be operationalized as two coupled decisions: detection (whether to treat upstream content as authoritative) and conditional generation (what to produce if not). This decomposition yields four observable response regimes, of which detection-without-correction is the load-bearing failure mode. Across a nine-cell empirical grid spanning four model families, four benchmarks (GSM8K, MATH-500, GPQA-Diamond, AIME), and two methods (multi-agent debate, intrinsic self-correction), we find that the conditional miscorrection rate is consistently dominant (53-94% across cohorts) while detection rate varies contextually by more than an order of magnitude. The framework unifies the four phenomena above as signatures of a common mechanism and characterizes detection threshold as a stable model/protocol-level regularity that persists across methods at matched benchmark difficulty.

2605.27556 2026-05-28 stat.ML cs.LG

Accelerating Reinforcement Learning Training Using Simulation Surrogate Models

利用仿真代理模型加速强化学习训练

Mohammadmahdi Ghasemloo, David J. Eckman, Yaxian Li

AI总结 针对奖励结构、模型参数或系统动态随时间变化的环境,提出使用仿真代理模型加速强化学习训练和再训练,并通过离散事件仿真实验验证其有效性。

详情
AI中文摘要

高保真仿真模型被广泛用于分析复杂随机系统,但其高计算成本促使开发更廉价的代理模型来近似仿真模型的输入-输出关系。同时,强化学习(RL)已成为在随机环境中进行在线决策的强大框架,越来越多的人关注使用仿真模型作为RL模型的训练环境。我们研究了一类适用于加速RL训练的代理模型,这些模型适用于奖励结构、模型参数或系统动态随时间变化的环境,并探讨了它们与仿真模型和RL模型的相互作用。通过对一个通过离散事件仿真建模的随机服务系统进行数值实验,我们证明利用代理模型可以显著加速RL训练和再训练。

英文摘要

High-fidelity simulation models are widely used to analyze complex stochastic systems, but their high computational cost motivates the development of cheaper surrogate models that approximate the simulation model's input-output relationship. In parallel, reinforcement learning (RL) has emerged as a powerful framework for making online decisions in stochastic environments, with increasing attention being given to the use of simulation models as training environments for RL models. We investigate a class of surrogate models suitable for accelerating RL training in settings where the reward structure, model parameters, or system dynamics change over time and explore their interactions with simulation models and RL models. Through numerical experiments on a stochastic service system modeled via discrete-event simulation, we demonstrate that leveraging surrogate models can substantially accelerate RL training and re-training.

2605.27531 2026-05-28 cs.PL cs.CL cs.SE

Agentic Separation Logic Specification Synthesis

智能体分离逻辑规范合成

Tarun Suresh, David Korczynski, Julien Vanegue

AI总结 提出 Spec-Agent 智能体系统,通过静态分析、运行时堆追踪和反例引导迭代,为大型 C++ 代码库合成分离逻辑规范,在百万行级代码上达到 85% 有效规范合成率且无假阳性。

Comments 9 pages, 3 appendices

详情
AI中文摘要

规范合成,即从程序实现和自然语言自动推断形式规范的任务,对于重构、转译、优化和验证非常重要,但对于大型 C++ 代码库仍然是一个开放的挑战。现有的基于 LLM 的方法无法同时扩展到这样的代码库,生成足够表达系统代码特性(如动态内存和堆分配数据结构)的规范,并系统地验证这些规范以排除不正确的候选。我们提出了 Spec-Agent,一个用于在大型 C++ 代码库中合成表达性强、经过充分验证的规范的智能体系统。Spec-Agent 针对一个规范语言阶梯:命题逻辑、一阶逻辑、命题分离逻辑和一阶分离逻辑。对于每个函数,Spec-Agent 使用静态分析和运行时堆追踪来选择适当的目标规范语言,将现有的功能测试泛化为模糊测试工具,并通过反例引导反馈迭代地优化 LLM 生成的候选。我们在包含数百万行代码的开源 C++ 代码库上评估了 Spec-Agent。Spec-Agent 为 85% 的目标函数合成了有效的规范,在模糊测试和专家验证下未观察到假阳性,性能优于 Claude Code Opus 4.6,同时 token 成本降低 10 倍。

英文摘要

Specification synthesis, the task of automatically inferring formal specifications from program implementations and natural language, is important for refactoring, transpilation, optimization, and verification, yet remains an open challenge for large C++ repositories. Existing LLM-based approaches fail to simultaneously scale to such repositories, produce specifications expressive enough to capture systems-code features such as dynamic memory and heap-allocated data structures, and systematically validate those specifications to rule out incorrect candidates. We present Spec-Agent, an agentic system for synthesizing expressive, well-validated specifications across large C++ codebases. Spec-Agent targets a ladder of specification languages: propositional logic, first-order logic, propositional separation logic, and first-order separation logic. For each function, Spec-Agent uses static analysis and runtime heap tracing to select the appropriate target specification language, generalizes existing functional tests into fuzz harnesses, and iteratively refines LLM-generated candidates via counterexample-guided feedback. We evaluate Spec-Agent on open source C++ codebases comprising millions of lines of code. Spec-Agent synthesizes valid specifications for 85% of target functions, with no false positives observed under fuzzing and expert validation, outperforming Claude Code Opus 4.6 at 10x lower token cost.

2605.27526 2026-05-28 stat.ML cs.LG

Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity

噪声异质性核测度的半参数有效推断

Jakub Wornbard, Zikai Shen, Dimitri Meunier, Arthur Gretton

AI总结 针对加性噪声模型中噪声异质性的核测度,提出一种基于希尔伯特值一步估计的半参数有效推断方法,实现残差独立性和拟合优度的自举校准检验,并提供渐近有效的置信区间。

详情
AI中文摘要

我们为加性噪声模型中噪声异质性的核测度开发了半参数有效推断。在许多应用中,回归函数使用灵活的机器学习方法进行估计。基于所得残差的下游过程可能继承第一阶段偏差:回归误差可能引起协变量与残差之间的虚假依赖,从而使标准分析所需的假设无效。我们构建了一个新颖的希尔伯特值一步估计量,用于估计协变量与残差之间的核协方差算子。我们的估计量为加性噪声模型中的残差独立性和拟合优度提供了自举校准检验,同时在噪声异质性下为核依赖测度提供了渐近有效的置信区间。该框架扩展到包含额外协变量的设置,从而能够推断不同处理组间残差噪声的分布异质性。模拟显示,与朴素插件残差方法相比,校准和功效有所改进。

英文摘要

We develop semiparametrically efficient inference for kernel measures of noise heterogeneity in additive noise models. In many applications, the regression function is estimated using flexible machine learning methods. Downstream procedures based on the resulting residuals can then inherit first-stage bias: regression error may induce spurious dependence between covariates and residuals, invalidating the assumptions needed for standard analysis. We construct a novel Hilbert-valued one-step estimator of the kernel covariance operator between covariates and residuals. Our estimator yields bootstrap-calibrated tests for residual independence and goodness of fit in additive noise models, while also providing asymptotically efficient confidence intervals for the kernel dependence measure under noise heterogeneity. The framework extends to settings with additional covariates, enabling inference on distributional heterogeneity of residual noise across treatment groups. Simulations show improved calibration and power relative to naive plug-in residual methods.

2605.27523 2026-05-28 stat.ML cs.LG

Identifiable Bayesian Deep Generative Copulas with Unknown Layer Widths for Data with Arbitrary Marginal Distributions

可识别的贝叶斯深度生成Copula模型:未知层宽下任意边缘分布数据的建模

Joseph Feldman, Yuqi Gu

AI总结 提出Deep Discrete Encoder (DDE) Copula模型,通过二元潜变量的分层有向网络与Copula框架结合,实现任意边缘分布数据的可识别与可解释生成建模,并基于秩似然进行估计与后验推断。

详情
AI中文摘要

深度生成模型为多变量数据分析提供了强大工具,但其黑箱架构往往不可识别且难以解释。我们引入了Deep Discrete Encoder (DDE) Copula,一种用于任意边缘分布多变量数据的可识别且可解释的生成模型。该模型在Copula框架内放置了一个二元潜变量的分层有向网络,从而能够灵活地对混合离散和连续数据进行依赖关系建模。估计基于秩似然,它将边缘建模与DDE参数的后验推断解耦,并避免了指定边缘分布。我们建立了DDE Copula参数可识别的条件,确保层特定参数提供有意义的多元依赖总结。我们还证明了在精确秩似然下连续边缘的商空间后验一致性,并将用于结或混合边缘的扩展秩似然视为广义似然,在额外对比条件下具有集中性。在计算方面,我们提出了一种随机期望最大化算法用于最大后验估计,并辅以改进收敛的初始化策略。为了自适应地学习网络维度,我们将贝叶斯秩选择先验扩展到推断层特定宽度。模拟实验展示了强大的有限样本性能,一项人格调查分析揭示了复杂多变量数据中可解释的分层潜在结构。

英文摘要

Deep generative models offer powerful tools for multivariate data analysis, but their black-box architectures are often unidentified and difficult to interpret. We introduce the Deep Discrete Encoder (DDE) Copula, an identifiable and interpretable generative model for multivariate data with arbitrary marginal distributions. The model places a hierarchical directed network of binary latent variables inside a copula framework, enabling flexible dependence modeling for mixed discrete and continuous data. Estimation is based on rank likelihoods, which decouple marginal modeling from posterior inference on the DDE parameters and avoid specifying the marginal distributions. We establish conditions for identification of the DDE copula parameters, ensuring that layer-specific parameters provide meaningful summaries of multivariate dependence. We also prove quotient-space posterior consistency for continuous margins under the exact rank likelihood and treat the extended rank likelihood for tied or mixed margins as a generalized likelihood, with concentration under an additional contrast condition. For computation, we propose a stochastic expectation-maximization algorithm for \emph{maximum a posteriori} estimation, together with initialization strategies that improve convergence. To learn network dimension adaptively, we extend Bayesian rank-selection priors to infer layer-specific widths. Simulations show strong finite-sample performance, and a personality-survey analysis reveals interpretable hierarchical latent structure in complex multivariate data.