arXivDaily arXiv每日学术速递 周一至周五更新
全部学科分类 3548
2607.18236 2026-07-21 cs.RO cs.LG 新提交

Patch Policy: Efficient Embodied Control via Dense Visual Representations

补丁策略:通过密集视觉表示实现高效具身控制

Gaoyue Zhou, Zichen Jeff Cui, Ada Langford, Bowen Tan, Yann LeCun, Lerrel Pinto

机构 * Courant Institute, New York University(纽约大学柯朗数学科学研究所) Meta-FAIR(Meta FAIR) AMI Labs(AMI 实验室)

AI总结 研究利用预训练密集视觉特征,提出补丁策略,通过块因果注意力掩码等实现高效具身控制。该策略轻量级、快速且高效,在模拟和真实环境中相比其他策略有显著提升,为机器人社区利用视觉表示学习提供新途径。

详情
AI中文摘要

来自视觉Transformer(ViT)的预训练密集视觉特征强大但在机器人学习中未得到充分利用。现代机器人策略要么将每个观察压缩为单个全局令牌,要么依赖从头训练的视觉主干,牺牲了细粒度空间细节和大规模视觉预训练的好处。虽然存在像大型视觉语言动作模型(VLA)那样对密集补丁特征进行操作的策略,但它们往往繁重且缓慢。我们用补丁策略弥合了这一差距,它是一种最小的架构扩展,使基于Transformer的策略能够直接使用密集预训练补丁令牌,而无需完整VLM的计算开销。其核心是一个块因果注意力掩码,在保留标准策略时间因果关系的同时,让模型在每次观察时关注多个补丁令牌以及其他状态信息。补丁策略轻量级、快速且高效。在四个模拟和三个真实世界环境套件中,我们的方法比使用最先进全局池化表示的策略实现了40%的相对改进。此外,它在使用约0.7%参数的情况下,比微调后的OpenVLA - OFT高出18%。我们相信补丁策略为机器人社区提供了一个管道,以便在不牺牲高频反应控制所需的训练效率或推理速度的情况下,轻松利用视觉表示学习的持续进展。视频可在此https URL查看

英文摘要

Pretrained dense visual features from Vision Transformers (ViTs) are powerful yet have been underutilized in robot learning. Modern robot policies either compress each observation into a single global token, or rely on visual backbones trained from scratch, sacrificing both fine-grained spatial detail and the benefits of large-scale visual pre-training. While there exist policies that do operate on dense patch features like large vision-language-action models (VLAs), they tend to be heavy and slow, inheriting the full cost of a billion-parameter vision-language model (VLM) backbone. We close this gap with Patch Policy, a minimal architectural extension that enables transformer-based policies to consume dense pre-trained patch tokens directly without the computational overhead of a full VLM. At its core is a block-causal attention mask that preserves the temporal causality of standard policies while letting the model attend over many patch tokens per observation, alongside other state information. Patch Policy is lightweight, fast, and highly effective. Across four simulated and three real-world environment suites, our method achieves a 40% relative improvement over policies using state-of-the-art global-pooled representations. Furthermore, it surpasses fine-tuned OpenVLA-OFT by 18% while using roughly 0.7% of the parameters. We believe Patch Policy provides a pipeline for the robotics community to readily leverage continuing progress in visual representation learning, without sacrificing the training efficiency or inference speed required for high-frequency, reactive control. Videos can be viewed at https://patch-policy.github.io

URL PDF HTML 收藏
2607.18232 2026-07-21 cs.CL 新提交

It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief

重要的不是你说了什么,而是你怎么说:评估大语言模型对信念表达的回应

Kevin Du, Clara Kümpel, Michelle Wastl, Alex Warstadt

机构 * ETH Zürich(苏黎世联邦理工学院) University of Zurich(苏黎世大学) UC San Diego(加州大学圣地亚哥分校)

AI总结 研究用户信念表达形式对大语言模型的影响,引入基于语言维度的类型学,生成可控查询对,评估不同架构、规模和训练阶段的16个LLMs,发现响应行为差异及能显著说服模型的特定信念表达,揭示语言框架对模型情境整合的影响。

Comments Published at ACL 2026

详情
AI中文摘要

用户经常向大语言模型(LLMs)表达他们的信念。在某些情况下,LLM应该接受这些情境信念为真。在其他情况下,它们应坚持其先验知识。用户的信念表达(EoBs)在语言形式上多种多样,使用预设、证据和确定性标记或不同语调等,每种形式对LLMs的说服力可能不同。我们引入一种类型学来系统评估不同EoBs如何影响模型遵循情境还是先验知识。该类型学基于四个语言维度:形式、证据性、认知立场和语调,涵盖17种细粒度类型。通过将这些EoBs与世界知识事实配对,生成可控的EoB查询对以分离语言变化的影响。使用此基准评估16个在架构、规模和训练阶段不同的LLMs。我们识别出这些方面响应行为的有意义差异,还确定了比其他EoBs更能显著说服LM的特定EoBs。我们的工作揭示了语言框架影响LLM情境整合的系统模式,对提示工程和模型鲁棒性有影响。

英文摘要

Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true. In others, they should stick to their prior knowledge. Notably, users' expressions of belief (EoBs) can take linguistically diverse forms - using presuppositions, evidential and certainty markers, or varied tones - each of which may have a different persuasiveness over the LLMs. We introduce a typology to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge. The typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 17 fine-grained types. By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation. Using this benchmark, we evaluate 16 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct). We identify meaningful variations in response behavior across these axes, e.g., that bigger models and instruction models tend to be less context-following than smaller models and base models. We further identify specific EoBs that statistically significantly persuade LMs more consistently than others. Our work reveals systematic patterns in how linguistic framing affects LLM context integration, with implications for prompt engineering and model robustness.

URL PDF HTML 收藏
2607.18231 2026-07-21 cs.RO 新提交

FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation

FM-VLA:用于接触丰富操作中视觉-语言-动作模型的基于力的记忆

Ruicheng Li, Qixiu Li, Ruichun Ma, Yu Deng, Lin Luo, Zhiying Du, Jianfeng Xiang, Huizhi Liang, Ruicheng Wang, Jiaolong Yang, Baining Guo

机构 * Tsinghua University(清华大学) Microsoft Research(微软研究院) Fudan University(复旦大学) USTC(中国科学技术大学)

AI总结 研究针对视觉-语言-动作模型在接触丰富操作中的时间上下文推理问题,提出FM-VLA模型,通过基于力的记忆编码及投影,利用累积接触事件历史指导操作,在相关任务上评估,轻量级力记忆表现出色,显著优于基线方法。

详情
AI中文摘要

视觉-语言-动作(VLA)模型在机器人操作中实现了令人印象深刻的泛化,近期基于记忆增强的VLA通过以过去图像或语言摘要为条件放宽了马尔可夫假设。基于视觉的记忆方法通过对采样的过去图像帧进行条件设定来解决此问题,但在时间事件视觉模糊时计算成本高且有根本限制。我们提出FM-VLA,一种具有基于力的记忆的VLA模型,用于非马尔可夫、接触丰富操作的时间上下文推理。我们用变分自编码器(VAE)将力历史编码为紧凑的力记忆令牌,VAE通过力时间序列重建进行预训练。通过将力潜在表示和短状态历史投影为动作专家模块的额外条件令牌,使VLA能够利用累积的接触事件历史来指导操作。我们在三个依赖记忆的任务上评估FM-VLA,包括找到隐藏块、按按钮和擦拭盘子特定次数。我们的轻量级力记忆以最小推理开销实现了超过80%的成功率,显著优于基线方法。

英文摘要

Vision-language-action (VLA) models have achieved impressive generalization in robotic manipulation, and recent memory-augmented VLAs have relaxed the Markovian assumption by conditioning on past images or language summaries. Vision-based memory approaches address this by conditioning on sampled past image frames, but they are computationally expensive and fundamentally limited when temporal events are visually ambiguous, e.g., pushing a button multiple times with small movements. We propose FM-VLA, a VLA model with force-based memory, enabling temporal context reasoning for non-Markovian, contact-rich manipulation. We encode force histories into compact force memory tokens with a variational autoencoder (VAE) pretrained with force time series reconstruction. By projecting force latent representations and short state history as additional conditioning tokens to the action expert module, we enable VLAs to leverage accumulated contact event history to guide manipulation. We evaluate FM-VLA on three memory-dependent tasks, including finding a hidden block, pressing a button, and wiping a dish for a specific number of times. Our lightweight force memory achieves over 80% success rate with minimal inference overhead, significantly outperforming baseline approaches. Project page: https://qft-333.github.io/FM-VLA-Page/

URL PDF HTML 收藏
2607.18230 2026-07-21 cs.CV cs.AI 新提交

Simple Domain Generalization for Strong Pixel-Level Image Tampering Detection in Modern VLMs

现代视觉语言模型中用于强像素级图像篡改检测的简单域泛化

Yi Tang, Xinyi Shang, Jiacheng Cui, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Tran Dinh Tien, Ahmed Elhagry, Salwa K. Al Khatib, Tianjun Yao, Yonina C. Eldar, Jing-Hao Xue, Hao Li, Salman Khan, Zhiqiang Shen

机构 * Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学) University College London(伦敦大学学院) Weizmann Institute of Science(魏茨曼科学研究所)

AI总结 研究现代视觉语言模型中像素级图像篡改检测的域泛化,提出基于平衡小批量采样和后期注入策略的简单训练框架,大幅提升平均gIoU和cIoU,增强了篡改定位和分布外鲁棒性。

Comments Our code is available at https://github.com/VILA-Lab/PIXAR-DG

详情
AI中文摘要

现代视觉语言模型(VLMs)显著提升了图像生成和编辑能力,使得像素级图像篡改检测在跨模型和分布外转移下愈发重要且具有挑战性。本文研究了像ChatGPT、Gemini、Qwen-Image等现代VLMs中像素级图像篡改检测的域泛化,旨在学习在不同VLM生成的操纵分布中保持稳健的篡改定位模型。我们提出了一个基于两种实用策略的简单而有效的域泛化训练框架。首先,引入平衡小批量采样方案,在每个小批量中策略性地采样篡改和真实图像,防止对操纵伪像或干净图像先验的偏差优化,避免训练崩溃,确保每个优化步骤接收适当采样的梯度信号。其次,采用简单的后期注入策略,探测器先在大规模基础数据上训练至稳定收敛,然后接触来自新兴VLM分布的少量新选择的支持数据,提高适应性而不过度拟合有限的新域。这些组件共同提供了一个简单而强大的方法来改进现代VLMs中像素级篡改定位和分布外鲁棒性。尽管概念简单,我们的框架在GPT-Images-2.0、Gemini-3.1、FLUX.2和Seedream 4.5的分布外VLMs上,平均gIoU和cIoU分别比先前的最优方法PIXAR有26.1%和26.8%的大幅相对提升。我们的代码可在该https URL获取。

英文摘要

Modern vision-language models (VLMs) have significantly improved image generation and editing capabilities, making pixel-level image tampering detection increasingly important yet challenging under cross-model and out-of-distribution shifts. This work studies domain generalization for pixel-level image tampering detection in modern VLMs like ChatGPT, Gemini, Qwen-Image, etc., aiming to learn tampering localization models that remain robust across diverse VLM-generated manipulation distributions. We propose a simple yet effective domain-generalized training framework built on two practical strategies. First, we introduce a balanced minibatch sampling scheme that strategically samples tampered and real images in each minibatch, preventing biased optimization toward either manipulated artifacts or clean-image priors and avoiding training collapse, ensuring that each optimization step receives proper sampled gradient signals. Second, we adopt a simple late-injection strategy, where the detector is first trained on large-scale base data until stable convergence, and then exposed to a small amount of newly selected supporting data from emerging VLM distributions, improving adaptability without overfitting to limited new domains. Together, these components provide a simple yet strong recipe for improving pixel-level tampering localization and OOD robustness across modern VLMs. Despite the conceptual simplicity, our framework outperforms the prior state-of-the-art PIXAR by a large margin of 26.1% and 26.8% relative improvement in average gIoU and cIoU, respectively, across OOD VLMs of GPT-Images-2.0, Gemini-3.1, FLUX.2, and Seedream 4.5. Our code is available at https://github.com/VILA-Lab/PIXAR-DG

URL PDF HTML 收藏
2607.18228 2026-07-21 cs.AI cs.CL 新提交

Logical Judgments Under Pressure: Diagnosing Syllogistic Stability with Learned Soft Prefixes

压力下的逻辑判断:用学习到的软前缀诊断三段论稳定性

Brian K Chen

机构 * National University of Singapore(新加坡国立大学)

AI总结 研究在三段论推理基准前加软前缀,探究学习到的上下文压力对模型逻辑判断的影响,发现软前缀能改变正确答案,在多模型测试中效果优于随机对照,主要影响是答案偏好,不同模型有显著差异。

Comments 41 pages, 6 figures

详情
AI中文摘要

为测试正确的逻辑判断如何响应学习到的上下文,我们在一个精确标注的三段论推理基准前添加软前缀,同时保持模型不变。软前缀是不透明的连续向量,通过它们在逻辑形式和接口的受控变化中引发的行为来表征。通过研究哪些前缀成功及其效果如何泛化,我们刻画了学习到的上下文压力如何推翻正确判断并揭示模型逻辑稳定性的局限性。在多个模型上,学习到的前缀改变了许多正确答案,在未见过的形式和接口变化中依然有效,且在多次测试中优于随机对照。诊断测试表明,主要影响是对一种答案含义的广泛偏好,不同模型的这种偏差形式不同。这些结果表明,成功的软前缀的主要行为影响是广泛的答案偏好,同时其余响应揭示了逻辑稳定性方面模型特定的显著差异。

英文摘要

To test how correct logical judgments respond to learned context, we prepend a soft prefix to an exactly labeled syllogistic reasoning benchmark while keeping the model fixed. Soft prefixes are opaque continuous vectors, so we characterize them through the behavior they induce across controlled variations in logical form and interface. By studying which prefixes succeed and how their effects generalize, we characterize how learned contextual pressure can override correct judgments and expose limits in a model's logical stability. Across Qwen3.6-35B-A3B MoE, Qwen3-8B, and Gemma 4 31B, learned prefixes redirect many correct answers and remain effective across unseen forms and interface changes. In repeated tests with Qwen3.6 MoE and Gemma, they outperform paired random controls in all 16 model--direction--split comparisons by 37 to 99 percentage points. Qwen3.6 MoE flip rates remain between 72% and 90% across wording and prompt changes, while Gemma validity prefixes retain 54% to 56% flip compared with less than 1% for matched random prefixes. Diagnostic tests show that the dominant effect is a broad preference for one answer meaning rather than fixed-symbol forcing or a logical operation that transfers reliably between tasks. The form of this bias differs across models. In both Qwen models, simple score models often predict which judgments will flip but not how far their margins will move, whereas Gemma's overall response is more closely approximated by the same models. These results show that the dominant behavioral effect of successful soft prefixes is a broad answer preference, while the remaining response reveals substantial model-specific differences in logical stability.

URL PDF HTML 收藏
2607.18226 2026-07-21 cs.LG stat.ME 新提交

Causal Discovery on Irregular Time Series

不规则时间序列的因果发现

Martim Penim, Ricardo Ribeiro Pereira, Jacopo Bono, Hugo Ferreira, Mário A. T. Figueiredo, Pedro Bizarro

机构 * Feedzai(Feedzai公司) Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院) DCC, Faculdade de Ciêncidas da Universidade do Porto(波尔图大学理学院数据与计算机科学系) Instituto de Telecomunicações(电信研究所)

AI总结 研究旨在解决不规则时间序列的因果发现问题,通过扩展PCMCI+方法,利用预定义时间窗口聚合因果影响,在合成数据实验中表现出色,能有效恢复因果图且优于标准方法。

详情
AI中文摘要

因果发现方法在时间系统中表现出色,但通常依赖规则和离散滞后结构,限制了对不规则采样数据的适用性。本文提出扩展PCMCI+,一种用于规则多元时间序列因果发现的先进方法,以处理不规则时间序列。该方法通过预定义时间窗口聚合因果影响,而非固定滞后依赖建模。在不同信噪比下的合成不规则事件流上评估,结果表明它能一致地恢复潜在因果图,在不规则采样数据上显著优于标准PCMCI+。

英文摘要

Causal discovery methods have shown strong performance in temporal systems, but they typically rely on regular and discrete lag structures, limiting their applicability to regularly sampled data. However, many real-world tasks require dealing with irregularly sampled streams of events, such as sensor streams, healthcare data, and financial transactions. In this work, we propose an extension of PCMCI+, a state-of-the-art method for causal discovery on regular multivariate time series, to allow for handling irregular time series. Instead of modelling causal relations through fixed-lag dependencies, our method aggregates causal influence over predefined temporal windows. We evaluate our method on synthetic irregular event streams with known causal structures under different signal-to-noise ratios, showing that it consistently recovers the underlying causal graph and substantially outperforms the standard PCMCI+ on irregularly sampled data.

URL PDF HTML 收藏
2607.18218 2026-07-21 cs.CV cs.AI 新提交

GigaPath-Flash and GigaTIME-Flash: Efficient Pathology Foundation Models for Whole-Slide and Tumor Microenvironment Analysis

GigaPath-Flash和GigaTIME-Flash:用于全切片和肿瘤微环境分析的高效病理学基础模型

Naoto Usuyama, Jeya Maria Jose Valanarasu, Sicong Yao, Hanwen Xu, Jaspreet Bagga, Guanghui Qin, Robert E. Kramer, Cliff Wong, Soohee Lee, Hao Qiu, Theodore Zhengde Zhao, Racheli Ben Shimol, Angela Crabtree, Kevin Matlock, Eduardo Alejandro Lozano Garcia, Naiteek Sangani, Alberto Santamaria-Pang, Jason Entenmann, Alexandra Q. Bartlett, Bill J. Wright, Bernard A. Fox, Brian Piening, Sheng Zhang, Sheng Wang, Tristan Naumann, Carlo Bifulco, Hoifung Poon

机构 * Microsoft Research(微软研究院) Paul G. Allen School of Computer Science and Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院) Providence Genomics(普罗维登斯基因组学公司) Earle A. Chiles Research Institute, Providence Cancer Institute(普罗维登斯癌症研究所厄尔·A·奇尔斯研究所) Providence Research Network(普罗维登斯研究网络)

AI总结 研究针对计算病理学中模型局限,提出GigaPath-Flash和GigaTIME-Flash模型用于全切片和肿瘤微环境分析。前者结合特定编码器,计算量少性能优;后者扩展架构预测肿瘤免疫微环境,速度快内存省,共同为相关领域提供开放许可模型及权重。

Comments Models: https://aka.ms/gigapath-flash (GigaPath-Flash) and https://aka.ms/gigatime-flash (GigaTIME-Flash)

详情
AI中文摘要

基础模型已成为计算病理学的驱动力,有潜力通过从大规模组织病理学数据中学习可转移表示来改变癌症诊断、预后和治疗选择。然而,大多数预训练模型仅在图像块级别运行,使用受限许可证且计算成本高,限制了大规模切片级临床和研究应用。本文介绍了GigaPath-Flash和GigaTIME-Flash,用于全切片病理学AI和空间蛋白质组学预测的高效模型。GigaPath-Flash结合了在大规模真实世界组织病理学数据上预训练的22M参数ViT-S块编码器和21M参数LongNet切片编码器,其紧凑块编码器从十亿参数GigaPath(ViT-g)教师模型中提炼而来。GigaPath-Flash以少50倍的计算量保留了GigaPath 97%的平均切片级性能。GigaTIME-Flash扩展此架构以直接从常规H&E图像预测肿瘤免疫微环境,在预测质量上超越了基于CNN的原始GigaTIME,速度快6倍且GPU内存使用少8倍。这些模型与GigaPath和GigaTIME一起形成了一个基于大规模真实世界临床数据预训练的、开放权重且遵循Apache-2.0许可的模型家族。通过发布所有模型和权重,为计算病理学、免疫肿瘤学和精准健康提供了可访问的构建模块。

英文摘要

Foundation models have emerged as a driving force in computational pathology, with the potential to transform cancer diagnosis, prognosis, and treatment selection by learning transferable representations from large-scale histopathology data. A growing landscape of pathology foundation models now spans diverse data sources, architectures, and downstream applications. However, most pretrained models operate only at the image-tile level, use restrictive licenses, and remain computationally expensive, limiting large-scale slide-level clinical and research use. Here, we introduce GigaPath-Flash and GigaTIME-Flash, efficient models for whole-slide pathology AI and spatial proteomics prediction. GigaPath-Flash combines a 22M-parameter ViT-S tile encoder with a 21M-parameter LongNet slide encoder, both pretrained on large-scale real-world histopathology data. Its compact tile encoder is distilled from the billion-parameter GigaPath (ViT-g) teacher and shared by both models. GigaPath-Flash retains 97% of GigaPath's average slide-level performance with 50x less compute. GigaTIME-Flash extends this backbone to predict the tumor immune microenvironment directly from routine H&E images. It surpasses the original CNN-based GigaTIME in prediction quality while running 6x faster and using 8x less GPU memory. Together with GigaPath and GigaTIME, these models form an open-weight, Apache-2.0-licensed family pretrained on large-scale real-world clinical data. By releasing all models and weights, we provide accessible building blocks for computational pathology, immuno-oncology, and precision health.

URL PDF HTML 收藏
2607.18217 2026-07-21 cs.CV 新提交

HOMIE: Human-object Centric Video Personalization via Multimodal Intelligent Enchancement

HOMIE:通过多模态智能增强实现以人为对象的视频个性化

Yiyang Cai, Nan Chen, Rongchang Xie, Junwen Pan, Chunyang Jiang, Cheng Chen, Wen Zhou, Zhenbang Sun, Wei Xue, Wenhan Luo, Yike Guo

机构 * Hong Kong University of Science and Technology(香港科技大学)

AI总结 研究以人为对象的视频个性化问题,提出HOMIE框架,统一处理主体间和主体内输入设置。通过更好的MLLM集成策略、自注意力中的全局多模态引导及模态参考嵌入,在多任务中达最优性能。

Comments 28 pages, 14 figures

详情
AI中文摘要

以人为对象的视频个性化(HOCVP)是主题驱动视频生成中的核心任务。现有方法存在两个关键局限。多数关注主体间个性化的方法难以在高主体保真度与人和多样物体间准确交互模式间平衡,尤其物体为抽象概念如图标时。虽主体内参考有望增强保真度,但多数现有工作缺乏理解潜在对应关系的机制。为应对这些挑战,我们提出HOMIE框架,统一处理主体间和主体内输入设置。与先前方法相比,HOMIE提出更好的MLLM集成策略,在不影响文本编码器可控性或产生高成本重新对齐的情况下提取参考级关系知识。具体而言,我们在自注意力中引入全局多模态引导,更好地对齐MLLM语义特征与VAE令牌;还提出模态参考嵌入来区分MLLM特征和VAE令牌并关联主体内参考图像令牌。大量实验验证了该方法在各种HOCVP任务中达到了当前最优性能。

英文摘要

Human-object centric video personalization (HOCVP) is a core task within subject-driven video generation. However, existing methods suffer from two key limitations. First, most approaches focusing on inter-subject personalization still struggle to strike a balance between high subject fidelity and accurate interaction patterns between humans and diverse objects, especially when objects represent abstract concepts such as logos. Second, while intra-subject references (e.g., OCR maps, multi-view inputs) are expected to enhance subject fidelity, most existing works lack mechanisms to understand such latent correspondence. To address both challenges, we propose HOMIE, an HOCVP framework that tackles both inter- and intra-subject input settings in a unified manner. Compared to previous approaches, HOMIE proposes a better MLLM integration strategy to extract knowledge of reference-level relationships without compromising the controllability of text encoders or incurring costly re-alignment. Specifically, we introduce global multimodal guidance within self-attention to better align MLLM-derived semantic features with VAE tokens. Furthermore, we propose modality-reference embedding to differentiate tokens from MLLM features and VAE tokens and associate intra-subject reference image tokens. Extensive experiments validate that our method achieves state-of-the-art performance across various HOCVP tasks. Project Page: https://yiyangcai.github.io/homie-page.github.io/

URL PDF HTML 收藏
2607.18210 2026-07-21 cs.RO 新提交

Optimization of sim-to-real transfer in the humanoid robot NICO

人形机器人NICO中模拟到现实转换的优化

Juraj Gavura, Igor Farkaš

机构 * Comenius University Bratislava(布拉迪斯拉发夸美纽斯大学)

AI总结 研究针对人形机器人模拟到现实转换中因定位误差妨碍抓取的问题,通过添加多种检测和定位方法及校正,形成基于校准的抓取管道,并实现视觉反馈模型,提高了抓取成功率。

Comments 12 pages, 8 figures, accepted to International Conference on Artificial Neural Networks 2026, Neurorobotics workshop

详情
AI中文摘要

机器人抓取需要视觉感知、物体定位、逆运动学和手部控制之间的精确协调。模拟中规划的动作在物理机器人上执行时,模拟到现实的差距会导致小定位误差,妨碍成功抓取。此前研究引入低成本触觉校准方法提高了人形机器人NICO的二维到达精度。本文通过添加基于YOLO的物体和手部检测、利用机器人内置低分辨率鱼眼相机的立体视觉定位以及抓取执行的特定任务校正,将该方法从到达扩展到桌面物体抓取。这些组件形成了一种新颖的基于校准的抓取管道,无需RGB-D相机、动作捕捉或外部跟踪系统。还实现了视觉反馈模型,在抓取前将机器人手与检测到的物体对齐。结果表明,完全非线性校准模型在校准区域内性能最佳,视觉反馈模型在整个桌面工作空间的总体抓取成功率最高。

英文摘要

Robotic grasping requires accurate coordination between visual perception, object localization, inverse kinematics, and hand control. However, when movements planned in simulation are executed on a physical robot, the sim-to-real gap can cause small positioning errors that prevent successful grasping. In our previous work, we introduced a low-cost haptic calibration method that improved 2D reaching accuracy of the humanoid robot NICO. In this paper, we extend this approach from reaching to tabletop object grasping by adding YOLO-based object and hand detection, stereo vision-based localization using the robot's built-in low-resolution fisheye cameras, and task-specific corrections for grasp execution. Together, these components form a novel calibration-based grasping pipeline that does not require RGB-D cameras, motion capture, or external tracking systems. We also implemented a visual feedback model that aligns the robot hand with the detected object before grasping. Our results show that the fully nonlinear calibration model achieved the best performance inside the calibrated area, while the visual feedback model achieved the highest overall grasping success across the full tabletop workspace.

URL PDF HTML 收藏
2607.18200 2026-07-21 cs.RO cs.AI 新提交

Learning Adaptive Safety Margins for Visual Navigation

学习用于视觉导航的自适应安全裕度

Junyi Hu, Shuaihang Yuan, Geeta Chandra Raju Bethala, Anthony Tzes, Yi Fang

机构 * New York University Abu Dhabi(纽约大学阿布扎比分校)

AI总结 研究针对室内机器人视觉导航中安全裕度校准问题,提出上下文条件安全评论家方法,通过分解的三个互补项学习自适应间隙偏好排序扩散提议,经训练和提炼后在导航任务中表现出色,还能转移到人形机器人。

详情
AI中文摘要

在杂乱的室内空间中,机器人失败往往不是因为无法生成无碰撞路径,而是固定安全裕度校准不当。基于扩散的规划器能从自中心RGB-D生成多样轨迹候选,但可靠选择仍是瓶颈。我们提出一种上下文条件安全评论家,学习自适应间隙偏好来对扩散提议排序,分解为三个互补项:安全项、效率项和距离约束匹配项。在模拟中用特权ESDF几何训练评论家,通过两阶段师生过程提炼为仅感知选择器。在PointGoal导航任务中,该方法在强扩散、优化和RL基线中取得最高成功率和成功加权路径长度,还能转移到Unitree G1人形机器人且无需特定任务调整。

英文摘要

Robots in cluttered indoor spaces often fail not because they cannot generate collision-free paths, but because a fixed safety margin is mis-calibrated: conservative margins cause detours and timeouts, while permissive margins lead to near-boundary shortcuts under perception bias. Diffusion-based planners propose diverse trajectory candidates from egocentric RGB-D, yet reliable selection remains the bottleneck. We propose a context-conditioned safety critic that learns an adaptive clearance preference for ranking diffusion proposals, decomposed into three complementary terms: (i) a safety term with a clearance-budget penalty and a control-barrier-function residual for waypoint- and transition-wise safety, (ii) an efficiency term combining a smoothness penalty with a safety-gated detour-ratio penalty that avoids detours without incentivizing risky shortcuts, and (iii) a distance-constraint matching term that anchors the learned budget to realized ESDF clearances to prevent margin collapse. We train the critic with privileged ESDF geometry in simulation and distill it into a perception-only selector via a two-stage teacher-student procedure. On PointGoal navigation in HM3D and MP3D, including cross-dataset transfer, our method achieves the highest success rate (SR) and success weighted by path length (SPL) among strong diffusion, optimization, and RL baselines. Trained purely in simulation, it transfers to a Unitree G1 humanoid and navigates cluttered indoor scenes without task-specific tuning.

URL PDF HTML 收藏
2607.18199 2026-07-21 cs.CL cs.LG 新提交

PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

PPL-Factory:从语言建模到推理的任务感知和预算感知数据选择

Hang Zhang, Warren J. Gross

机构 * McGill University(麦吉尔大学)

AI总结 研究如何从语言建模到推理进行任务和预算感知的数据选择,提出PPL-Factory框架,结合任务感知困惑度得分与数据预算感知标准,实验表明该框架在GSM8K等任务上仅用少量训练集就能超越其他方法,有效提升微调效率。

Comments 13 pages, 4 figures, 5 tables

详情
AI中文摘要

并非所有训练样本对大语言模型微调的贡献都相同。选择信息丰富的训练样本可降低计算成本并保持下游性能。现有数据选择方法多依赖间接启发式,效果因任务而异。基于困惑度的数据选择可估计样本难度,但现有方法忽略语言建模和推理任务学习目标差异。本文提出PPL-Factory,结合任务感知困惑度得分和数据预算感知选择标准。实验表明,PPL-Factory在GSM8K上仅用1%训练集就超越其他方法,用10%数据时在GSM8K和MATH上均有出色表现,证明该方法对高效微调有效且适用。

英文摘要

Not all training samples contribute equally to large language model fine-tuning. Selecting informative training samples can reduce the computational cost while preserving downstream performance. Many existing data selection methods rely on indirect heuristics, such as data quality, diversity or reasoning trace length. However, the effectiveness of these fixed criteria is task-dependent and difficult to generalize across diverse downstream tasks. Perplexity-based data selection provides a simple and model-aware solution to estimate the sample difficulty, but existing approaches typically score the entire training sequence and ignore the difference in learning objectives of language modeling and reasoning tasks. In this paper, we propose PPL-Factory, a simple and interpretable data selection framework that combines task-aware perplexity-based scores and data budget-aware selection criteria. Experiments on GSM8K demonstrate that PPL-Factory outperforms other state-of-the-art data selection methods using only $1\%$ of the training set. With $10\%$ of the data, PPL-Factory exceeds full-data fine-tuning accuracy by 0.9 on GSM8K and 4.8 on MATH. Overall, our results demonstrate that task-aware and budget-aware perplexity-based selection provides an effective and applicable approach for efficient fine-tuning.

URL PDF HTML 收藏
2607.18198 2026-07-21 cs.LG cs.CV 新提交

Three-Body Scattering for Generative Modeling

用于生成建模的三体散射

Peng Sun, Zhenglin Cheng, Deyuan Liu, Jun Xie, Xinyi Shang, Tao Lin

机构 * Westlake University(西湖大学) Zhejiang University(浙江大学) University College London(伦敦大学学院)

AI总结 研究提出三体散射建模(TBSM)用于生成,将能量距离转化为特定相互作用,为单步生成器提供监督。通过在线跟踪条件期望减少噪声,在ImageNet-256上训练单步生成器取得良好结果,确立了跟踪散射作为高维单步生成的途径。

Comments 31 pages, 5 figures, and 4 tables. Code: https://github.com/sp12138/TBSM

详情
AI中文摘要

现代生成模型通常依赖对抗性判别器、规定的噪声到数据路径或自回归分解。相反,我们表明适当的分布能量可以诱导样本级运动,并为单步生成器提供直接回归监督。用于生成的三体散射建模(TBSM)将能量距离转化为每个射弹的固定大小相互作用:每个射弹被一个真实源吸引并被一个独立生成的源排斥。基于射弹及其条件,其期望等于$\frac12D_E^2(P_{\theta},Q)$的2-瓦瑟斯坦梯度流速度。一批$B$个冻结目标事件产生$O(B)$个样本级损失,每个损失使用一个参考作为其条件,而不是像漂移模型等方法使用的小批量全对场。在线跟踪这种条件期望可以减少场噪声。使用冻结图像特征中的散射,TBSM在ImageNet-256上训练单步生成器,在NFE = 1时,像素空间的PixelDiT-XL实现FID = 2.23,潜在空间的DiT-XL实现FID = 1.63。我们提供了一个设计图,将扩散相关监督、类似漂移的动力学和类似GAN的目标联系起来。这些结果将跟踪散射确立为高维单步生成的一条途径。代码:此https URL。

英文摘要

Modern generative models typically rely on an adversarial critic, a prescribed noise-to-data path, or an autoregressive factorization. Instead, we show that a proper distributional energy can induce sample-level motion and provide direct regression supervision for a one-step generator. Three-Body Scattering Modeling (TBSM) for generation turns the energy distance into a constant-size per-projectile interaction: each projectile is attracted toward one real source and repelled from one independently generated source. Conditioned on the projectile and its condition, its expectation equals the $2$-Wasserstein gradient-flow velocity of $\frac12D_E^2(P_θ,Q)$. A batch of $B$ frozen-target events yields $O(B)$ sample-level losses, each using one reference for its condition instead of the minibatch-wide all-pairs field used by methods such as Drifting Models. Tracking this conditional expectation online can reduce field noise. Using scattering in frozen image features, TBSM trains one-step generators on ImageNet-256, achieving FID${}=2.23$ with pixel-space PixelDiT-XL and FID${}=1.63$ with latent-space DiT-XL at NFE${}=1$. We provide a design map relating diffusion-related supervision, Drift-like dynamics, and GAN-like objectives. These results establish tracked scattering as a route to high-dimensional one-step generation. Code: https://github.com/sp12138/TBSM.

URL PDF HTML 收藏
2607.18197 2026-07-21 cs.RO 新提交

Imitation of Arm Gestures by the Semi-Humanoid Robot NICO

半人形机器人NICO对手臂手势的模仿

Anastasiya Ihnatovich, Igor Farkaš

机构 * Faculty of Mathematics, Physics and Informatics Comenius University Bratislava(布拉迪斯拉发夸美纽斯大学数学、物理与信息学院)

AI总结 研究半人形机器人NICO模仿人类手臂手势的方法,基于解析几何和预训练模型,利用MediaPipe获取人体地标3D坐标计算关节角度并映射执行,实验表明该方法能从单目RGB输入产生模仿动作,也揭示了复杂姿势和手腕动作的局限。

Comments 15 pages, 7 figures, accepted to Human-Friendly Robotics workshop 2026

详情
AI中文摘要

无缝的人机交互(HRI)要求机器人具备多种感知和运动能力,其中之一是模仿人类手势。人形机器人因其拟人特征在HRI中具有优势。本文基于解析几何和预训练的MediaPipe姿态估计模型,开发了一个用于半人形机器人NICO模仿人类手臂手势的系统。对于每个输入的RGB帧,使用MediaPipe框架获取包括手臂关节和手部关键点在内的相关人体地标3D坐标。然后根据这些坐标利用推导的几何关系计算关节角度。最后,将计算出的角度正确映射到NICO的电机配置并按预定义运动序列执行。对六名不同身高参与者的几种代表性手臂手势进行的初步实验表明,该方法仅从单目RGB输入就能产生有意义的模仿动作,同时也突出了在更复杂姿势和与手腕相关运动中的局限性。

英文摘要

Seamless human-robot interaction (HRI) requires a number of perceptual and motor abilities from the robot, one of them being the imitation of human gestures. Humanoid robots have an advantage in HRI thanks to their anthropomorphic features. In this work, we develop a system for imitation of human arm gestures by the semi-humanoid robot NICO based on analytical geometry and a pretrained MediaPipe pose-estimation model. For each input RGB frame, 3D coordinates of relevant human body landmarks, including arm joints and hand keypoints, are obtained using the MediaPipe framework. Joint angles are then computed from these coordinates using derived geometric relations. Finally, the computed angles are properly mapped to NICO's motor configuration and executed in a predefined motion sequence. Preliminary experiments on several representative arm gestures with six participants of different height indicate that the proposed method can produce meaningful imitative motions from monocular RGB input only, while also highlighting limitations in more complex poses and wrist-related movements.

URL PDF HTML 收藏
2607.18195 2026-07-21 cs.CV cs.LG 新提交

Certified Training for Convolutional Perturbations

卷积扰动的认证训练

Benedikt Brückner, Alessio Lomuscio

机构 * Safe Intelligence(安全智能) Imperial College London(伦敦帝国理工学院)

AI总结 研究视觉模型受扰动问题,提出利用卷积扰动编码的认证训练方法,显著优于对抗训练,在CIFAR10上对运动模糊有超80%鲁棒准确率,且标准准确率相当。

详情
AI中文摘要

视觉模型易受运行时相机抖动引起的运动模糊等扰动影响,这阻碍了它们在关键应用中的部署。数据增强或对抗训练等方法缺乏形式安全保证。我们引入一种新颖的认证训练方法,利用卷积扰动的高效编码来训练可证明鲁棒的模型。该方法显著优于对抗训练,在CIFAR10上对合理强度运动模糊的鲁棒准确率超过80%,同时保持相当的标准准确率。

英文摘要

Vision models have been found to be susceptible to perturbations such as motion blur induced at runtime by a shaking camera. This impedes their deployment in critical applications since phenomena such as slightly blurred vision might lead to failures, for example an object detector missing objects. While methods such as data augmentation or Adversarial Training can improve empirical robustness, they lack formal safety guarantees, making it difficult to identify and mitigate hidden vulnerabilities. We introduce a novel Certified Training approach that leverages an efficient encoding of convolutional perturbations to train provably robust models. Our method significantly outperforms Adversarial Training, achieving, for example, over 80% robust accuracy against motion blur of reasonable intensity on CIFAR10 while maintaining comparable standard accuracy.

URL PDF HTML 收藏
2607.18189 2026-07-21 cs.SD eess.AS 新提交

Dense-Sparse Dynamic Time Warping for Customizing Piano Concerto Accompaniments

用于定制钢琴协奏曲伴奏的密集-稀疏动态时间规整

TJ Tsai, Kavi Dey, Yigitcan Ozer, Meinard Muller

机构 * Music Minus One(音乐减一公司)

AI总结 研究钢琴家定制MMO协奏曲伴奏的方法,提出密集-稀疏DTW,通过使用三种音频数据,解决频谱不匹配问题,建立数据收集与评估框架,实验表明该方法在定制伴奏录音上比复杂方法性能更好或相当。

Comments Published at ICASSP 2025

Journal ref Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025, pp. 1-5

详情
AI中文摘要

在本研究中,我们探索钢琴家如何定制音乐减一(MMO)协奏曲伴奏以匹配他们的演奏风格。绕过通常没有数字形式的符号乐谱需求,我们使用三种类型的音频数据:独奏钢琴录音、仅MMO管弦乐队录音以及钢琴和管弦乐队的混合录音(例如来自YouTube)。混合录音作为中介参考来对齐独奏和管弦乐队部分,仅通过时间尺度修改调整管弦乐队部分以与用户演奏同步。估计这些对齐的主要挑战是包含不同音乐部分的录音之间的频谱不匹配。受此应用场景启发,我们引入密集-稀疏动态时间规整(Dense-Sparse DTW),它是动态时间规整(DTW)的一种变体,旨在通过专注于对齐包含突出时间线索的选定音频帧子集来提高对齐对频谱不匹配的鲁棒性。我们从四个钢琴协奏曲乐章收集并标注数据,建立了一个生成和评估定制伴奏录音的框架。在此基准上,我们表明密集-稀疏DTW比基于源分离和频谱减法技术的更复杂方法具有更好或相当的性能。

英文摘要

In this study, we explore how pianists can customize Music Minus One (MMO) concerto accompaniments to match their playing style. Bypassing the need for a symbolic score, often not available digitally, we use three types of audio data: solo piano recordings, MMO orchestra-only recordings, and mixed recordings of both piano and orchestra (e.g., from YouTube). The mixed recording serves as an intermediary reference to align the solo and orchestra parts, with only the orchestral part being adjusted through time-scale modification to synchronize with the user's playing. The main challenge with estimating these alignments is the spectral mismatch between recordings containing different musical parts. Motivated by this application scenario, we introduce Dense-Sparse DTW, a variant of Dynamic Time Warping (DTW) that is designed to improve robustness of alignments to spectral mismatch by focusing on aligning a selected subset of audio frames containing prominent timing cues. We collect and annotate data from four piano concerto movements and establish a framework for generating and evaluating customized accompaniment recordings. On this benchmark, we show that Dense-Sparse DTW has better or comparable performance than more complex approaches based on source separation and spectral subtraction techniques.

URL PDF HTML 收藏
2607.18181 2026-07-21 cs.CL cs.SE 新提交

VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design

VEHBench:用于大语言模型辅助振动能量采集器设计的阶段局部诊断基准测试

Depeng Su, Yuyu Luo, Guobiao Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))

AI总结 研究针对无电池物联网中振动能量采集器设计,引入VEHBench基准测试,评估大语言模型在耦合物理设计不同阶段的表现,通过763个任务及四个设计角色测试,发现模型能力依赖阶段,为评估、选择等工程大语言模型提供阶段感知基础。

详情
AI中文摘要

无电池物联网需要在耦合物理约束下对振动能量采集器进行迭代设计,而大语言模型正成为工程工作流程的接口层。然而,现有的工程基准测试主要评估最终工件的有效性,对于大语言模型在耦合物理设计的不同阶段的表现提供的见解有限。我们引入了VEHBench,这是一个用于大语言模型辅助振动能量采集器设计的工程原生诊断基准测试,具有763个基于文献的任务,并由一个分析物理预言机评分。VEHBench评估四个设计角色:规范分类、验证器引导搜索、损坏状态恢复和策略条件选择。实验结果表明,大语言模型的能力强烈依赖于阶段:没有一个单一模型能在整个工作流程中始终占据主导地位,并且响应控制配置文件揭示了不同设计角色之间不同的行为模式。因此,VEHBench为评估、选择、路由和改进基于验证器的工程大语言模型提供了一个阶段感知基础。基准工件可在这个https URL上获取。

英文摘要

Battery-free Internet of Things (IoT) requires iterative design of vibration energy harvesters (VEHs) under coupled physical constraints, while LLMs are emerging as interface layers for engineering workflows. However, existing engineering benchmarks primarily assess final artifact validity, offering limited insights into how LLMs behave across different stages of coupled physical design. We introduce VEHBench, an engineering-native diagnostic benchmark for LLM-assisted VEH design, featuring 763 literature-grounded tasks scored by an analytical physical oracle. VEHBench evaluates four design roles: specification triage, verifier-guided search, corrupted-state recovery, and policy-conditioned selection. Experimental results reveal that LLM capability is strongly stage-dependent: no single model consistently dominates the entire workflow, and response-control profiles expose distinct behavioral patterns across design roles. VEHBench thus provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs. The benchmark artifact is available at https://huggingface.co/datasets/AnonymousVehbench/vehbench

URL PDF HTML 收藏
2607.18171 2026-07-21 cs.LG 新提交

FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications

FlashRT:用于引导智能体部署实时多模态应用的智能体框架

Krish Agarwal, Zhuoming Chen, Yanyuan Qin, Zhenyu Gu, Atri Rudra, Beidi Chen

机构 * Carnegie Mellon University(卡内基梅隆大学) AMD(超威半导体公司) University at Buffalo(纽约州立大学水牛城分校)

AI总结 研究实时多模态应用部署难题,提出FlashRT智能体框架。通过新范式引导编码智能体多阶段转换,将参考实现转为高效部署,在不同GPU上显著提升性能,尤其在专家优化不成熟平台更具扩展性。

详情
AI中文摘要

实时多模态应用,如语音智能体和交互式视频生成,将异构模型组合成管道,其高效部署需要针对放置、流和模型内并行性做出特定于应用的决策。现有服务系统和自动并行编译器的转换有限且工作量假设固定。我们提出了FlashRT,一种智能体框架,它引导编码智能体将简单的开发者编写的参考实现提升为优化的多GPU部署,灵活权衡延迟和吞吐量等目标指标。通过新的程序链范式,FlashRT指导通用编码智能体进行多阶段转换过程,包括将参考转换为中间表示以捕获数据依赖和持久状态范围,通过顺序解释器验证此IR,并进行静态分析以识别候选转换。然后,智能体在测量门控优化循环下迭代实现、验证和基准测试每个候选,以产生跨越不同硬件预算的有效部署。在各种应用中,FlashRT将参考实现转换为高效部署,在NVIDIA B200 GPU上延迟降低高达约70倍,吞吐量提高2.8倍。在AMD MI355X GPU上,FlashRT在匹配峰值延迟降低的同时将峰值吞吐量提高到3.6倍,表明在专家优化不太成熟的平台上,智能体驱动的优化更具可扩展性。实际上,对于Qwen3 - Omni文本到音频推理,与AMD MI355X上的专家vLLM - Omni实现相比,FlashRT将响应延迟降低了65%。

英文摘要

Real-time multimodal applications, including voice agents and interactive video generation, compose heterogeneous models into pipelines whose efficient deployment requires application-specific decisions about placement, streaming, and intra-model parallelism. Existing serving systems and auto-parallelism compilers commit to limited transformations and fixed workload assumptions, so achieving high performance on a new application requires hand-crafting an efficient implementation. We present FlashRT, an agent harness that guides coding agents to lift simple developer-written reference implementations into optimized multi-GPU deployments that flexibly weigh target metrics like latency and throughput. Using a new chain-of-program paradigm, FlashRT directs a generic coding agent through a multi-pass transformation process where an agent transforms the reference into an intermediate representation (IR) to capture data dependencies and persistent-state scopes, validates this IR via a sequential interpreter, and performs static analyses to identify candidate transformations. Then, the agent iteratively implements, verifies, and benchmarks each candidate under a measurement-gated optimization loop to produce effective deployments that span different hardware budgets. Across various applications, including video world models and multimodal LLMs, FlashRT converts reference implementations into highly efficient deployments, delivering up to ~70x latency reduction and 2.8x throughput improvement on NVIDIA B200 GPUs. On AMD MI355X GPUs, FlashRT matches the peak latency reduction while increasing peak throughput improvement to 3.6x, demonstrating that agent-driven optimization can be more scalable on platforms with less mature expert optimization. In fact, for Qwen3-Omni text-to-audio inference, FlashRT reduces response latency by 65% compared to the expert vLLM-Omni implementation on AMD MI355X.

URL PDF HTML 收藏
2607.18163 2026-07-21 cs.LG cs.AI 新提交

OR Else: A Differentiable Trust Region for Policy Optimization

OR 否则:用于策略优化的可微信赖域

Chinmay Rane, Kanishka Tyagi, Michael Manry

机构 * Quantiphi Inc(昆蒂菲公司) Self Machines Inc(自机器公司) The University of Texas at Arlington(德克萨斯大学阿灵顿分校)

AI总结 研究 PPO 和 GRPO 中裁剪代理目标导数突变问题,提出用输出重置(OR)规则优化。通过对比实验,在广义优势估计下 PPO-OR 有更高奖励模型得分,组相对优势下 GRPO-OR 虽平均得分未升但差异更小,OR 改变了优化行为但奖励效果有别。

Comments 22 pages, 5 figures

详情
AI中文摘要

本文研究的近端策略优化算法(PPO)和广义信赖域策略优化算法(GRPO)基线使用裁剪代理目标,其有利方向饱和会导致标量目标导数出现突变。我们探讨输出重置(OR)这种平滑单边饱和规则是否能为大语言模型训练后提供有用的替代方案。PPO-OR 和 GRPO-OR 在展开相对令牌对数比率空间中用 OR 平方边际损失替换裁剪策略项;优势符号决定更新方向,令牌越过有利边际后直接 OR 残差为零。我们在广义优势估计(GAE)下将 PPO-clip 与 PPO-OR比较,在组相对优势下将 GRPO 与 GRPO-OR 比较,使用具有一个共享奖励模型且每种方法三个种子的 Anthropic hh-rlhf 上的 Llama-3.2-1B-Instruct。在 GAE 下,PPO-OR 的平均最终训练时奖励模型得分比 PPO-clip 高 0.305,种子间差异更大。在组相对优势下,GRPO-OR 平均得分不更高,但差异更小,终端 OR 残差接近零且过冲分数下降,而匹配的 GRPO 裁剪目标轨迹仍变化。两种组相对方法的展开到当前对数比率位移比 GAE 方法大得多,OR 不能持续减少它。因此,OR 在两种匹配比较中改变了优化行为,但观察到的奖励效果不同。在 G = 2 时,GRPO-OR 诊断结果未转化为奖励得分提升。更大的组是否会改变这一结果尚待研究。报告的分数是训练时奖励模型测量值,而非保留的人类偏好性能。

英文摘要

PPO and the GRPO baseline studied here use clipped surrogate objectives whose favorable-direction saturation introduces an abrupt change in the scalar objective's derivative. We ask whether Output Reset (OR), a smooth one-sided saturation rule, offers a useful alternative for large language model post-training. PPO-OR and GRPO-OR replace the clipped policy term with an OR squared-margin loss in rollout-relative token log-ratio space; the advantage sign determines the update direction, and a token contributes zero direct OR residual after crossing the favorable margin. We compare PPO-clip with PPO-OR under generalized advantage estimation (GAE), and GRPO with GRPO-OR under group-relative advantages, using \texttt{Llama-3.2-1B-Instruct} on Anthropic \texttt{hh-rlhf} with one shared reward model and three seeds per method. Under GAE, PPO-OR has a mean final training-time reward-model score $0.305$ higher than PPO-clip, with a larger observed across-seed spread. Under group-relative advantages, GRPO-OR does not have a higher mean score, but shows a smaller observed spread, a near-zero terminal OR residual, and a declining overshoot fraction, while the matched GRPO clipped-objective trace remains variable. Both group-relative methods exhibit substantially larger rollout-to-current log-ratio displacement than the GAE methods, and OR does not consistently reduce it. Thus, OR changes optimization behavior in both matched comparisons, but the observed reward effect differs between them. At $G=2$, the GRPO-OR diagnostics do not translate into a reward-score gain. Whether larger groups change this outcome remains open. The reported scores are training-time reward-model measurements, not held-out human-preference performance.

URL PDF HTML 收藏
2607.18162 2026-07-21 cs.LG 新提交

The Calibration Channel Determines the Bayes-Error Proxy: An Exact Law for Temperature-Induced Distortion

校准通道决定贝叶斯误差代理:温度诱导失真的精确定律

Shreyas Pradeepkumar Khandale

机构 * Binghamton University (SUNY)(纽约州立大学宾汉姆顿分校) School of Computing(计算机学院)

AI总结 研究软标签贝叶斯误差估计器在概率非真实后验时的脆弱性,通过刻画温度缩放对代理的影响,证明相关恒等式并推导封闭形式,在多数据集上验证,给出失真预测,强调代理值与概率产生机制结合才有意义。

Comments 6 pages, 2 figures. Code available at https://github.com/sherurox/bayes-proxy-channel

详情
AI中文摘要

石田等人的软标签贝叶斯误差估计器β(z)=E[min(z,1 - z)]直接从概率值标签估计二元任务的不可约误差。Ushio等人最近的工作表明,当概率不是真实后验时,该估计器很脆弱。我们通过精确刻画最常用的事后校准映射——温度缩放——如何扭曲代理来补充这项工作。我们证明了一个精确的、无模型的恒等式,将温度缩放后的代理简化为分类器的边缘分布,由此我们得到了温度的严格单调性以及从温度轴到开区间(0,1/2)的连续双射。在对数its的高斯模型下,我们进一步推导出整个代理与温度曲线的双参数封闭形式。在CIFAR - 10、Fashion - MNIST和SVHN(八个二元任务)上,代理在恒定测试误差下变化56倍到980倍,封闭形式将经验曲线的再现误差控制在0.018以内,并且使预期校准误差最小化的校准温度与任何稳定的代理值都不一致。我们的结果给出了失真的精确预测描述,强化了只有代理值与其产生概率的机制一起才有意义的实际建议。

英文摘要

The soft-label Bayes-error estimator beta(z) = E[min(z, 1-z)] of Ishida et al. estimates the irreducible error of a binary task directly from probability-valued labels. Recent work by Ushio et al. showed that this estimator is fragile when the probabilities are not the true posterior: even perfectly calibrated soft labels can yield a substantially inaccurate estimate, and they propose isotonic calibration as a consistent remedy. We complement that line of work by characterizing exactly how the most widely used post-hoc calibration map -- temperature scaling -- distorts the proxy. We prove an exact, model-free identity reducing the temperature-scaled proxy to the classifier's margin distribution, from which we obtain (i) strict monotonicity in the temperature and (ii) a continuous bijection from the temperature axis onto the open interval (0, 1/2), so that a fixed classifier -- with fixed decisions and fixed 0-1 error -- can be made to report any proxy value whatsoever. Under a Gaussian model of the logits we further derive a two-parameter closed form for the entire proxy-versus-temperature curve. Across CIFAR-10, Fashion-MNIST, and SVHN (eight binary tasks), the proxy varies by 56x to 980x at constant test error, the closed form reproduces the empirical curve to within 0.018, and the calibration temperature that minimizes the expected calibration error does not coincide with any stable proxy value. Our results give a precise, predictive account of the distortion whose existence motivates calibration-based remedies, and they reinforce the practical recommendation that a proxy value is meaningful only together with the mechanism that produced its probabilities.

URL PDF HTML 收藏
2607.18154 2026-07-21 cs.RO 新提交

World Translation: Minimizing Sim-to-Real Gap with Backward Dynamics Extraction and Unpaired Domain Translation

世界翻译:通过反向动力学提取和非配对域翻译最小化模拟与现实之间的差距

Xinchen Yao, Leixin Chang, Hua Chen

机构 * Zhejiang University(浙江大学) LimX Dynamics(LimX动力学公司)

AI总结 研究模拟与现实差距问题,提出“世界翻译”方法,利用模拟器和学习动力学互补优势,反向提取动力学信息并跨域转换,实验表明该方法能更准确建模,在特定情况下优势明显,还证实可改进策略转移。

Comments 8 pages, 8 figures

详情
AI中文摘要

在现实世界中部署经过模拟训练的机器人策略时,模拟与现实之间的差距仍然是一个基本挑战。实到模拟方法从现实角度缩小了这一差距,从现实数据中学习过渡动力学以构建更逼真的数字世界,其中学习到的动力学模型是主要实例。然而,此类方法面临部分可观测性问题,相同观测可能因不可观测因素而导致不同过渡。现有方法假定这些因素可从观测历史中恢复,但在观测历史无信息时可能失败。为解决此限制,我们提出了“世界翻译”,利用模拟器和学习到的动力学的互补优势。我们从观测到的过渡中反向提取不可观测的动力学信息,然后将此特征作为非配对域翻译问题在模拟与现实之间进行转换,以在转移域风格的同时保留动力学内容。在人形机器人、四足机器人和操纵器平台上的实验表明,我们的方法比基线实现了更准确的动力学建模,在无法从观测历史中恢复不可观测因素时收益最大。在Go2四足机器人上的实际机器人部署证实了改进的策略转移。

英文摘要

The gap between simulation and reality remains a fundamental challenge in deploying simulation-trained robotic policies in the real world. Real-to-sim methods narrow this gap from the real side, learning transition dynamics from real data to build a more realistic digital world. Learned dynamics models are their dominant instance. Such methods, however, face a partial observability problem: the same observation may branch to different transitions due to unobservable factors. Existing methods assume these factors can be recovered from observation history. However, this may fail whenever observation history is uninformative, such as a sudden contact event with no prior warning. To address this limitation, we propose \textit{World Translation}, which exploits a complementary strength of simulators and learned dynamics. Simulators are deterministic but physically imperfect, while learned models are accurate but underdetermined under partial observability. Rather than predicting transitions forward from history, we extract the unobservable dynamics information backward from an observed transition, then translate this feature across simulation and reality as an unpaired domain-translation problem that preserves dynamics content while transferring domain style. Experiments across humanoid, quadruped, and manipulator platforms show that our method achieves more accurate dynamics modeling than baselines, with the largest gains when unobservable factors cannot be recovered from observation history. Real-robot deployment on Go2 quadruped confirms improved policy transfer.

URL PDF HTML 收藏
2607.18153 2026-07-21 cs.CV 新提交

Robust Multimodal Dynamic Object Segmentation

鲁棒多模态动态目标分割

Zhe Xin, Hanzhi Chang, Penghui Huang, Yinian Mao, Guoquan Huang

机构 * Meituan UAV(美团无人机) University of Delaware(特拉华大学)

AI总结 研究针对动态目标分割难题,提出整合多模态线索的框架,设计结合Transformer与特征聚类模块的网络进行分类,引入新后处理方法,在动态目标分割和静态场景重建任务中取得最优性能。

Comments Accepted by IEEE International Conference on Robotics & Automation ICRA 2026

详情
AI中文摘要

动态目标分割在许多视觉应用中起着关键作用,如从动态视频进行静态场景重建。现有的基于光流的方法无法确保沿物体边界的静态/动态分割一致,基于3D重建的方法对重建误差高度敏感。为解决这些局限,我们提出一个动态目标分割框架,通过整合多模态线索生成精确完整的动态掩码。设计了结合Transformer架构与特征聚类聚合模块的网络进行多模态特征轨迹的静态/动态分类,还引入基于点查询的SAM后处理方法处理单个掩码内的多个物体。大量实验表明该方法在动态目标分割和静态场景重建任务中达到了当前最优性能。

英文摘要

Dynamic object segmentation plays a critical role in many visual applications such as static scene reconstruction from dynamic videos. However, existing optical flow-based methods fail to ensure consistent static/dynamic segmentation along object boundaries, while 3D reconstruction-based approaches are highly sensitive to reconstruction errors. To address these limitations, we present a dynamic object segmentation framework that can generate both precise and complete dynamic masks by integrating multimodal cues including 2D point tracks, 3D reconstruction, and semantic information. We design a network combining Transformer architectures with feature clustering aggregation modules to perform static/dynamic classification of multimodal feature trajectories. It enables the model to adaptively determine which type of feature should dominate based on the characteristics of each scene, while also mitigating the impact of feature degradation. Additionally, we introduce a novel point-query-based SAM post-processing method capable of handling multiple objects within a single mask. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in both dynamic object segmentation and static scene reconstruction tasks.

URL PDF HTML 收藏
2607.18151 2026-07-21 cs.CV 新提交

Plenoptic Condensation: A Novel Approach to Generalized Scene Reconstruction

全光凝聚:一种广义场景重建的新方法

Brevin Tilmon, Alex DeJournett, John Leffingwell, Scott Ackerson

机构 * Quidient(奎迪恩特)

AI总结 本文提出全光凝聚(PCon)的广义场景重建新方法,通过多阶段管道转换和凝聚场景元素,其现实模型实现空间变化表征能力。实验表明该方法在重建精度上超现有先进方法,如在“受损菲亚特”案例中能精准测量损伤,优于对比方法。

详情
AI中文摘要

我们提出了一种名为全光凝聚(PCon)的新型广义场景重建(GSR)方法。PCon采用多阶段重建管道,先将图像转换为低(表征)能力的“汤状”场景元素,再自适应地将“汤”凝聚为高能力的“结构化”元素,如锐利边缘和平滑反射表面。PCon的场景模型——现实模型(Relms)实现空间变化的表征能力,对高保真渲染、测量和场景理解至关重要。我们展示了用消费级手机相机和无人机拍摄的多个PCon野外重建实例。在“受损菲亚特”案例中,PCon与两种先进的GSR方法NeRO和RT - Splatting进行基准测试。结果显示,PCon重建汽车引擎盖的精度是先进方法的两倍多,且PCon的局部损伤轮廓误差为35微米(0.035毫米),而其他两种先进方法基本无法测量损伤。

英文摘要

We present a novel Generalized Scene Reconstruction (GSR) approach called Plenoptic Condensation (PCon). PCon uses a multi-stage reconstruction pipeline, initially converting images into "soupy" scene elements with low (representational) power, then adaptively condensing the "soup" into "structured" elements of higher power capable of efficiently representing, for example, sharp edges and smooth reflective surfaces. PCon scene models called Reality Models (Relms) enable spatially varying representational power, which is essential for high-fidelity rendering, measurement, and scene understanding. We showcase several in-the-wild PCon reconstructions captured with consumer phone cameras and drones. In one case called "Damaged Fiat", PCon is benchmarked against two state-of-the-art (SOTA) GSR methods: NeRO and RT-Splatting. Referring to Figure 1 below, PCon reconstructs the car hood more than twice as accurately as the SOTA methods. But more importantly, the local damage profile error for PCon is 35 um (0.035 mm), whereas the two other SOTA methods are essentially unable to measure the damage at all. Our project website is available at https://quidient.github.io/pcon-2026.html.

URL PDF HTML 收藏
2607.18150 2026-07-21 cs.CV cs.GR 新提交

Lossless-INR: Lossless Volumetric Implicit Neural Representations

无损 INR:无损体隐式神经表示

Kaiyuan Tang, Daniel Burke, Chaoli Wang

机构 * University of Notre Dame(圣母大学)

AI总结 研究针对体数据的无损 INR 框架,核心方法是基于位平面分解,将重建转为逐位二进制分类,结合八叉树块分区与三元特征网格网络,实现零误码率和逐位精确重建,利于逼真渲染与下游分析。

Comments Accepted in IEEE VIS 2026 (short paper)

详情
AI中文摘要

隐式神经表示(INR)方法提供连续的坐标到值的映射,并与直接体绘制自然集成,使其对表示体数据具有吸引力。然而,现有的基于 INR 的体数据方法本质上是有损的,即使是小的重建误差也会在渲染和下游分析中传播。在这项工作中,我们探索了 Lossless-INR,这是一个基于位平面分解的用于 3D 科学体数据的无损 INR 框架。通过将每个体素值分解为二进制位平面,我们将重建重新表述为逐位二进制分类,从而精确恢复简化为正确预测每个位。为了在保持表示紧凑的同时使这种优化易于处理,我们结合了一种八叉树块分区策略,该策略自适应地细分复杂区域,以及一个三元特征网格网络,其网格条目由一组三元值参数化。在各种体数据集上的实验表明,这种设计可以实现零误码率和逐位精确重建,从而能够以紧凑的表示进行逼真的渲染和下游分析。代码可在这个 https URL 上获取。

英文摘要

Implicit neural representation (INR) methods provide continuous coordinate-to-value mappings and integrate naturally with direct volume rendering, making them attractive for representing volumetric data. However, existing INR-based approaches for volumetric data are inherently lossy, and even small reconstruction errors can propagate through rendering and downstream analysis. In this work, we explore Lossless-INR, a lossless INR framework for 3D scientific volumetric data based on bit-plane decomposition. By decomposing each voxel value into binary bit-planes, we reformulate reconstruction as per-bit binary classification, so that exact recovery reduces to predicting every bit correctly. To make this optimization tractable while keeping the representation compact, we combine an octree block-partitioning strategy that adaptively subdivides complex regions with a ternary feature-grid network whose grid entries are parameterized by a ternary set of values. Experiments on diverse volumetric datasets show that this design can achieve zero bit-error rate and bit-exact reconstruction, enabling faithful rendering and downstream analysis with a compact representation. The code is available at https://github.com/TouKaienn/Lossless-INR.

URL PDF HTML 收藏
2607.18149 2026-07-21 cs.LG cs.AI 新提交

Differentiable Logic Gate Networks for Low-Latency EEG Classification on Edge Devices

用于边缘设备低延迟脑电图分类的可微逻辑门网络

Shyamal Y. Dharia, Stephen D. Smith, Camilo E. Valderrama

机构 * The University of Winnipeg(温尼伯大学) University of Manitoba(曼尼托巴大学) University of Calgary(卡尔加里大学)

AI总结 研究边缘设备上低延迟脑电图分类问题,提出可微逻辑门网络Diff-Logic,通过实验将其与MLP、BNN比较,结果表明Diff-Logic在痴呆筛查中表现优,推理时间稳定,确立其为资源受限脑机接口实用范式。

Comments Published in the Proceedings of the 39th Canadian Conference on Artificial Intelligence, PMLR 318, pages 377-391, 2026. Conference version: https://proceedings.mlr.press/v318/dharia26a.html

Journal ref Proceedings of the 39th Canadian Conference on Artificial Intelligence, PMLR 318:377-391, 2026

详情
AI中文摘要

边缘设备上的实时脑电图分类受传统神经网络浮点运算的限制。我们研究了可微逻辑门网络(Diff-Logic),它可将模型编译为通过按位CPU操作执行的纯布尔电路。通过在跨越两个分类任务(二元痴呆检测和3类情感识别)的四个脑电图数据集上进行严格的等参数实验,我们在四个复杂度级别(50k - 500k参数)将Diff-Logic与匹配容量的多层感知器(MLP)和二值化神经网络(BNN)基线进行比较。在痴呆筛查中,Diff-Logic的宏F1达到80.2%,比MLP基线高6.8%。在情感识别中,MLP保持适度性能优势,但在功率受限(7W)的英伟达Jetson Orin Nano CPU(单核)上部署时,延迟高2.3倍,模型大小大14倍。关键的是,Diff-Logic推理时间在模型规模增加10倍时几乎保持不变,在最大复杂度级别比MLP实现了2.9倍的峰值加速。我们的结果确立了基于逻辑的神经架构作为资源受限脑机接口的实用范式,在满足便携式边缘部署的延迟和内存限制的同时,实现了有竞争力或更优的性能。代码可在GitHub上获取:此https URL

英文摘要

Real-time EEG classification on edge devices is bottlenecked by the floating-point arithmetic of conventional neural networks. We investigated Differentiable Logic Gate Networks (Diff-Logic) as a hardware-native alternative that compiles models into pure Boolean circuits executable via bitwise CPU operations. Through rigorous iso-parameter experiments across four EEG datasets spanning two classification tasks, binary dementia detection and 3-class emotion recognition, we compared Diff-Logic against matched-capacity Multi-Layer Perceptron (MLP) and Binarized Neural Network (BNN) baselines at four complexity tiers (50k-500k parameters). On dementia screening, Diff-Logic achieved 80.2% Macro F1, outperforming the MLP baseline by 6.8%. On emotion recognition, the MLP retained a moderate performance advantage but incurred a 2.3$\times$ higher latency and 14$\times$ larger model size when deployed on a power-constrained (7W) Nvidia Jetson Orin Nano CPU (Single-core). Critically, Diff-Logic inference time remained nearly constant across a 10$\times$ increase in model scale, achieving a peak speedup of 2.9$\times$ over MLPs at the largest complexity tier. Our results establish logic-based neural architectures as a practical paradigm for resource-constrained brain-computer interfaces, achieving competitive or superior performance while natively satisfying the latency and memory constraints of portable edge deployment. Code is available on GitHub: https://github.com/Shyamal-Dharia/eeg-difflogic

URL PDF HTML 收藏
2607.18142 2026-07-21 cs.CV cs.AI cs.CL cs.MA 新提交

O-VAD: Industrial Video Anomaly Detection through Object-Centric Tracking and Reasoning

O-VAD:通过以对象为中心的跟踪和推理进行工业视频异常检测

Mei Yuan, Qi Long, Qifeng Wu, Zhenyang Li, Yizhou Zhao, Lei Wang, Yang Liu, Min Xu

机构 * Carnegie Mellon University(卡内基梅隆大学) University of Alabama at Birmingham(阿拉巴马大学伯明翰分校) Griffith University(格里菲斯大学)

AI总结 针对工业视频异常检测,现有基于VLM的方法在工业场景中性能不佳的问题,提出无训练的智能框架O-VAD,通过跟踪对象时空动态和推理状态轨迹来检测异常,在多数据集实验中优于多种方法且能提供可解释报告。

Comments Accepted to ECCV 2026

详情
AI中文摘要

工业视频异常检测(IVAD)旨在识别工业过程中的异常对象和事件,对现代制造和质量控制系统至关重要。现有的基于视觉语言模型(VLM)的异常推理方法在一般领域能检测开放式异常,但在工业场景中性能下降。为此引入无训练的智能框架O-VAD,无需特定领域知识,强调对象状态演变。它跟踪检测对象的时空动态和潜在变换,通过对象时间状态轨迹推理识别异常对象。该方法克服了依赖正常片段再训练或注入领域知识的现有方法的局限性。在三个IVAD数据集上的大量实验表明,O-VAD优于前沿VLM、智能框架和在各自数据集上微调的传统VAD方法,还能提供异常过程和类型的可解释报告。

英文摘要

Industrial Video Anomaly Detection (IVAD) aims to identify anomalous objects and events in an industrial process, which is crucial for modern manufacturing and quality control systems. Existing VLM-based anomaly reasoning methods are capable of detecting open-ended anomalies in general domains. However, their performance declines in industrial settings characterized by intricate object transformations, strict physics, and procedural constraints. To tackle the complexity of such interaction-intensive detection, we introduce a training-free agentic framework for anomaly detection free of domain-specific knowledge, emphasizing object state evolution like humans inspectors. It is designed to track spatial-temporal dynamics and underlying transformations of detected objects over time, and then reason over the object-wise temporal state trajectories to identify abnormal objects in grounded frames. Our method overcomes limitations of prior approaches that rely on retraining on normal clips or injecting domain knowledge as context for test-time inference. Extensive experiments on three IVAD datasets demonstrate that our method outperforms frontier VLMs, agentic frameworks, and traditional VAD methods fine-tuned on the respective datasets, while providing interpretable reports over anomaly processes and types.

URL PDF HTML 收藏
2607.18135 2026-07-21 cs.RO 新提交

Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds

艾萨克模拟到现实:基于强化学习的四足动物运动

Jordan Dowdy, Jean Chagas Vaz

机构 * University of Louisville(路易斯维尔大学)

AI总结 研究基于强化学习的四足动物运动,利用Isaac Sim和Isaac Lab训练,实现零样本模拟到现实策略,在宇树Go1硬件上验证,其速度跟踪性能佳,能从大干扰中恢复,实现特定线速度和角速度。

Comments 6 pages, 5 figures. Accepted manuscript. Published in the 2025 IEEE 21st International Conference on Automation Science and Engineering (CASE), pp. 2194-2199

Journal ref 2025 IEEE 21st International Conference on Automation Science and Engineering (CASE), pp. 2194-2199 (2025)

详情
AI中文摘要

近年来,基于学习的运动方法越来越受欢迎,展示了复杂腿部运动和全身控制的能力。强化学习(RL)作为运动的主要基于学习的方法,通常利用高性能模拟工具,提供可控且高效的训练和开发环境。然而,在模拟中表现良好的策略在物理系统上部署时经常遇到意外挑战,即模拟到现实的差距。这项工作提出了一个能够进行全身控制的强大RL运动框架。所提出的RL框架利用英伟达的新模拟工具Isaac Sim及其配套的RL框架Isaac Lab进行训练,实现了零样本模拟到现实策略。我们的策略在使用宇树Go1的物理硬件上进行了性能验证,实验结果表明,其速度跟踪性能与四足动物的集成控制器相似,具有更强的从大干扰中恢复的能力,实现了2.0 m/s的线速度和1.8 rad/s的角速度。

英文摘要

Learning-based approaches to locomotion have risen in popularity in recent years, showing the capability for complex legged locomotion and whole-body control. Reinforcement learning (RL), the primary learning-based approach for locomotion, often utilizes a high-performance simulation tool, providing a controlled and efficient training and development environment. However, policies that perform well in simulation frequently encounter unexpected challenges when deployed on a physical system, known as the sim-to-real gap. This work presents a robust RL locomotion framework capable of whole-body control. The proposed RL framework utilizes Nvidia's new set of simulation tools, Isaac Sim, and its companion RL framework, Isaac Lab, for training, achieving a zero-shot sim-to-real policy. The performance of our policy is validated on physical hardware using the Unitree Go1, with experimental results showing similar velocity tracking performance to the quadruped's integrated controller, with a greater ability to recover from large disturbances, and achieve linear velocities of 2.0 m/s and angular velocities of 1.8 rad/s.

URL PDF HTML 收藏
2607.18130 2026-07-21 cs.LG 新提交

Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning

用于参数高效微调的流形约束超连接

Valentijn Oldenburg, Floris de Kam, Bente Zuijdam, Lieve Eberson, Nicky van Zutphen, Stef de Wildt, Ivo Verhoeven

机构 * University of Amsterdam(阿姆斯特丹大学)

AI总结 研究将流形约束超连接作为参数高效微调新方法,包裹冻结主干。发现其与预训练设置作用不同,固定残差混合矩阵有益。独立时不比LoRA优,但mHC+LoRA组合在特定规模下有优势,确定残差路由是有前景的新型PEFT轴。

Comments Pre-print

详情
AI中文摘要

大多数参数高效微调(PEFT)方法调整权重或激活,使关键的Transformer组件之一(残差连接)不变。本文研究了作为一种新型PEFT方法的流形约束超连接(mHC),它是残差连接的推广,用学习到的残差路由模块包裹冻结的OLMo-2主干。研究发现mHC能微调冻结的Transformer,但其作用与原始预训练设置有根本不同,固定残差混合矩阵为单位矩阵通常能提高性能。作为独立方法,mHC并不总是优于LoRA,但在匹配的可训练参数预算下,mHC+LoRA组合能改善语言建模损失,并在1B和7B规模上显示出任务相关的基准收益。总体而言,结果表明残差路由是一个独特且有前景的新型PEFT轴。

英文摘要

Most parameter-efficient finetuning (PEFT) methods adapt weights or activations, thus leaving one of the key Transformer components unchanged: residual connections. This paper investigates Manifold-Constrained Hyper-Connections (mHC), a generalisation of residual connections, as a novel PEFT approach, wrapping frozen OLMo-2 backbones with learned residual routing modules. We find that mHC can finetune frozen Transformers, but that its role differs fundamentally from the original pre-training setting: in finetuning, fixing the residual mixing matrix to identity often improves performance. As a standalone PEFT method, mHC does not consistently outperform LoRA. However, at matched trainable parameter budgets, mHC+LoRA combinations improve language-modelling loss and show task-dependent benchmark gains at both 1B and 7B scale. Overall, our results identify residual routing as a distinct and promising novel PEFT axis.

URL PDF HTML 收藏
2607.18116 2026-07-21 cs.AI cs.CV cs.GR cs.MA cs.MM 新提交

SGA: Plug&Play Geometric Verification for Educational Video Synthesis

SGA:用于教育视频合成的即插即用几何验证

Lopez Jhon, Hinojosa Carlos, Ghanem Bernard

机构 * Universidad Industrial de Santander(桑坦德工业大学) King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学)

AI总结 研究针对教育视频合成中空间正确性和视觉清晰度难题,提出符号几何智能体SGA,通过拦截代码、提取场景图及针对性优化提升质量,引入MVQS,实验显示其在多配置下有效提升了视觉质量分数。

详情
AI中文摘要

近期工作利用大语言模型(LLMs)为教学动画生成可执行代码。但确保空间正确性和视觉清晰度仍具挑战,因现有框架忽视几何遮挡。我们提出符号几何智能体(SGA),用于以代码为中心的动画管道,拦截LLM生成的代码,执行部分提取符号场景图,检测冲突时应用针对性优化。还引入Manim视觉质量分数(MVQS)。实验表明SGA在多个配置中提升了MVQS。

英文摘要

Recent work leverages Large Language Models (LLMs) to generate executable code for pedagogical animations using libraries such as Manim. However, ensuring spatial correctness and visual legibility remains challenging, as existing frameworks emphasize pedagogical content while overlooking geometric occlusions. We propose the Symbolic Geometric Agent (SGA), a plug-and-play module for code-centric animation pipelines that intercepts LLM-generated code, performs partial execution to extract symbolic scene graphs, and applies targeted refinement when spatial conflicts are detected. We further introduce the Manim Visual Quality Score (MVQS), a deterministic rendering-free proxy for spatial integrity. Experiments on the MMMC-Code benchmark across four LLM backbones and two agentic pipelines show that SGA achieves a peak MVQS of 73.11 (Code2Video + GPT-5.1), corresponding to a 16.1% relative improvement over the raw baseline, and improves MVQS in 7 of 8 backbone x pipeline configurations.

URL PDF HTML 收藏
2607.18114 2026-07-21 cs.CL cs.AI cs.LG 新提交

How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

对齐调整如何塑造大语言模型中谄媚及相关线索诱导偏差的表征?

Prakhar Gupta, Terry Jingchen Zhang, Florent Draye, Bernhard Schölkopf, Zhijing Jin

机构 * University of Michigan(密歇根大学) Jinesis Lab, University of Toronto & Vector Institute(多伦多大学Jinesis实验室和向量研究所) Max-Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所) ELLIS Institute Tübingen(图宾根ELLIS研究所)

AI总结 研究大语言模型中谄媚及相关线索诱导偏差的位置,通过从隐藏状态提取偏差方向并经探测等三种方法三角测量,发现易感性由对齐调整而非预训练导致,还能适度去偏,偏差是对齐调整安装的不同因果活性方向。

详情
AI中文摘要

现代大语言模型极易受到输入提示中简单无形变化的影响:一个随意的暗示、一个标注错误的少样本示例或一个虚假的先前助手回复往往会使原本正确的答案翻转。我们研究这种涵盖谄媚及相关线索诱导偏差的易感性在模型内部的位置。跨越五个模型家族和七种BCT偏差类型,我们从隐藏状态中提取每个偏差方向,并通过三种方法进行三角测量:探测、留一数据集转移和因果干预。这种易感性很大程度上是由对齐调整而非预训练造成的:预训练的基础模型几乎不受这些偏差影响,其激活除了问题内容外没有特定线索信号。在对齐模型中,每个偏差成为一个单一连贯方向,我们既能解码又能引导,在每个测试家族中恢复无偏差答案。偏差在表征上保持 distinct,交叉偏差纠缠是特定于模型而非偏差类别的属性,行为相似的偏差也占据不同方向。相同的干预也是一种适度的去偏工具,在保留所有指令家族中大多数正确答案的同时,恢复了相当一部分偏差诱导的错误。因此,线索诱导偏差最好理解为对齐调整所安装的一组不同的、具有因果活性的方向,而不是大语言模型中的单一缺陷。

英文摘要

Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few-shot example, or a fake prior assistant turn often flips an originally correct answer. We study where this susceptibility, spanning sycophancy and related cue-induced biases, lives inside the model. Across five model families and seven BCT bias types, we extract a per-bias direction from hidden states and triangulate it through three measures: probing, leave-one-dataset-out transfer, and causal intervention. The susceptibility is largely installed by alignment tuning rather than pretraining: pretrained base models barely cave to these biases, and their activations carry no cue-specific signal beyond question content. Within aligned models, each bias becomes a single coherent direction that we can both decode and steer along, recovering the unbiased answer across every family we test. The biases stay representationally distinct, however: cross-bias entanglement is model-specific rather than a property of the bias category, and even behaviorally similar biases occupy different directions. The same intervention also serves as a modest debiasing tool, recovering a meaningful share of bias-induced errors while preserving most correct answers across all instruct families. Cue-induced bias is therefore best understood not as a single flaw in LLMs but as a family of distinct, causally active directions that alignment tuning installs.

URL PDF HTML 收藏
2607.18110 2026-07-21 cs.LG cs.CL 新提交

LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

大语言模型作为教练:不可验证任务的体验式学习

Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu, Xun Wu, Shaohan Huang, Furu Wei

机构 * Microsoft Research(微软研究院) Tsinghua University(清华大学) Peking University(北京大学)

AI总结 研究不可验证任务,提出体验式学习(EL),将LLM反馈模型从评判转为教练,通过提炼体验知识提供密集监督,在开放式任务上表现优于基于规则的RL,泛化性好且减轻奖励作弊。

详情
AI中文摘要

在开放式任务上的强化学习(RL)将基于大语言模型(LLM)的基于规则的评估压缩为标量奖励,丢弃了丰富的文本反馈,并将具有不同质量配置文件的响应混为一谈。我们提出了体验式学习(EL),它将反馈模型从作为评判的LLM重新用作作为教练的LLM。教练将其对每个策略响应的评估提炼为可转移的体验知识,该知识为教师模型提供条件,并通过策略上下文提炼被策略内化。与标量奖励相比,这个更高带宽的反馈通道提供了密集监督,并保留了高质量响应之间的细粒度偏好。在两个策略家族中,有来自策略本身或专有模型的反馈,EL在留出的和未见的开放式任务上始终优于基于规则的RL。值得注意的是,EL在训练分布之外具有更好的泛化能力,并减轻了奖励作弊。这些发现将体验知识确立为用于不可验证任务的训练后更丰富、更可泛化的学习信号。

英文摘要

Reinforcement learning (RL) on open-ended tasks compresses an LLM's rubric-based evaluation into a scalar reward, discarding rich textual feedback and conflating responses with distinct quality profiles. We propose Experiential Learning (EL), which repurposes the feedback model from an LLM-as-a-Judge into an LLM-as-a-Coach. The coach distills its assessment of each on-policy response into transferable experiential knowledge, which conditions a teacher model and is internalized by the policy through on-policy context distillation. Compared with scalar rewards, this higher-bandwidth feedback channel provides dense supervision and preserves fine-grained preferences among high-quality responses. Across two policy families, with feedback from the policy itself or a proprietary model, EL consistently outperforms rubric-based RL on held-out and unseen open-ended tasks. Notably, EL generalizes better beyond the training distribution, and mitigates reward hacking. These findings establish experiential knowledge as a richer and more generalizable learning signal for post-training on non-verifiable tasks.

URL PDF HTML 收藏