arXivDaily arXiv每日学术速递 周一至周五更新
2606.19969 2026-06-19 cs.DB cs.DC 新提交

The Bi-Channel Networking Paradigm for Database Systems in the Cloud

云数据库系统的双通道网络范式

Georg Kreuzmayr, Muhammad El-Hindi, Benjamin Wagner, Tobias Ziegler, Viktor Leis

AI总结 针对现代高速云网络中内核TCP栈成为数据库性能瓶颈的问题,提出双通道网络范式,将通信分离为高性能数据路径和可靠控制路径,结合用户空间UDP与内核TCP,在分布式shuffle和复制键值存储中实现高吞吐与低开销。

Comments Accepted to EDBT 2027 (Lille, France)

详情
AI中文摘要

当网络链路速度较慢时,云和分布式数据库系统可以依赖通用的内核抽象,并将网络通信视为黑盒。在当今快速云网络下,这种方法失效了:数据库性能受到内核TCP栈CPU开销的限制。用用户空间UDP替换TCP可以减少这种开销,但需要重新实现基本保证,如可靠性和有序性。为解决这一难题,数据库系统不应再将网络视为黑盒,而应将其与数据库操作协同设计。我们提出了数据库系统的双通道范式,将通信分为两个通道:一个用于延迟和带宽敏感操作的高性能数据路径,以及一个用于协调和恢复的可靠控制路径。我们通过结合用户空间UDP和基于内核的TCP来实现该范式,尽管其他协议栈组合也是可能的。这种设计利用了现代NIC的能力,同时保留了TCP的可靠性。我们在两个代表性场景中展示了该范式的效率和简洁性:一个分布式shuffle用三个CPU核饱和200 Gbit/s,以及一个每秒处理数百万条消息的复制键值存储。

英文摘要

When network links were slow, cloud and distributed database systems could rely on generic kernel abstractions and treat network communication as a black box. With today's fast cloud networks, this approach breaks down: database performance becomes limited by the CPU overhead of the kernel TCP stack. Replacing TCP with user-space UDP can reduce this overhead, but it requires reimplementing essential guarantees, such as reliability and ordering. To solve this conundrum, database systems should no longer treat networking as a black box but co-design it with database operations. We propose the bi-channel paradigm for database systems, which separates communication into two channels: A high-performance data path for latency- and bandwidth-sensitive operations, and a reliable control path for coordination and recovery. We implement the paradigm by combining user-space UDP and kernel-based TCP, though other stack combinations are possible. This design exploits modern NIC capabilities while preserving TCP's reliability. We demonstrate the paradigm's efficiency and simplicity in two representative settings: a distributed shuffle saturating 200 Gbit/s with three CPU cores, and a replicated key-value store processing millions of messages per second.

2606.19968 2026-06-19 cs.GT 新提交

Beyond Lower Quota: Avoiding Overrepresentation in Multi-Winner Voting

超越最低配额:避免多赢者投票中的过度代表

Anton Baychkov, Martin Lackner, Jan Maly, Oliviero Nardi, Jannik Peters

AI总结 本文提出避免过度代表的公理JUQ,引入复合Thiele规则并刻画满足该公理的Adams-AV规则,同时提出平衡避免不足与过度代表的公理JNQ。

Comments This is an extended version of the publication with the same name in the proceedings of EC 2026

详情
AI中文摘要

最近,在社会选择文献中,避免基于批准的多赢者投票中代表不足的问题受到了广泛关注。本文探讨了被广泛忽视的互补问题——避免过度代表。尽管这是一个具有具体应用的理想性质,但尚未被系统研究。直观上,过度代表发生在一个群体决定了委员会中不成比例的大部分席位,从而超过了该群体的配额。我们提出了一个强且吸引人的避免过度代表的公理,称为可证明的上限配额(JUQ)。我们引入了Thiele规则的一个推广——复合Thiele规则,并刻画了该类中满足我们公理的唯一规则。该规则Adams-AV自然地扩展了Adams分配方法,此前未被研究。此外,我们引入了一个满足JUQ的多项式时间规则。进一步,我们引入了有理由的接近配额(JNQ),这是一个平衡避免不足和过度代表的公理。它刻画了扩展Sainte-Laguë分配方法的唯一Thiele规则。最后,我们分析了我们的公理与已建立的比例性概念(如EJR+)的兼容性。

英文摘要

Recently, in the social choice literature, much attention has been given to the question of avoiding underrepresentation in approval-based multi-winner voting. In this paper, we explore the largely overlooked complementary question of avoiding overrepresentation. This has not been explored systematically, despite being a desirable property with concrete applications. Intuitively, overrepresentation happens when a group determines a disproportionately large part of the committee, thereby exceeding the group's quota. We formulate a strong and appealing axiom for avoiding overrepresentation, called justifiable upper quota (JUQ). We introduce a generalization of Thiele rules, composite Thiele rules, and characterize the unique rule in this class satisfying our axiom. This rule, Adams-AV, which naturally extends Adams' apportionment method, has not been studied before. Additionally, we introduce a polynomial-time rule that satisfies JUQ. Furthermore, we introduce justified near quota, an axiom that balances avoiding under- and overrepresentation. It characterizes the unique Thiele rule extending the Sainte-Laguë apportionment method. Finally, we analyze the compatibility of our axioms with established proportionality notions such as EJR+.

2606.19966 2026-06-19 cs.CV cs.LG 新提交

Semantic-Anchored Evidential Fusion for Domain-Robust Whole-Slide Survival Analysis

语义锚定证据融合用于域鲁棒的全切片生存分析

Yucheng Xing, Ling Huang, Pei Liu, Jingying Ma, Jiaqing Xu, Kai He, Mengling Feng

发表机构 * National University of Singapore(新加坡国立大学) Imperial College London(帝国理工学院) Hunan University(湖南大学)

AI总结 提出SAEFS框架,通过视觉问答提取语义锚点,结合双流证据提取和狄利克雷主观逻辑建模不确定性,实现跨域零样本生存分析,平均C-index提升10.2%。

详情
AI中文摘要

全切片图像(WSIs)广泛用于计算癌症预后。然而,现有方法主要关注域内性能,难以泛化到不同临床中心。这一局限性源于它们依赖像素级表示,极易受到染色协议和扫描硬件导致的域特定伪影影响。我们假设高级病理语义(如肿瘤分级和微环境结构)提供了域不变的语义表示,反映了人类病理学家的鲁棒诊断逻辑。因此,我们提出了语义锚定证据融合生存(SAEFS)框架,其中SAEFS通过视觉问答(VQA)从WSIs中推导语义锚点,采用双流WSI证据提取架构,使用基于狄利克雷的主观逻辑建模不确定性,并通过谨慎合取规则融合语义和视觉证据,以避免来自相关源的过度自信融合。仅在单一源域上训练并在四个未见域上进行零样本评估,SAEFS在预测准确性和可靠性上均一致优于最先进模型,平均C-index提升10.2%。定量分析进一步表明,VQA导出的语义特征比像素级特征表现出显著更低的跨中心差异,突显了其在跨中心临床应用中的鲁棒性。

英文摘要

Whole-slide images (WSIs) are widely used for computational cancer prognosis. However, most existing methods primarily focus on in-domain performance and fail to generalize across clinical centers. This limitation stems from their reliance on pixel-derived representations that are highly susceptible to domain-specific artifacts caused by staining protocols and scanner hardware. We hypothesize that high-level pathology semantics, such as tumor grade and micro-environmental architecture, provide a domain-invariant semantic representation that mirrors the robust diagnostic logic of human pathologists. Therefore, we propose a Semantic-Anchored Evidential Fusion Survival (SAEFS) framework, where SAEFS derives semantic anchors from WSIs via Visual Question Answering (VQA), employs a dual-stream WSI evidence extraction architecture, uses Dirichlet-based Subjective Logic to model uncertainty, and fuses semantic and visual evidence through a cautious conjunction rule to avoid overconfident fusion from correlated sources. Trained exclusively on one source domain and evaluated zero-shot across four unseen domains, SAEFS consistently outperforms state-of-the-art models both in prediction accuracy and reliability, improving the average C-index by 10.2%. Quantitative analyses further show that VQA-derived semantic features exhibit significantly lower cross-center divergence than pixel-derived features, highlighting their robustness for cross-center clinical applications.

2606.19965 2026-06-19 cs.CV cs.AI 新提交

ROSE: Benchmarking the Perception-to-Action Gap in Multimodal Models

ROSE:多模态模型中感知到行动差距的基准测试

Yihao Wang, Zijian He, Jie Ren, Keze Wang

发表机构 * Sun Yat-sen University(中山大学) Shaanxi Normal University(陕西师范大学)

AI总结 提出ROSE基准,通过固定视觉场景并变化区域约束与符号输出,测试多模态大模型在不同上下文中将相同视觉证据转化为所需行动的能力,发现模型性能下降高达44.5个百分点,揭示感知到行动的瓶颈。

Comments 29 pages, 11 figures

详情
AI中文摘要

多模态大语言模型(MLLMs)越来越被期望基于视觉信息采取行动,然而同一场景在不同任务上下文中可能需要不同的行动。模型能否可靠地将相同的视觉证据转化为当前上下文所需的行动?为了回答这个问题,我们引入了\textsc{ROSE}(\textbf{R}eference-conditioned \textbf{O}ddity and \textbf{S}ymbolic \textbf{E}xecution),一个受控基准,它在保持视觉场景固定的同时变化区域约束和所需的符号输出。通过耦合的计数和坐标行动任务,\textsc{ROSE}测试模型是否能够推断出隐含的多数参考,并在变化的上下文中基于由此产生的细粒度视觉证据采取行动。在九个最近的MLLMs中,从计数导向任务到区域条件行动的性能下降高达44.5个百分点,而人类表现达到98.8%。这种差距在成对的场景和区域中持续存在,即使同一模型在这些场景和区域上返回正确的计数,而全局点击和匹配的局部控制表明坐标定位仅解释了部分损失,揭示了在将共享视觉证据转化为上下文特定行动时存在一个独特的、模型相关的瓶颈。

英文摘要

Multimodal large language models (MLLMs) are increasingly expected to act on visual information, yet the same scene may require different actions under different task contexts. How reliably can a model turn the same visual evidence into the action required by the current context? To answer this question, we introduce \textsc{ROSE} (\textbf{R}eference-conditioned \textbf{O}ddity and \textbf{S}ymbolic \textbf{E}xecution), a controlled benchmark that holds the visual scene fixed while varying region constraints and required symbolic outputs. Through coupled counting and coordinate-action tasks, \textsc{ROSE} tests whether models can infer an implicit majority reference and act on the resulting fine-grained visual evidence under changing contexts. Across nine recent MLLMs, performance drops by as much as 44.5 percentage points from counting-oriented tasks to region-conditioned action, despite 98.8\% human performance. The gap persists on paired scenes and regions for which the same model returns the correct count, while global-click and matched local controls show that coordinate grounding explains only part of the loss, revealing a distinct, model-dependent bottleneck in turning shared visual evidence into context-specific actions.

2606.19964 2026-06-19 cs.LG cs.AR 新提交

Low-Energy Reduced RISC-V Instruction Subset Processor for Tsetlin Machine Inference at the Edge

用于边缘Tsetlin Machine推理的低能耗精简RISC-V指令子集处理器

Chanda Gupta, Sanidhya Bhatia, Shaurya Priyadarshi, Himani Panwar, Rishad Shafik, Sudip Roy

AI总结 针对Tsetlin Machine推理,提出一种领域专用RISC-V微处理器架构,通过指令精简和数据路径简化,在保持可编程性的同时实现高达98%的执行时间减少和29.7倍能耗降低。

Comments 6 pages, 6 Figures, Accepted in IEEE ISVLSI Conference 2026

详情
AI中文摘要

Tsetlin Machine (TM) 是一种基于逻辑的机器学习方法,依赖于简单的位运算和有限状态自动机,使其适用于边缘AI部署。最近的工作集中在基于Tsetlin Machine (TM) 的协处理器和加速器设计上。尽管这些设计实现了高性能,但它们通常依赖于紧密耦合的接口、微码风格的编程和外部主机处理器,限制了灵活性和编程简易性。在这项工作中,我们提出了一种面向TM推理的领域专用RISC-V微处理器架构和设计流程。利用RISC-V的模块化结构,我们设计了一个精简指令子集处理器,在保持可编程性的同时,针对TM工作负载提高了性能并降低了能耗。采用指令分析来指导指令精简,随后针对TM推理进行数据路径和控制路径的简化。在多个数据集上评估了基线RV32IM核心和所提出的精简核心,并与二值神经网络 (BNN) 进行比较,BNN由于在推理过程中依赖位运算而被用作硬件高效基线。结果表明,TM实现了相当或更高的准确率(例如,在CIFAR-2上高达88.18%,而BNN为60.0%),同时在多个数据集上执行时间减少了高达98%。此外,所提出的设计实现了平均29.7倍的能耗降低,证明了其在可编程且高效的边缘AI系统中的有效性。

英文摘要

Tsetlin Machine (TM) is a logic-based machine learning approach that relies on simple bitwise operations and finite-state automata, which makes it attractive for edge AI deployments. Recent work has focused on co-processor and accelerator designs based on Tsetlin Machines (TMs). Although these designs achieve high performance, they typically depend on tightly coupled interfaces, microcode-style programming, and external host processors, limiting flexibility and ease of programming. In this work, we present a domain-specific RISC-V microprocessor architecture and design flow tailored for TM inference. Leveraging the modular structure of RISC-V, we design a reduced instruction subset processor that retains programmability while targeting improved performance and lower energy consumption for TM workloads. Instruction profiling is employed to guide instruction reduction, followed by datapath and control path simplifications tailored to TM inference. Both the baseline RV32IM core and the proposed reduced core are evaluated across multiple datasets and compared with Binarized Neural Networks (BNNs), which serve as a hardware-efficient baseline due to their reliance on bitwise operations during inference. Results show that TM achieves comparable or higher accuracy (e.g., up to 88.18% on CIFAR-2 compared to 60.0% for BNN) while reducing execution time by up to 98% across multiple datasets. Furthermore, the proposed design achieves an average $29.7\times$ reduction in energy consumption, demonstrating its effectiveness for programmable and efficient edge AI systems.

2606.19961 2026-06-19 cs.CV 新提交

Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation

解决潜在扩散模型中RGB到SWIR图像翻译的细节瓶颈

Kaili Wang, Martin Dimitrievski, Jose Maria Salvador, Ben Stoffelen, David Van Hamme, Lore Goetschalckx

发表机构 * imec imec-IPI-Ghent University(imec-IPI-根特大学) Yale University(耶鲁大学)

AI总结 针对潜在扩散模型在RGB到SWIR图像翻译中丢失空间细节的问题,提出源条件自编码器和可学习引导编码器两种轻量级改进,在驾驶场景下将检测mAP提升至2倍,小目标提升3.4倍,并达到最优FID。

详情
AI中文摘要

潜在扩散模型(LDM)能够高效地进行图像到图像的翻译,但在压缩过程中丢弃了精细的空间细节,从而降低了下游感知任务的性能。我们识别出两个瓶颈:自编码器(丢失空间信息)和条件路径(通过朴素下采样进一步退化源信号)。我们提出了两种轻量级、与骨干网络无关的修复方法:源条件自编码器(SCAE),通过跳跃连接将高分辨率源特征注入解码器;以及可学习引导编码器(LGE),用学习到的条件信号替代朴素下采样。在驾驶场景的RGB到SWIR翻译任务上,使用两种去噪骨干网络(U-Net和DiT)进行评估,我们的方法在潜在扩散基线基础上将检测mAP提升了高达2倍,小目标(COCO-small,<32^2像素^2)上提升高达3.4倍,同时达到了最先进的FID。我们进一步表明FID与检测性能相关性较差,从而激励多轴评估。结果零样本泛化到公开的RASMD基准。我们将公开发布带有标注的测试数据、所有检查点和训练代码。

英文摘要

Latent diffusion models (LDMs) enable efficient image-to-image translation but discard fine spatial details during compression, degrading downstream perception tasks. We identify two bottlenecks: the autoencoder, which loses spatial information, and the conditioning pathway, which further degrades the source signal through naive downsampling. We propose two lightweight, backbone-agnostic fixes: a Source-Conditioned Autoencoder (SCAE) that injects high-resolution source features into the decoder via skip connections, and a Learnable Guidance Encoder (LGE) that replaces naive downsampling with a learned conditioning signal. Evaluated on RGB-to-SWIR translation for driving scenes with two denoiser backbones (U-Net and DiT), our approach improves detection mAP by up to 2x over the latent diffusion baseline, with up to 3.4x gains on small objects (COCO-small, <32^2 px^2), while achieving state-of-the-art FID. We further show that FID and detection performance are poorly correlated, motivating multi-axis evaluation. Results generalise zero-shot to the public RASMD benchmark. We will publicly release test data with annotations, all checkpoints, and training code.

2606.19960 2026-06-19 cs.IR 新提交

Stellar: Scalable Multimodal Document Retrieval for Natural Language Queries

Stellar:面向自然语言查询的可扩展多模态文档检索

Yuxiang Guo, Zhonghao Hu, Yuren Mao, Yuhang Liu, Congcong Ge, Xiaolu Zhang, Jun Zhou, Yunjun Gao

AI总结 提出Stellar框架,通过磁盘存储令牌级文档嵌入并动态加载候选嵌入,结合词汇表示过滤和高效磁盘支持的后交互,在保持检索效果的同时将内存开销和查询延迟降低1-2个数量级。

详情
AI中文摘要

多模态文档检索——从大型语料库中选择最相关的多模态文档以回答自然语言查询——在检索增强生成(RAG)系统中扮演着重要角色。最先进的方法使用多个令牌级嵌入来表示每个文档和查询,并通过后交互实现高效性。然而,这种多向量表示在检索过程中会产生大量内存开销,导致可扩展性差,阻碍了实际部署。在本文中,我们提出了Stellar,一个可扩展的多模态文档检索框架,它将令牌级文档嵌入存储在磁盘上,仅将少量候选嵌入加载到内存中进行后交互。Stellar包含两个关键组件:(i)基于词汇表示的过滤(LRF),它微调多模态大语言模型(MLLM)作为稀疏编码器,以产生高质量的词汇表示,从而实现高效且有效的文档过滤,显著减少候选集;(ii)高效的磁盘支持后交互(DLI),它设计了一种基于平衡聚类算法的磁盘令牌嵌入存储布局,并通过简单有效的成本模型动态地将必要的令牌嵌入加载到内存中。在四个真实世界基准和一个新提出的大规模数据集上的大量实验表明,与现有方法相比,Stellar在不影响检索效果的情况下,将内存开销和查询延迟降低了1-2个数量级。

英文摘要

Multimodal document retrieval--selecting the most relevant multimodal document from a large corpus to answer a natural language query--plays an essential role in Retrieval-Augmented Generation (RAG) systems. State-of-the-art methods represent each document and query with multiple token-level embeddings and use late interaction to achieve high effectiveness. However, such multi-vector representations incur substantial memory overhead during retrieval, leading to poor scalability and hindering real-world deployment. In this paper, we present Stellar, a scalable multimodal document retrieval framework that stores token-level document embeddings on disk and loads only a small set of candidate embeddings into memory for late interaction. Stellar comprises two key components: (i) Lexical Representation-based Filtering (LRF), which fine-tunes a Multimodal Large Language Model (MLLM) as a sparse encoder to produce high-quality lexical representations, enabling efficient and effective document filtering to significantly reduce the candidate set; (ii) Efficient Disk-backed Late Interaction (DLI), which designs an on-disk token embedding storage layout guided by a balanced clustering algorithm, and dynamically loads only the necessary token embeddings into memory using a simple yet effective cost model. Extensive experiments on four real-world benchmarks and a newly presented large-scale dataset demonstrate that Stellar reduces memory overhead and query latency by 1-2 orders of magnitude compared to existing methods without compromising retrieval effectiveness.

2606.19958 2026-06-19 cs.CV 新提交

SketchKeyAnime: Reference-anchored Sparse Key-Sketch Animation Synthesis

SketchKeyAnime:基于参考锚点的稀疏关键草图动画合成

Meixi Li, Xianlin Zhang, Yue Zhang, Xueming Li

发表机构 * Beijing University of Posts and Telecommunications(北京邮电大学)

AI总结 提出SketchKeyAnime视频扩散框架,通过双分支条件机制和可学习门控的草图交叉注意力,从单张参考RGB图像和稀疏关键草图生成结构可控、外观一致且时间连贯的动画,在Sakuga-42M数据集上显著优于基线方法。

详情
AI中文摘要

传统动画制作严重依赖手工绘制和迭代细化,特别是关键姿势设计、中间帧生成和角色着色。虽然现有的动画和视频生成方法取得了显著进展,但它们通常依赖于RGB边界帧、密集的帧级条件或完整的草图序列,限制了在低成本输入条件下的适用性。我们提出了SketchKeyAnime,一个视频扩散框架,用于从稀疏关键草图输入生成结构可控、外观一致且时间连贯的动画。给定单个参考RGB图像和几个按时间索引的关键草图,SketchKeyAnime引入了一种双分支条件机制,以编码局部几何约束以及语义-时间上下文。它利用草图交叉注意力,通过可学习门控融合参考图像和草图条件,并加入自适应加权损失以加强对关键草图帧和线条艺术区域的监督。在Sakuga-42M的Aesthetic子集上的实验结果表明,我们的方法始终优于代表性的动画插值和草图引导生成基线。与最佳基线相比,SketchKeyAnime将EDMD降低了31.9%,FVD降低了9.5%,展示了卓越的草图保真度和时间连贯性,同时在大多数定量指标上实现了最佳整体性能。这些结果验证了所提出的框架,并突显了其在低成本、高度可控动画创作中的潜力。

英文摘要

Traditional animation production relies heavily on manual drawing and iterative refinement, particularly for key-pose design, in-betweening, and character coloring. While existing animation and video generation methods have made notable progress, they typically depend on RGB boundary frames, dense frame-wise conditions, or complete sketch sequences, limiting their applicability under low-cost input conditions. We present SketchKeyAnime, a video diffusion framework for generating structurally controllable, appearance-consistent, and temporally coherent animations from sparse key-sketch inputs. Given a single reference RGB image and a few temporally indexed key sketches, SketchKeyAnime introduces a dual-branch conditioning mechanism to encode local geometric constraints alongside semantic-temporal context. It leverages Sketch Cross Attention to fuse reference image and sketch conditions with learnable gating, and incorporates an Adaptive Weighted Loss to strengthen supervision on key-sketch frames and line-art regions. Experimental results on the Aesthetic subset of Sakuga-42M show that our approach consistently outperforms representative animation interpolation and sketch-guided generation baselines. Compared to the best-performing baseline, SketchKeyAnime reduces EDMD by 31.9\% and FVD by 9.5\%, demonstrating superior sketch fidelity and temporal coherence, while achieving the best overall performance across most quantitative metrics. These results validate the proposed framework and highlight its potential for low-cost, highly controllable animation creation.

2606.19957 2026-06-19 cs.CY 新提交

Modest, artistic, and radical solutions to the environmental impact of image-generating machine learning

图像生成机器学习的环境影响:温和、艺术与激进的解决方案

Laura U. Marks, Jess MacCormack, Kehui Li

AI总结 针对图像生成ML的高能耗问题,从计算机工程、媒体研究和艺术角度探索非精确计算、小模型、低精度硬件等解决方案,并提出真实成本核算。

Comments Paper in Proceedings of LIMITS 2026: 12th Workshop on Computing within Limits, 2026-06-23-25, Online

详情
AI中文摘要

机器学习常被宣称能提高信息通信技术的效率,但这种微小收益被数据中心和ML就绪设备的巨大碳、水和土地足迹所淹没。我们调查了ML应用在训练和推理中的电力消耗,重点关注电力密集型的图像生成。我们的团队由一名计算机工程师、一名媒体学者和一名艺术家组成,探索了包括非精确计算、微型语言模型、低精度硬件架构、有限容量硬件以及在设计阶段预测和缓解能源需求等解决方案。我们将概述正在进行的、使用非抓取数据的道德且美学上精致的微型图像生成器的工作。着眼于经济背景,我们将提出机器学习环境影响的真实成本核算,并表明效率标准是由信息通信技术的股东资本主义框架驱动的。

英文摘要

Machine learning is often touted to improve the efficiency of ICT, but that small gain is overwhelmed by the enormous carbon, water, and land footprints of data centers and ML-ready devices. We survey the electricity consumption of ML applications in training and inference, focusing on electricity-intensive image generation. Our team of a computer engineer, a media scholar, and an artist explore solutions including inexact computing; tiny language models; low-precision hardware architectures; hardware with limited capacity; and anticipating and mitigating energy demands at the design phase. We will sketch our work in progress of an ethical and aesthetically sophisticated tiny image generator using non-scraped data. Looking to the economic context, we will propose a true-cost accounting for the environmental impact of machine learning and suggest that the criterion of efficiency is driven by the shareholder-capitalist framing of ICT.

2606.19956 2026-06-19 cs.LG 新提交

Towards Graph-Based Deep Learning for Map Generalization: Insights from Building Footprints Simplification and Aggregation

基于图深度学习的制图综合:来自建筑足迹简化和聚合的见解

Yanning Wang, Zhiyong Zhou, Zhouyu Liu, Mengni Yu, Yu Feng

发表机构 * The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)) Zhejiang University(浙江大学) Mainz University of Applied Sciences(美因茨应用科学大学)

AI总结 本研究首次探索将图深度学习应用于建筑足迹简化(节点移动预测)和聚合(链接预测),评估了GCN、GAT和GraphSAGE等架构,发现GraphSAGE在链接预测上表现较好,但节点移动预测仍具挑战,且聚合比简化更复杂。

Comments 15 pages, 20 figures, 10 tables

详情
AI中文摘要

制图综合仍然是制图学的基本任务之一,特别是对于复杂建筑足迹的简化和聚合。本研究首次探索将基于图的深度学习应用于这两项任务,在统一的图学习框架中将简化重新表述为节点移动预测,将聚合重新表述为链接预测。我们在多尺度建筑数据集上评估了代表性的图神经网络架构(GCN、GAT和GraphSAGE),结果表明GraphSAGE在链接预测准确性方面表现出相对优势,同时也揭示了精确节点移动预测中持续存在的挑战。除了定量性能外,结果还强调聚合比简化带来更大的复杂性和挑战,突显了当前深度学习方法在制图综合中捕捉更高层次空间关系的困难。尽管存在数据不平衡和需要后处理等局限性,该研究为利用深度学习方法推进自动化制图综合提供了宝贵的见解和方法方向。

英文摘要

Map generalization remains one of the fundamental tasks in cartography, especially for the simplification and aggregation of complex building footprints. This study presents the first exploratory application of graph-based deep learning to both tasks, reformulating simplification as node movement prediction and aggregation as link prediction within a unified graph learning framework. We evaluate representative graph neural network architectures (GCN, GAT, and GraphSAGE) on multi-scale building datasets, showing that GraphSAGE demonstrates relative strengths in link prediction accuracy, while also revealing persistent challenges in precise node movement prediction. Beyond quantitative performance, the results highlight that aggregation poses greater complexity and challenges than simplification, underscoring the difficulty of capturing higher-level spatial relationships in map generalization with current deep learning approaches. Although limitations such as data imbalance and the need for post-processing remain, the study provides valuable insights and methodological directions for advancing automated map generalization with deep learning approaches.

2606.19950 2026-06-19 cs.CV cs.AI 新提交

Confidence Calibration for Multimodal LLMs: An Empirical Study through Medical VQA

多模态大语言模型的置信度校准:基于医学视觉问答的实证研究

Yuetian Du, Yucheng Wang, Ming Kong, Tian Liang, Qiang Long, Bingdi Chen, Qiang Zhu

发表机构 * College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院) School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院) Zhihui Medical Technology (Shanghai) Co., Ltd.(智汇医疗科技(上海)有限公司)

AI总结 针对多模态大语言模型在医学任务中置信度与准确性不匹配的问题,提出结合多策略融合询问与专家大语言模型评估的方法,在三个医学VQA数据集上将期望校准误差平均降低40%,提升了模型可靠性。

Comments Accepted by MICCAI 2025

详情
AI中文摘要

多模态大语言模型(MLLMs)在医学任务中展现出巨大潜力,但其引发的置信度常常与实际准确性不一致,可能导致误诊或忽略正确建议。本研究首次全面分析了医学MLLMs中准确性与置信度之间的关系。提出了一种新方法,将多策略融合询问(MS-FBI)与辅助专家大语言模型评估相结合,旨在改善医学视觉问答(VQA)中的置信度校准。实验表明,我们的方法在三个医学VQA数据集上将期望校准误差(ECE)平均降低了40%,显著增强了MLLMs的可靠性。研究结果强调了领域特定校准对医疗领域MLLMs的重要性,为AI辅助诊断提供了更可信的解决方案。

英文摘要

Multimodal Large Language Models (MLLMs) show great potential in medical tasks, but their elicited confidence often misaligns with actual accuracy, potentially leading to misdiagnosis or overlooking correct advice. This study presents the first comprehensive analysis of the relationship between accuracy and confidence in medical MLLMs. It proposes a novel method that combines Multi-Strategy Fusion-Based Interrogation (MS-FBI) with auxiliary expert LLM assessment, aiming to improve confidence calibration in Medical Visual Question Answering (VQA). Experiments demonstrate that our method reduces the Expected Calibration Error (ECE) by an average of 40\% across three Medical VQA datasets, significantly enhancing MLLMs' reliability. The findings highlight the importance of domain-specific calibration for MLLMs in healthcare, offering a more trustworthy solution for AI-assisted diagnosis.

2606.19949 2026-06-19 cs.CG 新提交

Semi-Automatic Correction of 3D Tubular Structure Skeletons via Component-Wise MST and Filtered Delaunay Triangulation

三维管状结构骨架的半自动校正:基于分量最小生成树与过滤Delaunay三角剖分

Ruoxuan Yang, Chuan Li

AI总结 提出一种半自动方法,通过用户选择源点和目标点,结合分量最小生成树和过滤Delaunay三角剖分,重建合理的中心线连接,校正骨架拓扑伪影。

Comments Accepted at ACM ICMR 2026

Journal ref In Proceedings of the International Conference on Multimedia Retrieval (ICMR '26), June 16--19, 2026, Amsterdam, Netherlands. ACM, New York, NY, USA, 10 pages

详情
AI中文摘要

从三维成像中对管状结构进行骨架化对于形态分析、运输或流动模拟以及包括血管网络、植物根系和神经连接组等领域的过程规划至关重要。然而,自动骨架提取常常引入拓扑伪影,例如邻近分支之间的错误连接以及由噪声或数据缺失引起的碎片化中心线。手动校正这些伪影可能耗时且易出错,尤其是在需要精确交互时。我们提出一种半自动校正方法,从最少的用户输入重建合理的中心线连接。给定用户选择的源点和目标点,我们的方法通过结合(i)用于稳定局部传播的分量最小生成树和(ii)用于桥接间隙和处理模糊连接点的过滤三维Delaunay边图来追踪路径。候选步骤根据考虑方向连续性、空间邻近性、分量一致性和目标导向进展的得分进行排序。输出是一个有序折线(或边序列),可作为建议的校正并集成到下游骨架后处理流程中。我们在C++中实现该系统,并基于Libigl提供交互式查看器,在脑血管数据集上展示了代表性的定性结果,包括校正典型的“交叉”和“点状”伪影。虽然我们目前的验证是定性的,但该方法轻量级,可作为实用的构建块,用于生物医学成像及相关领域中更全面的交互式校正流程。

英文摘要

Skeletonization of tubular structures from 3D imaging is essential for tasks such as morphometric analysis, transport or flow simulation, and procedural planning in domains including vascular networks, plant root systems, and neural connectomes. However, automatic skeleton extraction often introduces topological artifacts, such as erroneous connections between nearby branches and fragmented centerlines caused by noise or missing data. Correcting these artifacts manually can be time-consuming and error-prone, especially when precise interaction is required. We present a semi-automatic correction method that reconstructs a plausible centerline connection from minimal user input. Given a user-selected source and target point, our method traces a path by combining (i) component-wise minimum spanning trees for stable local propagation and (ii) a filtered 3D Delaunay edge graph for bridging gaps and handling ambiguous junctions. Candidate steps are ranked using a score that accounts for direction continuity, spatial proximity, component consistency, and target-directed progress. The output is an ordered polyline (or edge sequence) that can be used as a suggested correction and integrated into downstream skeleton post-processing workflows. We implement the system in C++ with an interactive viewer based on Libigl and demonstrate representative qualitative results on brain vessel datasets, including correction of typical "crossing" and "dotted" artifacts. While our current validation is qualitative, the method is lightweight and serves as a practical building block toward more comprehensive interactive correction pipelines in biomedical imaging and related domains.

2606.19948 2026-06-19 cs.AI 新提交

Advancing DialNav through Automatic Embodied Dialog Augmentation

通过自动具身对话增强推进DialNav

Leekyeung Han, Sangwon Jung, Hyunji Min, Jinseong Jeong, Minyoung Kim, Paul Hongsuck Seo

发表机构 * Korea University(高丽大学) Trillion Labs

AI总结 提出自动生成管道构建大规模RAINbow数据集(238K episodes),结合双策略训练和定位模型,在DialNav任务上实现成功率显著提升(Val Seen +89%,Val Unseen +100%)。

Comments 29 pages, 9 figures

详情
AI中文摘要

对于能够进行物理交互的具身智能体,创建和理解对话的能力对于确保安全性和有效性至关重要。虽然DialNav~\cite{han2025dialnav}为真实感室内导航中的对话-执行循环提供了整体评估框架,但其性能仍受限于训练数据的严重稀缺(2K episodes)。为解决这一问题,我们提出了一种自动生成管道,并构建了\textbf{RAINbow}数据集,这是一个包含238K episodes的大规模训练数据集,用于DialNav。我们的管道将现有的VLN数据集转换为多轮对话,并创建了成本高效且高质量的数据集。然后,我们引入了两项额外的互补性进展以充分释放数据潜力:(1)双策略训练,一种导航训练方案,用于使导航训练与动态对话-导航循环对齐;(2)一个利用VLN知识的定位模型。通过结合这些互补性解决方案,我们的模型在\textbf{Val Seen}(58.24,\textbf{+89\%})和\textbf{Val Unseen}(29.05,\textbf{+100\%})两个分割上的成功率均大幅超越基线,建立了新的最优水平。

英文摘要

For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav~\cite{han2025dialnav} provides a framework for holistic evaluation of the dialog--execution loop in photorealistic indoor navigation, its performance remains limited by a critical scarcity of training data (2K episodes). To address this, we propose an automatic generation pipeline, and construct the \textbf{RAINbow} dataset, a large-scale training dataset with 238K episodes for DialNav. Our pipeline converts existing VLN datasets into multi-turn dialog and creates cost-efficient and high-quality dataset. Then, we introduce two additional complementary advances to unlock the data's full potential: (1) Dual-Strategy Training, a navigation training scheme to align the navigation training with the dynamic dialog-navigation loop, and (2) a localization model that leverages VLN knowledge. By combining these complementary solutions, our model substantially outperforms the baseline in success rate on both \textbf{Val Seen} (58.24, \textbf{+89\%}) and \textbf{Val Unseen} (29.05, \textbf{+100\%}) splits, establishing a new state of the art.

2606.19946 2026-06-19 cs.CL cs.LG 新提交

GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs

GEMS: 几何约束使LLM中多语义叠加成为可能

Yu Deng

AI总结 提出GEMS方法,通过范数保持加权叠加、目标注意力路径注入和实时正交化两个几何约束,解决无训练多方向激活干预中的分布偏差和方向干扰问题,在GSM8K上保持98%准确率。

Comments 30 pages, 5 figures, 20 tables. Code and logs are available at: https://github.com/LuLu663939/gems-multi-semantic-steering

详情
AI中文摘要

激活引导通过在推理时修改中间隐藏状态来控制模型行为,无需重新训练。现有方法仅处理单方向注入;当多个语义方向无约束叠加时,模型崩溃。我们证明这种崩溃分解为两个独立作用的来源:分布偏差(加法扰动在层间累积范数并将激活推出训练分布)和方向干扰(非正交语义向量叠加时相互抑制)。这两个来源定义了任何无训练多方向干预必须满足的设计约束。作为这些原则的一个实例,我们提出GEMS,一种无训练方法,将每个来源映射到相应的几何约束:针对分布偏差的范数保持加权叠加和目标注意力路径注入,以及针对方向干扰的实时正交化。在GSM8K上,注入三个并发非数学方向保持98%的准确率(基线92%),而无约束加法崩溃至4%;在Wikitext-2上,相同注入仅导致2.2%的PPL增加。组件消融隔离了每个约束的因果作用,层级探针确认正交化信号通过FFN路径存活并以语义特异性到达输出分布。定性引导效果跨架构从3B到31B迁移。

英文摘要

Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining. Existing methods handle only single-direction injection; when multiple semantic directions are superposed without constraints, the model collapses. We show that this collapse decomposes into two independently acting sources: distributional deviation, where additive perturbations accumulate in norm across layers and drive activations outside the training distribution, and directional interference, where non-orthogonal semantic vectors mutually dampen when superposed. These two sources define the design constraints that any training-free multi-directional intervention must address. As one instantiation of these principles, we propose GEMS, a training-free method that maps each source to a corresponding geometric constraint: norm-preserving weighted superposition and targeted attention-pathway injection for distributional deviation, and real-time orthogonalization for directional interference. On GSM8K, injecting three concurrent non-mathematical directions preserves accuracy at 98% (baseline 92%), while unconstrained addition collapses to 4%; on Wikitext-2, the same injection incurs only 2.2% PPL increase. Component ablation isolates the causal role of each constraint, and layer-level probes confirm that orthogonalized signals survive the FFN pathway and reach the output distribution with semantic specificity. Qualitative steering effects transfer across architectures from 3B to 31B.

2606.19944 2026-06-19 cs.CV 新提交

Timage: A Generative Text-in-Image Paradigm for Fine-Tuning Vision-Language Models

Timage: 一种用于微调视觉语言模型的文本嵌入图像生成范式

Yifeng Wu, Huimin Huang, Ruiluo Wu, Chunyi Lin, Guanhua Chen, Xian Wu, Wang Song, Ruize Han

发表机构 * Fudan University(复旦大学) Shenzhen University of Advanced Technology(深圳先进技术大学) Tencent Jarvis Lab(腾讯贾维斯实验室) Southern University of Science and Technology(南方科技大学)

AI总结 提出Timage范式,通过约束薛定谔桥将查询文本作为排版覆盖层嵌入图像,以显式空间锚点引导模型关注,在不侵蚀骨干能力前提下提升细粒度空间推理性能。

Comments ECCV

详情
AI中文摘要

多模态大语言模型(MLLMs)在细粒度空间推理中常丢失正确图像区域,因为文本查询很少携带明确的几何锚点进入像素域。现有补救方法要么重新调整模型权重,要么用冗长指令填充提示,但都无法在不侵蚀骨干通用能力的情况下可靠地将语言定位到正确的视觉坐标。我们提出Timage,一种将多模态理解重新定义为输入层面对齐问题的范式:查询被绘制为排版覆盖层直接叠加在图像上。该覆盖层的放置和外观由约束薛定谔桥(cSB)生成,这是一种熵最优传输采样器,将布局合成分解为两个耦合的随机阶段。第一阶段——区域搜索,将噪声向查询对齐的图像区域传输,同时遵守硬遮挡屏障以保护显著前景内容;第二阶段——外观塑造,通过“墨水预算”正则化调整字形大小,使渲染文本保持可读和视觉平衡。生成的覆盖层作为显式注意力信标,引导模型沿空间语义聚焦。在VMCBench基准上,Timage搭配7B骨干模型明显超越更大的专有系统和参数调优基线。该研究将审慎的输入重构定位为一种强大的、架构中立的杠杆,以增强多模态推理。

英文摘要

Multimodal Large Language Models (MLLMs) often lose track of the right image regions during fine-grained spatial reasoning, because a textual query rarely carries any explicit geometric anchor into the pixel domain. Prevailing remedies either rewire the model's weights or pad the prompt with verbose instructions, yet neither reliably pins the language to the correct visual coordinates without eroding the backbone's general competence. We introduce Timage, a paradigm that recasts multimodal understanding as an alignment problem solved at the input: the query is drawn, as a typeset overlay, onto the image itself. The placement and appearance of this overlay are produced by a Constrained Schrödinger Bridge (cSB), an entropic optimal-transport sampler that factorizes layout synthesis into two coupled stochastic stages. The first stage, Region Search, transports noise toward query-aligned image zones while obeying a hard occlusion barrier that protects salient foreground content; the second stage, Appearance Shaping, sizes the glyphs through an ``ink-budget'' regularizer so that the rendered text stays legible and visually balanced. The resulting overlay behaves as an explicit attention beacon that channels the model's focus along spatial semantics. On the VMCBench suite, Timage paired with a modest 7B backbone clearly overtakes far larger proprietary systems as well as parameter-tuned baselines. The study positions deliberate input reconstruction as a powerful, architecture-neutral lever for strengthening multimodal reasoning.

2606.19941 2026-06-19 cs.LG 新提交

Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifolds

组合性在窄深度-连接性区域中涌现:架构约束与解流形

Dat H. Do, Rushi Shah, Duc V. Le, Dianbo Liu

发表机构 * National University of Singapore(新加坡国立大学) University of Twente(特温特大学)

AI总结 研究发现组合性仅在特定稀疏网络和特定深度区间涌现,提出基于相似性的剪枝和深度预测方法,并用理论框架解释原因。

详情
AI中文摘要

组合性被认为是泛化的基础,使模型能够在新颖组合中重用有意义的原语。然而,使用标准梯度优化训练的模型很少且通常仅微弱地表现出组合内部结构,并且尚不清楚这种组合性如何或为何形成。在这项工作中,我们表明组合性在一个狭窄的连接性-深度最佳点涌现。沿着连接性轴,组合性仅出现在某些特定稀疏网络中,严重依赖于保留哪些连接而非仅权重的稀疏性。沿着深度轴,组合性在一个狭窄的、目标依赖的区域内涌现,在特定深度达到峰值,而更浅和更深的网络都失败。当深度或连接性条件被违反时,梯度下降会静默地收敛到破碎解而非组合解。为了发现并利用这种涌现,我们引入了(i)基于相似性的剪枝(SP)以恢复组合连接性,以及(ii)一个启发式深度预测器以估计组合性最可能出现的深度。最后,我们通过基于组合稀疏性、体积比论证和特征干扰界限的理论框架支持这些实证发现,解释了为什么组合解仅在狭窄的深度-连接性区域内可达。

英文摘要

Compositionality is believed to be the foundation for generalization, enabling models to reuse meaningful primitives in novel combinations. Yet, models trained with standard gradient-based optimization rarely, and often only weakly, exhibit compositional internal structure, and it remains unclear how or why such compositionality forms. In this work, we show that compositionality emerges in a narrow connectivity-depth sweet spot. Along the connectivity axis, compositionality only appears in some specifically sparse networks, heavily depends on which connections remain rather than on weights' sparsity alone. Along the depth axis, compositionality emerges within a narrow, target-dependent regime, peaking at specific depths, while both shallower and deeper networks fail. When either the depth or connectivity condition is violated, gradient descent silently converges to fractured solutions rather than compositional ones. To discover and exploit this emergence, we introduce (i) similarity-based pruning (SP) to recover compositional connectivity and (ii) a heuristic depth predictor to estimate where compositionality is most likely to appear. Finally, we support these empirical findings with a theoretical framework based on compositional sparsity, volume-ratio arguments, and feature-interference bounds, explaining why compositional solutions are reachable only in a narrow depth-connectivity regime.

2606.19939 2026-06-19 cs.CV 新提交

DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation

DiffMath:面向手写数学表达式生成的符号与图感知潜在扩散Transformer

Wei Pan, Xuhan Zheng, Yilin Shi, Huiguo He, Hiuyi Cheng, Dezhi Peng, Minghui Liao, Lianwen Jin

发表机构 * South China University of Technology(华南理工大学) Huawei Technologies Co., Ltd.(华为技术有限公司)

AI总结 提出DiffMath框架,利用LaTeX层次结构作为先验,通过关系抽象语法树、结构保持潜在表示和条件去噪,无需位置监督即可生成结构一致的手写数学表达式。

详情
AI中文摘要

手写数学表达式生成(HMEG)由于数学表达式的复杂二维布局和长程结构依赖而具有挑战性。现有方法通常依赖显式空间监督,如符号级边界框,这导致高标注成本并限制可扩展性。在这项工作中,我们提出了DiffMath,一个符号与图感知的潜在扩散框架,利用LaTeX固有的层次结构作为结构先验,消除了位置监督的需求。首先,我们设计了关系抽象语法树(RelAST),一种面向生成的表示,将MathML树蒸馏为紧凑的三元组序列[S, R, D],其中每个标记直接编码符号身份、空间关系或嵌套深度。其次,我们引入了MathVAE,通过符号感知和关系感知的感知正则化学习保持结构的潜在表示,确保潜在空间同时捕获字符语义和空间拓扑。第三,MathDiT在这个结构化潜在空间中进行条件去噪,并通过自适应层归一化(AdaLN)进一步由全局符号计数先验引导,以改善结构一致性。实验表明,DiffMath生成结构一致的手写表达式,在现有方法上实现了优越性能,并通过合成数据增强提高了下游OCR模型的准确性。

英文摘要

Handwritten Mathematical Expression Generation (HMEG) is challenging due to the complex two-dimensional layouts and long-range structural dependencies of mathematical expressions. Existing methods typically rely on explicit spatial supervision, such as symbol-level bounding boxes, which incurs high annotation costs and limits scalability. In this work, we propose DiffMath, a symbol- and graph-aware latent diffusion framework that leverages the hierarchical structure inherent in LaTeX as a structural prior, eliminating the need for positional supervision. First, we design a Relational Abstract Syntax Tree (RelAST), a generation-oriented representation that distills MathML trees into compact triplet sequences [S, R, D], where each token directly encodes a symbol identity, spatial relation, or nesting depth. Second, we introduce MathVAE, which learns structure-preserving latent representations through symbol-aware and relation-aware perceptual regularization, ensuring that the latent space captures both character semantics and spatial topology. Third, MathDiT performs conditional denoising in this structured latent space, further guided by a global symbol-count prior via Adaptive Layer Normalization (AdaLN) to improve structural coherence. Experiments show that DiffMath produces structurally consistent handwritten expressions, achieves superior performance over existing methods, and improves the accuracy of downstream OCR models through synthetic data augmentation.

2606.19938 2026-06-19 cs.CV cs.AI 新提交

Triangular Consistency as a Universal Constraint for Learning Optical Flow

三角一致性作为光流学习的通用约束

Yi Xiao, Carlos Rodriguez Coronel, Jing Zhan, Haniyeh Ehsani Oskouie, Alex Wong, Dong Lao

发表机构 * Louisiana State University(路易斯安那州立大学) University of California, Los Angeles(加州大学洛杉矶分校) Yale University(耶鲁大学)

AI总结 提出三角一致性约束,通过组合两个光流诱导第三个光流并强制三者一致,适用于不同网络架构、监督类型和数据集,在监督、无监督和迁移学习中均提升性能。

Comments Accepted by ECCV 2026

详情
AI中文摘要

我们提出三角一致性作为光流的第一性原理约束,该约束与网络架构、监督类型和数据集无关,适用于图像对和多帧设置。这个简单但强大的约束是通过组合两个光流来诱导第三个光流,并强制三者之间的一致性。组合的光流可能来自:(i) 图像对,产生循环一致性;(ii) 多个视频帧,通过时间链产生更长范围的运动;或 (iii) 图像对与受控合成变换相结合,这成为数据增强。这种三角一致性引入的计算开销可忽略不计,且不需要额外的标注。由于它直接源自光流的几何特性,不依赖于模型特定的假设,因此可作为光流训练的“通用”即插即用组件。实验表明,在监督、无监督和迁移学习设置中均有一致的改进。

英文摘要

We propose triangular consistency as a first-principled constraint for optical flow, which is agnostic to network architecture, supervision type, and dataset, and applies to both image-pair and multi-frame settings. This simple but powerful constraint is to compose two flows to induce a third flow and enforce consistency among the three. The composed flows may arise from (i) image pairs, yielding cycle consistency; (ii) multiple video frames, producing longer-range motion through temporal chaining; or (iii) image pairs combined with controlled synthetic transformations, which becomes data augmentation. This triangular consistency introduces negligible computational overhead and requires no additional annotations. Since it is derived directly from the geometry of optical flow, it does not rely on model-specific assumptions and serves as a ``universal'' plug-and-play component for optical flow training. Experiments show consistent improvement across supervised, unsupervised, and transfer learning settings.

2606.19937 2026-06-19 cs.CR 新提交

AutoTam: Specifying Secure Protocol Implementations with Tamarin Model Generation

AutoTam: 通过 Tamarin 模型生成指定安全协议实现

Johannes Wilson, Mikael Asplund, Niklas Johansson

AI总结 提出一种语言优先方法,通过领域特定语言实现协议并自动生成 Tamarin 模型,验证迹属性并保证其传递到实现,同时集成符号执行分析内存安全,在签名 Diffie-Hellman 和 WireGuard 协议上验证了安全性和互操作性。

Comments 19 pages, 5 figures

详情
AI中文摘要

形式化验证是确保密码协议安全性的重要但具有挑战性的任务。虽然现代协议验证工具显著减少了验证工作量,但对于没有形式化验证背景的从业者来说,建模仍然具有挑战性。此外,将验证结果转移到具体的协议实现需要专业知识。在本文中,我们提出了一种新颖的语言优先方法,通过使用领域特定语言进行协议实现来验证迹属性。我们针对 Tamarin 证明器进行验证,并证明验证的通用迹属性可以转换回实现。我们还集成了符号执行以分析协议实现的内存安全性。我们使用我们的工具实现并生成了签名 Diffie-Hellman 协议和 WireGuard VPN 协议的准确模型。当使用我们的解释器时,我们的 WireGuard 实现与现有实现可互操作,并达到了可接受的性能。我们通过符号执行和生成的 Tamarin 模型的验证相结合,正式证明了我们的实现是安全的。

英文摘要

Formal verification is a challenging but important task for ensuring the security of cryptographic protocols. While modern protocol verification tools significantly reduce verification effort, modelling remains challenging to practitioners without a background in formal verification. In addition, transferring verification results to a concrete protocol implementation requires expert knowledge. In this paper, we present a novel language-first method for verification of trace properties using a domain-specific language for protocol implementations. We target the Tamarin prover for verification, and we prove that verified universal trace properties translate back to the implementation. We additionally integrate symbolic execution in order to analyse the memory safety of protocol implementations. We use our tool to implement and generate accurate models for a signed Diffie-Hellman protocol, and for the WireGuard VPN protocol. Our WireGuard implementation is interoperable with existing implementations when using our interpreter, and achieves acceptable performance. We formally prove our implementations secure using a combination of symbolic execution and verification of the generated Tamarin models.

2606.19936 2026-06-19 cs.LO cs.MM 新提交

Prismriver: Formalization of Music Theory and Algorithmic Composition in Lean 4

Prismriver:Lean 4 中音乐理论与算法作曲的形式化

Leni Aniva, Claire Wang

AI总结 使用 Lean 4 形式化音乐理论,实现可验证的算法作曲与伴奏生成,并支持音乐结构的单子分析。

详情
AI中文摘要

音乐理论遵循丰富的数学规则和对称性。这些对称性遵循数学结构,可以在证明助手的精确语言中进行验证和表达。在本文中,我们提出了 Prismriver,一个在 Lean 4 中形式化的音乐理论。通过在 Lean 4 中形式化音乐理论,我们为可验证的算法作曲和伴奏生成打开了大门。我们还实现了对音乐结构中单子分析的支持。

英文摘要

Music theory obeys a rich set of mathematical rules and symmetries. These symmetries follow mathematical structure which can be verified and expressioned in the precise language of a proof assistant. In this paper, we present Prismriver, a formalization of music theory in Lean 4. By formalizing music theory in Lean 4, we open the door to verifiable algorithmic composition and accompaniment generation. We also enable the analysis of monadic analysis of structures in music.

2606.19935 2026-06-19 cs.AI 新提交

PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation

PhysDrift: 弥合人形机器人共语动作生成中的具身差距

Zhangzhao Liang, Xiaofen Xing, Mingyue Yang, Wenlve Zhou, Xiangmin Xu

发表机构 * South China University of Technology(华南理工大学) DexForce Technology(DexForce科技公司) Foshan University(佛山大学)

AI总结 针对人形机器人共语动作生成中人体运动流形与机器人具身约束不匹配的问题,提出IK-EER框架和PhysDrift模型,直接预测可执行关节轨迹,提升运动对齐、物理合理性和实时交互能力。

详情
AI中文摘要

人形机器人需要共语动作,这些动作不仅要富有表现力且与语音对齐,还要在具身约束下物理可执行。现有的共语动作生成流程主要是以人为中心的:首先以人体表示(如SMPL-X)生成动作,随后重定向到人形机器人。在这项工作中,我们识别出这种范式中的基本具身差距,即人体运动流形与人形机器人具身约束之间的不匹配在运动转移和物理执行过程中破坏了具身一致性。通过广泛分析,我们表明尽管重定向可以保留粗粒度的运动语义,但它显著压缩了运动多样性并削弱了韵律-动作同步,限制了富有表现力的人形机器人行为。为解决此问题,我们首先提出IK-EER,一种保留韵律的人形机器人运动策展框架,在重定向过程中联合优化运动学可行性和语音-运动时间对齐。基于策展的机器人原生运动数据集,我们进一步引入PhysDrift,一种具身感知的共语动作生成框架,直接预测可执行的人形机器人关节轨迹,无需依赖中间人体表示。与传统的以人为中心的流程不同,PhysDrift在训练和推理过程中都保持具身一致性,同时加入物理正则化以稳定机器人运动动态。大量实验和真实世界人形机器人部署表明,具身感知的机器人原生生成显著改善了语音-运动对齐、物理合理性、运动平滑性、推理效率和实时交互能力。

英文摘要

Humanoid robots require co-speech motions that are not only expressive and speech-aligned, but also physically executable under embodiment constraints. Existing co-speech generation pipelines are predominantly human-centric: motions are first generated in human-body representations such as SMPL-X and subsequently retargeted to humanoid robots. In this work, we identify a fundamental embodiment gap in this paradigm, where the mismatch between human motion manifolds and humanoid embodiment constraints disrupts embodiment consistency during motion transfer and physical execution. Through extensive analysis, we show that although retargeting can preserve coarse motion semantics, it significantly compresses motion diversity and weakens prosody-motion synchronization, limiting expressive humanoid behaviors. To address this problem, we first propose IK-EER, a prosody-preserving humanoid motion curation framework that jointly optimizes kinematic feasibility and speech-motion temporal alignment during retargeting. Building upon the curated robot-native motion dataset, we further introduce PhysDrift, an embodiment-aware co-speech motion generation framework that directly predicts executable humanoid joint trajectories from speech without relying on intermediate human-body representations. Unlike conventional human-centric pipelines, PhysDrift maintains embodiment consistency throughout both training and inference while incorporating physical regularization to stabilize robot motion dynamics. Extensive experiments and real-world humanoid deployment demonstrate that embodiment-aware robot-native generation substantially improves speech-motion alignment, physical plausibility, motion smoothness, inference efficiency, and real-time interaction capability.

2606.19934 2026-06-19 cs.CV cs.AI 新提交

Speeding up the annotation process in semantic segmentation industrial applications

加速工业应用中的语义分割标注过程

Marta Fernandez-Moreno, Margarita Guerrero, Rosalia Rementeria, Pablo Mesejo, Raul Moreno

发表机构 * Department of Computer Science and Artificial Intelligence, Andalusian Research Institute in Data Science and Computational Intelligence, DaSCI, University of Granada(格拉纳达大学计算机科学与人工智能系,安达卢西亚数据科学与计算智能研究所,DaSCI) Department of Computer Science and Automatic Control, National Distance Education University (UNED)(国立远程教育大学计算机科学与自动控制系)

AI总结 本文利用无监督算法将材料科学中语义分割的标注时间从170小时降至37小时(减少78%),并发布了最大的公开钢微观结构分割数据集。

详情
AI中文摘要

当前的机器学习模型通常需要大量且标注良好的数据集。然而,标注过程常常成为瓶颈,随着复杂性的增加,人为错误的机会也更高。在此背景下,本文旨在利用无监督算法提高工业材料科学中复杂语义分割问题的数据标注效率。以往的研究量化了标注时间,并探索了无监督方法。但据我们所知,这是首次量化无监督算法加速标注过程程度的研究。我们旨在验证这一繁琐过程可以加速的程度,重点关注涉及高分辨率图像每个像素标注的语义分割任务,例如材料科学中的微观结构表征挑战。具体来说,我们证明通过使用无监督计算机视觉算法,标注过程所需的时间可以从170小时减少到37小时,实现了约78%的减少。我们处理的数据集包括尺寸为1280x959和960x703的大图像,这进一步增加了标注任务的复杂性。尽管存在这些挑战,我们创建并共享了迄今为止最大的公开钢微观结构分割数据集,在MIT许可下提供,并具有永久DOI,为该领域贡献了一个完全标注的高分辨率数据集。此外,这是首次将从头开始标注的时间(以往研究中的常见方法)与使用这些无监督算法作为预标注步骤时的标注时间进行比较。此外,我们提供了一个在此数据集上训练的深度学习模型,该模型经过领域专家验证,并部署在工业环境中,作为该公共数据集的初始基准。

英文摘要

Current machine learning models commonly require large and well-annotated datasets. However, the annotation process often becomes a bottleneck, with increased complexity leading to higher chances of human errors. Within this context, our goal in this paper is to leverage unsupervised algorithms to improve data annotation efficiency for complex semantic segmentation problems in industrial materials science. Previous research has quantified labeling time and others explored unsupervised methods. However, to the best of our knowledge, this is the first study to quantify how much unsupervised algorithms accelerate the labeling process. We aim to validate the extent to which this laborious process can be accelerated, focusing on semantic segmentation tasks that involve annotating each pixel of high-resolution images, such as the microstructure characterization challenge in materials science. Specifically, we demonstrate that by using unsupervised computer vision algorithms, the time required for the labeling process can be reduced from 170 hours to 37 hours, achieving an approximate reduction of 78\%. The dataset we work with includes large images of dimensions 1280x959 and 960x703, which further increases the complexity of the annotation task. Despite these challenges, we create and share the largest public steel microstructure segmentation dataset to date, available under MIT License with permanent DOI, contributing a fully annotated, high-resolution dataset to the field. Additionally, this is the first work to compare the labeling time from scratch (a common approach in previous studies) to the labeling time when using these unsupervised algorithms as a pre-annotation step. Furthermore, we provide a Deep Learning model trained on this dataset, validated by field experts, and deployed in an industrial setting, serving as an initial benchmark for this public dataset.

2606.19932 2026-06-19 cs.CV cs.AI 新提交

Spatial-Aware Reduction Framework: Towards Efficient and Faithful Visual State Space Models

空间感知缩减框架:迈向高效且忠实的视觉状态空间模型

Jindi Lv, Aoyu Li, Yuhao Zhou, Zheng Zhu, Xiaofeng Wang, Qing Ye, Yueqi Duan, Wentao Feng, Jiancheng Lv

发表机构 * Sichuan University(四川大学) Tsinghua University(清华大学)

AI总结 提出STORM框架,通过保持空间结构完整性解决视觉Mamba模型在token缩减时的性能崩溃问题,无需训练即可实现高精度剪枝。

Comments Accepted by ICML 2026

详情
AI中文摘要

Mamba在建模长视觉序列方面表现出强大的效率。然而,当将token缩减应用于结构增强的Mamba变体时,这些模型会出现严重的性能崩溃。我们将这种退化归因于现有缩减方法在空间上的不可知性,这违反了选择性扫描机制所需的二维结构前提。在这项工作中,我们提出了STORM,一个空间感知的token缩减框架,旨在在压缩过程中保持结构完整性。STORM将缩减重新表述为对空间单元的结构化操作,强制局部约束以保持网格拓扑和邻域一致性。作为一个即插即用模块,STORM无需任何训练即可为现有缩减流程赋予明确的空间感知能力。实验结果表明,STORM在无训练设置下,在多种视觉Mamba骨干网络上实现了最先进的剪枝精度。值得注意的是,STORM在VMamba上实现了显著的精度恢复,在top-1准确率上比先前方法高出63.3%。同时,STORM在PlainMamba上仅造成1.0%的准确率下降,达到了与ViT相当的性能。

英文摘要

Mamba demonstrates strong efficiency in modeling long visual sequences. However, when token reduction is applied to structurally enhanced Mamba variants, these models exhibit a severe performance collapse. We attribute this degradation to the spatially agnostic nature of existing reduction methods, which violate the two-dimensional structural premise required by the selective scanning mechanism. In this work, we propose STORM, a spatial-aware token reduction framework designed to maintain structural integrity throughout the compression process. STORM reformulates reduction into a structured operation on spatial units, enforcing localized constraints to maintain both grid topology and neighborhood coherence. As a plug-and-play module, STORM equips existing reduction pipelines with explicit spatial awareness without any training. Empirical results demonstrate that STORM achieves state-of-the-art pruning accuracy across diverse vision Mamba backbones under training-free settings. Notably, STORM delivers a substantial accuracy recovery on VMamba, outperforming prior methods by up to 63.3\% in top-1 accuracy. Meanwhile, STORM incurs only a 1.0\% accuracy drop on PlainMamba, achieving performance comparable to ViT.

2606.19931 2026-06-19 cs.MA 新提交

Blame is easier than praise: Measuring off-ball defensive performance in football

责备比表扬更容易:衡量足球中的无球防守表现

Jonas Bischofberger, Runqing Ma, Pascal Bauer, Kilian Arnsmeyer, Arnold Baca

AI总结 提出基于防守压力区(DPA)的球员参与度评分,将预期威胁的事件级变化归因于个体,以衡量足球无球防守表现,并在跨性别和跨赛事数据集上验证其有效性。

详情
AI中文摘要

足球运动员的防守表现通常通过有限的行动(如抢断和拦截)来衡量,而他们通过位置行为的持续影响此前很少被研究。我们将此问题表述为多智能体时空轨迹上的归因问题,没有球员级别的真实标签,其中事件级别的预期威胁变化被分配给个体。我们提出了一个框架,使用从防守压力区(DPA)计算的球员参与度评分来执行此归因。通过计算自动检测的团队结构内的角色条件基线,我们可以确定每个防守者对通过任意传球创造的威胁的预期责任。该方法的有效性和鲁棒性在独特的广泛跨性别和跨赛事数据集上进行了评估,包括来自男子世界杯64场比赛、女子德甲116场比赛和男子德丙336场比赛的位置和事件数据。在没有真实标签的情况下,我们提出了一个评估协议,将多个相对较弱的代理组合成稳健的总结分数。我们发现,与最佳基于行动的指标相比,有效性分数提高了大约一个标准差,并证明许多流行指标的有效性有限。对高价值行动的“责备”与外部评级和市场价值显示出特别强的相关性,使其成为足球中第一个可靠衡量定位错误的已发表指标。本工作所有代码均公开可用,以支持可重复性和进一步研究。

英文摘要

The defensive performance of football players is commonly measured through a limited number of actions like tackles and interceptions while their continuous impact through positional behaviour has hardly been studied before. We formulate this problem as an attribution over multi-agent spatiotemporal trajectories without player-level ground truth labels, where event-level changes of expected threat are distributed among individuals. We propose a framework that performs this attribution using player involvement scores calculated from defensive pressure areas (DPAs). By computing role-conditioned baselines within automatically detected team structures, we can determine each defender's expected responsibility for threat created through arbitrary passes. The validity and robustness of this approach are evaluated on a uniquely extensive cross-gender and cross-competition data set, including positional and event data from 64 matches of the men's World Cup, 116 matches of the women's German Bundesliga and 336 matches of the men's German 3. Liga. In the absence of a ground truth, we propose an evaluation protocol that combines multiple relatively weak proxies into robust summary scores. We find a validity score that is improved by around 1 standard deviation compared to the best action-based metric and demonstrate that many popular measures show limited validity. The "blame" for conceding high-value actions shows especially strong correlations with external ratings and market values, making it the first published metric in football to reliably measure positioning errors. All code underlying this work is publicly available to support reproducibility and further research.

2606.19930 2026-06-19 cs.HC 新提交

MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization

MobileForge:基于分层反馈引导策略优化的移动GUI智能体免标注适配

Guangyi Liu, Pengxiang Zhao, Gao Wu, Yiwen Yin, Mading Li, Liang Liu, Congxiao Liu, Zhang Qi, Mengyan Wang, Liang Guo, Yong Liu

AI总结 提出MobileForge系统,通过MobileGym环境实现任务生成与评估,结合分层反馈引导策略优化(HiFPO)将轨迹结果、步骤反馈和修正提示转化为步骤级GRPO更新,实现移动GUI智能体免标注适配,在AndroidWorld上达到67.2% Pass@3。

Comments Project page: https://mobile-forge.github.io/

详情
AI中文摘要

基于MLLM的移动GUI智能体在UI理解和动作执行方面取得了显著进展,但将它们适配到真实目标应用仍然成本高昂,因为移动应用数量众多、频繁更新,且难以用人工编写的任务、演示或奖励标签覆盖。现有的免标注GUI学习减少了人工监督,但缺乏将目标应用探索、课程挖掘、轨迹执行和反馈连接起来的统一基础,而策略优化通常依赖于孤立的轨迹和难以转化为可靠改进信号的粗粒度奖励。我们提出MobileForge,一个用于移动GUI智能体的免标注适配系统。MobileForge包含MobileGym,它将任务生成和轨迹评估基于真实移动应用交互,以及分层反馈引导策略优化(HiFPO),它将轨迹结果、步骤级过程反馈和修正提示转化为提示上下文化的步骤级GRPO更新。仅使用自动生成的免标注适配数据,MobileForge将Qwen3-VL-8B适配到AndroidWorld上67.2%的Pass@3,接近使用封闭数据的GUI专用GUI-Owl-1.5-8B基础模型的69.0%。MobileForge适配的ForgeOwl-8B进一步在AndroidWorld上达到77.6%的Pass@3,在域外MobileWorld GUI-only分割上达到41.0%的成功率,在我们的评估中建立了最强的开放数据移动GUI智能体。代码、数据和训练模型将在该URL发布。

英文摘要

MLLM-based mobile GUI agents have made substantial progress in UI understanding and action execution, but adapting them to real target apps remains costly because mobile apps are numerous, frequently updated, and hard to cover with human-written tasks, demonstrations, or reward labels. Existing annotation-free GUI learning reduces manual supervision, yet lacks a unified substrate connecting target-app exploration, curriculum mining, rollout execution, and feedback, while policy optimization often relies on isolated rollouts and coarse rewards that are hard to convert into reliable improvement signals. We present MobileForge, an annotation-free adaptation system for mobile GUI agents. MobileForge consists of MobileGym, which grounds task generation and rollout evaluation in real mobile app interaction, and Hierarchical Feedback-Guided Policy Optimization (HiFPO), which turns trajectory outcomes, step-level process feedback, and corrective hints into hint-contextualized step-level GRPO updates. Using only automatically generated annotation-free adaptation data, MobileForge adapts Qwen3-VL-8B to 67.2% Pass@3 on AndroidWorld, close to the closed-data GUI-specialized GUI-Owl-1.5-8B base model at 69.0%. The MobileForge-adapted ForgeOwl-8B further reaches 77.6% Pass@3 on AndroidWorld and 41.0% success on the out-of-domain MobileWorld GUI-only split, establishing the strongest open-data mobile GUI agent in our evaluation. Code, data, and trained models will be released at https://mobile-forge.github.io/.

2606.19929 2026-06-19 cs.RO 新提交

Motor Angular Speed Preintegration for Multirotor UAV State Estimation

多旋翼无人机状态估计中的电机角速度预积分

Matěj Petrlík, Filip Novák, Robert Pěnička, Martin Saska

AI总结 针对无人机振动导致IMU精度下降的问题,提出基于电机转速加速度预积分的方法,替代IMU进行状态传播,并构建因子用于图优化,结合LiDAR形成MAS-LO算法,相比LIO-SAM位置精度提升28%,速度精度提升65%。

详情
AI中文摘要

精确的状态估计对于实现无人机的敏捷和近障碍飞行所需的紧密反馈控制至关重要。最先进的方法融合慢速位姿测量与高频惯性测量以获得精确的状态估计。然而,来自无人机上IMU的惯性测量会受到旋转螺旋桨振动的退化,导致估计状态的精度下降。我们提出了一种基于电机转速加速度预积分的新方法。我们展示了以这种方式获得的加速度可以单独用于状态传播,在不包含IMU的情况下实现更好的精度。此外,我们提出了一个由预积分电机转速组成的因子,可以直接用于因子图优化框架。我们将该因子与LiDAR测量结合,提出电机角速度LiDAR里程计(MAS-LO)算法,用于精确状态估计,并开源该算法。最后,我们与最先进的惯性算法LIO-SAM进行估计精度评估,结果显示位置估计精度提升28%,速度估计精度提升65%,测量延迟降低14%,并且对错误参数值具有高鲁棒性。

英文摘要

A precise state estimate is crucial for a tight feedback control that enables agile and near-obstacle flights of UAVs. The state-of-the-art methods fuse slow pose measurements with high-frequency inertial measurements to obtain a precise state estimate. However, the inertial measurements from the IMU onboard the UAV are degraded by vibrations from spinning propellers and the precision of the estimated state suffers. We propose a novel approach based on the preintegration of accelerations obtained from motor speeds. We show that the accelerations obtained in this manner can be used for state propagation on their own to achieve better precision without including the IMU. Further, we propose a factor composed of the preintegrated motor speeds that can be directly employed in factor graph optimization frameworks. We combine our factor with LiDAR measurements into the proposed Motor Angular Speed LiDAR Odometry (MAS-LO) algorithm for precise state estimation, which we open-source. Lastly, we evaluate the estimation precision against a state-of-the-art inertial algorithm LIO-SAM to show 28% improvement in position and 65% in velocity estimation accuracy, 14% lower measurement lag, and high robustness to wrong parameter values.

2606.19928 2026-06-19 cs.RO 新提交

SWAP: Symmetric Equivariant World-Model for Agile Robot Parkour

SWAP: 用于敏捷机器人跑酷的对称等变世界模型

Kaixin Lan, Ze Wang, Hongyi Li, Lei Jiang, Chaojie Fu, Chengkai Su, Choi Lam Wong, Yongbin Jin, Hongtao Wang

发表机构 * Center for X-Mechanics, Zhejiang University(浙江大学交叉力学中心) ZJU-Hangzhou Global Scientific and Technology Innovation Center(浙江大学杭州国际科创中心) Mirrorme Technology Co., Ltd.(魔镜科技有限公司)

AI总结 提出SWAP框架,将对称等变性嵌入世界模型和演员-评论家网络,实现四足机器人跑酷记录突破(跨越2.13米间隙、攀爬1.63米平台),并展现出对未见镜像地形的几何泛化与零样本迁移能力。

详情
AI中文摘要

虽然潜在世界模型能够实现极限跑酷所需的主动预测,但其纯数据驱动的特性迫使它们将左右对称交互冗余编码为独立模式。这增加了学习负担并阻碍了几何规律性的捕获,限制了潜在空间对下游策略的效率。为了解决这个问题,我们提出了SWAP,一个端到端的等变对称世界模型。该框架将对称性直接嵌入到世界模型和演员-评论家网络中。在真实世界测试中,机器人跨越了2.13米的间隙并攀爬了1.63米的高台,打破了四足机器人跑酷的记录。此外,该框架对未见过的镜像地形展现出鲁棒的几何泛化能力,并在多种户外环境中具有卓越的零样本迁移能力。这些结果表明,对称等变性是推动学习型腿式运动物理极限的有效结构先验。

英文摘要

While latent world models enable the proactive predictions required for extreme parkour, their purely data-driven nature forces them to redundantly encode left-right symmetric interactions as independent patterns. This inflates the learning burden and hinders the capture of geometric regularities, restricting the latent space's efficiency for downstream policies. To address this, we propose SWAP, an end-to-end equivariant symmetric world model. This framework embeds symmetry directly into both the world model and the actor-critic networks. In real-world tests, the robot leaps across a 2.13 m gap and climbs a 1.63 m platform, breaking records for quadruped parkour. Furthermore, the framework exhibits robust geometric generalization to unseen mirrored terrains and exceptional zero-shot transferability across diverse outdoor environments. These results demonstrate that symmetry equivariance is an effective structural prior for pushing the physical boundaries of learned legged locomotion.

2606.19927 2026-06-19 cs.CV 新提交

CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs

CARE: 面向视频多模态大语言模型的自适应推理长度的能力感知奖励塑形

Chengwen Liu, Hao Peng, Jisheng Dang, Hong Peng, Bin Hu, Tat-Seng Chua

发表机构 * School of Information Science and Engineering, Lanzhou University(兰州大学信息科学与工程学院) School of Medical Technology, Beijing Institute of Technology(北京理工大学医学技术学院) School of Computing, National University of Singapore(新加坡国立大学计算机学院)

AI总结 提出CARE框架,通过能力感知奖励塑形自适应优化推理长度,利用指数移动平均估计能力并分阶段调整奖励偏好,结合批次归一化和后验放大器提升效率与准确性。

详情
AI中文摘要

在多模态视频推理中,基于强化学习的方法通常依赖简单且不灵活的推理长度控制策略,无法适应模型不断变化的能力。这种不匹配可能在早期阶段抑制必要的探索,而在模型变得更有能力后鼓励冗余推理和低效解码。本文提出CARE,一种用于多模态推理中自适应推理长度优化的能力感知奖励塑形框架。具体来说,CARE通过通过率的指数移动平均维护平滑的能力估计,并利用它将训练路由到渐进阶段,将奖励偏好从探索导向的长形式推理转向效率导向的简洁推理。为避免将冗长与内在任务复杂性混淆,CARE进一步使用批次级统计归一化推理努力,并引入后验放大器以增强对历史上困难样本上意外强性能的奖励信号。所提出的机制无缝集成到GRPO训练流程中,且不增加额外推理开销。在多个视频推理和通用视频理解基准上的大量实验表明,CARE持续提高推理准确性,稳定强化学习,并显著提升令牌效率。此外,CARE在训练过程中展现出推理长度的特征性倒U型轨迹,并在收敛时产生更短但信息更丰富的推理轨迹,表明推理预算的有效自适应分配。我们在以下网址提供CARE框架和实验的源代码:此https URL。

英文摘要

In multimodal video reasoning, reinforcement learning-based methods typically rely on simplistic and inflexible reasoning-length control strategies that fail to adapt to the model's evolving competence. This mismatch may suppress necessary exploration at early stages, while encouraging redundant reasoning and inefficient decoding once the model becomes more competent. In this paper, we propose CARE, a competence-aware reward shaping framework for adaptive reasoning length optimization in multimodal reasoning. Specifically, CARE maintains a smoothed competence estimate via an exponential moving average of pass rates, and uses it to route training into progressive stages that shift the reward preference from exploration-oriented long-form reasoning to efficiency-oriented concise reasoning. To avoid conflating verbosity with intrinsic task complexity, CARE further normalizes reasoning effort with batch-level statistics, and introduces a posterior amplifier to strengthen reward signals for unexpectedly strong performance on historically difficult samples. The proposed mechanism is seamlessly integrated into the GRPO training pipeline and incurs no additional inference-time overhead. Extensive experiments on multiple video reasoning and general video understanding benchmarks demonstrate that CARE consistently improves reasoning accuracy, stabilizes reinforcement learning, and significantly enhances token efficiency. Moreover, CARE exhibits a characteristic inverted-U trajectory of reasoning length during training, and yields shorter yet more informative reasoning traces at convergence, indicating effective adaptive allocation of reasoning budget. We provide the source code for our proposed CARE framework and experiments at https://github.com/1Pansy/Video-CARE.

2606.19926 2026-06-19 cs.HC 新提交

MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management

MemGUI-Agent: 一种具有主动上下文管理的端到端长时移动GUI智能体

Guangyi Liu, Gao Wu, Congxiao Liu, Pengxiang Zhao, Liang Liu, Mading Li, Qi Zhang, Mengyan Wang, Liang Guo, Yong Liu

AI总结 提出MemGUI-Agent,通过主动上下文管理机制(ConAct)将上下文管理作为一等动作,解决长时任务中提示膨胀和关键信息稀释问题,在8B模型上达到最佳性能。

Comments 33 pages, 6 figures. Project page: https://memgui-agent.github.io/

详情
AI中文摘要

基于MLLM的移动GUI智能体在短时任务上取得了显著进展,但在需要跨多步和应用转换保留中间事实的长时任务上仍不可靠。我们将此限制归因于ReAct风格的提示,它被动地累积每一步的记录,导致提示膨胀和关键跨应用事实的稀释。为了解决这个问题,我们引入了MemGUI-Agent,一种具有主动上下文管理的端到端长时移动GUI智能体。MemGUI-Agent建立在Context-as-Action (ConAct)之上,它将上下文管理作为与选择UI动作相同的策略发出的一等动作。ConAct不是被动地追加历史,而是维护三个结构化的上下文字段:折叠的动作历史、折叠的UI状态和最近的步骤记录,在保持上下文紧凑的同时保留关键的UI事实。为了使主动上下文管理跨模型规模可学习,我们构建了MemGUI-3K,一个包含2956条轨迹的数据集,带有完整的ConAct注释,用于监督训练和离线分析。在MemGUI-3K上训练8B模型产生了MemGUI-8B-SFT,一个8B的MemGUI-Agent,它在MemGUI-Bench上实现了最佳的开源8B性能,并泛化到分布外的MobileWorld基准测试。代码、数据和训练好的模型将在以下网址发布:https://this URL。

英文摘要

MLLM-based mobile GUI agents have made substantial progress on short-horizon tasks, yet remain unreliable on long-horizon tasks that require retaining intermediate facts across many steps and app transitions. We attribute this limitation to ReAct-style prompting, which passively accumulates per-step records, leading to prompt explosion and dilution of critical cross-app facts. To address this, we introduce MemGUI-Agent, an end-to-end long-horizon mobile GUI agent with proactive context management. MemGUI-Agent is built on Context-as-Action (ConAct), which casts context management as first-class actions emitted by the same policy that selects UI actions. Instead of passively appending history, ConAct maintains three structured context fields: folded action history, folded UI state, and recent step record, preserving critical UI facts while keeping context compact. To make proactive context management learnable across model scales, we construct MemGUI-3K, a 2,956-trajectory dataset with full ConAct annotations for supervised training and offline analysis. Training an 8B model on MemGUI-3K produces MemGUI-8B-SFT, an 8B MemGUI-Agent that achieves the best open-data 8B performance on MemGUI-Bench and generalizes to the out-of-distribution MobileWorld benchmark. Code, data, and trained models will be released at https://memgui-agent.github.io/.

2606.19924 2026-06-19 cs.AI 新提交

The Tao of Agency: Autotelic AI, Embedded Agency and Dissolution of the Self

主体之道:自生目标人工智能、嵌入主体与自我的消解

Aritra Sarkar

AI总结 本文探讨自生目标AI中主体生成自身目标的问题,通过内在动机、资源驱动先验、因果干预学习、稳态和嵌入性等概念,揭示嵌入性虽必要但不充分,并指出核心难题在于主体如何生成并相对化自我,最后提出量子表述、哲学解读和基于LLM的具体实现。

详情
AI中文摘要

大多数人工智能系统建立在目标由设计者外生指定的假设上。探索当主体开始生成自身目标时会发生什么,开启了自生目标AI领域。主体不仅应追求目标,还应发现目标。本文通过内在动机、资源驱动先验、因果干预学习、稳态和嵌入性追溯其后果;发现嵌入性是自生目标主体性的必要但不充分条件。嵌入性将主体个体化,但代价是揭示这种个体化并非唯一,相同的动力学允许许多有效划分,每个划分定义了一个不同的候选自我。因此,自生目标AI最深层次的问题不在于主体如何生成目标,而在于主体如何生成并相对化目标所归属的自我。主体必须相信自身的边界才能行动,并看穿该边界才能理解。我们将这些发展整合到一个统一框架中,并沿三个方向扩展:量子表述(其中主体-环境切割成为物理的)、针对非二元沉思传统的哲学解读,以及基于LLM的具体主体实现。

英文摘要

Most artificial intelligence systems are built on the assumption that goals are exogenous and specified by the designer. Exploring what happens when an agent begins generating its own goals opens the field of autotelic AI. Agents are expected not merely to pursue objectives but to discover them. In this article, we trace its consequences through intrinsic motivation, resource-driven priors, causal-interventional learning, homeostasis, and embeddedness; the last of which is found to be a necessary but not sufficient condition for autotelic agency. Embeddedness individuates the agent at the cost of revealing that the individuation is non-unique, such that the same dynamics admit many valid partitions, each defining a different candidate self. The deepest problem with autotelic AI is therefore not how the agent generates goals, but how it generates and relativizes the self to which the goals are assigned. The agent must believe in its own boundary in order to act, and see through that boundary in order to understand. We consolidate these developments into a single framework and extend it along three directions: a quantum formulation in which the agent-environment cut becomes physical, a philosophical reading against non-dual contemplative traditions, and a concrete LLM-based agentic instantiation.