FlashbackCL: Mitigating Temporal Forgetting in Federated Learning
FlashbackCL:缓解联邦学习中的时间遗忘
Mubarak A. Ojewale, Adriana E. Chis, Jorge M. Cortes-Mendoza, Bernardo Pulido-Gaytan, Horacio Gonzalez-Velez
AI总结 针对联邦学习中客户端数据分布随时间漂移导致的时间遗忘问题,提出FlashbackCL方法,通过时间衰减标签计数、类别平衡水库采样重放和服务器端主动核心集筛选,在CIFAR-10上相对Flashback提升6.9%-10.0%,时间遗忘减少68%。
详情
基础模型和边缘模型的联邦学习(FL)越来越多地部署在客户端数据分布随时间漂移的场景中,然而现有的遗忘缓解方法假设每个客户端的分布是平稳的。Flashback是近期最强的针对跨客户端(空间)遗忘的FL方法,它使用单调累积的每类标签计数作为知识代理;该代理在时间分布漂移下会失准,并将全局模型锚定在过时的类别平衡上。我们通过一个与协议级波动隔离的每阶段指标形式化定义了FL中的时间遗忘,并提出了Flashback Continual Learning(FlashbackCL),它是Flashback的即插即用扩展,包含:(i) 时间衰减的标签计数;(ii) 具有类别平衡水库采样(CBRS)的设备感知重放缓冲区;(iii) 在公共蒸馏集上的服务器端主动核心集筛选。结果表明,在具有50个客户端和三种受控时间漂移模式的CIFAR-10上,FlashbackCL相对于Flashback实现了6.9%至10.0%的相对改进,同时将时间遗忘减少了高达68%。一项5变体消融实验表明CBRS重放是关键组件。FlashbackCL在平稳CIFAR-100上也比Flashback提高了3.5个百分点,表明类别平衡重放同样正则化了空间异质性和时间漂移。
Federated Learning (FL) of foundation and edge models increasingly targets deployments where client data distributions drift over time, yet existing forgetting-mitigation methods assume each client's distribution is stationary. Flashback, the strongest recent FL method against cross-client (spatial) forgetting, uses monotonically accumulating per-class label counts as a knowledge proxy; this proxy becomes miscalibrated under temporal distribution shift and anchors the global model to an outdated class balance. We formalise temporal forgetting in FL with a per-phase metric isolated from protocol-level fluctuations and propose Flashback Continual Learning (FlashbackCL), a drop-in extension of Flashback with (i) temporally-decayed label counts; (ii) a device-aware replay buffer with Class-Balanced Reservoir Sampling (CBRS); and (iii) server-side active coreset curation on the public distillation set. The results show that FlashbackCL achieves 6.9% to 10.0% relative improvement relative to Flashback, on CIFAR-10 with 50 clients and three controlled temporal shift modes, while simultaneously reducing temporal forgetting by up to 68%. A 5-variant ablation identifies CBRS replay as the critical component. FlashbackCL also improves Flashback by 3.5 points on stationary CIFAR-100, suggesting that class-balanced replay regularises spatial heterogeneity as well as temporal shift.