Privacy Protection Against Personalized Text-to-Image Synthesis via Cross-image Consistency Constraints
针对个性化文本到图像合成的跨图像一致性约束隐私保护
Guanyu Wang, Kailong Wang, Yihao Huang, Mingyi Zhou, Geguang Pu, Li Li
AI总结 提出跨图像反个性化框架,通过强制扰动图像间的风格一致性并采用动态比率调整策略,增强对扩散模型个性化攻击的抵抗能力。
详情
扩散模型和个性化技术的快速发展使得仅凭少量公开图像就能重建个人肖像成为可能。虽然这种能力赋能了各种创意应用,但也带来了严重的隐私问题,因为攻击者可以利用它生成高度逼真的冒充图像。为应对这些威胁,反个性化方法被提出,通过向已发布图像添加对抗性扰动来破坏个性化模型的训练。然而,现有方法很大程度上忽视了个性化固有的多图像特性,而是采用一种朴素的独立应用扰动策略(如同在单图像设置中常见的那样)。这忽略了利用图像间关系实现更强隐私保护的机会。因此,我们倡导从群体层面看待针对个性化的隐私保护。具体而言,我们引入了跨图像反个性化(CAP),一种通过强制扰动图像间的风格一致性来增强对个性化抵抗能力的新型框架。此外,我们开发了一种动态比率调整策略,可在攻击迭代过程中自适应地平衡一致性损失的影响。在经典CelebHQ和VGGFace2基准上的大量实验表明,CAP显著改进了现有方法。
The rapid advancement of diffusion models and personalization techniques has made it possible to recreate individual portraits from just a few publicly available images. While such capabilities empower various creative applications, they also introduce serious privacy concerns, as adversaries can exploit them to generate highly realistic impersonations. To counter these threats, anti-personalization methods have been proposed, which add adversarial perturbations to published images to disrupt the training of personalization models. However, existing approaches largely overlook the intrinsic multi-image nature of personalization and instead adopt a naive strategy of applying perturbations independently, as commonly done in single-image settings. This neglects the opportunity to leverage inter-image relationships for stronger privacy protection. Therefore, we advocate for a group-level perspective on privacy protection against personalization. Specifically, we introduce Cross-image Anti-Personalization (CAP), a novel framework that enhances resistance to personalization by enforcing style consistency across perturbed images. Furthermore, we develop a dynamic ratio adjustment strategy that adaptively balances the impact of the consistency loss throughout the attack iterations. Extensive experiments on the classical CelebHQ and VGGFace2 benchmarks show that CAP substantially improves existing methods.