2606.13767
2026-06-15
cs.LG
cs.AI
cs.IT
math.IT
新提交
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
超越LoRA:稀疏诱导的适应更好吗?
Elijah Cadenhead, Cristian McGee, Xin Li, El Houcine Bergou, Aritra Dutta
发表机构
*
School of Data, Mathematical and Statistical Sciences, University of Central Florida, United States(中佛罗里达大学数据、数学与统计科学学院)
;
College of Computing, Mohammed VI Polytechnic University (UM6P), Morocco(穆罕默德六世理工大学计算机学院)
;
Department of Computer Science, University of Central Florida, United States(中佛罗里达大学计算机科学系)
AI总结
本文提出Cheap LoRA (cLA)及其变体,通过在LoRA中引入稀疏性实现参数高效微调,理论推导泛化误差界,实验表明在多种任务上性能与参数匹配基线相当,同时减少训练时间和峰值GPU内存。