NeuronMLP: Efficient LLM Inference via Singular Value Decomposition Compression and Tiling on AWS Trainium
NeuronMLP:通过AWS Trainium上的奇异值分解压缩和分块实现高效的LLM推理
机构 * University of California, Merced USA ; University of Wisconsin, Madison USA ; Yotta Labs \& University of California, Merced USA ; University of California, Merced ; University of Wisconsin, Madison ; Yotta Labs \& University of California, Merced
AI总结 本文提出NeuronMLP,基于奇异值分解压缩和分块技术优化AWS Trainium上的LLM推理,通过定制化技术减少数据移动,提升SRAM带宽,实现1.35倍的内核加速和1.21倍的端到端推理加速。
Comments 12 pages, 8 figures