AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
AMMA: 一种面向低延迟1M上下文注意力服务的多芯片片内存架构
Zhongkai Yu, Haotian Ye, Chenyang Zhou, Ohm Rishabh Venkatachalam, Zaifeng Pan, Zhengding Hu, Junsung Kim, Won Woo Ro, Po-An Tsai, Shuyi Pei, Yangwook Kang, Yufei Ding
机构
*
University of California, San Diego(加州大学圣迭戈分校)
;
Columbia University(哥伦比亚大学)
;
Yonsei University(延世大学)
;
NVIDIA(NVIDIA公司)
;
Samsung Semiconductor, Inc.(三星半导体公司)
机构
*
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
;
University of California San Diego(加州大学圣地亚哥分校)
;
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
;
Fudan University(复旦大学)
;
Shanghai Innovation Institute(上海创新研究院)
;
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)