LLM Serving Optimization with Variable Prefill and Decode Lengths
具有可变预填和解码长度的LLM服务优化
Meixuan Wang, Yinyu Ye, Zijie Zhou
机构
*
Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
;
Department of Management Science and Engineering, Stanford University(斯坦福大学管理科学与工程系)
;
Department of Industrial Engineering and Decision Analytics, HKUST(香港科技大学工业工程与决策分析系)
专题命中
效率与部署
:LLM(title,abstract);large language model(abstract);language model(abstract);分类 cs.AI、cs.LG
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
你只有一份工作:利用LLM隐藏表示进行每任务量化
Amit LeVi, Raz Lapid, Rom Himelstein, Chaim Baskin, Ravid Shwartz Ziv, Avi Mendelson
机构
*
Technion—Israel Institute of Technology(技术ion—以色列理工学院)
;
Zenity(Zenity公司)
;
DeepKeep(DeepKeep公司)
;
Ben-Gurion University of the Negev(巴伊兰大学)
;
New York University(纽约大学)
Research Entity Extraction and Topic Detection from UKRI Grant Proposals
从UKRI资助提案中提取研究实体和检测主题
Xingran Ruan, Angelo Salatino, Rosa Filgueira, Kara Moraw, Alexandru Marcoci, Gemma Derrick, Sarah Callaghan
机构
*
EPCC, University of Edinburgh, UK(埃克塞特大学埃平中心,英国)
;
Knowledge Media Institute, The Open University, UK(开放大学知识媒体研究所,英国)
;
Institute for Technology and Humanity, University of Cambridge, UK(剑桥大学技术与人类研究所,英国)
;
Centre for Higher Education Transformations, School of Education, University of Bristol, UK(布里斯托大学教育学院高等教育转型中心,英国)
;
University of Oxford, UK(牛津大学,英国)
SpecMind: Cognitively Inspired, Interactive Multi-Turn Framework for Postcondition Inference
SpecMind:基于认知的、交互式多轮框架用于后置条件推断
Cuong Chi Le, Minh V. T Pham, Tung Vu Duy, Cuong Duc Van, Huy N. Phan, Hoang N. Phan, Tien N. Nguyen
机构
*
The University of Texas at Dallas(德克萨斯大学达拉斯分校)
;
FPT Software AI Center(FPT软件AI中心)
;
VinUniversity(文大学)
;
Nanyang Technological University(南洋理工大学)
专题命中
效率与部署
:LLM(abstract);large language model(abstract);language model(abstract);prompting(abstract)
Aligning Language Model Benchmarks with Pairwise Preferences
将语言模型基准与成对偏好对齐
Marco Gutierrez, Xinyi Leng, Hannah Cyberey, Jonathan Richard Schwarz, Ahmed Alaa, Thomas Hartvigsen
机构
*
School of Data Science, University of Virginia(弗吉尼亚大学数据科学学院)
;
Imperial College London(伦敦帝国理工学院)
;
Thomson Reuters Foundational Research(汤姆森路透基础研究)
;
Department of Electrical Engineering and Computer Science, UC Berkeley and UCSF(伯克利大学电气工程与计算机科学系及旧金山大学)
机构
*
Tsinghua University(清华大学)
;
Pengcheng Laboratory(鹏城实验室)
;
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
;
Xiaomi EV(小米电动车)
;
National University of Singapore(新加坡国立大学)