UB-SMoE: Universally Balanced Sparse Mixture-of-Experts for Resource-adaptive Federated Fine-tuning of Foundation Models
UB-SMoE:面向资源自适应联邦微调的通用平衡稀疏专家混合模型
Van-Tuan Tran, Hong-Hanh Nguyen-Le, Marco Ruffini, Merim Dzaferagic
机构
*
School of Computer Science and Statistics, Trinity College Dublin, Ireland(特里尼蒂学院都柏林分校计算机科学与统计学学院)
;
School of Computer Science, University College Dublin, Ireland(都柏林大学学院计算机科学学院)
机构
*
Department of Automation, BNRist, Tsinghua University, Beijing, China(自动化系、BNRist、清华大学、北京、中国)
;
School of Computer Science and Technology, Xidian University(计算机科学与技术学院、西安电子科技大学)
;
Beijing Academy of Artificial Intelligence(北京人工智能研究院)
;
Chinese PLA General Hospital(中国人民解放军总医院)
SLASH the Sink: Sharpening Structural Attention Inside LLMs
SLASH the Sink: 在大语言模型中 sharpening 结构性注意力
Yiming Liu, Bin Lu, Xinbing Wang, Chenghu Zhou, Meng Jin
机构
*
Shanghai Jiao Tong University(上海交通大学)
;
Institute of Geographical Science and Natural Resources Research(地理科学与自然资源研究所)
;
Chinese Academy of Sciences(中国科学院)
专题命中
指令微调
:LLM(abstract_cn);large language model(abstract);language model(abstract);分类 cs.AI
Fine-tuning an ECG Foundation Model to Predict Coronary CT Angiography Outcomes
微调一种心电图基础模型以预测冠状动脉CT血管造影结果
Yujie Xiao, Qinghao Zhao, Gongzheng Tang, Hao Zhang, Zhuoran Kan, Deyun Zhang, Jun Li, Guangkun Nie, Xiaocheng Fang, Haoyu Wang, Shun Huang, Tong Liu, Jian Liu, Kangyin Chen, Shenda Hong
机构
*
Institute of Medical Technology, Peking University Health Science Center(北京大学人民医院医学技术研究所)
;
National Institute of Health Data Science, Peking University(北京大学国家健康数据科学研究院)
;
Department of Cardiology, Peking University People’s Hospital(北京大学人民医院心内科)
;
Tianjin Key Laboratory of Ionic-Molecular Function of Cardiovascular Disease, Department of Cardiology, Tianjin Institute of Cardiology, The Second Hospital of Tianjin Medical University(天津医科大学心血管离子-分子功能重点实验室,天津心脏病学研究院,天津医科大学第二医院心内科)
;
Heart Voice Medical Technology(心声医疗科技)
;
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院)
Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
战略性过参数化以实现通用的低秩适应
Jing Gao, Zhong-Yi Lu, Pan Zhang, Ze-Feng Gao
机构
*
School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study, UCAS, Hangzhou 310024, China(1 基础物理与数学科学学院,杭州先进研究院,UCAS,杭州 310024,中国)
;
School of Physical Sciences, University of Chinese Academy of Sciences, No. 19A Yuquan Road, Beijing 100049, China(2 物理科学学院,中国科学院大学,玉泉路19A号,北京 100049,中国)
;
CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China(3 中国科学院理论物理重点实验室,理论物理研究所,中国科学院,北京 100190,中国)
;
School of Physics and Key Laboratory of Quantum State Construction and Manipulation (Ministry of Education), Renmin University of China, Beijing 100872, China(4 物理学院和量子态构造与操控(教育部)重点实验室,中国人民大学,北京 100872,中国)
专题命中
指令微调
:large language model(abstract);language model(abstract);分类 cs.AI、cs.LG
Training-Free Zero-Shot Temporal Action Detection with Vision-Language Models
无需训练的零样本时序动作检测与视觉-语言模型
Chaolei Han, Hongsong Wang, Jidong Kuang, Lei Zhang, Jie Gui
机构
*
Southeast University School of Cyber Science and Engineering(东南大学网络安全科学与工程学院)
;
Southeast University School of Computer Science and Engineering(东南大学计算机科学与工程学院)
;
Nanjing Normal University School of Electrical Engineering and Automation(南京师范大学电气工程与自动化学院)
GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining
GLT-PEFT: 基于门控李-塔克参数高效微调的阿尔茨海默病诊断方法(结合海马体分割预训练)
Guanghua He, Hancan Zhu, Gaohang Yu, An Zhang
机构
*
Department of Mathematics, Hangzhou Dianzi University(杭州电子科技大学数学系)
;
School of Mathematics, Physics and Information, Shaoxing University(绍兴大学数学与物理学院)
;
Department of Mathematics, Zhejiang University of Science and Technology(浙江科技学院数学系)
机构
*
University of Manchester(曼彻斯特大学)
;
National University of Singapore(新加坡国立大学)
;
New York University(纽约大学)
;
University of Minnesota(明尼苏达大学)
;
Halmstad University(哈姆斯塔德大学)
;
Harvard University(哈佛大学)
;
University of Florida(佛罗里达大学)
专题命中
指令微调
:large language model(abstract);language model(abstract);分类 cs.LG
机构
*
School of Computer Science and Engineering, Beihang University, Beijing, China(北航计算机科学与工程学院)
;
School of Computer Science, Peking University, Beijing, China(北京大学计算机科学学院)
;
School of Computer Science and Technology, Beijing Institute of Technology, Beijing(北京理工大学计算机科学与技术学院)
;
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳)计算机科学与技术学院)
;
Baidu Inc., Beijing, China(百度公司)
机构
*
Odisha University of Technology and Research(奥里萨理工大学)
;
National Taiwan University of Science and Technology(台湾科技大学)
;
East China Normal University(东华大学)
;
Macau University of Science and Technology(澳门科技大学)
机构
*
New York University Tandon School of Engineering(纽约大学Tandon工程学院)
;
University of California, San Diego(加州大学圣地亚哥分校)
;
Stanford University(斯坦福大学)
;
ServiceNow Research(ServiceNow研究)
;
Mila - Quebec AI Institute(魁北克AI研究院)
;
Université de Montréal(蒙特利尔大学)
专题命中
后训练与偏好优化
:LLM(summary_cn,abstract);SLM(summary_cn,abstract);language model(abstract);small language model(abstract)