Attention Sinks and Outliers in Attention Residuals
注意力沉底与注意力残差中的异常值
Haozheng Luo, Haoran Dai, Shaoyang Zhang, Xi Chen, Eric Hanchen Jiang, Yijiang Li, Jingyuan Huang, Chenghao Qiu, Chenwei Xu, Zhenyu Pan, Haotian Zhang, Binghui Wang, Yan Chen
机构
*
Department of Computer Science, Northwestern University(西北大学计算机科学系)
;
Department of Computer Science and Engineering, University of Michigan(密歇根大学计算机科学与工程系)
;
Department of Statistics and Data Science, University of California Los Angeles(加州大学洛杉矶分校统计与数据科学系)
;
Department of Electrical and Computer Engineering, University of California San Diego(加州圣地亚哥大学电气与计算机工程系)
;
Department of Computer Science, Rutgers University-New Brunswick(新泽西州立大学鲁特学院计算机科学系)
;
Department of Computer Science and Engineering, Texas A&M University(德克萨斯农工大学计算机科学与工程系)
;
Department of Computer Science, Columbia University(哥伦比亚大学计算机科学系)
Conv-FinRe: A Conversational and Longitudinal Benchmark for Utility-Grounded Financial Recommendation
Conv-FinRe:一种用于实用导向财务推荐的对话和纵向基准
Yan Wang, Yi Han, Lingfei Qian, Yueru He, Xueqing Peng, Dongji Feng, Zhuohan Xie, Vincent Jim Zhang, Rosie Guo, Fengran Mo, Jimin Huang, Yankai Chen, Xue Liu, Jian-Yun Nie
机构
*
Georgia Institute of Technology(佐治亚理工学院)
;
Columbia University(哥伦比亚大学)
;
California State University(加州州立大学)
;
University of Montreal(蒙特利尔大学)
;
The University of Manchester(曼彻斯特大学)
;
McGill University(麦吉尔大学)
FinAuditing: A Financial Taxonomy-Structured Multi-Document Benchmark for Evaluating LLMs
FinAuditing: 一个基于财务分类结构的多文档基准,用于评估LLMs
Yan Wang, Keyi Wang, Shanshan Yang, Jaisal Patel, Jeff Zhao, Fengran Mo, Xueqing Peng, Lingfei Qian, Yankai Chen, Víctor Gutiérrez-Basulto, Jimin Huang, Guojun Xiong, Xiao-Yang Liu, Xue Liu, Jian-Yun Nie
机构
*
Columbia University(哥伦比亚大学)
;
Stevens Institute of Technology(史蒂文斯理工学院)
;
Rensselaer Polytechnic Institute(拉特格斯理工学院)
;
University of Montreal(蒙特利尔大学)
;
McGill University(麦吉尔大学)
;
MBZUAI(麦吉尔大学人工智能研究所)
;
Cardiff University(卡迪夫大学)
;
The University of Manchester(曼彻斯特大学)
;
Harvard University(哈佛大学)
机构
*
Northeastern University(东北大学)
;
University of Southern California(南加州大学)
;
Stony Brook University(石溪大学)
;
Independent Researcher(独立研究者)
;
Ohio State University(俄亥俄州立大学)
;
University of Notre Dame(Notre Dame 大学)
;
Columbia University(哥伦比亚大学)
机构
*
Georgia Institute of Technology(佐治亚理工学院)
;
Columbia University(哥伦比亚大学)
;
California State University(加州州立大学)
;
University of Montreal(蒙特利尔大学)
;
Carnegie Mellon University(卡内基梅隆大学)
;
Rensselaer Polytechnic Institute(莱斯利理工学院)
;
The University of Manchester(曼彻斯特大学)
;
Harvard University(哈佛大学)
Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents
面向LLM推理和AI代理的吞吐量最优调度算法
J. G. Dai, Tianze Deng, Yueying Li, Tianyi Peng
机构
*
School of Operations Research and Information Engineering, Cornell University(Cornell大学运筹学与信息工程学院)
;
Operations Management, Booth School of Business, University of Chicago(芝加哥大学博斯商学院运营管理系)
;
Decision, Risk and Operations, Columbia Business School, Columbia University(哥伦比亚大学哥伦比亚商学院决策、风险与运营系)
Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment
LLM代理能成为CFO吗?在不确定的企业环境中评估长期资源分配
Yi Han, Yan Wang, Lingfei Qian, Haohang Li, Yupeng Cao, Yueru He, Xueqing Peng, Nanhan Shen, Yitao Xu, Yankai Chen, Dongji Feng, Jimin Huang, Xue Liu, Jian-Yun Nie, Sophia Ananiadou
机构
*
Georgia Institute of Technology(佐治亚理工学院)
;
The Fin AI
;
Stevens Institute of Technology(史蒂文斯理工学院)
;
Columbia University(哥伦比亚大学)
;
George Mason University(乔治·马歇尔大学)
;
McGill University(麦吉尔大学)
;
Mohamed bin Zayed University of Artificial Intelligence(莫扎大学人工智能学院)
;
California State University, Monterey Bay(加州州立大学蒙特雷湾分校)
;
University of Manchester(曼彻斯特大学)
;
Mila – Quebec Artificial Intelligence Institute(魁北克人工智能研究所)
;
Université de Montréal(蒙特利尔大学)
机构
*
Columbia University(哥伦比亚大学)
;
Columbia University School of Professional Studies(哥伦比亚大学专业研究学院)
;
Department of Statistics(统计学系)
;
Columbia University Department of Computer Science(哥伦比亚大学计算机科学系)
Gradient Dynamics of Attention: How Cross-Entropy Sculpts Bayesian Manifolds
注意力的梯度动力学:交叉熵如何塑造贝叶斯流形
Naman Agarwal, Siddhartha R. Dalal, Vishal Misra
机构
*
Columbia University(哥伦比亚大学)
;
Columbia University School of Professional Studies(哥伦比亚大学专业研究学院)
;
Department of Statistics(统计学系)
;
Columbia University Department of Computer Science(哥伦比亚大学计算机科学系)
机构
*
Columbia University(哥伦比亚大学)
;
Columbia University School of Professional Studies(哥伦比亚大学专业研究学院)
;
Department of Statistics(统计学系)
;
Columbia University Department of Computer Science(哥伦比亚大学计算机科学系)
Learning more physically realistic dynamics in machine-learning based weather forecasting with latent-space constraints
在基于机器学习的天气预报中通过隐空间约束学习更符合物理现实的动力学
Hang Fan, Yi Xiao, Yongquan Qu, Juan Nathaniel, Fenghua Ling, Ben Fei, Lei Bai, Pierre Gentine
机构
*
Department of Earth and Environmental Engineering, Columbia University(哥伦比亚大学地球与环境工程系)
;
Learning the Earth with Artificial Intelligence and Physics (LEAP) Center, Columbia University(人工智能与物理联合地球学习中心(LEAP), 哥伦比亚大学)
;
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
;
The Chinese University of Hong Kong(香港中文大学)
;
Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
Wavelet Flow Matching for Multi-Scale Physics Emulation
小波流匹配用于多尺度物理模拟
Gabriele Accarino, Juan Nathaniel, Carla Roesch, Pierre Gentine, Sara Shamekh, Duncan Watson-Parris, Viviana Acquaviva
机构
*
Department of Earth and Environmental Engineering(地球与环境工程系)
;
Columbia University(哥伦比亚大学)
;
University of Edinburgh(爱丁堡大学)
;
Courant Institute of Mathematical Sciences(数学科学学院)
;
New York University(纽约大学)
;
Scripps Institution of Oceanography(斯克里普斯海洋研究所)
;
Halıcıoğlu Data Science Institute(哈利奇数据科学研究所)
;
University of California San Diego(加州大学圣地亚哥分校)
;
CUNY New York City College of Technology(纽约市立大学纽约技术学院)
;
Lamont-Doherty Earth Observatory(拉蒙特-多伊蒂地球观测站)
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosures
FinReporting: 一种用于跨司法管辖区财务披露本地化报告的代理工作流
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, Yu Chen, Xue, Liu, Preslav Nakov, Zhuohan Xie
机构
*
MBZUAI
;
The University of Tokyo(东京大学)
;
Meiji Gakuin University(明治大学)
;
McGill University(麦吉尔大学)
;
Kyoto University(京都大学)
;
Columbia University(哥伦比亚大学)
;
University of California, Berkeley(加州大学伯克利分校)
CTF4Nuclear: Common Task Framework for Nuclear Fission and Fusion Models
CTF4Nuclear: 用于核裂变和核聚变模型的通用任务框架
Stefano Riva, Carolina Introini, Antonio Cammi, Dean Price, Alexey Yermakov, Yue Zhao, Philippe M. Wyder, Judah Goldfeder, Jan Williams, Amy Sara Rude, Matteo Tomasetto, Joe Germany, Joseph Bakarji, Georg Maierhofer, Miles Cranmer, J. Nathan Kutz
机构
*
Autodesk Research(Autodesk研究院)
;
Department of Energy, Nuclear Engineering Division, Politecnico di Milano(能源部,核工程系,米兰理工学院)
;
Nuclear Science and Engineering, Massachusetts Institute of Technology(核科学与工程,麻省理工学院)
;
Department of Applied Mathematics, University of Washington(应用数学系,华盛顿大学)
;
Department of Electrical and Computer Engineering, University of Washington(电气与计算机工程系,华盛顿大学)
;
High Performance Machine Learning, SURF(高性能机器学习,SURF)
;
Distyl AI
;
Department of Computer Science, Columbia University(计算机科学系,哥伦比亚大学)
;
Department of Mechanical Engineering, University of Washington(机械工程系,华盛顿大学)
;
Department of Mechanical Engineering, Politecnico di Milano(机械工程系,米兰理工学院)
;
Department of Mathematics, American University in Beirut(数学系,贝鲁特美国大学)
;
Department of Mechanical Engineering, American University in Beirut(机械工程系,贝鲁特美国大学)
;
Department of Applied Mathematics and Theoretical Physics, University of Cambridge(应用数学与理论物理系,剑桥大学)
Hybrid LLM-based Intelligent Framework for Robot Task Scheduling
基于混合大语言模型的智能机器人任务调度框架
Swayamjit Saha, Subhabrata Das, Haonan Duan, Xiao-Yang Liu
机构
*
Department of Computer Science and Engineering, Mississippi State University(密苏里州立大学计算机科学与工程系)
;
Graduate School of Arts and Sciences, Columbia University(哥伦比亚大学研究生院)
;
Consumer and Community Banking, JPMorgan Chase(摩根大通消费与社区银行业)
;
Department of Data Science, Columbia University(哥伦比亚大学数据科学系)
;
Department of Electrical Engineering, Columbia University(哥伦比亚大学电气工程系)
Croissant Baker: Metadata Generation for Discoverable, Governable, and Reusable ML Datasets
饼干师:用于可发现、可治理和可重用的机器学习数据集元数据生成
Rafi Al Attrach, Rajna Fani, Sebastian Lobentanzer, Joan Giner-Miguelez, Debanshu Das, Varuni H. K., Nobin Sarwar, Rajat Ghosh, Anwai Archit, Surbhi Motghare, Christina Conrad Parry, Luis Oala, Lara Grosso, Joaquin Vanschoren, Steffen Vogler, Sujata Goswami, Eric S. Rosenthal, Marzyeh Ghassemi, Matthew McDermott, Tom Pollard
机构
*
Technical University of Munich(慕尼黑技术大学)
;
Massachusetts Institute of Technology(麻省理工学院)
;
Helmholtz Munich(海德堡-慕尼黑研究所以及医学中心)
;
Barcelona Supercomputing Center(巴塞罗那超级计算中心)
;
Google(谷歌)
;
Couchbase(Couchbase公司)
;
University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)
;
Nutanix(Nutanix公司)
;
Georg-August-University Göttingen(哥廷根大学)
;
Salesforce(Salesforce公司)
;
Sage Bionetworks(Sage Bionetworks公司)
;
Dotphoton(Dotphoton公司)
;
Harvard University(哈佛大学)
;
Eindhoven University of Technology(埃因霍温理工大学)
;
Bayer AG(拜耳公司)
;
Independent Researcher(独立研究者)
;
Massachusetts General Hospital(麻省总医院)
;
Columbia University(哥伦比亚大学)
A cross-species neural foundation model for end-to-end speech decoding
跨物种神经基础模型用于端到端语音解码
Yizi Zhang, Linyang He, Chaofei Fan, Tingkai Liu, Han Yu, Trung Le, Jingyuan Li, Scott Linderman, Lea Duncker, Francis R Willett, Nima Mesgarani, Liam Paninski
机构
*
Columbia University(哥伦比亚大学)
;
Stanford University(斯坦福大学)
;
Microsoft(微软公司)
;
University of Washington(华盛顿大学)
Energy-Regularized Sequential Model Editing on Hyperspheres
超球面能量正则化的序列模型编辑
Qingyuan Liu, Jia-Chen Gu, Yunzhi Yao, Hong Wang, Nanyun Peng
机构
*
Columbia University(哥伦比亚大学)
;
University of California, Los Angeles(加州大学洛杉矶分校)
;
Zhejiang University(浙江大学)
;
University of Science and Technology of China(中国科学技术大学)
Cross Modality Image Translation In Medical Imaging Using Generative Frameworks
利用生成框架在医学影像中实现跨模态图像翻译
Giulia Romoli, Alessia Capoccia, Filippo Ruffini, Francesco Di Feola, Luca Boldrini, Arturo Chiti, Renato Cuocolo, Tugba Akinci D'Antonoli, Fatemeh Darvizeh, Marcello Di Pumpo, Bradley J. Erickson, Liu Fang, Deborah Fazzini, Paola Feraco, Fabrizia Gelardi, Francesco Gossetti, Ana Isabel Hernáiz Ferrer, Michail E. Klontzas, Seyedmehdi Payabvash, Katrine Riklund, Sara N. Strandberg, Valerio Guarrasi, Paolo Soda
机构
*
Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University(诊断与介入部门、放射物理、生物医学工程,乌梅大学)
;
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma(人工智能与计算机系统单位,工程部门,罗马生物医学学院)
;
Vita-Salute San Raffaele University(维塔-萨拉特·桑拉法埃莱大学)
;
Department of Medicine, Surgery and Dentistry, University of Salerno(医学、外科和牙科部门,萨勒诺大学)
;
Division of Diagnostic and Interventional Neuroradiology, Department of Radiology, University Hospital Basel(诊断和介入神经放射学部门,放射学部门,巴塞尔大学医院)
;
Department of Pediatric Radiology, University Children’s Hospital Basel(儿科放射学部门,巴塞尔儿童医院)
;
Department of Life Science and Public Health, Università Cattolica del Sacro Cuore(生命科学与公共健康部门,圣心大学)
;
Athinoula A. Martinos Center for Biomedical Imaging(阿提诺拉A·马里诺斯生物医学成像中心)
;
Artificial Intelligence and Translational Imaging (ATI) Lab, Department of Radiology, School of Medicine, University of Crete(人工智能与转化成像(ATI)实验室,放射学部门,医学院,克里特大学)
;
Division of Radiology, Department of Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institute(放射学部门,临床科学、介入和科技(CLINTEC)部门,卡罗林斯卡研究所)
;
Columbia University Medical Center(哥伦比亚大学医学中心)
;
Department of Diagnostics and intervention, Diagnostic radiology, Umeå University(诊断与介入部门,诊断放射学,乌梅大学)
Amortized Guidance for Image Inpainting with Pretrained Diffusion Models
基于预训练扩散模型的图像修复的 amortized 指导
Yilie Huang, Xun Yu Zhou
机构
*
Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027, USA(工业工程与运筹学系,哥伦比亚大学,纽约,NY 10027,美国)
;
Department of Industrial Engineering and Operations Research & Data Science Institute, Columbia University, New York, NY 10027, USA(工业工程与运筹学系及数据科学研究所,哥伦比亚大学,纽约,NY 10027,美国)
机构
*
Brandeis University(布雷纳大学)
;
University of California, Los Angeles(加州大学洛杉矶分校)
;
Georgetown University(乔治城大学)
;
Columbia University(哥伦比亚大学)
;
Stevens Institute of Technology(史蒂文斯理工学院)