Forecasting Downstream Performance of LLMs With Proxy Metrics
通过代理指标预测大语言模型的下游性能
Arkil Patel, Siva Reddy, Marius Mosbach, Dzmitry Bahdanau
机构
*
Mila – Quebec AI Institute & McGill University(魁北克AI研究院与麦吉尔大学)
;
CIFAR AI Chair(CIFAR人工智能主席)
;
ServiceNow Research Periodic Labs(ServiceNow研究周期实验室)
机构
*
New York University Tandon School of Engineering(纽约大学Tandon工程学院)
;
University of California, San Diego(加州大学圣地亚哥分校)
;
Stanford University(斯坦福大学)
;
ServiceNow Research(ServiceNow研究)
;
Mila - Quebec AI Institute(魁北克AI研究院)
;
Université de Montréal(蒙特利尔大学)
VectorGym: A Multitask Benchmark for SVG Code Generation, Sketching, and Editing
VectorGym:一个多任务基准用于SVG代码生成、草图和编辑
Juan Rodriguez, Haotian Zhang, Abhay Puri, Tianyang Zhang, Rishav Pramanik, Meng Lin, Xiaoqing Xie, Marco Terral, Darsh Kaushik, Aly Shariff, Perouz Taslakian, Spandana Gella, Sai Rajeswar, David Vazquez, Christopher Pal, Marco Pedersoli
机构
*
ServiceNow Research(ServiceNow 研究院)
;
Mila, Quebec AI Institute(Mila 魁北克人工智能研究所)
;
Columbia University(哥伦比亚大学)
;
University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
;
Stony Brook University(石溪大学)
;
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
;
Minzu University of China(中央民族大学)
;
University of Waterloo(滑铁卢大学)
;
Canada CIFAR AI Chair(加拿大 CIFAR 人工智能讲席)
;
Computer Vision Center(计算机视觉中心)
CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
CUA-Suite:大规模人工标注的视频演示用于计算机使用代理
Xiangru Jian, Shravan Nayak, Kevin Qinghong Lin, Aarash Feizi, Kaixin Li, Patrice Bechard, Spandana Gella, Sai Rajeswar
机构
*
ServiceNow
;
University of Waterloo(多伦多大学)
;
Mila
;
Université de Montréal(蒙特利尔大学)
;
McGill University(麦吉尔大学)
;
University of Oxford(牛津大学)
;
National University of Singapore(新加坡国立大学)
DRBench: A Realistic Benchmark for Enterprise Deep Research
DRBench:企业深度研究的现实基准
Amirhossein Abaskohi, Tianyi Chen, Miguel Muñoz-Mármol, Curtis Fox, Amrutha Varshini Ramesh, Étienne Marcotte, Xing Han Lù, Nicolas Chapados, Spandana Gella, Peter West, Giuseppe Carenini, Christopher Pal, Alexandre Drouin, Issam H. Laradji
机构
*
ServiceNow Research(ServiceNow 研究所)
;
University of British Columbia(不列颠哥伦比亚大学)
;
Mila – Quebec AI Institute(魁北克人工智能研究院)
;
McGill University(麦吉尔大学)
;
Polytechnique Montreal(蒙特利尔理工学院)
;
Canada CIFAR AI Chair(加拿大 CIFAR 人工智能主席)
Joshua Kazdan, Noam Levi, Rylan Schaeffer, Jessica Chudnovsky, Abhay Puri, Bo He, Mehmet Donmez, Sanmi Koyejo, David Donoho
机构
*
Department of Statistics, Stanford University
;
AI4Science, EPFL
;
Department of Computer Science, Stanford University
;
ServiceNow Research
;
Department of XXX, University of YYY, Location, Country
;
School of ZZZ, Institute of WWW, Location, Country
;
École Polytechnique F\'ed\'erale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland
WildSVG: Towards Reliable SVG Generation Under Real-Word Conditions
WildSVG:迈向真实世界条件下可靠的SVG生成
Marco Terral, Haotian Zhang, Tianyang Zhang, Meng Lin, Xiaoqing Xie, Haoran Dai, Darsh Kaushik, Pai Peng, Nicklas Scharpff, David Vazquez, Joan Rodriguez
机构
*
QuiverAI
;
Columbia University(哥伦比亚大学)
;
Illinois Institute of Technology(伊利诺伊理工学院)
;
Mila - Quebec Artificial Intelligence Institute(魁北克人工智能研究所)
;
University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
;
ServiceNow Research(ServiceNow研究)
JEF-Hinter: Leveraging Offline Knowledge for Improving Web Agents Adaptation
利用离线知识提升网络代理适应性的JEF-Hinter
Hadi Nekoei, Aman Jaiswal, Patrice Bechard, Oleh Shliazhko, Orlando Marquez Ayala, Mathieu Reymond, Massimo Caccia, Alexandre Drouin, Sarath Chandar, Alexandre Lacoste
机构
*
ServiceNow AI Research(ServiceNow AI研究机构)
;
Chandar Research Lab(Chandar研究实验室)
;
Mila -- Quebec AI Institute(魁北克人工智能研究院)
;
Dalhousie University(达尔豪斯大学)
;
Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI
GEO-Bench-2:从性能到能力,重新思考地理空间AI的评估
Naomi Simumba, Nils Lehmann, Paolo Fraccaro, Hamed Alemohammad, Geeth De Mel, Salman Khan, Manil Maskey, Nicolas Longepe, Xiao Xiang Zhu, Hannah Kerner, Juan Bernabe-Moreno, Alexandre Lacoste
机构
*
IBM Research Europe(IBM欧洲研究院)
;
Technical University Munich(慕尼黑技术大学)
;
Clark University(克拉克大学)
;
MBZUAI
;
NASA Impact(NASA影响计划)
;
ESA Φ \Phi -lab(欧洲航天局Φ实验室)
;
Arizona State University(亚利桑那州立大学)
;
ServiceNow AI Research(ServiceNow人工智能研究)
Posets and Bounded Probabilities for Discovering Order-inducing Features in Event Knowledge Graphs
偏序集与有界概率用于事件知识图谱中诱导顺序的特征发现
Christoffer Olling Back, Jakob Grue Simonsen
机构
*
Department of Computer Science, University of Copenhagen(哥本哈根大学计算机科学系)
;
ServiceNow Denmark Aps(ServiceNow丹麦公司)
;
IT University of Copenhagen(哥本哈根信息技术大学)