Stefan Birr, Tobias Huelden, Mones Raslan, Adele Gouttes, Andreas Schmitt, Mateusz Koren, Johannes Stephan, Robert Streek, Manuel Kunz, Tim Januschowski
OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning
OfficeQA Pro:一个企业级端到端 grounded 推理基准测试
Krista Opsahl-Ong, Arnav Singhvi, Jasmine Collins, Ivan Zhou, Cindy Wang, Ashutosh Baheti, Owen Oertell, Jacob Portes, Sam Havens, Erich Elsen, Michael Bendersky, Matei Zaharia, Xing Chen
机构
*
Databricks AI Research(Databricks人工智能研究)
AI总结
OfficeQA Pro 是一个企业级基准测试,评估 AI 代理在大规模文档语料库上进行端到端 grounded 推理的能力,发现结构化文档表示可显著提升性能。
Comments24 pages, 16 figures. Introduces the OfficeQA Pro benchmark for grounded reasoning over enterprise documents
Jonathan D. Chang, Andrew Drozdov, Shubham Toshniwal, Owen Oertell, Alexander Trott, Jacob Portes, Abhay Gupta, Pallavi Koppol, Ashutosh Baheti, Sean Kulinski, Ivan Zhou, Irene Dea, Krista Opsahl-Ong, Simon Favreau-Lessard, Sean Owen, Jose Javier Gonzalez Ortiz, Arnav Singhvi, Xabi Andrade, Cindy Wang, Kartik Sreenivasan, Sam Havens, Jialu Liu, Peyton DeNiro, Wen Sun, Michael Bendersky, Jonathan Frankle
CommentsAccepted at IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024 as a highlight. Code available at https://github.com/mariannerakic/tyche
Predicting Task Performance with Context-aware Scaling Laws
Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang
机构
*
UC Santa Cruz(加州大学圣克ruz分校)
;
Washington University in St. Louis(华盛顿大学圣路易斯分校)
;
Databricks(Databricks公司)
;
Google DeepMind(谷歌DeepMind)
;
UC Berkeley(加州大学伯克利分校)