Comments7 pages, 2 figures, 5 tables. Oral paper at the 2nd Workshop on Epistemic Intelligence in Machine Learning (EIML@ICML 2026), Seoul, South Korea
CommentsPublished in the Proc. 1st Symposium on Artificial Intelligence throughout the Human-Centered Design Process (https://dl.gi.de/handle/20.500.12116/48536). Winner of the Best Paper Award
Journal refProc. 1st Symposium on Artificial Intelligence throughout the Human-Centered Design Process 2026 (AI-HCD)
IMProofBench: Benchmarking AI on Research-Level Mathematical Proof Generation
IMProofBench:在研究级数学证明生成上对人工智能进行基准测试
Johannes Schmitt, Gergely Bérczi, Jasper Dekoninck, Jeremy Feusi, Tim Gehrunger, Raphael Appenzeller, Pieter Belmans, Alessio Bottini, Jim Bryan, João Camarneiro, Ana Cannas da Silva, Niklas Canova, Ana-Maria Castravet, Timo de Wolff, Claudio Fontanari, Filippo Gaia, Baran Hashemi, Daniel Holmes, David Holmes, Aitor Iribar Lopez, Victor Jaeck, Martina Jørgensen, Steven Kelk, Martijn Kool, Stefan Kuhlmann, Adam Kurpisz, Johannes Lengler, Chiara Meroni, Ingmar Metzler, Martin Möller, Samuel Muñoz-Echániz, David Muñoz-Lahoz, Robert Nowak, Georg Oberdieck, Daniel Platt, Dylan Possamaï, Gabriel Ribeiro, Aluna Rizzoli, Daria Sakhanda, Raúl Sánchez Galán, Zheming Sun, Diaaeldin Taha, Josef Teichmann, Richard P. Thomas, Henk van der Pol, Michel van Garrel, Charles Vial, Ignacio Barros, Benjamin Doerr, Peter Grünwald, Henry Liu, David Martins, Aleksandar Mijatović, Sergej Monavari, Marc Roth, Patrick Schnider, Yannik Schuler, Pim Spelier, Yuuji Tanaka, Ronald van Luijk
机构
*
ETH Zurich(苏黎世联邦理工学院)
;
Aarhus University(奥胡斯大学)
专题命中
推理与问题求解
:large language model(abstract);language model(abstract);分类 cs.CL
Commentsv2: benchmark expanded from 39 to 77 problems; evaluation extended to 14 models including GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6; new analyses (IRT-based score aggregation, inter-rater reliability, tool/token usage, non-agentic ablation); contributor author list updated
CommentsAccepted for publication at the 2026 IEEE International Conference on Cyber Security and Resilience (IEEE CSR), Lisbon, Portugal, August 3-5, 2026. 8 pages, 1 figure
Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry
重新思考大语言模型作为评判器:通过语义能力不对称利用小语言模型进行表示作为评判器
Zhuochun Li, Yong Zhang, Ming Li, Yuelyu Ji, Yiming Zeng, Ning Cheng, Yun Zhu, Yanmeng Wang, Shaojun Wang, Jing Xiao, Daqing He
机构
*
Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)
;
University of Pittsburgh(匹兹堡大学)
;
University of Maryland, College Park(马里兰大学学院公园分校)
;
University of Connecticut(康涅狄格大学)
专题命中
评测与基准
:LLM(title,abstract);language model(title,abstract);small language model(title);large language model(abstract)
Dropping Just a Handful of Preferences Can Change Top Large Language Model Rankings
丢弃少量偏好可以改变大型语言模型的排名
Jenny Y. Huang, Yunyi Shen, Dennis Wei, Tamara Broderick
机构
*
Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology(麻省理工学院电子工程与计算机科学系)
;
MIT-IBM Watson AI Lab(MIT-IBM沃森人工智能实验室)
;
IBM Research(IBM研究院)
专题命中
评测与基准
:LLM(summary_cn,abstract);large language model(title);language model(title);分类 cs.LG