大模型推理能力

Flex-TravelPlanner: A Benchmark for Flexible Planning with Language Agents

Juhyun Oh, Eunsu Kim, Alice Oh

机构 * School of Computing（计算机学院） ； KAIST（韩国科学技术院） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.CL

OpenThoughts: Data Recipes for Reasoning Models

Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao, John Yang, Shreyas Pimpalgaonkar, Kartik Sharma, Charlie Cheng-Jie Ji, Yichuan Deng, Sarah Pratt, Vivek Ramanujan, Jon Saad-Falcon, Jeffrey Li, Achal Dave, Alon Albalak, Kushal Arora, Blake Wulfe, Chinmay Hegde, Greg Durrett, Sewoong Oh, Mohit Bansal, Saadia Gabriel, Aditya Grover, Kai-Wei Chang, Vaishaal Shankar, Aaron Gokaslan, Mike A. Merrill, Tatsunori Hashimoto, Yejin Choi, Jenia Jitsev, Reinhard Heckel, Maheswaran Sathiamoorthy, Alexandros G. Dimakis, Ludwig Schmidt

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments https://www.openthoughts.ai/blog/ot3. arXiv admin note: text overlap with arXiv:2505.23754 by other authors

MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models

Huanqia Cai, Yijun Yang, Winston Hu

机构 * Tencent Hunyuan Team（腾讯文言团队） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Toward Scientific Reasoning in LLMs: Training from Expert Discussions via Reinforcement Learning

Ming Yin, Yuanhao Qu, Ling Yang, Le Cong, Mengdi Wang

机构 * Princeton University（普林斯顿大学） ； Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning

Peiqi Sui, Juan Diego Rodriguez, Philippe Laban, Dean Murphy, Joseph P. Dexter, Richard Jean So, Samuel Baker, Pramit Chaudhuri

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL 2025 main

Reasoning-Table: Exploring Reinforcement Learning for Table Reasoning

Fangyu Lei, Jinxiang Meng, Yiming Huang, Tinghong Chen, Yun Zhang, Shizhu He, Jun Zhao, Kang Liu

机构 * Institute of Automation, CAS（中国科学院自动化研究所） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Work in progress

Deep Temporal Reasoning in Video Language Models: A Cross-Linguistic Evaluation of Action Duration and Completion through Perfect Times

Olga Loginova, Sofía Ortega Loguinova

机构 * University of Trento（特伦托大学） ； Maastricht University（马斯特里赫特大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations

Pardis Sadat Zahraei, Ali Emami

机构 * Independent Researcher（独立研究者） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted to Findings of ACL 2025

SafeTy Reasoning Elicitation Alignment for Multi-Turn Dialogues

Martin Kuo, Jianyi Zhang, Aolin Ding, Louis DiValentin, Amin Hass, Benjamin F Morris, Isaac Jacobson, Randolph Linderman, James Kiessling, Nicolas Ramos, Bhavna Gopal, Maziyar Baran Pouyan, Changwei Liu, Hai Li, Yiran Chen

机构 * Center for Computational Evolutionary Intelligence, Duke University（计算进化智能中心，杜克大学） ； Accenture, USA（Accenture美国公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM

Bowen Dong, Minheng Ni, Zitong Huang, Guanglei Yang, Wangmeng Zuo, Lei Zhang

机构 * Harbin Institute of Technology（哈尔滨工业大学） ； The Hong Kong Polytechnic University（香港理工大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

CaReAQA: A Cardiac and Respiratory Audio Question Answering Model for Open-Ended Diagnostic Reasoning

Tsai-Ning Wang, Lin-Lin Chen, Neil Zeghidour, Aaqib Saeed

机构 * Eindhoven University of Technology（埃因霍温理工大学） ； Kyutai France（Kyutai法国分公司） ； Eindhoven Artificial Intelligence Systems Institute（埃因霍温人工智能系统研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments Accepted at AHLI CHIL 2025

STRICTA: Structured Reasoning in Critical Text Assessment for Peer Review and Beyond

Nils Dycke, Matej Zečević, Ilia Kuznetsov, Beatrix Suess, Kristian Kersting, Iryna Gurevych

机构 * UKP Lab, ATHENE National Research Center for Applied Cybersecurity, Technical University of Darmstadt（UKP实验室，ATHENE国家应用网络安全研究中心，德累斯顿技术大学） ； AIML Lab, Hessian Center for AI, Technical University of Darmstadt（AIML实验室，黑森人工智能中心，德累斯顿技术大学） ； Synthetic RNA Biology, Technical University of Darmstadt（合成RNA生物学，德累斯顿技术大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2025

Beyond Context to Cognitive Appraisal: Emotion Reasoning as a Theory of Mind Benchmark for Large Language Models

Gerard Christopher Yeo, Kokil Jaidka

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 9 pages, 3 figures

Breakpoint: Scalable evaluation of system-level reasoning in LLM code agents

Kaivalya Hariharan, Uzay Girit, Atticus Wang, Jacob Andreas

机构 * MIT（麻省理工学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments 21 pages, 14 figures

DeepSeek-R1 vs. o3-mini: How Well can Reasoning LLMs Evaluate MT and Summarization?

Daniil Larionov, Sotaro Takeshita, Ran Zhang, Yanran Chen, Christoph Leiter, Zhipin Wang, Christian Greisinger, Steffen Eger

机构 * University of Mannheim（曼海姆大学） ； University of Technology Nuremberg（纽伦堡技术大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

FCMR: Robust Evaluation of Financial Cross-Modal Multi-Hop Reasoning

Seunghee Kim, Changhyeon Kim, Taeuk Kim

机构 * Hanyang University（翰阳大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL 2025

Scalable Complexity Control Facilitates Reasoning Ability of LLMs

Liangkai Hang, Junjie Yao, Zhiwei Bai, Tianyi Chen, Yang Chen, Rongjie Diao, Hezhou Li, Pengxiao Lin, Zhiwei Wang, Cheng Xu, Zhongwang Zhang, Zhangchen Zhou, Zhiyu Li, Zehao Lin, Kai Chen, Feiyu Xiong, Yaoyu Zhang, Weinan E, Hongkang Yang, Zhi-Qin John Xu

机构 * Institute of Natural Sciences, School of Mathematical Sciences, Shanghai Jiao Tong University（上海交通大学数学科学学院自然科学院） ； MOE-LSC, School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院） ； Center for Machine Learning Research, School of Mathematical Sciences, Peking University（北京大学数学科学学院机器学习研究中心） ； Center for LLM, Institute for Advanced Algorithms Research, Shanghai（上海高级算法研究所大语言模型中心） ； MemTensor (Shanghai) Technology Co., Ltd.（MemTensor（上海）科技有限公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

PhyX: Does Your Model Have the "Wits" for Physical Reasoning?

Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, Zhongwei Wan, Kai Zhang, Wendong Xu, Jing Xiong, Ping Luo, Wenhu Chen, Chaofan Tao, Zhuoqing Mao, Ngai Wong

机构 * The University of Hong Kong（香港大学） ； University of Michigan（密歇根大学） ； Independent（独立研究者） ； University of Toronto（多伦多大学） ； University of Waterloo（滑铁卢大学） ； The Ohio State University（俄亥俄州立大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning

Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, Bo Li

机构 * University of Georgia（佐治亚大学） ； University of Chicago（芝加哥大学） ； UIUC（伊利诺伊大学香槟分校） ； University of Texas at Austin（德克萨斯大学奥斯汀分校） ； University of California, Berkeley（加州大学伯克利分校） ； Emory University（埃默里大学） ； Virtue AI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments Accepted to ICML 2025

GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoning

Qingchen Yu, Zifan Zheng, Ding Chen, Simin Niu, Bo Tang, Feiyu Xiong, Zhiyu Li

机构 * MemTensor (Shanghai) Technology Co., Ltd.（MemTensor（上海）科技有限公司） ； University of Sydney（悉尼大学） ； Research Institute of China Telecom（中国电信研究院） ； Renmin University of China（中国人民大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2025

RAG-Zeval: Towards Robust and Interpretable Evaluation on RAG Responses through End-to-End Rule-Guided Reasoning

Kun Li, Yunxiang Li, Tianhua Zhang, Hongyin Luo, Xixin Wu, James Glass, Helen Meng

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models

Sohyun An, Ruochen Wang, Tianyi Zhou, Cho-Jui Hsieh

机构 * University of California, Los Angeles（加州大学洛杉矶分校） ； University of Maryland, College Park（马里兰大学学院公园分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Work In Progress

Reasoning Is Not All You Need: Examining LLMs for Multi-Turn Mental Health Conversations

Mohit Chandra, Siddharth Sriraman, Harneet Singh Khanuja, Yiqiao Jin, Munmun De Choudhury

机构 * Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 34 pages, 5 figures, 30 tables

rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset

Yifei Liu, Li Lyna Zhang, Yi Zhu, Bingcheng Dong, Xudong Zhou, Ning Shang, Fan Yang, Mao Yang

机构 * Microsoft Research Asia（微软亚洲研究院） ； Dalian University of Technology（大连理工大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Beyond Templates: Dynamic Adaptation of Reasoning Demonstrations via Feasibility-Aware Exploration

Yong Wu, Weihang Pan, Ke Li, Chen Binhui, Ping Li, Binbin Lin

机构 * College of Software Technology, Zhejiang University（浙江大学软件技术学院） ； Fullong Technology, Ningbo, Zhejiang, China（宁波 fuller 技术有限公司） ； Ningbo Zhoushan Port Co., Ltd., Ningbo, Zhejiang, China（宁波舟山港股份有限公司） ； School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

GIFARC: Synthetic Dataset for Leveraging Human-Intuitive Analogies to Elevate AI Reasoning

Woochang Sim, Hyunseok Ryu, Kyungmin Choi, Sungwon Han, Sundong Kim

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMs

Haochuan Wang, Xiachong Feng, Lei Li, Yu Guo, Zhanyue Qin, Dianbo Sui, Lingpeng Kong

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution

Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, Xing Zhou, Dongrui Liu, Ling Yang, Yue Wu, Kaixuan Huang, Shilong Liu, Hongru Wang, Mengdi Wang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 9 pages, 3 figures

Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Markets

Simpson Zhang, Tennison Liu, Mihaela van der Schaar

机构 * DAMTP, University of Cambridge（剑桥大学应用数学与理论物理系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments *Zhang & Liu contributed equally

How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translation

Yongshi Ye, Biao Fu, Chongxuan Huang, Yidong Chen, Xiaodong Shi

机构 * Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能学院） ； School of Informatics, Xiamen University（厦门大学信息学院） ； Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University),Ministry of Culture and Tourism（福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室（厦门大学），文化部） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL