大模型推理能力

FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging

Zichen Tang, Haihong E, Ziyan Ma, Haoyang He, Jiacheng Liu, Zhongjun Yang, Zihua Rong, Rongjin Li, Kun Ji, Qing Huang, Xinyang Hu, Yang Liu, Qianhe Zheng

机构 * Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted by ACL 2025 Main Conference

Journal ref Proc. ACL 2025 (Volume 1: Long Papers), pages 15721-15749

Evaluating Multi-Hop Reasoning in Large Language Models: A Chemistry-Centric Case Study

Mohammad Khodadad, Ali Shiraee Kasmaee, Mahdi Astaraki, Nicholas Sherck, Hamidreza Mahyar, Soheila Samiee

机构 * Department of Computational Science and Engineering（计算科学与工程系） ； McMaster University（麦斯特大学） ； BASF Canada Inc（巴斯夫加拿大公司） ； BASF Corporation（巴斯夫公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

REALM-Bench: A Benchmark for Evaluating Multi-Agent Systems on Real-world, Dynamic Planning and Scheduling Tasks

Longling Geng, Edward Y. Chang

机构 * Stanford University（斯坦福大学） 纠错

专题命中 推理评测 ：planning(title,abstract)；分类 cs.AI

Comments 24 pages, 8 figures, 28 tables, 7 listings

Coherent Multimodal Reasoning with Iterative Self-Evaluation for Vision-Language Models

Wenjie Luo, Ruocheng Li, Shanshan Zhu, Julian Perry

机构 * Fujian University of Technology（福建工程大学） ； Delta University for Science and Technology（科技大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

The Multi-Round Diagnostic RAG Framework for Emulating Clinical Reasoning

Penglei Sun, Yixiang Chen, Xiang Li, Xiaowen Chu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos

Jiashuo Yu, Yue Wu, Meng Chu, Zhifei Ren, Zizheng Huang, Pei Chu, Ruijie Zhang, Yinan He, Qirui Li, Songze Li, Zhenxiang Li, Zhongying Tu, Conghui He, Yu Qiao, Yali Wang, Yi Wang, Limin Wang

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Nanjing University（南京大学） ； Shanghai Innovation Institute（上海创新研究院） ； Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments ICCV2025

Causal Reasoning in Pieces: Modular In-Context Learning for Causal Discovery

Kacper Kadziolka, Saber Salehkaleybar

机构 * Leiden Institute of Advanced Computer Science (LIACS)（莱顿先进计算机科学研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Reasoning Language Models for Root Cause Analysis in 5G Wireless Networks

Mohamed Sana, Nicola Piovesan, Antonio De Domenico, Yibin Kang, Haozhe Zhang, Merouane Debbah, Fadhel Ayed

机构 * Huawei, France（华为，法国） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

DrugPilot: LLM-based Parameterized Reasoning Agent for Drug Discovery

Kun Li, Zhennan Wu, Shoupeng Wang, Jia Wu, Shirui Pan, Wenbin Hu

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 29 pages, 8 figures, 2 tables

LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs

Da-Chen Lian, Ri-Sheng Huang, Pin-Er Chen, Chunki Lim, You-Kuan Lin, Guan-Yu Tseng, Zi-Cheng Yang, Zhen-Yu Lin, Pin-Cheng Chen, Shu-Kai Hsieh

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 42p, 17f, 10t. Revisions: Merged paragraphs in Intro to emphasize contributions. Clarified benchmark design (Sec 3.5.1). Added single-agent, OpenAI-guided & 6-round experiments (Sec 5.2). Note: we only ran each experiment once; statistical tests are needed for strong claims. Revised Sec 6. Added acknowledgements, 2 new co-authors, and corrected typos/grammar

Reasoning Beyond the Obvious: Evaluating Divergent and Convergent Thinking in LLMs for Financial Scenarios

Zhuang Qiang Bok, Watson Wei Khong Chua

机构 * Deep Insight Labs（深洞察实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Accepted by Agentic & GenAI Evaluation KDD2025: KDD workshop on Evaluation and Trustworthiness of Agentic and Generative AI Models https://kdd-eval-workshop.github.io/genai-evaluation-kdd2025/

Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning

Xinyao Liu, Diping Song

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

AURA: A Multi-Modal Medical Agent for Understanding, Reasoning & Annotation

Nima Fathi, Amar Kumar, Tal Arbel

机构 * Center for Intelligent Machines, McGill University, Montreal, Canada（麦吉尔大学智能机器中心，加拿大蒙特利尔） ； Mila - Quebec AI institute, Montreal, Canada（魁北克AI研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments 9 pages, 3 figures, International Conference on Medical Image Computing and Computer-Assisted Intervention

CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos

Xuchen Li, Xuzhao Li, Shiyu Hu, Kaiqi Huang, Wentao Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments Preprint, Under review

X-Intelligence 3.0: Training and Evaluating Reasoning LLM for Semiconductor Display

Xiaolin Yan, Yangxing Liu, Jiazhang Zheng, Chi Liu, Mingyu Du, Caisheng Chen, Haoyang Liu, Ming Ding, Yuan Li, Qiuping Liao, Linfeng Li, Zhili Mei, Siyu Wan, Li Li, Ruyi Zhong, Jiangling Yu, Xule Liu, Huihui Hu, Jiameng Yue, Ruohui Cheng, Qi Yang, Liangqing Wu, Ke Zhu, Chi Zhang, Chufei Jing, Yifan Zhou, Yan Liang, Dongdong Li, Zhaohui Wang, Bin Zhao, Mingzhou Wu, Mingzhong Zhou, Peng Du, Zuomin Liao, Chao Dai, Pengfei Liang, Xiaoguang Zhu, Yu Zhang, Yu Gu, Kun Pan, Yuan Wu, Yanqing Guan, Shaojing Wu, Zikang Feng, Xianze Ma, Peishan Cheng, Wenjuan Jiang, Jing Ba, Huihao Yu, Zeping Hu, Yuan Xu, Zhiwei Liu, He Wang, Zhenguo Lin, Ming Liu, Yanhong Meng

机构 * TCL Corporate Research（TCL企业研究） ； TCL China Star Optoelectronic Technology Co Ltd.（TCL中国星光电技术有限公司） ； National Center of Technology Innovation for Display（显示技术创新国家中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Technical Report

Commonsense Reasoning in Arab Culture

Abdelrahman Sadallah, Junior Cedric Tonga, Khalid Almubarak, Saeed Almheiri, Farah Atif, Chatrine Qwaider, Karima Kadaoui, Sara Shatnawi, Yaser Alesh, Fajri Koto

机构 * Mohamed bin Zayed University of Artificial Intelligence（莫扎伊德·本·泽德人工智能大学） ； SDAIA ； Al-Balqa Applied University（巴尔加应用大学） ； Khalifa University（卡利法大学） ； HUMAIN, Data and AI Models（HUMAIN、数据与人工智能模型） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments ACL 2025 - Main

Adaptive Multi-Agent Reasoning via Automated Workflow Generation

Humza Sami, Mubashir ul Islam, Pierre-Emmanuel Gaillardon, Valerio Tenace

机构 * PrimisAI ； University of Utah（犹他大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

COREVQA: A Crowd Observation and Reasoning Entailment Visual Question Answering Benchmark

Ishant Chintapatla, Kazuma Choji, Naaisha Agarwal, Andrew Lin, Hannah You, Charles Duong, Kevin Zhu, Sean O'Brien, Vasu Sharma

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

FormulaOne: Measuring the Depth of Algorithmic Reasoning Beyond Competitive Programming

Gal Beniamini, Yuval Dor, Alon Vinnikov, Shir Granot Peled, Or Weinstein, Or Sharir, Noam Wies, Tomer Nussbaum, Ido Ben Shaul, Tomer Zekharya, Yoav Levine, Shai Shalev-Shwartz, Amnon Shashua

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Revisiting Reliability in the Reasoning-based Pose Estimation Benchmark

Junsu Kim, Naeun Kim, Jaeho Lee, Incheol Park, Dongyoon Han, Seungryul Baek

机构 * UNIST（全南大学） ； NVIDIA Foundation Models Lab（NVIDIA基础模型实验室） ； MODULABS ； NAVER AI Lab（NAVER AI实验室） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments To be presented as a poster at MMFM 2025

TransEvalnia: Reasoning-based Evaluation and Ranking of Translations

Richard Sproat, Tianyu Zhao, Llion Jones

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Tackling the Abstraction and Reasoning Corpus with Vision Transformers: the Importance of 2D Representation, Positions, and Objects

Wenhao Li, Yudong Xu, Scott Sanner, Elias Boutros Khalil

机构 * University of Toronto（多伦多大学） ； Vector Institute for Artificial Intelligence（人工智能向量研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Journal ref Transactions on Machine Learning Research (TMLR), 07/2025, https://openreview.net/forum?id=Al72Fp0rCg

MSA at ImageCLEF 2025 Multimodal Reasoning: Multilingual Multimodal Reasoning With Ensemble Vision Language Models

Seif Ahmed, Mohamed T. Younes, Abdelrahman Moustafa, Abdelrahman Allam, Hamza Moustafa

机构 * October University for Modern Sciences and Arts（现代科学与艺术大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs

Florian Grötschla, Luis Müller, Jan Tönshoff, Mikhail Galkin, Bryan Perozzi

机构 * ETH Zurich（苏黎世联邦理工学院） ； RWTH Aachen University（亚琛工业大学） ； Google Research（谷歌研究院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.LG

Comments Preprint

Automating Expert-Level Medical Reasoning Evaluation of Large Language Models

Shuang Zhou, Wenya Xie, Jiaxi Li, Zaifu Zhan, Meijia Song, Han Yang, Cheyenna Espinoza, Lindsay Welton, Xinnie Mai, Yanwei Jin, Zidu Xu, Yuen-Hei Chung, Yiyun Xing, Meng-Han Tsai, Emma Schaffer, Yucheng Shi, Ninghao Liu, Zirui Liu, Rui Zhang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments 22 pages,6 figures

Can Input Attributions Explain Inductive Reasoning in In-Context Learning?

Mengyu Ye, Tatsuki Kuribayashi, Goro Kobayashi, Jun Suzuki

机构 * Tohoku University（东北大学） ； MBZUAI ； RIKEN（日本研究机构） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Findings of ACL 2025

A Fuzzy Supervisor Agent Design for Clinical Reasoning Assistance in a Multi-Agent Educational Clinical Scenario Simulation

Weibing Zheng, Laurah Turner, Jess Kropczynski, Murat Ozer, Seth Overla, Shane Halse

机构 * University of Cincinnati（辛辛那提大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 6 pages, 3 figures, 1 table. 2025 IFSA World Congress NAFIPS Annual Meeting

Judging the Judges: Can Large Vision-Language Models Fairly Evaluate Chart Comprehension and Reasoning?

Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Ahmed Masry, Mizanur Rahman, Amran Bhuiyan, Mir Tafseer Nayeem, Shafiq Joty, Enamul Hoque, Jimmy Huang

机构 * York University（约克大学） ； University of Alberta（阿尔伯塔大学） ； Nanyang Technological University（南洋理工大学） ； Salesforce AI Research（Salesforce AI研究） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2025 Industry Track

M$^3$-Med: A Benchmark for Multi-lingual, Multi-modal, and Multi-hop Reasoning in Medical Instructional Video Understanding

Shenxi Liu, Kan Li, Mingyang Zhao, Yuhang Tian, Bin Li, Shoujun Zhou, Hongliang Li, Fuxia Yang

机构 * Beijing Institute of Technology（北京理工大学） ； The Hong Kong Polytechnic University（香港理工大学） ； Chinese Academy of Sciences（中国科学院） ； Air Force Hospital of Southern Theater Command of PLA（中国人民解放军南部战区空军医院） ； Shenzhen Traditional Chinese Medicine Hospital（深圳中医药医院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 19 pages, 8 figures, 7 tables

MMLU-Reason: Benchmarking Multi-Task Multi-modal Language Understanding and Reasoning

Guiyao Tie, Xueyang Zhou, Tianhe Gu, Ruihang Zhang, Chaoran Hu, Sizhe Zhang, Mengqu Sun, Yan Zhang, Pan Zhou, Lichao Sun

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI

Comments 39 pages, 28 figures, 4 tables