大模型推理能力

From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models

Jue Zhang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

机构 * Microsoft（微软） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP'25 (Main)

RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks

Amit Agarwal, Hitesh Laxmichand Patel, Srikant Panda, Hansa Meghwani, Jyotika Singh, Karan Dua, Paul Li, Tao Sheng, Sujith Ravi, Dan Roth

机构 * Oracle AI 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in EMNLP 2025

Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs

Zhipeng Yang, Junzhuo Li, Siyu Xia, Xuming Hu

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； The Hong Kong University of Science and Technology（香港科技大学） ； Southeast University（东南大学） 纠错

专题命中 推理评测 ：chain-of-thought(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2025 Main

One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning

Sualeha Farid, Jayden Lin, Zean Chen, Shivani Kumar, David Jurgens

机构 * University of Michigan（密歇根大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 22 pages, 11 figures, 6 tables

JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning

Huanghai Liu, Quzhe Huang, Qingjing Chen, Yiran Hu, Jiayu Ma, Yun Liu, Weixing Shen, Yansong Feng

机构 * School of Law, Tsinghua University（清华大学法学院） ； Wangxuan Institute of Computer Technology, Peking University（北京大学王璇计算机技术研究所） ； Department of Legal Studies, University of Bologna（博洛尼亚大学法律系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Cross-Cultural Transfer of Commonsense Reasoning in LLMs: Evidence from the Arab World

Saeed Almheiri, Rania Hossam, Mena Attia, Chenxi Wang, Preslav Nakov, Timothy Baldwin, Fajri Koto

机构 * Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2025 - Findings

Brittleness and Promise: Knowledge Graph Based Reward Modeling for Diagnostic Reasoning

Saksham Khatwani, He Cheng, Majid Afshar, Dmitriy Dligach, Yanjun Gao

机构 * University of Colorado Boulder（科罗拉多大学博尔德分校） ； University of Colorado Anschutz（科罗拉多大学安舒茨分校） ； University of Wisconsin - Madison（威斯康星大学麦迪逊分校） ； Loyola University Chicago（芝加哥洛约拉大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MSCoRe: A Benchmark for Multi-Stage Collaborative Reasoning in LLM Agents

Yuzhen Lei, Hongbin Xie, Jiaxing Zhao, Shuangxue Liu, Xuan Song

机构 * Jilin University（吉林大学） ； Southern University of Science and Technology（南方科技大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 5 figures

Agentic AI with Orchestrator-Agent Trust: A Modular Visual Classification Framework with Trust-Aware Orchestration and RAG-Based Reasoning

Konstantinos I. Roumeliotis, Ranjan Sapkota, Manoj Karkee, Nikolaos D. Tselikas

机构 * University of the Peloponnese, Department of Informatics and Telecommunications（希腊皮埃蒙特大学信息与电信系） ； Cornell University, Department of Biological and Environmental Engineering（康奈尔大学生物与环境工程系） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Seeing Culture: A Benchmark for Visual Reasoning and Grounding

Burak Satar, Zhixin Ma, Patrick A. Irawan, Wilfried A. Mulyawan, Jing Jiang, Ee-Peng Lim, Chong-Wah Ngo

机构 * Singapore Management University（新加坡管理大学） ； Bandung Institute of Technology（班达理工大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 Main Conference, https://seeingculture-benchmark.github.io/

GRID: Graph-based Reasoning for Intervention and Discovery in Built Environments

Taqiya Ehsan, Shuren Xia, Jorge Ortiz

机构 * Rutgers University（罗格斯大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

ASTRA: A Negotiation Agent with Adaptive and Strategic Reasoning via Tool-integrated Action for Dynamic Offer Optimization

Deuksin Kwon, Jiwon Hae, Emma Clift, Daniel Shamsoddini, Jonathan Gratch, Gale M. Lucas

机构 * University of Southern California（美国南加州大学） ； St. Olaf College（圣奥拉夫学院） ； Northwestern University（西北大学） ； USC for Institute of Creative Technologies（美国南加州大学创意技术研究所） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 (Main Conference)

Spatial Audio Motion Understanding and Reasoning

Arvind Krishna Sridhar, Yinyi Guo, Erik Visser

机构 * Qualcomm Technologies Inc.（高通技术公司） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 5 pages, 2 figures, 3 tables

MovieCORE: COgnitive REasoning in Movies

Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh, Ying Cheng, Hung-Ting Su, Yung-Hao Tang, Shang-Hong Lai, Winston H. Hsu

机构 * National Taiwan University（国立台湾大学） ； NVIDIA ； National Tsing Hua University（国立清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for EMNLP'2025 Main Conference (Oral Presentation). Project Page: https://joslefaure.github.io/assets/html/moviecore.html

Assessing Large Language Models on Islamic Legal Reasoning: Evidence from Inheritance Law Evaluation

Abdessalam Bouchekif, Samer Rashwani, Heba Sbahi, Shahd Gaben, Mutaz Al-Khatib, Mohammed Ghaly

机构 * Hamad Bin Khalifa University（哈马德·本·卡西姆大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 7 Tables, Code: https://github.com/bouchekif/inheritance_evaluation

FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning

Liang Hu, Jianpeng Jiao, Jiashuo Liu, Yanle Ren, Zhoufutu Wen, Kaiyuan Zhang, Xuanliang Zhang, Xiang Gao, Tianci He, Fei Hu, Yali Liao, Zaiyuan Wang, Chenghao Yang, Qianyu Yang, Mingren Yin, Zhiyuan Zeng, Ge Zhang, Xinyi Zhang, Xiying Zhao, Zhenwei Zhu, Hongseok Namkoong, Wenhao Huang, Yuwen Tang

机构 * Columbia Business School（哥伦比亚商学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments 29 pages

Zero-shot Graph Reasoning via Retrieval Augmented Framework with LLMs

Hanqing Li, Kiran Sheena Jyothi, Henry Liang, Sharika Mahadevan, Diego Klabjan

机构 * Northwestern University（西北大学） ； EXL Service（EXL服务） ； Vail Systems（Vail系统） ； Netflix 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables

Matteo Marcuzzo, Alessandro Zangari, Andrea Albarelli, Jose Camacho-Collados, Mohammad Taher Pilehvar

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to EMNLP 2025 Main Conference

EthicsMH: A Pilot Benchmark for Ethical Reasoning in Mental Health AI

Sai Kartheek Reddy Kasu

机构 * IIIT Dharwad（德瓦德理工学院） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Automated MCQA Benchmarking at Scale: Evaluating Reasoning Traces as Retrieval Sources for Domain Adaptation of Small Language Models

Ozan Gokdemir, Neil Getty, Robert Underwood, Sandeep Madireddy, Franck Cappello, Arvind Ramanathan, Ian T. Foster, Rick L. Stevens

机构 * Argonne National Laboratory（阿贡国家实验室） ； The University of Chicago（芝加哥大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments This manuscript has been accepted for publication at the Supercomputing 25 (SC '25) Conference (Frontiers in Generative AI for HPC Science and Engineering: Foundations, Challenges, and Opportunities Workshop) in St. Louis, MO, USA on November 16th, 2025. It will appear in the SC25 Workshop Proceedings after that date

GeoGuess: Multimodal Reasoning based on Hierarchy of Visual Information in Street View

Fenghua Cheng, Jinxiang Wang, Sen Wang, Zi Huang, Xue Li

机构 * The University of Queensland（昆士兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Updated version

Optimizing Length Compression in Large Reasoning Models

Zhengxiang Cheng, Dongping Chen, Mingyang Fu, Tianyi Zhou

机构 * umd（马里兰大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 7 figures, 4 tables

MaRVL-QA: A Benchmark for Mathematical Reasoning over Visual Landscapes

Nilay Pande, Sahiti Yerramilli, Jayant Sravan Tamarapalli, Rynaa Grover

机构 * Waymo ； Google（谷歌） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reasoning Language Model for Personalized Lung Cancer Screening

Chuang Niu, Ge Wang

机构 * Department of Biomedical Engineering, School of Engineering, Center for Computational Innovations, Center for Biotechnology & Interdisciplinary Studies（生物医学工程系、工程学院、计算创新中心、生物技术与跨学科研究所以及中心） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.AI、cs.LG

Reverse-Engineered Reasoning for Open-Ended Generation

Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, Wangchunshu Zhou, Jiazhan Feng, Wanjun Zhong, Wei Ye, Tong Yang, Wenhao Huang, Ge Zhang, Fangzhen Lin

机构 * Hong Kong University of Science and Technology（香港科技大学） ； Peking University（北京大学） ； Tsinghua University（清华大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Preprint

CVPD at QIAS 2025 Shared Task: An Efficient Encoder-Based Approach for Islamic Inheritance Reasoning

Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Hichem Telli, Cosimo Distante, Abdenour Hadid

机构 * University of the Basque Country UPV/EHU（巴斯克大学UPV/EHU） ； Institute of Applied Sciences and Intelligent Systems – CNR（应用科学与智能系统研究所 – CNR） ； Laboratory of LESIA, University of Biskra（LESIA实验室，比斯克拉大学） ； Sorbonne University Abu Dhabi, UAE（阿布扎赫尔索邦大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.LG

Evaluating Large Language Models for Financial Reasoning: A CFA-Based Benchmark Study

Xuan Yao, Qianteng Wang, Xinbo Liu, Ke-Wei Huang

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

FRIDA to the Rescue! Analyzing Synthetic Data Effectiveness in Object-Based Common Sense Reasoning for Disaster Response

Mollie Shichman, Claire Bonial, Austin Blodgett, Taylor Hudson, Francis Ferraro, Rachel Rudinger

机构 * University of Maryland, College Park（马里兰大学学院市分校） ； Army Research Lab（陆军研究实验室） ； Oak Ridge Associated Universities（橡树岭关联大学） ； University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 3 figures, 5 tables

LongReasonArena: A Long Reasoning Benchmark for Large Language Models

Jiayu Ding, Shuming Ma, Lei Cui, Nanning Zheng, Furu Wei

机构 * IAIR, Xi’an Jiaotong University（IAIR，西安交通大学） 纠错

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI

TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models

Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, Arman Cohan

专题命中 推理评测 ：reasoning(title,abstract)；分类 cs.CL、cs.AI