大模型推理能力

Large language models show fragile cognitive reasoning about human emotions

大型语言模型在人类情感认知推理中的脆弱性

Sree Bhattacharyya, Evgenii Kuriabov, Lucas Craig, Tharun Dilliraj, Reginald B. Adams,, Jia Li, James Z. Wang

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

AI总结 本文探讨大型语言模型是否能通过认知维度而非标签进行情感推理，提出CoRE基准测试，发现模型在情感认知结构上有系统性关系，但与人类判断存在偏差且在不同上下文中不稳定。

Comments Under Review, a version was presented at WiML Workshop @ NeurIPS 2025

SpeakRL: Synergizing Reasoning, Speaking, and Acting in Language Models with Reinforcement Learning

SpeakRL: 在语言模型中协同推理、说话与行动以增强强化学习

Emre Can Acikgoz, Jinoh Oh, Jie Hao, Joo Hyuk Jeon, Heng Ji, Dilek Hakkani-Tür, Gokhan Tur, Xiang Li, Chengyuan Ma, Xing Fan

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校） ； Amazon Alexa（亚马逊Alexa） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

AI总结 SpeakRL通过强化学习提升语言模型的对话能力，通过奖励主动互动提高任务完成效率。

LLM-Powered Text-Attributed Graph Anomaly Detection via Retrieval-Augmented Reasoning

通过检索增强推理的大型语言模型驱动的文本属性图异常检测

Haoyan Xu, Ruizhi Qian, Zhengtao Yao, Ziyi Liu, Li Li, Yuqi Li, Yanshu Li, Wenqing Zheng, Daniele Rosa, Daniel Barcklow, Senthil Kumar, Jieyu Zhao, Yue Zhao

专题命中 推理评测 ：reasoning(title)；分类 cs.AI、cs.LG

AI总结 本文提出基于检索增强推理的LLM驱动文本属性图异常检测框架，通过生成语义连贯但上下文不一致的异常节点，提升异常检测效果。

Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning

Qi Luo, Xiaonan Li, Tingshuo Fan, Xinchi Chen, Xipeng Qiu

机构 * Fudan University（复旦大学） ； Shanghai Innovation Institute（上海创新研究院） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

DocFinQA: A Long-Context Financial Reasoning Dataset

Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai, Michael Krumdick, Charles Lovering, Chris Tanner

机构 * Kensho Technologies（肯斯霍技术公司） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments 13 pages

Multimedia-Aware Question Answering: A Review of Retrieval and Cross-Modal Reasoning Architectures

Rahul Raja, Arpita Vats

机构 * Carnegie Mellon University（卡内基梅隆大学） ； Boston University（波士顿大学） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Comments In Proceedings of the 2nd ACM Workshop in AI-powered Question and Answering Systems (AIQAM '25), October 27-28, 2025, Dublin, Ireland. ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3746274.3760393

LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation

Joseph Enguehard, Morgane Van Ermengem, Kate Atkinson, Sujeong Cha, Arijit Ghosh Chowdhury, Prashanth Kallur Ramaswamy, Jeremy Roghair, Hannah R Marlowe, Carina Suzana Negreanu, Kitty Boxall, Diana Mincu

机构 * Robin AI ； Amazon Web Services（亚马逊网络服务） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments Published in Natural Legal Language Processing - EMNLP Workshop 2025

GRILE: A Benchmark for Grammar Reasoning and Explanation in Romanian LLMs

Adrian-Marius Dumitran, Alexandra-Mihaela Danila, Angela-Liliana Dumitran

机构 * University of Bucharest Faculty of Mathematics and Computer Science（布加勒斯大学数学与计算机科学学院） 纠错

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Comments Accepted as long paper @RANLP2025

Harnessing Chain-of-Thought Metadata for Task Routing and Adversarial Prompt Detection

Ryan Marinelli, Josef Pichlmeier, Tamas Bisztray

专题命中 推理评测 ：chain-of-thought(title)；分类 cs.CL、cs.AI

Examining Two Hop Reasoning Through Information Content Scaling

David Johnston, Nora Belrose

专题命中 推理评测 ：reasoning(title)；分类 cs.AI、cs.LG

Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Data

Tianyi Huang, Elsa Fan

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments Accepted Paper (Oral Presentation) in the Workshop on the Social Impact of AI: Research, Diversity and Inclusion Frameworks at AAAI 2025

Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics

Daniel J. H. Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, Sai Chaitanya Tadepalli

专题命中 推理评测 ：reasoning(title)；分类 cs.AI、cs.LG

Comments 48 pages, 4 figures

metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models

Alex Kipnis, Konstantinos Voudouris, Luca M. Schulze Buschoff, Eric Schulz

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Comments accepted for publication at ICLR 2025

Reasoning Over the Glyphs: Evaluation of LLM's Decipherment of Rare Scripts

Yu-Fei Shih, Zheng-Lin Lin, Shu-Kai Hsieh

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Comments 7 pages, 3 figures

TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planning

Xiang Li, Yunshi Lan, Chao Yang

专题命中 推理评测 ：planning(title)；分类 cs.CL、cs.AI

MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning

Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan S. Ilgen, Emma Pierson, Pang Wei Koh, Yulia Tsvetkov

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments 29 pages, 12 figures

AHP-Powered LLM Reasoning for Multi-Criteria Evaluation of Open-Ended Responses

Xiaotian Lu, Jiyi Li, Koh Takeuchi, Hisashi Kashima

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments Accepted for EMNLP 2024 Findings

Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference

Siddhartha Dalal, Vishal Misra

专题命中 推理评测 ：reasoning(title)；分类 cs.AI、cs.LG

OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models

Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, Yulan He

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Comments ACL 2024

Lissard: Long and Simple Sequential Reasoning Datasets

Mirelle Bueno, Roberto Lotufo, Rodrigo Nogueira

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.AI

Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog

Mayank Mishra, Danish Contractor, Dinesh Raghu

专题命中 推理评测 ：reasoning(title)；分类 cs.CL、cs.LG

Disentangling Pictorial Cue Understanding from Language Bias in VLMs via Depth Ordering Task

通过深度排序任务解构视觉语言模型中的图像线索理解与语言偏差

Yiqian Liu, Iuliia Kotseruba, John K. Tsotsos

机构 * York University（约克大学） ； University of Guelph（圭尔夫大学） 纠错

专题命中 推理评测 ：CoT(abstract,abstract_cn)；chain-of-thought(abstract)

AI总结 提出深度排序与异常检测任务，结合控制图像线索和语言表达，量化视觉语言模型的深度感知能力，发现模型对深度线索利用不足且存在语言偏差。

Comments 15 pages, 7 figures, accepted to ECCV 2026 (30 pages, 13 figures, supplementary materials included)

BashCoder-R1: Towards Robust and Explainable Bash Code Generation with Robustness-Aware Group Relative Policy Optimization

BashCoder-R1: 面向鲁棒且可解释的Bash代码生成——鲁棒感知组相对策略优化

Lei Yu, Peng Wang, Jia Xu, Jingyuan Zhang, Xin Wang, Jiajia Ma, Li Yang, Changzhi Deng, Zenghua Wang, Fengjun Zhang

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract_cn)

AI总结 提出BashCoder-R1框架，结合持续预训练、长思维链监督微调和鲁棒感知组相对策略优化，在BashBench基准上实现高语法通过率、低鲁棒警告率和功能完整率，显著超越DeepSeek-V3.2。

Comments Accepted to ISSTA 2026

SmellCC: A Tool for Automated Code Smells Remediation

SmellCC：一种用于自动修复代码异味的工具

Xiaoting Zhang, Yujie Zhang, Zhipeng Gao, Xing Hu, Xin Xia

专题命中 推理评测 ：CoT(abstract,abstract_cn)；chain-of-thought(abstract)

AI总结 SmellCC是一款Visual Studio Code扩展，结合SonarQube与LLM流水线，采用思维链和少样本学习，可一键修复Python前10种常见代码异味，清洁率96.8%、准确率91.3%，能提升软件可维护性。

JoyAI-Talker: Full-Duplex Speech Interactive Large Model Built for Empathetic Voice Agents

JoyAI-Talker：面向共情语音智能体的全双工语音交互大模型

Yinhao Bai, Jinming Chen, Yafeng Chen, Wei Deng, Boya Dong, Nan Duan, Yu Gu, Weisheng Han, Yankun Huang, Ming Ke, Hao Li, Jingdong Li, Xiangyu Liang, Ning Liu, Yuan Liu, Ji Miao, Jiaqi Wang, Qi Wang, Wenchao Wang, Yuxuan Wang, Zhenfang Wang, Zhangyu Xiao, Chao Xue, Hongfei Xue, Fan Yu, Tianyi Zhang, Yuan Zhang, Yuqi Zhang, Lin Zhu

专题命中 推理评测 ：CoT(abstract,abstract_cn)；reasoning(abstract)

AI总结 JoyAI-Talker采用Thinker-Talker架构等技术，实现兼具强推理能力与共情交互的全双工语音对话系统，在基准测试及用户打断、背景语音场景下表现优异。

LeAct: Learning to Reason from Expert Actions

LeAct：从专家行动中学习推理

Ziran Yang, Chengshuai Shi, Raj Ghugare, Benjamin Eysenbach, Karthik Narasimhan, Chi Jin

机构 * Princeton University（普林斯顿大学） 纠错

专题命中 推理评测 ：reasoning(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 研究如何从专家行动中恢复思维链，提出LeAct方法，通过优化潜在变量，让学生为专家行动采样候选思维链并保留有效链。该方法在多个博弈和机器人基准测试中表现出色，使专家系统成为基础模型推理教师的新来源。

Comments 27 pages, 3 figures, 11 tables

Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth

使用确定性真实数据评估长文本生成中语言模型的不确定性

Ido Amit, Ido Galil, Ran El-Yaniv

机构 * Technion（技术学院） ； Nvidia（英伟达） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 研究针对长文本生成中语言模型不确定性评估难题，引入含单一确定性长文本真值的SALT基准，通过对50多个模型分析揭示关键见解，如置信函数作用、错误驱动因素及推理影响等，为风险关键应用提供参考。

Comments Accepted to the 43rd International Conference on Machine Learning (ICML 2026). Code available at https://github.com/IdoAmit198/SALT

Beyond 'One Language, One Script': Quantifying Orthographic Bias in Multilingual VLMs with PuMVR

超越“一种语言，一种文字”：用PuMVR量化多语言视觉语言模型中的正字法偏差

Prabhjot Singh, Bhushan Pawar, Madhu Reddiboina

机构 * RediMinds Inc.（RediMinds公司） ； The University of Texas at Austin（德克萨斯大学奥斯汀分校） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 提出PuMVR基准，通过旁遮普语三种文字的图像推理任务，量化多语言视觉语言模型中的文字依赖偏差，发现文字一致性率低至24.8%，视觉输入无法消除偏差。

Comments 22 pages, 4 figures. Accepted to the 4th Workshop on Cross-Cultural Considerations in NLP (C3NLP) @ ACL 2026

SCOPE: A Lightweight-training LLM Framework for Air Traffic Control Readback Monitoring

SCOPE：一种用于空中交通管制复诵监控的轻量训练LLM框架

Qihan Deng, Minghua Zhang, Yang Yang, Zhenyu Gao

机构 * Department of Mechanical and Aerospace Engineering, The Hong Kong University of Science and Technology（香港科学与技术大学机械与航空航天工程系） ； School of Electronic and Information Engineering, Beihang University（北航电子与信息工程学院） ； State Key Laboratory of CNS/ATM（国家空管自动化系统实验室） 纠错

专题命中 推理评测 ：reasoning(abstract,abstract_cn)；分类 cs.CL、cs.AI、cs.LG

AI总结 提出SCOPE框架，通过冻结LLM结合插件式开放集分类器和上下文学习机制，实现高效准确的空管复诵监控，在少样本设置下开放集检测准确率达91.05%，异常纠正率96.63%。

Eureka: Intelligent Feature Engineering for Enterprise AI Cloud Resource Demand Prediction

Eureka：面向企业AI云资源需求预测的智能特征工程

Hangxuan Li, Renjun Jia, Xuezhang Wu, Yunjie Qian, Zeqi Zheng, Xianling Zhang

机构 * Alibaba Cloud Computing Co. Ltd, Hangzhou, China（阿里云计算有限公司，杭州，中国） ； School of Computer Science, Fudan University, Shanghai, China（复旦大学计算机学院，上海，中国） ； School of Computer Science and Technology, Tongji University, Shanghai, China（同济大学计算机科学与技术学院，上海，中国） ； Independent Researcher, United States（独立研究员，美国） 纠错

专题命中 推理评测 ：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 提出Eureka框架，将特征工程视为智能体代码生成问题，通过专家代理、LLM特征工厂和自演化对齐引擎三阶段，自动生成可执行特征代码，在医疗、金融、社交等7个公开基准及阿里云GPU资源需求预测中显著提升性能。

Comments accepted at NeurIPS 2025 Workshop, DASFAA 2026 (International Conference on Database Systems for Advanced Applications)

Journal ref Database Systems for Advanced Applications (DASFAA 2026), Lecture Notes in Computer Science, vol. 16540, pp. 528-540, Springer