大模型推理能力 - arXivDaily 专题

2305.15054 2023-10-23 cs.CL cs.LG 81%

A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis

Alessandro Stolfo, Yonatan Belinkov, Mrinmaya Sachan

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments EMNLP 2023. 18 pages, 19 figures

2305.14628 2023-10-23 cs.CL cs.AI 81%

Getting MoRE out of Mixture of Language Model Reasoning Experts

Chenglei Si, Weijia Shi, Chen Zhao, Luke Zettlemoyer, Jordan Boyd-Graber

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023 Findings

2310.12426 2023-10-20 cs.CL cs.AI 81%

MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models

Deepak Nathani, David Wang, Liangming Pan, William Yang Wang

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at EMNLP 2023 Main Conference, Camera Ready

2305.12477 2023-09-21 cs.CL cs.AI 81%

GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts

Jessica López Espejel, El Hassane Ettifouri, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, Walid Dahhane

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted for publication at Elsevier's Natural Language Processing Journal

2307.07951 2023-07-18 cs.AI cs.CL 81%

MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning

Zhenwen Liang, Dian Yu, Xiaoman Pan, Wenlin Yao, Qingkai Zeng, Xiangliang Zhang, Dong Yu

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

2210.12023 2023-06-09 cs.CL cs.LG 81%

A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models

Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, Bernhard Schölkopf, Mrinmaya Sachan

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.LG

Comments ACL 2023. A shorter version of the paper was accepted at the MATH-AI Workshop at NeurIPS 2022. 15 pages, 8 figures

2306.00784 2023-06-02 cs.CL cs.AI 81%

Interpretable Math Word Problem Solution Generation Via Step-by-step Planning

Mengxue Zhang, Zichao Wang, Zhichao Yang, Weiqi Feng, Andrew Lan

专题命中数学推理：planning(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to The 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)

2211.12668 2022-11-24 cs.CL cs.AI cs.IR 81%

DyRRen: A Dynamic Retriever-Reranker-Generator Model for Numerical Reasoning over Tabular and Textual Data

Xiao Li, Yin Zhu, Sichen Liu, Jiangzhou Ju, Yuzhong Qu, Gong Cheng

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL、cs.AI

Comments 9 pages, accepted by AAAI 2023

2101.06223 2022-03-17 cs.LG cs.AI cs.LO 81%

LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning

Yuhuai Wu, Markus Rabe, Wenda Li, Jimmy Ba, Roger Grosse, Christian Szegedy

专题命中数学推理：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Published as a conference paper at ICML 2021 (16 pages)

2103.12329 2021-03-24 cs.LG cs.AI cs.CV 81%

Contrastive Reasoning in Neural Networks

Mohit Prabhushankar, Ghassan AlRegib

专题命中数学推理：reasoning(title,abstract)；分类 cs.AI、cs.LG

Comments Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence

2509.26306 2026-07-07 cs.AI 版本更新 80%

Interactive Learning for LLM Reasoning

为LLM推理设计的交互学习

Hehai Lin, Shilei Cao, Sudong Wang, Haotian Wu, Minzhi Li, Linyi Yang, Juepeng Zheng, Chengwei Qin

机构 * The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））； Sun Yat-sen University（中山大学）； Southern University of Science and Technology（南方科技大学）； National University of Singapore（新加坡国立大学）

专题命中数学推理：reasoning(title,abstract)；分类 cs.AI

AI总结本文提出ILR框架，通过动态交互和感知校准提升LLM独立问题解决能力，实验表明其在多个基准测试中优于单agent学习。

Comments The code is available at https://github.com/linhh29/Interactive-Learning-for-LLM-Reasoning

2603.21558 2026-06-01 cs.AI 80%

Reliable Self-Improvement Training by Verifying Reasoning, Not Just Answers

可靠的自改进训练：验证推理过程，而不仅仅是答案

Xinyu Zhang

机构 * Anyscale

专题命中数学推理：reasoning(title,abstract)；分类 cs.AI

AI总结针对自改进训练中因依赖最终答案正确性导致推理错误累积的问题，提出VSI框架，通过步骤级结构验证（如符号计算检查算术步骤）筛选训练数据，在GSM8K上实现持续准确率提升（80.5%→91.0%）。

Comments Accepted at ICLR 2026 Workshop LLM Reasoning. 10 pages, 3 figures, 5 tables

2511.15694 2025-11-20 cs.LG 80%

The Impact of Quantization on Large Reasoning Model Reinforcement Learning

量化对大推理模型强化学习的影响

Medha Kumar, Zifei Xu, Xin Wang, Tristan Webb

机构 * Pennsylvania State University（宾夕法尼亚州立大学）； d-Matrix

专题命中数学推理：reasoning(title,abstract)；分类 cs.LG

AI总结本文研究了量化对大推理模型强化学习的影响，发现量化感知训练对学习过程有负面影响，而PTQ和QLoRA能提升性能。

Comments Accepted to the NeurIPS 2025 Efficient Reasoning Workshop

2510.01265 2026-03-03 cs.LG cs.AI cs.CL 80%

RLP: Reinforcement as a Pretraining Objective

RLP：将强化学习作为预训练目标

Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi

机构 * NVIDIA（英伟达）； Carnegie Mellon University（卡内基梅隆大学）； Boston University（波士顿大学）； Stanford University（斯坦福大学）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；verifier(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 RLP通过将强化学习的探索精神引入预训练阶段，提升模型在数学和科学任务中的推理能力。

Comments ICLR 2026 camera ready

2601.06052 2026-01-22 cs.CL cs.AI cs.LG 80%

Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization

基于单领域到全领域泛化的强化学习进行思维链压缩

Hanyu Li, Jiangshan Duo, Bofei Gao, Hailin Zhang, Sujian Li, Xiaotie Deng, Liang Zhao

机构 * CFCS, School of Computer Science, Peking University（计算机学院，北京大学）； State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）； LLM-Core Xiaomi（小米LLM-Core）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结本文提出了一种基于单领域到全领域泛化的强化学习方法，通过样本级软强化学习压缩思维链推理，有效减少响应长度并提升跨领域泛化能力。

2509.23232 2026-01-13 cs.LG cs.AI cs.CL 80%

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts

SPEC-RL: 通过推测性回放加速在线策略学习

Bingshuai Liu, Ante Wang, Zijun Min, Liang Yao, Haibo Zhang, Yang Liu, Xu Han, Peng Li, Anxiang Zeng, Jinsong Su

机构 * School of Informatics, Xiamen University（厦门大学信息学院）； LLM Team, Shopee Pte. Ltd.（Shopee公司语言模型团队）； Tsinghua University（清华大学）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；math reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

AI总结 SPEC-RL通过整合推测解码技术，有效减少强化学习回放阶段的计算开销，提升大模型推理能力。

Comments fixed typos

2510.09714 2025-10-17 cs.CL cs.AI cs.LG 80%

All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Language

Shiyuan Guo, Henry Sleight, Fabien Roger

机构 * Anthropic Fellows Program（Anthropic Fellow项目）； Constellation ； Anthropic

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Version 2: updated related works section on LLM steganography

2509.08814 2025-10-17 cs.LG cs.AI cs.CL 80%

Merge-of-Thought Distillation

Zhanming Shen, Zeyu Qin, Zenan Huang, Hao Chen, Jiaqi Hu, Yihong Zhuang, Guoshan Lu, Gang Chen, Junbo Zhao

机构 * Zhejiang University（浙江大学）； Inclusion AI, Ant Group（蚂蚁集团 inclusion AI）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

2509.25267 2025-10-01 cs.LG cs.AI cs.CL 80%

Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learning

Jiexi Xu

机构 * University of California, Irvine School of Information & Computer Science（加州大学尔湾分校信息与计算机科学学院）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 13 pages, 2 figures, 2 tables

2503.18892 2025-08-07 cs.LG cs.AI cs.CL 80%

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild

Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, Junxian He

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published as a conference paper at COLM 2025

2502.00691 2025-07-21 cs.AI cs.CL cs.LG 80%

To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximization

Haozhe Wang, Long Li, Chao Qu, Fengming Zhu, Weidi Xu, Wei Chu, Fangzhen Lin

机构 * Hong Kong University of Science and Technology（香港科技大学）； INFLY Tech（INFLY科技）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2025

2505.22312 2025-05-30 cs.LG cs.AI cs.CL 80%

Skywork Open Reasoner 1 Technical Report

Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, Yahui Zhou

机构 * Skywork Open Reasoner 1

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

2505.14852 2025-05-22 cs.CL cs.AI cs.LG 80%

EasyMath: A 0-shot Math Benchmark for SLMs

Drishya Karki, Michiel Kamphuis, Angelecia Frey

机构 * Assistantslab（助手实验室）

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；math reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages, 9 figures, 8 tables

2501.04040 2025-02-11 cs.CL cs.AI cs.LG cs.NE 80%

A Survey on Large Language Models with some Insights on their Capabilities and Limitations

Andrea Matarazzo, Riccardo Torlone

专题命中数学推理：reasoning(abstract)；CoT(abstract)；planning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 174 pages, to be submitted to a journal in a shorter version. It includes figures taken from papers by other authors. All the sources have been referenced. arXiv admin note: text overlap with arXiv:2303.18223 by other authors

2306.13549 2024-12-02 cs.CV cs.AI cs.CL cs.LG 80%

A Survey on Multimodal Large Language Models

Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, Enhong Chen

专题命中数学推理：reasoning(abstract)；CoT(abstract)；math reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted for publication in National Science Review. Project page:https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models

2409.12122 2024-09-19 cs.CL cs.AI cs.LG 80%

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement

An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, Zhenru Zhang

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

2403.14624 2024-08-20 cs.CV cs.AI cs.CL cs.LG 80%

MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, Hongsheng Li

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by ECCV 2024, 46 Pages, Benchmark Project Page: https://mathverse-cuhk.github.io

2402.06457 2024-08-15 cs.LG cs.AI cs.CL 80%

V-STaR: Training Verifiers for Self-Taught Reasoners

Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, Rishabh Agarwal

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；verifier(abstract)；分类 cs.CL、cs.AI、cs.LG

2608.16554 2026-08-18 cs.CL 新提交 79%

Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

询问、条件化或弃权：针对缺失前提推理的强化学习

Yongqi Tong, Zhenyu Zhang, Zimi Liu, Kewei Fu, Mingli Song, Haofei Zhang, Junshao Zhang, Hong Zhu, Jiang-Ming Yang, Xin Zhang, Jianshe Li

机构 * Ant International（蚂蚁国际）； Zhejiang University（浙江大学）； Dingtalk, Alibaba Group（阿里巴巴集团钉钉）； Ant Group（蚂蚁集团）

专题命中数学推理：reasoning(title,abstract)；分类 cs.CL

AI总结本研究提出ACA-RL框架，结合MPB基准，训练模型应对缺失前提的推理任务，可询问、条件化或弃权，提升欠定问题处理能力。

2608.15445 2026-08-18 cs.AI 新提交 79%

Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization

测量位置混淆优化下的奖励黑客行为与推理-答案解耦

Suyash Maniyar, Armaan Sandhu, Abhishek Mishra

机构 * University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）

专题命中数学推理：reasoning(title,abstract)；分类 cs.AI

AI总结该研究针对位置混淆优化下的奖励黑客与推理-答案解耦问题，通过GRPO训练语言模型，发现模型会学习答案位置捷径，推理与答案解耦，且该捷径可跨域迁移，解耦率等指标可区分能力损失与捷径。

Comments Accepted at the AI Measurement Science Workshop, COLM 2026

1. 数学推理 3218 篇

A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis

Getting MoRE out of Mixture of Language Model Reasoning Experts

MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models

GPT-3.5, GPT-4, or BARD? Evaluating LLMs Reasoning Ability in Zero-Shot Setting and Performance Boosting Through Prompts

MinT: Boosting Generalization in Mathematical Reasoning via Multi-View Fine-Tuning

A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models

Interpretable Math Word Problem Solution Generation Via Step-by-step Planning

DyRRen: A Dynamic Retriever-Reranker-Generator Model for Numerical Reasoning over Tabular and Textual Data

LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning

Contrastive Reasoning in Neural Networks

Interactive Learning for LLM Reasoning

Reliable Self-Improvement Training by Verifying Reasoning, Not Just Answers

The Impact of Quantization on Large Reasoning Model Reinforcement Learning

RLP: Reinforcement as a Pretraining Objective

Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization

SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts

All Code, No Thought: Current Language Models Struggle to Reason in Ciphered Language

Merge-of-Thought Distillation

Dynamic Policy Induction for Adaptive Prompt Optimization: Bridging the Efficiency-Accuracy Gap via Lightweight Reinforcement Learning

SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild

To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximization

Skywork Open Reasoner 1 Technical Report

EasyMath: A 0-shot Math Benchmark for SLMs

A Survey on Large Language Models with some Insights on their Capabilities and Limitations

A Survey on Multimodal Large Language Models

Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement

MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?

V-STaR: Training Verifiers for Self-Taught Reasoners

Ask, Condition or Abstain: Reinforcement Learning for Missing-Premise Reasoning

Measuring Reward Hacking and Reasoning-Answer Decoupling Under Position-Confounded Optimization