大模型推理能力

Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models

Jonas Zausinger, Lars Pennig, Anamarija Kozina, Sean Sdahl, Julian Sikora, Adrian Dendorfer, Timofey Kuznetsov, Mohamad Hagog, Nina Wiedemann, Kacper Chlodny, Vincent Limbach, Anna Ketteler, Thorben Prein, Vishwa Mohan Singh, Michael Morris Danziger, Jannis Born

机构 * Technical University of Munich（慕尼黑技术大学） ； Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML）） ； IBM Research Europe（IBM欧洲研究院） ； ETH Zurich（苏黎世联邦理工学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2025

COPO: Consistency-Aware Policy Optimization

Jinghang Han, Jiawei Chen, Hang Shao, Hao Ma, Mingcheng Li, Xintian Shen, Lihao Zheng, Wei Chen, Tao Wei, Lihua Zhang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

A Comparative Study of Specialized LLMs as Dense Retrievers

Hengran Zhang, Keping Bi, Jiafeng Guo

机构 * Key Laboratory of Network Data Science and Technology（网络数据科学与技术重点实验室） ； Institute of Computing Technology（计算技术研究所） ； Chinese Academy of Sciences（中国科学院） ； State Key Laboratory of AI Safety（人工智能安全国家重点实验室） ； University of Chinese Academy of Science（中国科学院大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by CCIR25 and published by Springer LNCS or LNAI

LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation

Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein

机构 * University of Maryland（马里兰大学） ； Tsinghua University（清华大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments COLM 2025

Self-Evolving Critique Abilities in Large Language Models

Zhengyang Tang, Ziniu Li, Zhenyang Xiao, Tian Ding, Ruoyu Sun, Benyou Wang, Dayiheng Liu, Fei Huang, Tianyu Liu, Bowen Yu, Junyang Lin

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； Shenzhen Research Institute of Big Data（深圳大数据研究院） ； Shenzhen International Center for Industrial and Applied Mathematics（深圳国际工业与应用数学中心） ； Qwen Team, Alibaba Inc.（阿里云通义团队） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted by COLM 2025

The Policy Cliff: A Theoretical Analysis of Reward-Policy Maps in Large Language Models

Xingcheng Xu

机构 * Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts

Yifan Zhang, Yifan Luo, Yang Yuan, Andrew C Yao

机构 * Tsinghua University（清华大学） ； Shanghai Qi Zhi Institute（上海启智研究院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Published in ACL 2025 Findings

Executable Functional Abstractions: Inferring Generative Programs for Advanced Math Problems

Zaid Khan, Elias Stengel-Eskin, Archiki Prasad, Jaemin Cho, Mohit Bansal

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Project Page: https://zaidkhan.me/EFAGen/

Integrating External Tools with Large Language Models to Improve Accuracy

Nripesh Niketan, Hadj Batatia

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 9 pages, 3 figures, 2 tables. Extended version of paper published in Proceedings of International Conference on Information Technology and Applications, Springer Nature Singapore, 2025, pp. 409-421. This version includes additional experimental results comparing against GPT-4o, LLaMA-Large, Mistral-Large, and Phi-Large, expanded evaluation methodology, and enhanced analysis

Journal ref Proceedings of International Conference on Information Technology and Applications, Springer Nature Singapore, 2025, pp. 409-421

StepHint: Multi-level Stepwise Hints Enhance Reinforcement Learning to Reason

Kaiyi Zhang, Ang Lv, Jinpeng Li, Yongbo Wang, Feng Wang, Haoyuan Hu, Rui Yan

机构 * GSAI, Renmin University of China（清华大学） ； Peking University（北京大学） ； Ant Group（蚂蚁集团） ； SCS, Wuhan University（武汉大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Quantifying artificial intelligence through algorithmic generalization

Takuya Ito, Murray Campbell, Lior Horesh, Tim Klinger, Parikshit Ram

机构 * Mathematics & Theoretical Computer Science Department, T.J. Watson Research Center, IBM Research（数学与理论计算机科学系，T.J.沃森研究中心，IBM研究） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

RePO: Replay-Enhanced Policy Optimization

Siheng Li, Zhanhui Zhou, Wai Lam, Chao Yang, Chaochao Lu

机构 * The Chinese University of Hong Kong（香港中文大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Project Page: https://github.com/SihengLi99/RePO

Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit

Charles Goddard, Fernando Fernandes Neto

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

A Complexity-Based Theory of Compositionality

Eric Elmoznino, Thomas Jiralerspong, Yoshua Bengio, Guillaume Lajoie

机构 * Mila – Quebec AI Institute（魁北克人工智能研究所） ； Université de Montréal（蒙特利尔大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning

Ziyu Wan, Yunxiang Li, Xiaoyu Wen, Yan Song, Hanjing Wang, Linyi Yang, Mark Schmidt, Jun Wang, Weinan Zhang, Shuyue Hu, Ying Wen

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； University of British Columbia（不列颠哥伦比亚大学） ； University College London（伦敦大学学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, Bing Liu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICLR 2025, Spotlight Paper, In The Thirteenth International Conference on Learning Representations, 2025

Journal ref The Thirteenth International Conference on Learning Representations (2025)

TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluation

Jialin Ouyang

机构 * Columbia University（哥伦比亚大学） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ACL 2025 Main Conference

InductionBench: LLMs Fail in the Simplest Complexity Class

Wenyue Hua, Tyler Wong, Sun Fei, Liangming Pan, Adam Jardine, William Yang Wang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 25 pages, 10 figures, more details including examples and prompts are added

DeepCritic: Deliberate Critique with Large Language Models

Wenkai Yang, Jingwen Chen, Yankai Lin, Ji-Rong Wen

机构 * Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学首都人工智能学院） ； School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Work in progress. Data and models are available at https://github.com/RUCBM/DeepCritic

Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models

Michael Noukhovitch, Shengyi Huang, Sophie Xhonneux, Arian Hosseini, Rishabh Agarwal, Aaron Courville

机构 * Mila Quebec AI Institute（魁北克AI研究所） ； Université de Montréal（蒙特利尔大学） ； Allen Institute for AI（人工智能研究院） ； Google Deepmind（谷歌DeepMind） ； Canada CIFAR AI Chair（加拿大CIFAR人工智能主席） 纠错

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted at ICLR 2025, code at https://github.com/mnoukhov/async_rlhf, integrated into the open-instruct library https://github.com/allenai/open-instruct

A Dual-Space Framework for General Knowledge Distillation of Large Language Models

Xue Zhang, Songming Zhang, Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 19 pages, 9 figures, 11 tables, under review. Code is available at: https://github.com/songmzhang/DSKDv2. arXiv admin note: text overlap with arXiv:2406.17328

MegaMath: Pushing the Limits of Open Math Corpora

Fan Zhou, Zengzhi Wang, Nikhil Ranjan, Zhoujun Cheng, Liping Tang, Guowei He, Zhengzhong Liu, Eric P. Xing

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 26 pages, 15 figures, 22 tables

Teaching LLMs for Step-Level Automatic Math Correction via Reinforcement Learning

Junsong Li, Jie Zhou, Yutao Yang, Bihao Zhan, Qianjun Pan, Yuyang Ding, Qin Chen, Jiang Bo, Xin Lin, Liang He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

AgentRxiv: Towards Collaborative Autonomous Research

Samuel Schmidgall, Michael Moor

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, Tieniu Tan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Camera-Ready Version for ICLR 2025; technical corrections to previous version

xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference

Maximilian Beck, Korbinian Pöppel, Phillip Lippe, Richard Kurle, Patrick M. Blies, Günter Klambauer, Sebastian Böck, Sepp Hochreiter

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code available at: https://github.com/NX-AI/xlstm and https://github.com/NX-AI/xlstm-jax

Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi-ling Chen, Qi Dai, Xiyang Dai, Ruchao Fan, Mei Gao, Min Gao, Amit Garg, Abhishek Goswami, Junheng Hao, Amr Hendy, Yuxuan Hu, Xin Jin, Mahmoud Khademi, Dongwoo Kim, Young Jin Kim, Gina Lee, Jinyu Li, Yunsheng Li, Chen Liang, Xihui Lin, Zeqi Lin, Mengchen Liu, Yang Liu, Gilsinia Lopez, Chong Luo, Piyush Madan, Vadim Mazalov, Arindam Mitra, Ali Mousavi, Anh Nguyen, Jing Pan, Daniel Perez-Becker, Jacob Platin, Thomas Portet, Kai Qiu, Bo Ren, Liliang Ren, Sambuddha Roy, Ning Shang, Yelong Shen, Saksham Singhal, Subhojit Som, Xia Song, Tetyana Sych, Praneetha Vaddamanu, Shuohang Wang, Yiming Wang, Zhenghao Wang, Haibin Wu, Haoran Xu, Weijian Xu, Yifan Yang, Ziyi Yang, Donghan Yu, Ishmam Zabir, Jianwen Zhang, Li Lyna Zhang, Yunan Zhang, Xiren Zhou

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 39 pages

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Y. X. Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve?

Zhenheng Tang, Xiang Liu, Qian Wang, Peijie Dong, Bingsheng He, Xiaowen Chu, Bo Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs

Hao Sun, Yunyi Shen, Jean-Francois Ton, Mihaela van der Schaar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG