大模型推理能力 - arXivDaily 专题

2505.20869 2025-05-28 cs.AI 70%

Step-Wise Formal Verification for LLM-Based Mathematical Problem Solving

Kuo Zhou, Lu Zhang

机构 * Peking University（北京大学）

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

2406.19976 2025-05-27 cs.LG math.OC 70%

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

Rui Pan, Dylan Zhang, Hanning Zhang, Xingyuan Pan, Minrui Xu, Jipeng Zhang, Renjie Pi, Xiaoyu Wang, Tong Zhang

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）； The Hong Kong University of Science and Technology（香港科学与技术大学）

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.LG

Comments ACL 2025

2504.07288 2025-04-15 cs.CL 70%

MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning

Yangning Li, Zihua Lan, Lv Qingsong, Yinghui Li, Hai-Tao Zheng

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

2504.07316 2025-04-15 cs.CL 70%

Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization

Shujin Wu, Cheng Qian, Yi R. Fung, Paul Pu Liang, Heng Ji

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

2504.02349 2025-04-04 cs.LG 70%

Large (Vision) Language Models are Unsupervised In-Context Learners

Artyom Gadetsky, Andrei Atanov, Yulun Jiang, Zhitong Gao, Ghazal Hosseini Mighan, Amir Zamir, Maria Brbic

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.LG

Comments ICLR 2025 camera-ready

2406.09044 2025-03-04 cs.CL 70%

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, Yun Chen

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments This paper has been accepted at NAACL 2025. Code is available at: https://github.com/sufenlp/MiLoRA

2502.13516 2025-02-20 cs.AI 70%

SPPD: Self-training with Process Preference Learning Using Dynamic Value Margin

Hao Yi, Qingyang Li, Yulan Hu, Fuzheng Zhang, Di Zhang, Yong Liu

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.AI

2501.16513 2025-01-31 cs.CL 70%

Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Models

Sudarshan Kamath Barkur, Sigurd Schacht, Johannes Scholl

专题命中数学推理：reasoning(abstract)；planning(abstract)；分类 cs.CL

Comments Corrected Version - Solved Some Issues with reference compilation by latex

2501.10979 2025-01-31 cs.LG 70%

Control LLM: Controlled Evolution for Intelligence Retention in LLM

Haichao Wei, Yunxiang Ren, Zhoutong Fu, Aman Lunia, Yi-Lin Chen, Alice Leung, Ya Xu

专题命中数学推理：reasoning(abstract)；CoT(abstract)；分类 cs.LG

Comments 8 pages

2310.19740 2025-01-22 cs.CL 70%

Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks

Qintong Li, Leyang Cui, Lingpeng Kong, Wei Bi

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments We release our resources at \url{https://github.com/qtli/CoEval}

2412.19361 2024-12-30 cs.CL 70%

Dynamic Skill Adaptation for Large Language Models

Jiaao Chen, Diyi Yang

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

2412.16838 2024-12-24 cs.CL 70%

Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions

Hang Li, Tianlong Xu, Kaiqi Yang, Yucheng Chu, Yanling Chen, Yichi Song, Qingsong Wen, Hui Liu

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

Comments 12 pages, 4 figures

2412.07386 2024-12-11 cs.CL 70%

Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic

Alan Sun, Ethan Sun, Warren Shepard

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments 10 pages, 5 figures

2411.01807 2024-11-05 cs.DB cs.AI 70%

Can Language Models Enable In-Context Database?

Yu Pan, Hongfeng Yu, Tianjiao Zhao, Jianxin Sun

专题命中数学推理：reasoning(abstract)；planning(abstract)；分类 cs.AI

2401.09395 2024-11-05 cs.CL 70%

Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions

Pengfei Hong, Navonil Majumder, Deepanway Ghosal, Somak Aditya, Rada Mihalcea, Soujanya Poria

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments With o1 and GPT-4o results. Reformatted the data and presented more analysis

2410.18035 2024-10-24 cs.CL 70%

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning

Jingfan Zhang, Yi Zhao, Dan Chen, Xing Tian, Huanran Zheng, Wei Zhu

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2024 Findings. arXiv admin note: substantial text overlap with arXiv:2405.18203

2410.10348 2024-10-15 cs.CL 70%

Augmenting In-Context-Learning in LLMs via Automatic Data Labeling and Refinement

Joseph Shtok, Amit Alfassy, Foad Abo Dahood, Eliyahu Schwartz, Sivan Doveh, Assaf Arbelle

专题命中数学推理：reasoning(abstract)；CoT(abstract)；分类 cs.CL

2410.04751 2024-10-08 cs.CV cs.CL 70%

Intriguing Properties of Large Language and Vision Models

Young-Jun Lee, Byungsoo Ko, Han-Gyu Kim, Yechan Hwang, Ho-Jin Choi

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments Code is available in https://github.com/passing2961/IP-LLVM

2408.02970 2024-08-07 cs.CL 70%

EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization

Zhaopeng Feng, Zijie Meng, Zuozhu Liu

专题命中数学推理：chain-of-thought(abstract)；CoT(abstract)；分类 cs.CL

2408.01779 2024-08-06 cs.CL 70%

MathLearner: A Large Language Model Agent Framework for Learning to Solve Mathematical Problems

Wenbei Xie, Donglin Liu, Haoran Yan, Wenjie Wu, Zongyang Liu

专题命中数学推理：reasoning(abstract)；chain-of-thought(abstract)；分类 cs.CL

2407.16831 2024-07-25 cs.AI 70%

Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design

Jared Quincy Davis, Boris Hanin, Lingjiao Chen, Peter Bailis, Ion Stoica, Matei Zaharia

专题命中数学推理：reasoning(abstract)；verifier(abstract)；分类 cs.AI

2402.19255 2024-07-03 cs.CL 70%

GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers

Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, Wei Bi

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments ACL 2024

2407.00390 2024-07-02 cs.CL 70%

Advancing Process Verification for Large Language Models via Tree-Based Preference Learning

Mingqian He, Yongliang Shen, Wenqi Zhang, Zeqi Tan, Weiming Lu

专题命中数学推理：reasoning(abstract)；verifier(abstract)；分类 cs.CL

2404.03820 2024-06-24 cs.CL 70%

CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues

Makesh Narsimhan Sreedhar, Traian Rebedea, Shaona Ghosh, Jiaqi Zeng, Christopher Parisien

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

2312.06585 2024-04-19 cs.LG 70%

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, Abhishek Kumar, Alex Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura Culp, Lechao Xiao, Maxwell L. Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yundi Qian, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, Noah Fiedel

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.LG

Comments Accepted to TMLR. Camera-ready version. First three authors contributed equally

2404.04817 2024-04-09 cs.CL 70%

FRACTAL: Fine-Grained Scoring from Aggregate Text Labels

Yukti Makhija, Priyanka Agrawal, Rishi Saket, Aravindan Raghuveer

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

Comments 22 pages, 1 figure

2402.13064 2024-02-21 cs.CL 70%

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, Yuxian Gu, Xin Cheng, Xun Wang, Si-Qing Chen, Li Dong, Wei Lu, Zhifang Sui, Benyou Wang, Wai Lam, Furu Wei

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Work in progress

2402.12851 2024-02-21 cs.CL 70%

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models

Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, Kang Liu

专题命中数学推理：reasoning(abstract)；math reasoning(abstract)；分类 cs.CL

2311.05821 2023-11-13 cs.CL 70%

Let's Reinforce Step by Step

Sarah Pan, Vladislav Lialin, Sherin Muckatira, Anna Rumshisky

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

2305.07004 2023-10-24 cs.CL 70%

Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting

Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, Furu Wei

专题命中数学推理：reasoning(abstract)；logical reasoning(abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 Findings

1. 数学推理 3220 篇

Step-Wise Formal Verification for LLM-Based Mathematical Problem Solving

ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting

MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuning

Alice: Proactive Learning with Teacher's Demonstrations for Weak-to-Strong Generalization

Large (Vision) Language Models are Unsupervised In-Context Learners

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

SPPD: Self-training with Process Preference Learning Using Dynamic Value Margin

Deception in LLMs: Self-Preservation and Autonomous Goals in Large Language Models

Control LLM: Controlled Evolution for Intelligence Retention in LLM

Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks

Dynamic Skill Adaptation for Large Language Models

Ask-Before-Detection: Identifying and Mitigating Conformity Bias in LLM-Powered Error Detector for Math Word Problem Solutions

Algorithmic Phase Transitions in Language Models: A Mechanistic Case Study of Arithmetic

Can Language Models Enable In-Context Database?

Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning

Augmenting In-Context-Learning in LLMs via Automatic Data Labeling and Refinement

Intriguing Properties of Large Language and Vision Models

EC-Guide: A Comprehensive E-Commerce Guide for Instruction Tuning and Quantization

MathLearner: A Large Language Model Agent Framework for Learning to Solve Mathematical Problems

Networks of Networks: Complexity Class Principles Applied to Compound AI Systems Design

GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers

Advancing Process Verification for Large Language Models via Tree-Based Preference Learning

CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

FRACTAL: Fine-Grained Scoring from Aggregate Text Labels

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models

Let's Reinforce Step by Step

Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting