大模型推理能力

Language Models Use Trigonometry to Do Addition

Subhash Kantamneni, Max Tegmark

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

LLMs can see and hear without any training

Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen, Ishan Misra, Rohit Girdhar

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Code: https://github.com/facebookresearch/MILS

Evolutionary Optimization of Model Merging Recipes

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, David Ha

专题命中 数学推理 ：reasoning(abstract)；math reasoning(abstract)

Comments Authors' submitted version before final edits. Published in Nature Machine Intelligence on January 27, 2025: https://www.nature.com/articles/s42256-024-00975-8

Journal ref Nat Mach Intell (2025)

Stream Aligner: Efficient Sentence-Level Alignment via Distribution Induction

Hantao Lou, Jiaming Ji, Kaile Wang, Yaodong Yang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments AAAI Alignment Track 2025 Poster

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection

Xutao Liao, Shaohui Li, Yuhui Xu, Zhi Li, Yu Liu, You He

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Channel Merging: Preserving Specialization for Merged Experts

Mingyang Zhang, Jing Liu, Ganggui Ding, Xinyi Yu, Linlin Ou, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments accepted by AAAI 2025

Augmenting Math Word Problems via Iterative Question Composing

Haoxiong Liu, Yifan Zhang, Yifan Luo, Andrew Chi-Chih Yao

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, Chuang Gan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024

Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads

Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Joanna Matthiesen, Kevin Smith, Joshua B. Tenenbaum

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted at NeurIPS 2024 (Datasets and Benchmarks Track)

A Careful Examination of Large Language Model Performance on Grade School Arithmetic

Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Charlotte Zhuang, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele Lunati, Summer Yue

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 2024 NeurIPS Camera Ready (Datasets and Benchmarks Track)

Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?

Kuei-Chun Kao, Ruochen Wang, Cho-Jui Hsieh

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Journal ref https://aclanthology.org/2024.findings-emnlp.980/

3-in-1: 2D Rotary Adaptation for Efficient Finetuning, Efficient Batching and Composability

Baohao Liao, Christof Monz

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to NeurIPS 2024. Code: https://github.com/BaohaoLiao/road

Implicit Personalization in Language Models: A Systematic Study

Zhijing Jin, Nils Heil, Jiarui Liu, Shehzaad Dhuliawala, Yahang Qi, Bernhard Schölkopf, Rada Mihalcea, Mrinmaya Sachan

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024 Findings

MathPile: A Billion-Token-Scale Pretraining Corpus for Math

Zengzhi Wang, Xuefeng Li, Rui Xia, Pengfei Liu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 43 pages. Accepted by NeurIPS 2024. https://github.com/GAIR-NLP/MathPile/

ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models

Jing Liu, Ruihao Gong, Mingyang Zhang, Yefei He, Jianfei Cai, Bohan Zhuang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Tech report

NVLM: Open Frontier-Class Multimodal LLMs

Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Fixed the typos. For more information, please visit our project page at: https://research.nvidia.com/labs/adlr/NVLM-1

SBoRA: Low-Rank Adaptation with Regional Weight Updates

Lai-Man Po, Yuyang Liu, Haoxuan Wu, Tianqi Zhang, Wing-Yin Yu, Zhuohan Wang, Zeyu Jiang, Kun Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 16 pages, 4 figures

Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together

Dilara Soylu, Christopher Potts, Omar Khattab

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments EMNLP 2024

Revisiting the Superficial Alignment Hypothesis

Mohit Raghavendra, Vaskar Nath, Sean Hendryx

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

Ruoyu Wang, Xiaoxuan Li, Lina Yao

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

The Perfect Blend: Redefining RLHF with Mixture of Judges

Tengyu Xu, Eryk Helenowski, Karthik Abinav Sankararaman, Di Jin, Kaiyan Peng, Eric Han, Shaoliang Nie, Chen Zhu, Hejia Zhang, Wenxuan Zhou, Zhouhao Zeng, Yun He, Karishma Mandyam, Arya Talabzadeh, Madian Khabsa, Gabriel Cohen, Yuandong Tian, Hao Ma, Sinong Wang, Han Fang

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments submitted to conference

ControlMath: Controllable Data Generation Promotes Math Generalist Models

Nuo Chen, Ning Wu, Jianhui Chang, Jia Li

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages

Leveraging Large Language Models for Solving Rare MIP Challenges

Teng Wang, Wing-Yin Yu, Ruifeng She, Wenhan Yang, Taijie Chen, Jianping Zhang

专题命中 数学推理 ：chain-of-thought(abstract)；分类 cs.CL、cs.AI、cs.LG

OccamLLM: Fast and Exact Language Model Arithmetic in a Single Step

Owen Dugan, Donato Manuel Jimenez Beneto, Charlotte Loh, Zhuo Chen, Rumen Dangovski, Marin Soljačić

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems

Tian Ye, Zicheng Xu, Yuanzhi Li, Zeyuan Allen-Zhu

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments arXiv admin note: text overlap with arXiv:2407.20311

ReGAL: Refactoring Programs to Discover Generalizable Abstractions

Elias Stengel-Eskin, Archiki Prasad, Mohit Bansal

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments ICML 2024 Camera-Ready; First two authors contributed equally; Code: https://github.com/esteng/regal_program_learning

Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks

Andrew Gambardella, Yusuke Iwasawa, Yutaka Matsuo

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

ReFT: Representation Finetuning for Language Models

Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, Christopher Potts

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments preprint

Prompt Selection and Augmentation for Few Examples Code Generation in Large Language Model and its Application in Robotics Control

On Tai Wu, Frodo Kin Sun Chan, Zunhao Zhang, Yan Nei Law, Benny Drescher, Edmond Shiao Bun Lai

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments 17 pages, 4 figures

CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets

Lifan Yuan, Yangyi Chen, Xingyao Wang, Yi R. Fung, Hao Peng, Heng Ji

专题命中 数学推理 ：reasoning(abstract)；分类 cs.CL、cs.AI、cs.LG

Comments Accepted to ICLR 2024. Code is available at https://github.com/lifan-yuan/CRAFT