Knowledge Distillation from Large Reasoning Models to Compact Student Models: A Case Study on the John O Bryan Mathematics Competition
从大型推理模型到紧凑学生模型的知识蒸馏:以约翰·O·布莱恩数学竞赛为例
机构 * Northern Kentucky University(北肯塔基大学)
专题命中 数学推理 :reasoning(title,abstract);CoT(abstract,abstract_cn);chain-of-thought(abstract);分类 cs.AI、cs.LG
AI总结 本文研究从大型推理模型DeepSeek-R1向紧凑学生模型Qwen2.5-7B的知识蒸馏,通过构建思维链训练语料库微调,使学生在竞赛数据集上准确率提升4.76个百分点,并发现响应长度对数学推理质量至关重要。
Comments 15 pages, 3 figures, 7 tables. Code and data available at https://github.com/TempGaurab/Distillation.John-O-Bryan