SamatNext v0.2-B: An Exploratory Study of RMS-Normalized Hybrid Decoders for Curriculum Retention in Small Code Models
SamatNext v0.2-B: 针对小型代码模型中课程保留的RMS归一化混合解码器的探索性研究
专题命中 代码评测 :code model(title);分类 cs.CL、cs.LG
AI总结 提出SamatNext v0.2-B混合解码器,在课程微调中通过RMS归一化和输出缩放校准交替使用差分注意力层与简化线性状态混合器,在Python代码课程上相比Transformer基线显著提升保留率,但长期早期阶段保留仍较弱。
Comments 12 pages, 3 tables. Technical report. Code and reproducibility artifacts: https://github.com/samat2003/samatnext-v0.1/tree/samatnext-v02-lsm-rmsnorm. v2 adds an AI-assisted software development disclosure; no changes to main results