States Hidden in Hidden States: Implicit Discrete State Representations Emerge in LLMs' Hidden States
隐藏状态中隐藏的状态:大型语言模型的隐藏状态中出现隐式离散状态表示
专题命中 推理评测 :chain-of-thought(abstract);分类 cs.CL
AI总结 本研究探索LLMs的符号计算能力,发现其隐藏状态中存在隐式离散状态表示(IDSRs),最强模型可直接计算最多15个加数的求和,但当前开源模型的IDSRs存在损耗会导致输出错误。
Comments 12 pages, 12 figures. Revised manuscript with public code and reproducibility artifacts; clarified the evaluation protocol; corrected figure labels and chance baselines, the attention-bridge description, cross-references, and probe notation. No new experiments