Newly Important Is Not Newly Built: What Reasoning Post-Training Adds Inside a Language Model, Why the Evidence Separates Representation From Wiring From Deployment, and the Cross-Model Tests That Would Show an Attention Head Is New
- 1SONYTECH
Abstract
(c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0.
Life of Research
Reasoning models are built by post-training a pretrained model with supervised fine-tuning on teacher traces, with reinforcement learning against a verifier, or with both, and a running dispute asks whether this builds new machinery or redirects machinery the base model already had. A 2025 circuit study gave the first side a concrete form: attention heads that are absent from the base model's reasoning circuit and present after post-training, many of them after distillation and supervised fine-tuning, fewer and less stable after reinforcement learning. In this analysis the author reviews and synthesises published results to show that "new" has at least three separable meanings inside a model - a new representation, new wiring that consumes a representation, and a new policy for when existing behaviour is deployed - and that the measurements now in the record each speak to one of them. A head that enters a thresholded attribution circuit is newly important; whether it is newly built is a further question that the comparison as run cannot answer, because the reference model differs from the distilled model's actual parent, the circuit is defined by contrasting the two models' own response styles, and zero-ablation effects in the same study do not separate emergent heads from base-circuit heads or from heads taken from an unrelated task. The strongest cross-model evidence points to a regime split at the level of deployment: base-model steering vectors directed by the thinking model's own gating recover about 76% of the accuracy gap for reinforcement-learned models and about 11% for distilled ones, and a gap-matched pair computed from that study's table preserves the split. At the level of representation, a direction taken from the base model drives backtracking in a distilled model but not in the base. The paper consolidates these values in two tables and a figure, gives a classification procedure, and names five cross-model tests that would show whether a post-training head is new.
Related
- Normalization Couples What the Objective Separates: Representation Selection in Group Composition Networks
- Feature-aware attention convolutional neural network for efficient pulmonary fibrosis classification from chest CT images with FPGA deployment
- Dynamic heterogeneous graph attention network for spatiotemporal correlation modeling of mine microseismic events and intelligent rockburst risk level reasoning
- EDGRU: edge-guided dual graph reasoning U-Net with spatial graph attention for infrared small target detection
- Reliability gated cross-modal attention with shear reparameterized directional reasoning for RGB-thermal salient object detection
BibTeX
@misc{mahendrakar2026newly,
title = {{Newly Important Is Not Newly Built: What Reasoning Post-Training Adds Inside a Language Model, Why the Evidence Separates Representation From Wiring From Deployment, and the Cross-Model Tests That Would Show an Attention Head Is New}},
author = {Pranay Mahendrakar},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23536189},
url = {https://zenodo.org/records/23536189},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Newly Important Is Not Newly Built: What Reasoning Post-Training Adds Inside a Language Model, Why the Evidence Separates Representation From Wiring From Deployment, and the Cross-Model Tests That Would Show an Attention Head Is New
AU - Pranay Mahendrakar
PY - 2026
DA - 2026/10/21
PB - Zenodo
DO - 10.5281/zenodo.23536189
UR - https://zenodo.org/records/23536189
AB - (c) 2026 Pranay Mahendrakar. Licensed under CC BY 4.0. Life of Research Reasoning models are built by post-training a pretrained model with supervised fine-tuning on teacher traces, with reinforcement learning against a verifier, or with both, and a running dispute asks whether this builds new machinery or redirects machinery the base model already had. A 2025 circuit study gave the first side a concrete form: attention heads that are absent from the base model's reasoning circuit and present after post-training, many of them after distillation and supervised fine-tuning, fewer and less stable after reinforcement learning. In this analysis the author reviews and synthesises published results to show that "new" has at least three separable meanings inside a model - a new representation, new wiring that consumes a representation, and a new policy for when existing behaviour is deployed - and that the measurements now in the record each speak to one of them. A head that enters a thresholded attribution circuit is newly important; whether it is newly built is a further question that the comparison as run cannot answer, because the reference model differs from the distilled model's actual parent, the circuit is defined by contrasting the two models' own response styles, and zero-ablation effects in the same study do not separate emergent heads from base-circuit heads or from heads taken from an unrelated task. The strongest cross-model evidence points to a regime split at the level of deployment: base-model steering vectors directed by the thinking model's own gating recover about 76% of the accuracy gap for reinforcement-learned models and about 11% for distilled ones, and a gap-matched pair computed from that study's table preserves the split. At the level of representation, a direction taken from the base model drives backtracking in a distilled model but not in the base. The paper consolidates these values in two tables and a figure, gives a classification procedure, and names five cross-model tests that would show whether a post-training head is new.
ER -