Breakdown of Local Denoising as Semantic Speciation
Abstract
The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.
Related
- Accelerating noisy variational quantum algorithms with physics-informed denoising networks
- Breakdown of the Wiedemann-Franz Law in an Interacting Quantum Hall Metamaterial
- Non-reciprocity and exchange-spring delay of domain-wall Walker breakdown in magnetic nanowires with azimuthal magnetization
- Bicarbonate/carbonic acid equilibria in aqueous solutions: Conformational Dynamics, Ion-Pair Speciation, and Free Energy Landscapes from Ab Initio Molecular Dynamics and Metadynamics
- Probabilistic denoising for reliable signal extraction in spectroscopy
BibTeX
@misc{liu2026breakdown,
title = {{Breakdown of Local Denoising as Semantic Speciation}},
author = {Guangkuo Liu and Mert Okyay and Yifan F. Zhang and Fangjun Hu and Rahul Nandkishore and Xun Gao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38176},
eprint = {2609.38176},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38176},
}
RIS
TY - UNPB
TI - Breakdown of Local Denoising as Semantic Speciation
AU - Guangkuo Liu
AU - Mert Okyay
AU - Yifan F. Zhang
AU - Fangjun Hu
AU - Rahul Nandkishore
AU - Xun Gao
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.38176
UR - https://arxiv.org/abs/2609.38176
AB - The dynamics of generative models exhibit two apparently distinct temporal windows: a speciation window, in which a sample commits to a semantic class, and a nonlocality window, in which local context windows become insufficient for generation. Motivated by evidence of their near-concurrence in a variety of frontier models, we investigate their relationship through the spatial distribution of semantic information. Under a "common cause" hypothesis, we prove that the nonlocality window must lie in the speciation window. This hypothesis postulates that semantic labels explain a fraction of the correlations between distant tokens, a condition that is natural for many real datasets. We further give conditions under which both windows shrink to a single limiting time as system size grows, defining a "phase transition", and verify this behavior analytically in Gaussian mixtures. Together, these results identify conditions under which semantic information explains the concurrence of speciation and nonlocality, connecting two complementary perspectives on the emergence of semantic structure in generative modeling.
ER -