Propagation of Chaos in Contextual Flow Maps
Abstract
We develop a quantitative statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps (CFMs): dynamical systems that evolve a distinguished token in the presence of a contextual measure across a stack of attention blocks. Within this framework, the finite-context model approximates an idealized infinite-context system in which the contextual measure is replaced by its underlying population, so that the context length n becomes a statistical resource. Exploiting the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos, we establish a forward bound controlling the deviation between the finite- and infinite-context CFMs uniformly along depth, and a backward bound controlling the deviation between the corresponding training trajectories uniformly across iterations of online gradient descent. Both bounds achieve the optimal Wasserstein rate n−1/d for general CFMs and parametric rate n−1/2 for a restricted class of CFMs that includes transformers as a special case. The analysis rests on a new Eulerian adjoint formulation of the loss gradient and stability estimates for the resulting forward--adjoint system, both of which may be of independent interest.
Related
- Uniform in time propagation of chaos for noisy mean-field coupled maps
- Quantitative Propagation of Chaos for Lindblad Dynamics
- On the uniform-in-time propagation of chaos for the Motsch-Tadmor model
- Mean-field branching SDEs: propagation of chaos, scaling limits and phase transitions
- Quantitative propagation of chaos for particle systems with memory and their long-time behaviour
BibTeX
@misc{chen2026propagation,
title = {{Propagation of Chaos in Contextual Flow Maps}},
author = {Shi Chen and Zhengjiang Lin and Kaizhao Liu and Philippe Rigollet},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.16747},
eprint = {2605.16747},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.16747},
}
RIS
TY - UNPB
TI - Propagation of Chaos in Contextual Flow Maps
AU - Shi Chen
AU - Zhengjiang Lin
AU - Kaizhao Liu
AU - Philippe Rigollet
PY - 2026
DA - 2026/10/30
PB - arXiv
DO - 10.48550/arxiv.2605.16747
UR - https://arxiv.org/abs/2605.16747
AB - We develop a quantitative statistical theory of transformers in the large-context regime by adopting the abstraction of contextual flow maps (CFMs): dynamical systems that evolve a distinguished token in the presence of a contextual measure across a stack of attention blocks. Within this framework, the finite-context model approximates an idealized infinite-context system in which the contextual measure is replaced by its underlying population, so that the context length n becomes a statistical resource. Exploiting the McKean--Vlasov structure of the dynamics and the classical machinery of propagation of chaos, we establish a forward bound controlling the deviation between the finite- and infinite-context CFMs uniformly along depth, and a backward bound controlling the deviation between the corresponding training trajectories uniformly across iterations of online gradient descent. Both bounds achieve the optimal Wasserstein rate n^−1/d for general CFMs and parametric rate n^−1/2 for a restricted class of CFMs that includes transformers as a special case. The analysis rests on a new Eulerian adjoint formulation of the loss gradient and stability estimates for the resulting forward--adjoint system, both of which may be of independent interest.
ER -