Generalization of Transformer-Based Neural Quantum States via In-Context Learning
Abstract
Neural quantum states based on modern deep learning architectures have emerged as powerful representations for quantum many-body systems. In particular, Transformer-based neural quantum states provide expressive models capable of capturing long-range correlations, and their empirical generalization performance has recently been demonstrated. However, a theoretical understanding of their generalization behavior remains largely unexplored. In this paper, we develop a theoretical framework to analyze the generalization properties of Transformer-based neural quantum states under in-context learning. We establish a rigorous inference-time generalization error bound in terms of mean squared error (MSE), showing that the pointwise prediction error decreases inversely with both the number of in-context examples and the depth of the Transformer. We further show that the Transformer depth required to achieve this guarantee scales only linearly with the system size--namely, the number of particles in continuous systems or the number of qudits in discrete systems. Building on this result, we extend our analysis to full quantum states formulated as rank-one density operators, and derive MSE-based generalization bounds over both continuous and discrete domains under physical constraints. Finally, numerical simulations corroborate our theoretical analysis.
Related
- Enhancing generalization in endwall film cooling prediction: Incorporating the superposition principle into transformer-based neural operators
- An all-transistor physical-learning transformer can be trained to perform in-context learning
- Variational study of the magnetization plateaus in the spin-1/2 kagome Heisenberg antiferromagnet: An approach from vision transformer neural quantum states
- Margin-Based Generalization for Quantum Machine Learning with Classical Data Encoding
- Experimental evidence of generalization in quantum machine learning in small-data regime
BibTeX
@misc{qin2026generalization,
title = {{Generalization of Transformer-Based Neural Quantum States via In-Context Learning}},
author = {Zhen Qin and Qing Qu and Alfred O. Hero III},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.03463},
eprint = {2610.03463},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.03463},
}
RIS
TY - UNPB
TI - Generalization of Transformer-Based Neural Quantum States via In-Context Learning
AU - Zhen Qin
AU - Qing Qu
AU - Alfred O. Hero III
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2610.03463
UR - https://arxiv.org/abs/2610.03463
AB - Neural quantum states based on modern deep learning architectures have emerged as powerful representations for quantum many-body systems. In particular, Transformer-based neural quantum states provide expressive models capable of capturing long-range correlations, and their empirical generalization performance has recently been demonstrated. However, a theoretical understanding of their generalization behavior remains largely unexplored. In this paper, we develop a theoretical framework to analyze the generalization properties of Transformer-based neural quantum states under in-context learning. We establish a rigorous inference-time generalization error bound in terms of mean squared error (MSE), showing that the pointwise prediction error decreases inversely with both the number of in-context examples and the depth of the Transformer. We further show that the Transformer depth required to achieve this guarantee scales only linearly with the system size--namely, the number of particles in continuous systems or the number of qudits in discrete systems. Building on this result, we extend our analysis to full quantum states formulated as rank-one density operators, and derive MSE-based generalization bounds over both continuous and discrete domains under physical constraints. Finally, numerical simulations corroborate our theoretical analysis.
ER -