Universal Quantum Transformer
Abstract
Classical continuous-space neural networks fundamentally struggle to lock into exact formal rules, whether mathematical, such as modular arithmetic and non-Abelian group algebra, or linguistic, such as systematic compositional generalization. To approximate these discrete logical rules, they often rely on massive parameter scaling, resulting in stochastic instability even after delayed generalization phenomena known as grokking. Here, we introduce the Universal Quantum Transformer (UQT), a novel, quantum-native computing architecture that uses the physical properties of multi-qubit systems as a universal inductive bias for exact algebraic and compositional reasoning. Rather than translating classical neural mechanisms, our framework relies entirely on parameterized geometric phase embedding and SU(2) wave-interference. We demonstrate that an identical quantum attention circuit, operating on a highly compact 5 or 6 qubit substrate with only 551 to 1,650 trainable parameters, exactly learns three highly distinct formal classes: cyclic modular arithmetic (ℤ11), non-Abelian algebra (the S4 permutation group), and systematic linguistic compositionality (the SCAN language). While standard classical models, including multi-layer perceptrons (MLPs) and Transformers, exhibit stochastic instability at convergence, the UQT achieves mathematically exact, deterministic generalization. We define this stricter regime as crystallization: a step beyond the well-known phenomenon of grokking. Finally, we deploy the UQT on noisy intermediate-scale quantum (NISQ) hardware, achieving 97.5% accuracy on IBM Quantum computers. These results demonstrate that the UQT provides a structurally suited inductive bias for exact formal reasoning that standard classical continuous-space architectures do not natively provide.
Related
- Variational study of the magnetization plateaus in the spin-1/2 kagome Heisenberg antiferromagnet: An approach from vision transformer neural quantum states
- All-fiber quantum current transformer based on nitrogen-vacancy centers
- Variational Quantum Transformer Architecture for Synthetic Language Generation
- Watching Quantum Models Think: Hilbert-Space Interpretability in Quantum Transformer Blocks
- Transformer-Based Time-Series Inference of Lindblad Dynamics in Open Quantum Systems
BibTeX
@misc{chung2026universal,
title = {{Universal Quantum Transformer}},
author = {Sungyong Chung and Alireza Talebpour},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.00045},
eprint = {2606.00045},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.00045},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Universal Quantum Transformer
AU - Sungyong Chung
AU - Alireza Talebpour
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2606.00045
UR - https://arxiv.org/abs/2606.00045
AB - Classical continuous-space neural networks fundamentally struggle to lock into exact formal rules, whether mathematical, such as modular arithmetic and non-Abelian group algebra, or linguistic, such as systematic compositional generalization. To approximate these discrete logical rules, they often rely on massive parameter scaling, resulting in stochastic instability even after delayed generalization phenomena known as grokking. Here, we introduce the Universal Quantum Transformer (UQT), a novel, quantum-native computing architecture that uses the physical properties of multi-qubit systems as a universal inductive bias for exact algebraic and compositional reasoning. Rather than translating classical neural mechanisms, our framework relies entirely on parameterized geometric phase embedding and SU(2) wave-interference. We demonstrate that an identical quantum attention circuit, operating on a highly compact 5 or 6 qubit substrate with only 551 to 1,650 trainable parameters, exactly learns three highly distinct formal classes: cyclic modular arithmetic (ℤ_11), non-Abelian algebra (the S_4 permutation group), and systematic linguistic compositionality (the SCAN language). While standard classical models, including multi-layer perceptrons (MLPs) and Transformers, exhibit stochastic instability at convergence, the UQT achieves mathematically exact, deterministic generalization. We define this stricter regime as crystallization: a step beyond the well-known phenomenon of grokking. Finally, we deploy the UQT on noisy intermediate-scale quantum (NISQ) hardware, achieving 97.5% accuracy on IBM Quantum computers. These results demonstrate that the UQT provides a structurally suited inductive bias for exact formal reasoning that standard classical continuous-space architectures do not natively provide.
ER -