Attention as Lattice Gauge Theory: Relations Interact Only Through Wilson Loops
Abstract
Relations compose through the entity they share - Alice is Bob's mother and Bob is Carol's father, so Alice is Carol's grandmother - and the order matters, since a father's mother is not a mother's father; verifying a composed relation means closing the loop. We show that if each entity's coordinate frame may be rotated independently (per-entity frame freedom), three things follow. In the pure-gauge sector, relations that share no cycle cannot interact at all. The only gauge-invariant score of a triangle is a class function of its holonomy H = Tij Tjk Tki - a Wilson loop. And the only equivariant update of an edge is U = Ψ(H) Tij with Ψ conjugation-equivariant. Edge attention by Wilson loops is therefore not one design among many but the unique form the symmetry permits. Three communities hold pieces of this form: the commuting-operator embeddings of knowledge graphs are its abelian (commuting) sector; AlphaFold's triangle update and the Edge Transformer have its shape - an edge updated through the third vertex - without its constraints; and lattice QCD builds it equivariantly on lattices. We instantiate the equivariant sub-family on general relational graphs as holonomy attention. With no training and composition alone, it infers unobserved S4 relations on sparse graphs at 0.88 with a fifth of the edges corrupted, at or slightly below specialist synchronization algorithms, where every commutative architecture is capped at 0.083; its holonomy score flags corrupted triangles at AUC 0.99 where commuting scores are capped at 0.76 by a second selection rule, and it separates clusters without labels.
Related
- One-loop unifying relations between Yang-Mills theory and the nonlinear sigma model from Berends-Giele currents
- Photon evolution through light-particle conversion in stochastic magnetic fields: Kramers--Kronig relations and universality
- Quantum computation of hadron scattering in a lattice gauge theory
- Elastic deuteron-deuteron scattering in the 5S2 channel within nuclear lattice effective field theory
- Octupole deformation in even-even Ra isotopes from covariant density functional theory with localized exchange terms in a three-dimensional lattice space
BibTeX
@misc{sarkar2026attention,
title = {{Attention as Lattice Gauge Theory: Relations Interact Only Through Wilson Loops}},
author = {Aakash Sarkar and Ananya Mukherjee and Motoshige Sato},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37656},
eprint = {2610.37656},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37656},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Attention as Lattice Gauge Theory: Relations Interact Only Through Wilson Loops
AU - Aakash Sarkar
AU - Ananya Mukherjee
AU - Motoshige Sato
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37656
UR - https://arxiv.org/abs/2610.37656
AB - Relations compose through the entity they share - Alice is Bob's mother and Bob is Carol's father, so Alice is Carol's grandmother - and the order matters, since a father's mother is not a mother's father; verifying a composed relation means closing the loop. We show that if each entity's coordinate frame may be rotated independently (per-entity frame freedom), three things follow. In the pure-gauge sector, relations that share no cycle cannot interact at all. The only gauge-invariant score of a triangle is a class function of its holonomy H = T_ij T_jk T_ki - a Wilson loop. And the only equivariant update of an edge is U = Ψ(H) T_ij with Ψ conjugation-equivariant. Edge attention by Wilson loops is therefore not one design among many but the unique form the symmetry permits. Three communities hold pieces of this form: the commuting-operator embeddings of knowledge graphs are its abelian (commuting) sector; AlphaFold's triangle update and the Edge Transformer have its shape - an edge updated through the third vertex - without its constraints; and lattice QCD builds it equivariantly on lattices. We instantiate the equivariant sub-family on general relational graphs as holonomy attention. With no training and composition alone, it infers unobserved S_4 relations on sparse graphs at 0.88 with a fifth of the edges corrupted, at or slightly below specialist synchronization algorithms, where every commutative architecture is capped at 0.083; its holonomy score flags corrupted triangles at AUC 0.99 where commuting scores are capped at 0.76 by a second selection rule, and it separates clusters without labels.
ER -