Do Neural Networks Learn Structure-Preserving Maps? A Case Study in Latent-to-Hilbert Embeddings

Abstract

We ask whether a neural network can learn a structure-preserving map from a compressed latent space to a Hilbert-space representation. Using an 8-dimensional autoencoder bottleneck on MNIST and n-qubit product-state targets from PCA-based angle encoding, we report four findings. Although the target angles are generated by a nonlinear sigmoid transformation of the latent projections, the resulting mapping is well approximated by a linear function over the observed latent distribution: linear regression from z to the true target angles achieves R² = 0.98, while regression to the MLP's recovered angles achieves R² = 0.91. The learned map's primary direction is strongly aligned with the target-induced direction, with cosine similarity $0.989$, while remaining nearly orthogonal to the input's principal direction, with cosine similarity $0.002$. The map is genuinely rank-4: removing any singular direction degrades inner-product preservation by $2.5−−5.9\times$ despite a singular-value spectrum with two dominant and two small values. The learned subspace does not coincide with the PCA basis used to construct the target, and different random seeds recover the same primary direction but diverge in higher ranks. Finally, kernel ridge regression with an RBF kernel outperforms a tuned MLP (IP error $0.0144$ vs.\ $0.0197$), suggesting that for approximately linear structure-preserving mappings, classical kernel methods may be a simpler and more effective alternative.

Structure-preserving Fourier Neural Operators for Long-time Cahn-Hilliard Dynamics under Coarse Temporal Supervision
Yingli Li, Y. C. Zhou
arXiv30 Sep 2026PreprintarXiv licenceFull textNumerical analysis & scientific computing Computational physicsarXiv
Dynamical Batching: How Class-Cycling Training Shapes the Internal Structure of Neural Networks
Nicolás Ratier-Werbin, Samuel Lozano, Miguel Ruiz-García
arXiv19 Oct 2026PreprintCC BY 4.0Full textCondensed matterarXiv
Efficient Representation of multicategorical local Hilbert spaces: nonlinear Restricted Boltzmann Machines to Kolmogorov-Arnold Networks
Abhiroop Lahiri
arXiv11 Sep 2026PreprintarXiv licenceFull textCondensed matter Quantum physics & quantum informationarXiv
The latent structure of diversification from relatedness and network embeddings
Niclas Frederic Sturm, Cristian Candia, Bruno Damásio +1
PLOS One22 Oct 2026Peer-reviewedCC BYFull textStatistics Statistical & nonlinear physicsPLOSEurope PMC
Many-body topology in parity-preserving tensor networks
Maksimilian Usoltcev, Nguyen Hanh Dung, Carolin Wille +2
arXiv30 Sep 2026PreprintCC BY 4.0Full textCondensed matter Quantum physics & quantum informationarXiv

Cite

BibTeX RIS

BibTeX

@misc{shahzad2026neural,
  title = {{Do Neural Networks Learn Structure-Preserving Maps? A Case Study in Latent-to-Hilbert Embeddings}},
  author = {Muhammad Adnan Shahzad},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.05297},
  eprint = {2610.05297},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.05297},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Do Neural Networks Learn Structure-Preserving Maps? A Case Study in Latent-to-Hilbert Embeddings
AU  - Muhammad Adnan Shahzad
PY  - 2026
DA  - 2026/10/04
PB  - arXiv
DO  - 10.48550/arxiv.2610.05297
UR  - https://arxiv.org/abs/2610.05297
AB  - We ask whether a neural network can learn a structure-preserving map from a compressed latent space to a Hilbert-space representation. Using an 8-dimensional autoencoder bottleneck on MNIST and n-qubit product-state targets from PCA-based angle encoding, we report four findings. Although the target angles are generated by a nonlinear sigmoid transformation of the latent projections, the resulting mapping is well approximated by a linear function over the observed latent distribution: linear regression from z to the true target angles achieves R^2 = 0.98, while regression to the MLP's recovered angles achieves R^2 = 0.91. The learned map's primary direction is strongly aligned with the target-induced direction, with cosine similarity $0.989$, while remaining nearly orthogonal to the input's principal direction, with cosine similarity $0.002$. The map is genuinely rank-4: removing any singular direction degrades inner-product preservation by $2.5−−5.9\times$ despite a singular-value spectrum with two dominant and two small values. The learned subspace does not coincide with the PCA basis used to construct the target, and different random seeds recover the same primary direction but diverge in higher ranks. Finally, kernel ridge regression with an RBF kernel outperforms a tuned MLP (IP error $0.0144$ vs.\ $0.0197$), suggesting that for approximately linear structure-preserving mappings, classical kernel methods may be a simpler and more effective alternative.
ER  -

Abstract

Related

Cite