Convergent evolution in neural representation space: emergent order in deep belief networks
4 affiliations
- 1Cognitive Computational Neuroscience Group, Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nürnberg (FAU), Erlangen, Germany
- 2Neuromodulation and Neuroprosthetics, University Hospital Mannheim, University Heidelberg, Mannheim, Germany
- 3Neuroscience Lab, University Hospital Erlangen, Erlangen, Germany
- 4BG Clinic Ludwigshafen, Ludwigshafen, Germany
Abstract
Deep Belief Networks (DBNs) learn hierarchical generative models without class supervision. We ask how this unsupervised hierarchy reorganizes data, whether it exposes latent class structure, and whether independently initialized networks discover corresponding internal representations. Across MNIST, Fashion-MNIST, and KMNIST, we follow class geometry and accessibility together with representational dimensionality, input reconstruction, and free generation. We find that global class organization generally strengthens with depth even though labels are never available during learning. Early layers frequently make class identity easier to extract, whereas deeper layers form increasingly compact representations in which related features are learned repeatedly. Global class geometry and classifier accessibility can nevertheless diverge, showing that a more ordered representation is not uniformly more informative at every class boundary. Independently trained DBNs converge to closely related depth-specific representation geometries and neuronal response features, indicating that the learned hierarchy is strongly constrained by the data and learning process. Deeper generative paths also produce broader populations of recognizable free samples while preserving less information about individual inputs. Together, these results show that unsupervised depth follows a reproducible path of representational reorganization in which class structure, accessibility, compression, reconstruction, and generation remain distinct but interacting properties.
Related
- Emergent Generalization by Representation Learning in Artificial Neural Networks
- Emergent Loop Geometry in Gauge-Equivariant Graph Neural Networks
- Puncture Evolution with Physics-Informed Neural Networks
- Repeated evolution of convergent iridescence in closely related species of Morpho butterflies living in sympatry
- Physical and emergent nonpairwise interactions in oscillator networks: from higher-order phase reduction to coupling design
BibTeX
@article{krauss2026convergent,
title = {{Convergent evolution in neural representation space: emergent order in deep belief networks}},
author = {Patrick Krauss and Patrick Krauss and Patrick Krauss and Patrick Krauss and Achim Schilling and Achim Schilling and Achim Schilling and Achim Schilling and Andreas K. Maier and Thomas Kinfe and Claus Metzner},
journal = {Frontiers in Complex Systems},
year = {2026},
month = oct,
doi = {10.3389/fcpxs.2026.1960494},
url = {https://doi.org/10.3389/fcpxs.2026.1960494},
note = {CC BY},
}
RIS
TY - JOUR
TI - Convergent evolution in neural representation space: emergent order in deep belief networks
AU - Patrick Krauss
AU - Patrick Krauss
AU - Patrick Krauss
AU - Patrick Krauss
AU - Achim Schilling
AU - Achim Schilling
AU - Achim Schilling
AU - Achim Schilling
AU - Andreas K. Maier
AU - Thomas Kinfe
AU - Claus Metzner
PY - 2026
DA - 2026/10/23
JO - Frontiers in Complex Systems
DO - 10.3389/fcpxs.2026.1960494
UR - https://doi.org/10.3389/fcpxs.2026.1960494
AB - Deep Belief Networks (DBNs) learn hierarchical generative models without class supervision. We ask how this unsupervised hierarchy reorganizes data, whether it exposes latent class structure, and whether independently initialized networks discover corresponding internal representations. Across MNIST, Fashion-MNIST, and KMNIST, we follow class geometry and accessibility together with representational dimensionality, input reconstruction, and free generation. We find that global class organization generally strengthens with depth even though labels are never available during learning. Early layers frequently make class identity easier to extract, whereas deeper layers form increasingly compact representations in which related features are learned repeatedly. Global class geometry and classifier accessibility can nevertheless diverge, showing that a more ordered representation is not uniformly more informative at every class boundary. Independently trained DBNs converge to closely related depth-specific representation geometries and neuronal response features, indicating that the learned hierarchy is strongly constrained by the data and learning process. Deeper generative paths also produce broader populations of recognizable free samples while preserving less information about individual inputs. Together, these results show that unsupervised depth follows a reproducible path of representational reorganization in which class structure, accessibility, compression, reconstruction, and generation remain distinct but interacting properties.
SN - 2813-6187
ER -