The Geometry of Manifold Learning: Singular foliations via Deep Learning models
Abstract
Deep learning algorithms have become widely used and successful across a wide range of tasks. A key challenge is determining whether these algorithms truly capture the intrinsic geometry of their training data in order to better understand what they learn from data. In this theoretical work, we explore a new geometric structure defined on the data space by a trained neural network through the Data Information Matrix (DIM), a variation of the Fisher Information Matrix. This new geometric structure is a foliation and the main focus of this work is to explore the non-smooth case of ReLU activation functions, where traditional geometric tools often fail. The foliation is defined through a distribution spanned by the columns of the DIM. We first prove that this distribution is non-smooth in the sense of distributions, rendering the existence of an associated foliation much more complex. To circumvent this issue, we introduce a smoothed version of this distribution that differs from the original only on a set of measure zero. Finally, we prove that this smoothed distribution induces a singular foliation on the data space, which is based on the data the network has been trained with. Our theoretical contributions provide a geometric framework to better understand how neural networks organize their input space in non-smooth settings, bridging differential geometry and deep learning theory.
Related
- Finite-Rank Lie Algebroids for Singular Foliations of Prescribed Vanishing Order
- Bounded Leaf Volumes and Analytic Quotients of Singular Holomorphic Foliations
- Singular Levi-flats and holomorphic foliations on projective spaces
- Limit cycles of singular algebraic foliations on the real projective plane
- Foliations with small singular set in arbitrary characteristic
BibTeX
@misc{novelli2026geometry,
title = {{The Geometry of Manifold Learning: Singular foliations via Deep Learning models}},
author = {Gabriele Novelli and Eliot Tron and Emanuele Latini and Rita Fioresi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37370},
eprint = {2610.37370},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37370},
}
RIS
TY - UNPB
TI - The Geometry of Manifold Learning: Singular foliations via Deep Learning models
AU - Gabriele Novelli
AU - Eliot Tron
AU - Emanuele Latini
AU - Rita Fioresi
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37370
UR - https://arxiv.org/abs/2610.37370
AB - Deep learning algorithms have become widely used and successful across a wide range of tasks. A key challenge is determining whether these algorithms truly capture the intrinsic geometry of their training data in order to better understand what they learn from data. In this theoretical work, we explore a new geometric structure defined on the data space by a trained neural network through the Data Information Matrix (DIM), a variation of the Fisher Information Matrix. This new geometric structure is a foliation and the main focus of this work is to explore the non-smooth case of ReLU activation functions, where traditional geometric tools often fail. The foliation is defined through a distribution spanned by the columns of the DIM. We first prove that this distribution is non-smooth in the sense of distributions, rendering the existence of an associated foliation much more complex. To circumvent this issue, we introduce a smoothed version of this distribution that differs from the original only on a set of measure zero. Finally, we prove that this smoothed distribution induces a singular foliation on the data space, which is based on the data the network has been trained with. Our theoretical contributions provide a geometric framework to better understand how neural networks organize their input space in non-smooth settings, bridging differential geometry and deep learning theory.
ER -