Learning the information contained in the microstructure of data distributions
Abstract
Understanding how much and where information is nested in sets of data obtained using different parameters and metrics is a typical problem in the analysis of high-dimensional datasets. Approaches based on relative data-driven metrics may be poorly quantitative and difficult to interpret, while accurate methods may be computationally expensive and impractical for large datasets. Here, we report an efficient, quantitative, and transferable method useful to this end, which we call "Micro-Information Analysis". Its foundation is based on the tight relationship that exists between the microscopic features of data distributions and the information they contain. The method performs preliminary unsupervised clustering, capturing microstructural features of data distributions such as the dominant Gaussian microdomains (microstates) present in them, their population, separation, and their residual non-Gaussian fraction. Leveraging the principle of clustering-and-minimization of Shannon entropy, this allows quantifying in bit (that is, in absolute metric) the information contained in the microstructure of the data, and how much information is nested in each Gaussian and non-Gaussian microdomain. The output of the analysis is a set of easily interpretable Information-Microstructure plots that allow to readily compare and rank data based on their information content and to graphically map the information breakdown among their various microenvironments in diagrams that are easy to interpret. We prove the efficiency of our method by analyzing and ranking data generated synthetically or from molecular simulations, demonstrating how "Micro-Information Analysis" is a physically-grounded quantitative tool useful for the study of physical systems from their data and for data science in general.
Related
- Prediction of aluminium distributions in zeolites using basin-hopping and machine learning potentials
- Tungsten Oxide-contained PEO Coatings on Ti–6Al‑4 V Alloys: Enhanced Surface Characteristics and Bioactivity for Bio-implant
- Foldability Is an Intrinsic Property of Protein Sequences: A Lattice Model Study with Information-Theoretic Approach and Unsupervised Machine Learning
- Folding and Function as Two Sides of One Coin: Statistical Mechanics, Information Theory, and Deep Learning of Binding, Catalysis, and Allostery in an Exactly Enumerable Lattice Protein
- Machine Learning Identifies an Information-Sufficiency Regime for Critical-Mineral Recovery
BibTeX
@misc{baldo2026learning,
title = {{Learning the information contained in the microstructure of data distributions}},
author = {Lucrezia L. Baldo and Chiara Lionello and Edward D. Donkor and Matteo Becchi and Giovanni M. Pavan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.22283},
eprint = {2610.22283},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22283},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning the information contained in the microstructure of data distributions
AU - Lucrezia L. Baldo
AU - Chiara Lionello
AU - Edward D. Donkor
AU - Matteo Becchi
AU - Giovanni M. Pavan
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2610.22283
UR - https://arxiv.org/abs/2610.22283
AB - Understanding how much and where information is nested in sets of data obtained using different parameters and metrics is a typical problem in the analysis of high-dimensional datasets. Approaches based on relative data-driven metrics may be poorly quantitative and difficult to interpret, while accurate methods may be computationally expensive and impractical for large datasets. Here, we report an efficient, quantitative, and transferable method useful to this end, which we call "Micro-Information Analysis". Its foundation is based on the tight relationship that exists between the microscopic features of data distributions and the information they contain. The method performs preliminary unsupervised clustering, capturing microstructural features of data distributions such as the dominant Gaussian microdomains (microstates) present in them, their population, separation, and their residual non-Gaussian fraction. Leveraging the principle of clustering-and-minimization of Shannon entropy, this allows quantifying in bit (that is, in absolute metric) the information contained in the microstructure of the data, and how much information is nested in each Gaussian and non-Gaussian microdomain. The output of the analysis is a set of easily interpretable Information-Microstructure plots that allow to readily compare and rank data based on their information content and to graphically map the information breakdown among their various microenvironments in diagrams that are easy to interpret. We prove the efficiency of our method by analyzing and ranking data generated synthetically or from molecular simulations, demonstrating how "Micro-Information Analysis" is a physically-grounded quantitative tool useful for the study of physical systems from their data and for data science in general.
ER -