Surprisingly high redundancy in electronic structure data across materials explained by low intrinsic dimensionality
4 affiliations
- 1Department of Mechanical and Aerospace Engineering, Michigan Technological University, Houghton, MI, USA
- 2Hopkins Extreme Materials Institute, Johns Hopkins University, Baltimore, MD, USA
- 3Department of Materials Science and Engineering, University of California, Los Angeles, CA, USA
- 4Center for Artificial Intelligence, Michigan Technological University, Houghton, MI, USA
Abstract
Machine learning (ML) models for electronic structure typically rely on large datasets generated by computationally expensive Kohn-Sham density functional theory calculations, as it is not known a priori which portions of the data are essential for accurate learning. Here, we reveal significant redundancies in electronic structure datasets across diverse material systems and attribute them to the low intrinsic dimensionality of the underlying data. We show that even random pruning can substantially reduce dataset size with minimal degradation in predictive accuracy. Moreover, a state-of-the-art coverage-based pruning strategy that samples data across all learning difficulties almost always preserves chemical accuracy and maintains model generalizability while using up to two orders of magnitude less data and reducing training time by a factor of three or more. We further demonstrate that the essential electronic structure information lies on a low-dimensional, non-linear manifold, providing a potential geometric explanation for the observed prunability. These observations are consistent with the predominance of local atomic environments in determining electronic properties, as suggested by nearsightedness arguments, and indicate that large-scale datasets may contain highly overlapping information. Our findings challenge the prevailing assumption that such extensive datasets are necessary for accurate ML-based electronic structure predictions and open a path toward identifying minimal, representative datasets for each material class.
Related
- Decoupling Nanofiller Dimensionality and Electronic Character in Gel Polymer Electrolytes for Supercapacitors
- Electronic crystals in layered materials
- Detection Defines Dephasing in Two-Dimensional Electronic Spectroscopy of Materials: Coherent Field Emission versus Incoherent Population Observables
- Next-generation electronic packaging materials: thermal-dielectric properties and design strategies in PI and PTFE-based composites
- VIBE-QC: an open-source, general-purpose electronic-structure code for molecules and solids developed with generative AI
BibTeX
@article{hossain2026surprisingly,
title = {{Surprisingly high redundancy in electronic structure data across materials explained by low intrinsic dimensionality}},
author = {Sazzad Hossain and Ponkrshnan Thiagarajan and Shashank Pathrudkar and S. Taylor and Abhijeet S. Gangan and Amartya Banerjee and Susanta Ghosh},
journal = {npj Computational Materials},
year = {2026},
month = oct,
doi = {10.1038/s41524-026-02361-x},
url = {https://doi.org/10.1038/s41524-026-02361-x},
note = {CC BY},
}
RIS
TY - JOUR
TI - Surprisingly high redundancy in electronic structure data across materials explained by low intrinsic dimensionality
AU - Sazzad Hossain
AU - Ponkrshnan Thiagarajan
AU - Shashank Pathrudkar
AU - S. Taylor
AU - Abhijeet S. Gangan
AU - Amartya Banerjee
AU - Susanta Ghosh
PY - 2026
DA - 2026/10/19
JO - npj Computational Materials
DO - 10.1038/s41524-026-02361-x
UR - https://doi.org/10.1038/s41524-026-02361-x
AB - Machine learning (ML) models for electronic structure typically rely on large datasets generated by computationally expensive Kohn-Sham density functional theory calculations, as it is not known a priori which portions of the data are essential for accurate learning. Here, we reveal significant redundancies in electronic structure datasets across diverse material systems and attribute them to the low intrinsic dimensionality of the underlying data. We show that even random pruning can substantially reduce dataset size with minimal degradation in predictive accuracy. Moreover, a state-of-the-art coverage-based pruning strategy that samples data across all learning difficulties almost always preserves chemical accuracy and maintains model generalizability while using up to two orders of magnitude less data and reducing training time by a factor of three or more. We further demonstrate that the essential electronic structure information lies on a low-dimensional, non-linear manifold, providing a potential geometric explanation for the observed prunability. These observations are consistent with the predominance of local atomic environments in determining electronic properties, as suggested by nearsightedness arguments, and indicate that large-scale datasets may contain highly overlapping information. Our findings challenge the prevailing assumption that such extensive datasets are necessary for accurate ML-based electronic structure predictions and open a path toward identifying minimal, representative datasets for each material class.
SN - 2057-3960
ER -