Trillion-scale integrated framework for high-throughput materials databases and seamless sharing
Abstract
Artificial intelligence is accelerating materials discovery, yet for complex systems such as high-entropy alloys, exhaustive exploration of their immense composition spaces is computationally intractable. Here we propose an integrated framework that enables large-scale exploration with reduced resource demands. Our framework uses stateless parallel computing with message queues and containerization, and a lakehouse architecture with columnar storage and metadata-based cloud sharing. Using this framework, we constructed a ten-trillion-scale database of six-principal-element high-entropy alloys in one week, totaling 17.5 terabytes and encompassing compositions, descriptors, and predicted properties. Compared with distributed frameworks, our approach cuts runtime-memory cost by 45%, storage by 90% versus row-based formats, and accelerates subset queries up to 180 times. A full database search completes in under eight minutes using less than four gigabytes of memory. Screening over 50 billion compositions with experimental validation across two alloy families confirms the practical utility of this framework for data-driven materials discovery.
Related
- Integrated High-Throughput Mass Spectrometry Framework for Multiplexed Substrate Screening in Enzyme Discovery and Development
- Advanced High-Throughput Characterization of Nanoporous Materials using Pore Graphs
- High-throughput design of catalytic materials for NOx reduction from aircraft emissions
- Efficiency and Accessibility Are Not Enough: Chemical-Validity Filtering in High-Throughput Screening of Organic Photovoltaic Materials
- High-throughput Materials-to-Device Discovery of Lead-free Double Perovskites Beyond Radiative-limit Metrics
BibTeX
@article{huang2026trillionscale,
title = {{Trillion-scale integrated framework for high-throughput materials databases and seamless sharing}},
author = {Xiaoya Huang and Yu Liu and Shuo Shi and Yuanyuan Zhang and Zengzeng Liang and Miao Zhou and Hanwei Fu and Lei Zheng and Peng Kang},
journal = {Nature Communications},
year = {2026},
month = oct,
doi = {10.1038/s41467-026-78223-3},
url = {https://www.nature.com/articles/s41467-026-78223-3},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Trillion-scale integrated framework for high-throughput materials databases and seamless sharing
AU - Xiaoya Huang
AU - Yu Liu
AU - Shuo Shi
AU - Yuanyuan Zhang
AU - Zengzeng Liang
AU - Miao Zhou
AU - Hanwei Fu
AU - Lei Zheng
AU - Peng Kang
PY - 2026
DA - 2026/10/07
JO - Nature Communications
DO - 10.1038/s41467-026-78223-3
UR - https://www.nature.com/articles/s41467-026-78223-3
AB - Artificial intelligence is accelerating materials discovery, yet for complex systems such as high-entropy alloys, exhaustive exploration of their immense composition spaces is computationally intractable. Here we propose an integrated framework that enables large-scale exploration with reduced resource demands. Our framework uses stateless parallel computing with message queues and containerization, and a lakehouse architecture with columnar storage and metadata-based cloud sharing. Using this framework, we constructed a ten-trillion-scale database of six-principal-element high-entropy alloys in one week, totaling 17.5 terabytes and encompassing compositions, descriptors, and predicted properties. Compared with distributed frameworks, our approach cuts runtime-memory cost by 45%, storage by 90% versus row-based formats, and accelerates subset queries up to 180 times. A full database search completes in under eight minutes using less than four gigabytes of memory. Screening over 50 billion compositions with experimental validation across two alloy families confirms the practical utility of this framework for data-driven materials discovery.
SN - 2041-1723
ER -