PhononBench:a large-scale phonon-based benchmark for dynamical stability in crystal generation
- 1Renmin University of China
Abstract
Recent advances in generative artificial intelligence have enabled crystal-design methods based on graph neural networks, diffusion models, and large language models. Existing evaluations commonly follow the stability–uniqueness–novelty (S.U.N.) framework, but typically assess stability using thermodynamic criteria, which do not fully capture dynamical stability. Although dynamical stability influences material synthesis and persistence, experimental realization also depends on anharmonicity, finite temperature, defects, disorder, and synthesis conditions. Its high computational cost has hindered systematic large-scale evaluation of generated crystals. Here, we introduce PhononBench, the first large-scale benchmark of dynamical stability in AI-generated crystals. Using the MatterSim interatomic potential, which provides density-functional-theory-level accuracy for phonon predictions across more than 10,000 materials, PhononBench enables efficient phonon calculations for 133,838 structures generated by seven leading crystal-generation models. Unless otherwise specified, all stability metrics use a phonon-frequency threshold of −0.1 THz. The results reveal a major limitation of current models: the average dynamical-stability rate is only 32.15%, while the best-performing model, MatterGen, reaches 45.05%. Further analyses show that, for MatterGen band-gap-conditioned generation, the stability rate remains 41.0% even under the optimal 0.5 eV condition. In space-group-controlled generation, higher-symmetry crystals are generally more stable, with cubic systems reaching 53.4%; nevertheless, the overall average is 44.7%. We identify 32,995 structures whose minimum phonon frequencies remain above the adopted threshold along sampled high-symmetry paths; these labels do not establish stability throughout the full Brillouin-zone volume. Multi-threshold sensitivity analyses and investigations of structural and physical factors further clarify the origins of dynamical instability. By establishing the first large-scale benchmark for dynamical stability, PhononBench systematically exposes limitations of current crystal-generation models and provides practical criteria for their evaluation and improvement. Together, these results create a scalable basis for comparing generative approaches, diagnosing failure modes, and guiding future model development toward crystals with improved physical plausibility and greater relevance to experimental materials discovery
Related
- Symmetry-aware Bayesian flow networks for crystal generation
- Symmetry-Aware Flow Matching for End-to-end Molecular Crystal Generation
- Modulating the polarization-resolved second harmonic generation response of a Sulfamic acid multicomponent crystal via amino acid coformer selection
- Phase-dependent Al leaching and crystal structure stability of CoAl2O4 in alkaline media
- Benchmarking Machine-Learning Interatomic Potentials for Dynamical Stability in Inorganic Semiconductor Nanocrystals: A CdSe Case Study
BibTeX
@article{han2026phononbencha,
title = {{PhononBench:a large-scale phonon-based benchmark for dynamical stability in crystal generation}},
author = {Xiao-Qi Han and Ze-Feng Gao and Wen-Kao Li and Peng‐Jie Guo and Zhong-Yi Lu},
journal = {AI for Science},
year = {2026},
month = aug,
doi = {10.1088/3050-287x/ae9ee4},
url = {https://doi.org/10.1088/3050-287x/ae9ee4},
note = {CC BY},
}
RIS
TY - JOUR
TI - PhononBench:a large-scale phonon-based benchmark for dynamical stability in crystal generation
AU - Xiao-Qi Han
AU - Ze-Feng Gao
AU - Wen-Kao Li
AU - Peng‐Jie Guo
AU - Zhong-Yi Lu
PY - 2026
DA - 2026/08/26
JO - AI for Science
DO - 10.1088/3050-287x/ae9ee4
UR - https://doi.org/10.1088/3050-287x/ae9ee4
AB - Recent advances in generative artificial intelligence have enabled crystal-design methods based on graph neural networks, diffusion models, and large language models. Existing evaluations commonly follow the stability–uniqueness–novelty (S.U.N.) framework, but typically assess stability using thermodynamic criteria, which do not fully capture dynamical stability. Although dynamical stability influences material synthesis and persistence, experimental realization also depends on anharmonicity, finite temperature, defects, disorder, and synthesis conditions. Its high computational cost has hindered systematic large-scale evaluation of generated crystals. Here, we introduce PhononBench, the first large-scale benchmark of dynamical stability in AI-generated crystals. Using the MatterSim interatomic potential, which provides density-functional-theory-level accuracy for phonon predictions across more than 10,000 materials, PhononBench enables efficient phonon calculations for 133,838 structures generated by seven leading crystal-generation models. Unless otherwise specified, all stability metrics use a phonon-frequency threshold of −0.1 THz. The results reveal a major limitation of current models: the average dynamical-stability rate is only 32.15%, while the best-performing model, MatterGen, reaches 45.05%. Further analyses show that, for MatterGen band-gap-conditioned generation, the stability rate remains 41.0% even under the optimal 0.5 eV condition. In space-group-controlled generation, higher-symmetry crystals are generally more stable, with cubic systems reaching 53.4%; nevertheless, the overall average is 44.7%. We identify 32,995 structures whose minimum phonon frequencies remain above the adopted threshold along sampled high-symmetry paths; these labels do not establish stability throughout the full Brillouin-zone volume. Multi-threshold sensitivity analyses and investigations of structural and physical factors further clarify the origins of dynamical instability. By establishing the first large-scale benchmark for dynamical stability, PhononBench systematically exposes limitations of current crystal-generation models and provides practical criteria for their evaluation and improvement. Together, these results create a scalable basis for comparing generative approaches, diagnosing failure modes, and guiding future model development toward crystals with improved physical plausibility and greater relevance to experimental materials discovery
SN - 3050-287X
ER -