Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence
Abstract
Deep generative models are widely reported to enable de novo crystal structure prediction (CSP), but their capability has not been measured consistently against template-based methods. We evaluate 12 representative generative CSP models, spanning latent-variable, diffusion, flow-matching, autoregressive, and manifold random-walk architectures, against TCSP 2.0 on 180 test structures and a leakage-controlled subset of 46. All methods use identical structure-matching, symmetry, and consensus criteria. Template retrieval is the strongest single method, reaching 68.3% top-1 success; symmetry-aware EquiCSP (66.4%) and Uni-3DAR (62.9%) form the next tier. However, comparison with TCSP 2.0 shows that most structures correctly predicted by generative models are also correctly predicted by template substitution. Thus, the set of structures uniquely reachable by generation is small, limiting its practical advantage for discovering structures outside existing prototype libraries. To test the source of this performance, we removed entire stoichiometric prototype families from the training set and retrained the strongest generative model. Accuracy declined by 50-78% across four families, establishing that performance is substantially prototype-dependent. A small minority of structures survived removal of their prototype family, demonstrating a real but limited retrieval-independent predictive capacity. Present generative CSP models therefore function largely as implicit, softer-edged prototype libraries rather than genuinely de novo predictors. Enlarging this residual capacity, rather than aggregate match rate alone, is the central open problem.
Related
- uFlowCSP: Crystal Structure Prediction using Mean flow generative models
- Generative modeling for fusion: a large-scale benchmark of stellarator fast-ion simulations
- An open benchmark for machine learning-based polymer property prediction
- Geometry-Controlled Initial-Density Dependence in Branching-Merging Exclusion Processes
- X2SBench: an open benchmark for evaluating crystal structure determination from powder diffraction
BibTeX
@misc{wei2026deep,
title = {{Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence}},
author = {Lai Wei and Rongzhi Dong and Ying Feng and Madeline Miklos and Jianjun Hu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.26502},
eprint = {2609.26502},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.26502},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Deep Generative Crystal Structure Prediction: A Benchmark Study and a Controlled Test of Prototype Dependence
AU - Lai Wei
AU - Rongzhi Dong
AU - Ying Feng
AU - Madeline Miklos
AU - Jianjun Hu
PY - 2026
DA - 2026/10/17
PB - arXiv
DO - 10.48550/arxiv.2609.26502
UR - https://arxiv.org/abs/2609.26502
AB - Deep generative models are widely reported to enable de novo crystal structure prediction (CSP), but their capability has not been measured consistently against template-based methods. We evaluate 12 representative generative CSP models, spanning latent-variable, diffusion, flow-matching, autoregressive, and manifold random-walk architectures, against TCSP 2.0 on 180 test structures and a leakage-controlled subset of 46. All methods use identical structure-matching, symmetry, and consensus criteria. Template retrieval is the strongest single method, reaching 68.3% top-1 success; symmetry-aware EquiCSP (66.4%) and Uni-3DAR (62.9%) form the next tier. However, comparison with TCSP 2.0 shows that most structures correctly predicted by generative models are also correctly predicted by template substitution. Thus, the set of structures uniquely reachable by generation is small, limiting its practical advantage for discovering structures outside existing prototype libraries. To test the source of this performance, we removed entire stoichiometric prototype families from the training set and retrained the strongest generative model. Accuracy declined by 50-78% across four families, establishing that performance is substantially prototype-dependent. A small minority of structures survived removal of their prototype family, demonstrating a real but limited retrieval-independent predictive capacity. Present generative CSP models therefore function largely as implicit, softer-edged prototype libraries rather than genuinely de novo predictors. Enlarging this residual capacity, rather than aggregate match rate alone, is the central open problem.
ER -