X2SBench: an open benchmark for evaluating crystal structure determination from powder diffraction
Abstract
Progress towards practical powder X-ray diffraction (PXRD) structure determination requires evaluation beyond small crystals and idealized patterns. X2SBench combines 152,587 simulated structure-pattern pairs with 591 curated measurements, extending evaluation to larger cells, lower-symmetry structures and diverse compositions. Fixed splits, defined inputs and common structural metrics support comparisons by crystal system, atom count and element count. Joint stratification locates weaknesses within these groups. On 558 paired targets, an X2SBench-fine-tuned model improves recovery from simulated patterns but loses accuracy on measured inputs, revealing a gap in experimental transfer. Background subtraction and smoothing improve recovery for one tested checkpoint, while augmentation and reference-lattice comparisons identify further opportunities for measurement adaptation and reliable cell estimation. An open platform provides data access and standardized result submission, establishing a shared basis for community evaluation and progress towards practical PXRD analysis.
Related
- OpenGray: a benchmark for evaluating the decisions of radiotherapy planning agents
- MedPhysBench: A Benchmark for Evaluating Language Models on Medical Physics Calculations and Error Detection
- Structure of niobium to 365 GPa using ultrafast x-ray diffraction and shock compression
- A 4D‐STEM Tomographic Framework Assisted by Object Tracking for Nanoparticle Structure Determination
- Energy Accuracy Is Not Enough: A Structure-Aware Benchmark and Evaluation Protocol for Quantum Architecture Search
BibTeX
@misc{gao2026x2sbench,
title = {{X2SBench: an open benchmark for evaluating crystal structure determination from powder diffraction}},
author = {Zhiyuan Gao and Juncheng Xiao and Shuchen Pu and Qi Li and Weida Wang and Shufei Zhang and Yong Yang and Shifeng Jin and Yunqi Cai and Hongming Weng},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.29751},
eprint = {2609.29751},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.29751},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - X2SBench: an open benchmark for evaluating crystal structure determination from powder diffraction
AU - Zhiyuan Gao
AU - Juncheng Xiao
AU - Shuchen Pu
AU - Qi Li
AU - Weida Wang
AU - Shufei Zhang
AU - Yong Yang
AU - Shifeng Jin
AU - Yunqi Cai
AU - Hongming Weng
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.29751
UR - https://arxiv.org/abs/2609.29751
AB - Progress towards practical powder X-ray diffraction (PXRD) structure determination requires evaluation beyond small crystals and idealized patterns. X2SBench combines 152,587 simulated structure-pattern pairs with 591 curated measurements, extending evaluation to larger cells, lower-symmetry structures and diverse compositions. Fixed splits, defined inputs and common structural metrics support comparisons by crystal system, atom count and element count. Joint stratification locates weaknesses within these groups. On 558 paired targets, an X2SBench-fine-tuned model improves recovery from simulated patterns but loses accuracy on measured inputs, revealing a gap in experimental transfer. Background subtraction and smoothing improve recovery for one tested checkpoint, while augmentation and reference-lattice comparisons identify further opportunities for measurement adaptation and reliable cell estimation. An open platform provides data access and standardized result submission, establishing a shared basis for community evaluation and progress towards practical PXRD analysis.
ER -