Reproducibility Limits of Public MS/MS Spectral Libraries: Irreproducibility, Outlier Spectra, and Indistinguishable Compounds Across One Million Reference Spectra
- 1Independent Researcher
Abstract
Compound annotation in untargeted metabolomics relies on matching an experimental MS/MS spectrum against a reference library, and the quality of such libraries is routinely summarized by the proportion of spectrum pairs exceeding a similarity threshold. We show that this class of metric is structurally invalid. Analyzing 1,022,449 spectra covering 66,843 compounds from GNPS, MS-DIAL, and MassBank of North America, we compute all within- and between-compound pairwise similarities and demonstrate that proportion-based consistency estimates are confounded by sampling depth as a matter of construction: the apparent high-consistency rate falls from 91% for compound groups supported by a single comparison to 5% for groups supported by more than one hundred. Consequently, any comparison of libraries, instrument platforms, or acquisition conditions that does not control depth measures coverage rather than quality — an apparent advantage of one instrument platform over another narrows substantially once depth is matched. We further identify a resolution floor that no improvement in library curation can remove: 31.7% of compounds share near-identical spectra (cosine ≥ 0.9) with at least one structurally distinct compound, so for these the identification is underdetermined by spectral matching alone. Against this corrected framework, 49.6% of 12,613,259 same-compound comparisons fall below the conventional 0.7 threshold and 6.4% of spectra behave as outliers within their own compound group. We release the collection annotated with spectrum-, group-, and compound-level metrics, each reported alongside the evidence supporting it rather than as a bare proportion, together with the pipeline as an open-source R package.
Related
- Identifiability Boundaries in Dynamic GPCR Pharmacology: When Desensitization, Internalization and Persistent Signaling Become Mechanistically Indistinguishable
- N-glycopedia: libraries for native N-glycan structural analysis
- DNA-Encoded Chemical Diversification of mRNA Display Libraries
- Hybrid Macrocyclic Peptides — Synthetic Strategies to Diversify and Cyclize Peptide Libraries in Phage Display Selections
- A proximity-driven and regioselective peptide bicyclization (PReP-Bicyc) approach for phage display libraries
BibTeX
@misc{kim2026reproducibility,
title = {{Reproducibility Limits of Public MS/MS Spectral Libraries: Irreproducibility, Outlier Spectra, and Indistinguishable Compounds Across One Million Reference Spectra}},
author = {Yongseok Kim},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15008195/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008195/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Reproducibility Limits of Public MS/MS Spectral Libraries: Irreproducibility, Outlier Spectra, and Indistinguishable Compounds Across One Million Reference Spectra
AU - Yongseok Kim
PY - 2026
DA - 2026/09/02
PB - ChemRxiv
DO - 10.26434/chemrxiv.15008195/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008195/v1
AB - Compound annotation in untargeted metabolomics relies on matching an experimental MS/MS spectrum against a reference library, and the quality of such libraries is routinely summarized by the proportion of spectrum pairs exceeding a similarity threshold. We show that this class of metric is structurally invalid. Analyzing 1,022,449 spectra covering 66,843 compounds from GNPS, MS-DIAL, and MassBank of North America, we compute all within- and between-compound pairwise similarities and demonstrate that proportion-based consistency estimates are confounded by sampling depth as a matter of construction: the apparent high-consistency rate falls from 91% for compound groups supported by a single comparison to 5% for groups supported by more than one hundred. Consequently, any comparison of libraries, instrument platforms, or acquisition conditions that does not control depth measures coverage rather than quality — an apparent advantage of one instrument platform over another narrows substantially once depth is matched. We further identify a resolution floor that no improvement in library curation can remove: 31.7% of compounds share near-identical spectra (cosine ≥ 0.9) with at least one structurally distinct compound, so for these the identification is underdetermined by spectral matching alone. Against this corrected framework, 49.6% of 12,613,259 same-compound comparisons fall below the conventional 0.7 threshold and 6.4% of spectra behave as outliers within their own compound group. We release the collection annotated with spectrum-, group-, and compound-level metrics, each reported alongside the evidence supporting it rather than as a bare proportion, together with the pipeline as an open-source R package.
ER -