Time-split validation and cross-platform transferability of an explainable MALDI-TOF MS machine-learning model for carbapenem-resistant Klebsiella pneumoniae screening
- 1Department of Clinical Laboratory Medicine, Suining Central Hospital
Abstract
MALDI-TOF MS-based machine learning may shorten the time to detection of carbapenem-resistant Klebsiella pneumoniae (CRKP), yet performance estimates often rest on random splits and spectrum-level evaluation, and transferability across instruments is rarely tested. In a laboratory-information-system-audited single-center cohort of 2,399 K. pneumoniae isolates (363 CRKP; 2,836 VITEK MS spectra; 2024–2026), we trained a LightGBM classifier on 1,766 isolates from 2024 to 2025 and evaluated it on 633 isolates from 2026. The out-of-fold area under the ROC curve (AUC) was 0.909 (95% CI 0.885–0.932); on the 2026 cohort the calibrated AUC was 0.833 (0.776–0.885), with an expected calibration error of 0.024 and a Brier score of 0.068. Thresholds derived independently from the training cohort gave a sensitivity of 0.493 on the 2026 cohort at both the Youden and the NPV-oriented threshold, with an observed NPV of 0.935; the higher sensitivity obtained with thresholds chosen on the 2026 cohort itself was not reproducible. Applying the model unchanged to 1,190 isolates from three other platforms (Autof, Bruker, Quantof) resulted in loss of essentially all discriminative ability (AUC 0.450–0.543); because those datasets also differ in population and label definition, the loss cannot be attributed to platform alone. SHAP analysis identified a dominant m/z 4517.5–4519.5 cluster. A deployable offline tool reproduced the audited performance (AUC 0.826, ECE 0.022), and a monitoring analysis on 564 spectra drawn from the same period was consistent with it. The model is not a stand-alone screening test: deployment is limited to the platform on which it was developed and validated, and requires ongoing monitoring. The labeled dataset is deposited in MassIVE (MSV000103080) for independent benchmarking.
Related
- Closely related ST11 Klebsiella pneumoniae isolates with transient attenuation and re-emergence of carbapenem resistance during prolonged hospitalization: a case report
- Genome-wide transposon screening uncovers phage resistance mechanisms and bidirectional effects on antibiotic susceptibility in carbapenem-resistant Klebsiella pneumoniae
- Capsule types driving carbapenem-resistant Klebsiella pneumoniae in Pakistan are largely absent from the phage literature
- PBT2 potentiates carbapenem antibiotics against diverse Klebsiella pneumoniae strains
- A central hub for convergence: multi-layer genomic networks reveal lineage-specific plasmid compatibility driving hypervirulence and carbapenem resistance in ST11 Klebsiella pneumoniae
BibTeX
@article{li2026timesplit,
title = {{Time-split validation and cross-platform transferability of an explainable MALDI-TOF MS machine-learning model for carbapenem-resistant Klebsiella pneumoniae screening}},
author = {Kun Li and Xue Yang and Long Li and Jie Wang},
journal = {Frontiers in Microbiology},
year = {2026},
month = oct,
doi = {10.3389/fmicb.2026.1985869},
url = {https://www.frontiersin.org/articles/10.3389/fmicb.2026.1985869/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Time-split validation and cross-platform transferability of an explainable MALDI-TOF MS machine-learning model for carbapenem-resistant Klebsiella pneumoniae screening
AU - Kun Li
AU - Xue Yang
AU - Long Li
AU - Jie Wang
PY - 2026
DA - 2026/10/28
JO - Frontiers in Microbiology
DO - 10.3389/fmicb.2026.1985869
UR - https://www.frontiersin.org/articles/10.3389/fmicb.2026.1985869/full
AB - MALDI-TOF MS-based machine learning may shorten the time to detection of carbapenem-resistant Klebsiella pneumoniae (CRKP), yet performance estimates often rest on random splits and spectrum-level evaluation, and transferability across instruments is rarely tested. In a laboratory-information-system-audited single-center cohort of 2,399 K. pneumoniae isolates (363 CRKP; 2,836 VITEK MS spectra; 2024–2026), we trained a LightGBM classifier on 1,766 isolates from 2024 to 2025 and evaluated it on 633 isolates from 2026. The out-of-fold area under the ROC curve (AUC) was 0.909 (95% CI 0.885–0.932); on the 2026 cohort the calibrated AUC was 0.833 (0.776–0.885), with an expected calibration error of 0.024 and a Brier score of 0.068. Thresholds derived independently from the training cohort gave a sensitivity of 0.493 on the 2026 cohort at both the Youden and the NPV-oriented threshold, with an observed NPV of 0.935; the higher sensitivity obtained with thresholds chosen on the 2026 cohort itself was not reproducible. Applying the model unchanged to 1,190 isolates from three other platforms (Autof, Bruker, Quantof) resulted in loss of essentially all discriminative ability (AUC 0.450–0.543); because those datasets also differ in population and label definition, the loss cannot be attributed to platform alone. SHAP analysis identified a dominant m/z 4517.5–4519.5 cluster. A deployable offline tool reproduced the audited performance (AUC 0.826, ECE 0.022), and a monitoring analysis on 564 spectra drawn from the same period was consistent with it. The model is not a stand-alone screening test: deployment is limited to the platform on which it was developed and validated, and requires ongoing monitoring. The labeled dataset is deposited in MassIVE (MSV000103080) for independent benchmarking.
SN - 1664-302X
ER -