Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated 50 min ago
Peer-reviewedCC BY 4.0International Journal of Molecular Sciences

What a Near-Perfect AUROC Does Not Show: Random-Panel and Label-Permutation Controls for a Sepsis Blood-Transcriptome Signature

  • Mikhail Kolev1,2
  • Irina Naskinova2
  • Meglena Lazarova3
  • Velin Klisurov4
  • Hristo Kalinov4
4 affiliations
  1. 1Department of Applied Computer Science and Mathematical Modelling, Faculty of Mathematics and Computer Science, University of Warmia and Mazury in Olsztyn, Słoneczna 54, 10-710 Olsztyn, Poland
  2. 2Department of Mathematics, University of Architecture, Civil Engineering and Geodesy (UACEG), 1 Hristo Smirnenski Blvd., 1164 Sofia, Bulgaria
  3. 3Department of Mathematical Modelling and Numerical Methods, Faculty of Applied Mathematics and Informatics, Technical University of Sofia, 8 “Kliment Ohridski” Blvd., 1000 Sofia, Bulgaria
  4. 4Faculty of Mathematics and Informatics, Sofia University “St. Kliment Ohridski”, 5 James Bourchier Blvd., 1164 Sofia, Bulgaria
Read full textPDFDOICite

Abstract

Sepsis blood-transcriptome signatures are usually judged against healthy controls. Two questions are rarely separated: whether the classifier exceeds chance, and whether its genes outperform an arbitrary list of equal size. We tested both on public microarray data (discovery GSE65682, n = 359; validation GSE57065, n = 53), using the 9983 of 11,222 genes measurable on both platforms. A 10-gene signature was selected by nested cross-validation and applied to validation without refitting. It separated sepsis from health perfectly (area under the receiver operating characteristic curve, AUROC, 1.000). The pipeline exceeded chance: no label permutation in 1000 matched it (p = 0.001). The genes did not outperform random lists: 12.5% of 2000 random 10-gene panels matched or exceeded the signature, as did 67.6% of panels drawn from the same 300 preselected candidates. Against non-infected intensive care patients, cross-fitted AUROC fell to 0.730 (95% confidence interval 0.642–0.818), unchanged after age and sex matching. At a threshold of 0.5, the cross-fitted model called 70 of these 75 patients septic. Selection was unstable: 200 bootstrap resamples gave a median of 6 genes, from 109 distinct genes. A near-perfect AUROC against healthy controls reflects an easy benchmark, not a specific gene list.

Related

  1. Transcriptome Analysis Identifies a Myeloid-Like Signature in Relapsed or Refractory B-Cell Acute Lymphoblastic Leukemia Patients With Poor Response to CD19 Chimeric Antigen Receptor T Cell Therapy.
    Shisen Wang, Wenbing Liu, Ying Wang +10
    Cancer medicinePeer-reviewedCC BY-NC-NDFull textEpidemiology & public healthEurope PMC
  2. Machine learning reveals a multimodal, transdiagnostic signature of emotion dysregulation vulnerability across patients, offspring, and controls
    Luigi F. Saccaro, Thomas Larrieu, Farnaz Delavari +6
    Molecular PsychiatryPeer-reviewedCC BY 4.0Full textPsychiatry & mental healthPMC
  3. Senescence-Associated Transcriptome Reversal Prioritizes Repurposing Candidates for Diabetic Kidney Disease: A Computational Study
    Duygu Aygüneş-Jafari, Zekeriya Düzgün
    BiomedicinesPeer-reviewedCC BYFull textGeneral medicineEurope PMC
  4. Single-cell transcriptome analysis highlights conserved transcriptional responses across multiple models of Retinitis Pigmentosa
    Santiago Negueruela, Martina Di Guida, Paola Brandi +1
    Genes & DiseasesPeer-reviewedCC BY 4.0Full textGeneral medicinePMC
  5. HDAC inhibitor treatment restores transcriptome, chromatin accessibility, and memory deficits in a mouse model for Down syndrome
    Cesar Sierra, René A.J. Crans, Mara Dierssen
    ISciencePeer-reviewedCC BY-NC-NDFull textEpidemiology & public healthEurope PMCCrossref

Cite

BibTeXRIS
BibTeX
@article{kolev2026what,
  title = {{What a Near-Perfect AUROC Does Not Show: Random-Panel and Label-Permutation Controls for a Sepsis Blood-Transcriptome Signature}},
  author = {Mikhail Kolev and Irina Naskinova and Meglena Lazarova and Velin Klisurov and Hristo Kalinov},
  journal = {International Journal of Molecular Sciences},
  year = {2026},
  month = oct,
  doi = {10.3390/ijms27209273},
  url = {https://www.mdpi.com/1422-0067/27/20/9273},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - What a Near-Perfect AUROC Does Not Show: Random-Panel and Label-Permutation Controls for a Sepsis Blood-Transcriptome Signature
AU  - Mikhail Kolev
AU  - Irina Naskinova
AU  - Meglena Lazarova
AU  - Velin Klisurov
AU  - Hristo Kalinov
PY  - 2026
DA  - 2026/10/01
JO  - International Journal of Molecular Sciences
DO  - 10.3390/ijms27209273
UR  - https://www.mdpi.com/1422-0067/27/20/9273
AB  - Sepsis blood-transcriptome signatures are usually judged against healthy controls. Two questions are rarely separated: whether the classifier exceeds chance, and whether its genes outperform an arbitrary list of equal size. We tested both on public microarray data (discovery GSE65682, n = 359; validation GSE57065, n = 53), using the 9983 of 11,222 genes measurable on both platforms. A 10-gene signature was selected by nested cross-validation and applied to validation without refitting. It separated sepsis from health perfectly (area under the receiver operating characteristic curve, AUROC, 1.000). The pipeline exceeded chance: no label permutation in 1000 matched it (p = 0.001). The genes did not outperform random lists: 12.5% of 2000 random 10-gene panels matched or exceeded the signature, as did 67.6% of panels drawn from the same 300 preselected candidates. Against non-infected intensive care patients, cross-fitted AUROC fell to 0.730 (95% confidence interval 0.642–0.818), unchanged after age and sex matching. At a threshold of 0.5, the cross-fitted model called 70 of these 75 patients septic. Selection was unstable: 200 bootstrap resamples gave a median of 6 genes, from 109 distinct genes. A near-perfect AUROC against healthy controls reflects an easy benchmark, not a specific gene list.
SN  - 1422-0067
ER  -