Skip to content
DraconiticResearch
Search Sources API
203,203 studies31 sourcesUpdated 6 min ago
Peer-reviewedCC BYNpj health systems

Paradoxical inflation of prediction model performance measures in minority patient subgroups: the role of upstream bias.

  • Rahul Ladhania1,2,3
  • Allister Z Ho4
  • Paramveer Dhillon5
  • Chad M Brummett6,7,8
  • Karandeep Singh9,10
  • Anne C Fernandez11
11 affiliations
  1. 1Department of Health Management & Policy, School of Public Health, University of Michigan, Ann Arbor, MI, USA. [email protected].
  2. 2Department of Biostatistics, School of Public Health, University of Michigan, Ann Arbor, MI, USA. [email protected].
  3. 3Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA. [email protected].
  4. 4Department of Bioinformatics, University of Michigan Medical School, Ann Arbor, MI, USA.
  5. 5School of Information, University of Michigan, Ann Arbor, MI, USA.
  6. 6Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA.
  7. 7Division of Pain Medicine, Department of Anesthesiology, University of Michigan Medical School, Ann Arbor, MI, USA.
  8. 8Opioid Research Institute, University of Michigan, Ann Arbor, MI, USA.
  9. 9Joan and Irwin Jacobs Center for Health Innovation, University of California, San Diego, CA, USA. [email protected].
  10. 10Division of Biomedical Informatics, Department of Medicine, University of California, San Diego, CA, USA. [email protected].
  11. 11Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI, USA.
Read full textPDFDOICite

Abstract

Predictive models in healthcare AI are expected to perform worse in underrepresented subgroups due to upstream biases such as underrepresentation, differential care access, and potential measurement error. In a previously published model predicting persistent postoperative opioid use, we surprisingly observed substantially higher model performance in Black patients (which comprised less than 5% of the sample) than in White patients. In this study, we used subgroup matching to investigate whether differences in patient characteristics contributed to the observed disparity, and replicate our findings in a second independent model. Across both models, performance differences - measured by AUC and other performance metrics - were attenuated after matching, suggesting that upstream biases may paradoxically inflate apparent model performance in minoritized patient groups. These findings challenge conventional assumptions about bias and highlight the importance of investigating performance differences using subgroup-level covariate alignment. Our results emphasize the need for careful evaluation of subgroup model metrics to ensure fair and trustworthy deployment of clinical AI.

Related

  1. Identifying subgroups with decreased performance on malaria rapid diagnostic tests in Malawi
    Geoffrey Guenther, Alick Sixpence, Charles Mangani +8
    PLOS Global Public HealthPeer-reviewedCC BY 4.0Full textMolecular biology & gene regulationGeneral biologyPLOSPMC
  2. The Impact of Age and Physical Assessment Measures on Performance Among Non-Weightbearing Walking Aids: A Randomized Clinical Trial
    Audrey Zucker-Levin, Lucy Senger
    HSS journal : the musculoskeletal journal of Hospital for Special SurgeryPeer-reviewedCC BYFull textSurgery & anaesthesiaEurope PMC
  3. Motor and non-motor symptoms impact on different domains of capacity- and performance-related mobility measures in Parkinson’s disease
    Edoardo Bianchini, Pascal Petit, Domiziana Rinaldi +10
    Npj Parkinson's DiseasePeer-reviewedCC BYFull textNeuroscienceEpidemiology & public healthHAL
  4. Association Between Pain and Functional Disability in Patients with Shoulder Tendinopathy: A Cross-Sectional Study Using Patient-Reported and Performance-Based Measures.
    Stefanos Drimytakis, Panagiotis-Nikolaos Vlachogiorgis, Anna Christakou
    Muscles (Basel, Switzerland)Peer-reviewedCC BYFull textSurgery & anaesthesiaEurope PMC
  5. Performance and response time in emotion recognition and mentalizing measures in patients with schizophrenia.
    Alicia Ruiz-Toca, Alba Hernández-Gómez, Nelson Andrade-González +5
    BMC psychiatryPeer-reviewedCC BY-NC-NDFull textPsychiatry & mental healthEurope PMC

Cite

BibTeXRIS
BibTeX
@article{ladhania2026paradoxical,
  title = {{Paradoxical inflation of prediction model performance measures in minority patient subgroups: the role of upstream bias.}},
  author = {Rahul Ladhania and Allister Z Ho and Paramveer Dhillon and Chad M Brummett and Karandeep Singh and Anne C Fernandez},
  journal = {Npj health systems},
  year = {2026},
  month = oct,
  doi = {10.1038/s44401-026-00158-0},
  url = {https://europepmc.org/article/MED/42907563},
  note = {CC BY},
}
RIS
TY  - JOUR
TI  - Paradoxical inflation of prediction model performance measures in minority patient subgroups: the role of upstream bias.
AU  - Rahul Ladhania
AU  - Allister Z Ho
AU  - Paramveer Dhillon
AU  - Chad M Brummett
AU  - Karandeep Singh
AU  - Anne C Fernandez
PY  - 2026
DA  - 2026/10/21
JO  - Npj health systems
DO  - 10.1038/s44401-026-00158-0
UR  - https://europepmc.org/article/MED/42907563
AB  - Predictive models in healthcare AI are expected to perform worse in underrepresented subgroups due to upstream biases such as underrepresentation, differential care access, and potential measurement error. In a previously published model predicting persistent postoperative opioid use, we surprisingly observed substantially higher model performance in Black patients (which comprised less than 5% of the sample) than in White patients. In this study, we used subgroup matching to investigate whether differences in patient characteristics contributed to the observed disparity, and replicate our findings in a second independent model. Across both models, performance differences - measured by AUC and other performance metrics - were attenuated after matching, suggesting that upstream biases may paradoxically inflate apparent model performance in minoritized patient groups. These findings challenge conventional assumptions about bias and highlight the importance of investigating performance differences using subgroup-level covariate alignment. Our results emphasize the need for careful evaluation of subgroup model metrics to ensure fair and trustworthy deployment of clinical AI.
SN  - 3005-1959
ER  -