Safety-Relevant Biomedical Machine Learning Should Adopt Stability-First Reporting
- 1The Universtiy of Sydney
Abstract
Biomedical Machine Learning (ML) is increasingly evaluated through Independent and Identically Distributed (IID) benchmarks, even when deployment involves irregular observation, missingness, distribution shift, and high failure costs. This position paper argues for a stability-oriented reporting standard for safety-relevant biomedical ML. The central requirement is not a specific model class or optimizer, but auditable evidence: performance should be reported under clinically plausible perturbations such as thinning, timestamp jitter, bursty missingness, channel dropout, and Signal-to-Noise Ratio (SNR) shift. When authors invoke training-stability diagnostics, those diagnostics should be hazard-linked: their assumptions, operational meaning, and relationship to stress-test degradation should be stated explicitly. We use controlled mechanism probes to illustrate two hazards that IID scores can hide: observation-process dependence under irregular sampling, and perturbation sensitivity associated with high-curvature training regimes such as Edge of Stability (EoS). We do not propose Neural ODEs, continuous-time models, Sharpness-Aware Minimization (SAM), or EoS metrics as default prescriptions. Instead, we propose a minimum reporting standard that makes robustness claims testable and supports escalation to more complex modelling or optimization only when stress tests justify the cost.
Related
- Effect Estimates, Effect Sizes, Confidence Intervals, and P-values: A Framework for Reporting and Interpreting Biomedical Research
- Internal Evaluation of a Machine Learning-Based Random Forest Model for Clinically Relevant Postoperative Pancreatic Fistula After Pancreaticoduodenectomy: Limited Incremental Value of Inflammatory–Nutritional Markers
- A machine learning model based on mPASS and imaging features for early assessment of clinically relevant outcomes in acute pancreatitis.
- An integrated deep learning and large language model for automated knee osteoarthritis grading and reporting from the Vietnam Osteoporosis Study
- Inferring clinically relevant molecular subtypes of pancreatic cancer from routine histopathology using deep learning
BibTeX
@misc{zainal2026safetyrelevant,
title = {{Safety-Relevant Biomedical Machine Learning Should Adopt Stability-First Reporting}},
author = {Z. A. Zainal and O. Kavehei and I. Aguilar and L. F. Herbozo Contreras and Z. Huang},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.30.26364447},
url = {https://www.medrxiv.org/content/10.64898/2026.09.30.26364447v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Safety-Relevant Biomedical Machine Learning Should Adopt Stability-First Reporting
AU - Z. A. Zainal
AU - O. Kavehei
AU - I. Aguilar
AU - L. F. Herbozo Contreras
AU - Z. Huang
PY - 2026
DA - 2026/09/30
PB - medRxiv
DO - 10.64898/2026.09.30.26364447
UR - https://www.medrxiv.org/content/10.64898/2026.09.30.26364447v1
AB - Biomedical Machine Learning (ML) is increasingly evaluated through Independent and Identically Distributed (IID) benchmarks, even when deployment involves irregular observation, missingness, distribution shift, and high failure costs. This position paper argues for a stability-oriented reporting standard for safety-relevant biomedical ML. The central requirement is not a specific model class or optimizer, but auditable evidence: performance should be reported under clinically plausible perturbations such as thinning, timestamp jitter, bursty missingness, channel dropout, and Signal-to-Noise Ratio (SNR) shift. When authors invoke training-stability diagnostics, those diagnostics should be hazard-linked: their assumptions, operational meaning, and relationship to stress-test degradation should be stated explicitly. We use controlled mechanism probes to illustrate two hazards that IID scores can hide: observation-process dependence under irregular sampling, and perturbation sensitivity associated with high-curvature training regimes such as Edge of Stability (EoS). We do not propose Neural ODEs, continuous-time models, Sharpness-Aware Minimization (SAM), or EoS metrics as default prescriptions. Instead, we propose a minimum reporting standard that makes robustness claims testable and supports escalation to more complex modelling or optimization only when stress tests justify the cost.
ER -