Random-split validation overstates the skill of machine-learning models of global coral bleaching severity
Abstract
Global compilations of coral bleaching surveys are increasingly used to train machine-learning models of bleaching severity, but such models are often evaluated with random train–test splits that ignore duplicated records, spatial dependence and differences among contributing survey programmes. We re-examined an earlier analysis of the Global Coral Bleaching Database (BCO-DMO dataset 773466) that reported a random-split random forest R² of 0.688. The result was reproducible, but it was obtained on 32,716 records, 34.5% of which duplicated another record exactly because Reef Check surveys are stored once per substrate category. After collapsing records to 21,527 sampling events, we compared random, site-grouped, spatial-block (1–10°), leave-one-realm-out and forward temporal validation for models using coordinates, thermal-stress metrics, other environmental variables, or indicators of the contributing data source. Random-forest R² fell from 0.60 (random) to 0.53 (new sites), 0.18 (1° blocks) and 0.02 (10° blocks), and was at most 0.06 in held-out realms. Coordinates alone predicted worse than the mean under all block designs. Indicators of the contributing source alone explained 0.35–0.37 of the variance under every cross-validation design, more than any environmental model under spatial validation. Within the single-protocol Reef Check subset, no predictor set achieved positive R² under spatial blocks. The all-predictor model retained moderate ability to rank events with at least 50% bleaching (AUC 0.73 under 5° blocks). These results indicate that the apparent skill of pooled bleaching-severity models largely reflects record duplication, interpolation between nearby surveys, and heterogeneity in how bleaching was measured, rather than transferable environment–bleaching relationships.
Related
- Prediction of late blight severity in a large panel of potato genotypes using low-altitude aerial images and machine learning methods
- Admission immune-inflammatory biomarker profiles for trauma severity stratification: an externally validated machine learning study
- Development and External Validation of an M2 Macrophage-Related Prognostic Signature in Breast Cancer Using Multi-Omics Data and Machine Learning.
- Rigorous nested cross-validation and external generalization-gap analysis of a machine-learning classifier for U-251 glioblastoma cytotoxicity
- Integrative network pharmacology, machine learning, and experimental validation reveal the protective potential and potential molecular mechanisms of orientin in DSS-induced colitis
BibTeX
@misc{mestry2026randomsplit,
title = {{Random-split validation overstates the skill of machine-learning models of global coral bleaching severity}},
author = {Arav Siddharth Mestry},
howpublished = {EarthArXiv},
year = {2026},
month = oct,
doi = {10.31223/x56c02},
url = {https://eartharxiv.org/repository/view/15638/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Random-split validation overstates the skill of machine-learning models of global coral bleaching severity
AU - Arav Siddharth Mestry
PY - 2026
DA - 2026/10/21
PB - EarthArXiv
DO - 10.31223/x56c02
UR - https://eartharxiv.org/repository/view/15638/
AB - Global compilations of coral bleaching surveys are increasingly used to train machine-learning models of bleaching severity, but such models are often evaluated with random train–test splits that ignore duplicated records, spatial dependence and differences among contributing survey programmes. We re-examined an earlier analysis of the Global Coral Bleaching Database (BCO-DMO dataset 773466) that reported a random-split random forest R² of 0.688. The result was reproducible, but it was obtained on 32,716 records, 34.5% of which duplicated another record exactly because Reef Check surveys are stored once per substrate category. After collapsing records to 21,527 sampling events, we compared random, site-grouped, spatial-block (1–10°), leave-one-realm-out and forward temporal validation for models using coordinates, thermal-stress metrics, other environmental variables, or indicators of the contributing data source. Random-forest R² fell from 0.60 (random) to 0.53 (new sites), 0.18 (1° blocks) and 0.02 (10° blocks), and was at most 0.06 in held-out realms. Coordinates alone predicted worse than the mean under all block designs. Indicators of the contributing source alone explained 0.35–0.37 of the variance under every cross-validation design, more than any environmental model under spatial validation. Within the single-protocol Reef Check subset, no predictor set achieved positive R² under spatial blocks. The all-predictor model retained moderate ability to rank events with at least 50% bleaching (AUC 0.73 under 5° blocks). These results indicate that the apparent skill of pooled bleaching-severity models largely reflects record duplication, interpolation between nearby surveys, and heterogeneity in how bleaching was measured, rather than transferable environment–bleaching relationships.
ER -