Towards reliable data augmentation in machine learning: Practices to prevent data leakage
- 1Cologne Lab for Artificial Intelligence and Smart Automation (CAISA), Institute of Product Development and Engineering Design (IPK), TH Köln – University of Applied Sciences, Cologne, Germany, Betzdorfer Straße 2, 50679, Cologne, Germany
Abstract
Data leakage, the unintentional use of illegitimate information during model training, fundamentally undermines the validity of machine learning evaluations. While well-documented in theory, a critical and pervasive manifestation, improper sequencing of data augmentation relative to dataset splitting remains a frequently overlooked issue in practice. This pre-split application of augmentation violates the core independence assumption between training and test data, allowing models to memorize augmentation artifacts rather than learn generalizable features. Through rigorous reproducibility case studies in agricultural computer vision, we quantify the severe inflationary effect of this methodological flaw. Re-evaluating the work of Joshi et al. with a correct split-then-augment protocol reveals a performance drop from 98.70% to 32.77% mAP50, a reduction of 65.93 percentage points. A second case study on rice pest detection (Yin et al.) confirms this pattern, with a performance decrease from 99.5% to 89.1% after correcting for data leakage and near-duplicates.This analysis identifies two distinct leakage pathways, categorized as process-induced (improper augmentation sequence) and data-inherent (undetected duplicates in benchmarks). Contextualizing our findings within broader literature, we show how this error is perpetuated by educational gaps, tool design, and academic incentives. To mitigate these risks, a comprehensive set of actionable best practices for researchers, reviewers and educators is proposed. These include rigorous data integrity protocols, enhanced methodological checklists in peer reviews and a paradigm shift towards pitfalls-based learning. Our work underscores that preventing data leakage is not merely a technicality but a prerequisite for reliable and reproducible machine learning, with implications for computer vision and beyond.
Related
- An optimized machine learning approach for reliable agronomic parameter prediction in precision farming systems
- Determining Reliable Neural Targets for enhancing Motor Sequence Learning: A preparatory study for focal tDCS
- A method for brain tumor detection and localization by means of deep learning and reliable prediction explainability
- Regional climate variability and rice yields prediction in Bangladesh using machine learning and deep learning models
- Window leakage in clinical EEG machine learning: matched subject-independent evaluation across ADHD and Parkinson cohorts
BibTeX
@article{gerz2026towards,
title = {{Towards reliable data augmentation in machine learning: Practices to prevent data leakage}},
author = {Fabian Gerz and Mohieddine Jelali},
journal = {Smart Agricultural Technology},
year = {2026},
month = aug,
doi = {10.1016/j.atech.2026.102462},
url = {https://doi.org/10.1016/j.atech.2026.102462},
note = {CC BY},
}
RIS
TY - JOUR
TI - Towards reliable data augmentation in machine learning: Practices to prevent data leakage
AU - Fabian Gerz
AU - Mohieddine Jelali
PY - 2026
DA - 2026/08/10
JO - Smart Agricultural Technology
DO - 10.1016/j.atech.2026.102462
UR - https://doi.org/10.1016/j.atech.2026.102462
AB - Data leakage, the unintentional use of illegitimate information during model training, fundamentally undermines the validity of machine learning evaluations. While well-documented in theory, a critical and pervasive manifestation, improper sequencing of data augmentation relative to dataset splitting remains a frequently overlooked issue in practice. This pre-split application of augmentation violates the core independence assumption between training and test data, allowing models to memorize augmentation artifacts rather than learn generalizable features. Through rigorous reproducibility case studies in agricultural computer vision, we quantify the severe inflationary effect of this methodological flaw. Re-evaluating the work of Joshi et al. with a correct split-then-augment protocol reveals a performance drop from 98.70% to 32.77% mAP50, a reduction of 65.93 percentage points. A second case study on rice pest detection (Yin et al.) confirms this pattern, with a performance decrease from 99.5% to 89.1% after correcting for data leakage and near-duplicates.This analysis identifies two distinct leakage pathways, categorized as process-induced (improper augmentation sequence) and data-inherent (undetected duplicates in benchmarks). Contextualizing our findings within broader literature, we show how this error is perpetuated by educational gaps, tool design, and academic incentives. To mitigate these risks, a comprehensive set of actionable best practices for researchers, reviewers and educators is proposed. These include rigorous data integrity protocols, enhanced methodological checklists in peer reviews and a paradigm shift towards pitfalls-based learning. Our work underscores that preventing data leakage is not merely a technicality but a prerequisite for reliable and reproducible machine learning, with implications for computer vision and beyond.
SN - 2772-3755
ER -