{ "abstract": "Reinforcement learning (RL) is a type of artificial intelligence for making optimal choices. In healthcare, researchers generally use offline RL (ORL), where models are trained and evaluated from retrospective observational data. To accommodate naturally irregular clinical records, researchers often resample the data into uniform time intervals before training (known as binning). However, binning may create counterfactual trajectories that do not represent clinical practice. We first quantified these distortions in patients receiving continuous insulin infusions in the MIMIC-IV intensive care database. Binning reversed the apparent causal order for over 70% of paired glucose measurements and insulin adjustments within the same interval, and frequently masked significant dysglycaemic events and merged distinct treatment decisions together. We then conducted an in silico clinical trial using 30 virtual patients with type 1 diabetes from the FDA-approved UVA/Padova simulator. The simulator was modified to include stochastic intervals between decisions and used to generate a training dataset for offline RL. We trained three ORL algorithms on both the unprocessed dataset and equivalent datasets resampled at 10-minute, 2-hour, and 4-hour intervals. When deployed back into the simulated environment, temporal resampling was found to reduce model performance by up to 60% relative to unprocessed data, with 4-hour binning causing all agents to perform worse than the dataset’s baseline. Retrospective evaluation on resampled data actively obscured this effect, predicting 1.5–3x better returns than agents achieved in practice. We recommend that future research in this area prioritises datasets with natural clinical timings between decisions, which may be a necessary step before these models can be safely deployed into patient care. Author summary: Artificial Intelligence (AI) models could help future doctors by recommending personalised treatments for patients. These models are currently trained and tested using historical patient records rather than real patients. Medical decisions happen at unpredictable intervals, but before training an AI model, researchers usually shift and align the timings of recorded decisions into a regular schedule (‘temporal resampling’). This makes training the AI models easier with standard techniques. We investigated whether this practice leads to lower quality models that would perform poorly in the real clinical world. We ran a virtual clinical trial using an FDA-approved diabetes simulator to test how well models that had been trained on resampled data perform in environments with unpredictable decision intervals. We found that models trained on artificially regular decisions perform worse than expected when trialled in environments with these irregular timings. Concerningly, standard tests used to predict model safety before clinical use do not detect this, producing falsely optimistic scores. We then assessed models trained on natural, unedited data. These models performed much better and the safety tests accurately predicted their true performance. We therefore recommend training and evaluating AI models using unedited data to preserve the authentic timing of clinical practice.", "abstract_shown": true, "affiliations": [ "University College London, London, United Kingdom", "University College London Hospitals NHS Foundation Trust, London, United Kingdom" ], "article_type": "Research Article", "arxiv": null, "author_total": 3, "authors": [ { "affs": [ 0 ], "name": "Thomas Frost" }, { "affs": [ 1 ], "name": "Hrisheekesh Vaidya" }, { "name": "Steve Harris" } ], "bibtex": "https://research.draconitic.com/paper/c2zfwhh8c86k0.bib", "cited_by": 2, "doi": "10.1371/journal.pdig.0001801", "fulltext_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001801", "id": "c2zfwhh8c86k0", "issns": [ "2767-3170" ], "landing_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001801", "language": null, "licence": "CC-BY-4.0", "licence_label": "CC BY 4.0", "licence_source": "journal", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "openalex", "1702|Machine Learning in Healthcare" ], [ "plos", "/Biology and life sciences/Anatomy/Body fluids/Blood/Blood sugar" ], [ "plos", "/Biology and life sciences/Biochemistry/Hormones/Insulin" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Learning" ], [ "plos", "/Biology and life sciences/Neuroscience/Learning and memory/Learning" ], [ "plos", "/Biology and life sciences/Physiology/Body fluids/Blood/Blood sugar" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Learning" ], [ "plos", "/Computer and information sciences/Artificial intelligence" ], [ "plos", "/Medicine and health sciences/Anatomy/Body fluids/Blood/Blood sugar" ], [ "plos", "/Medicine and health sciences/Endocrinology/Diabetic endocrinology/Insulin" ], [ "plos", "/Medicine and health sciences/Endocrinology/Endocrine disorders/Diabetes mellitus" ], [ "plos", "/Medicine and health sciences/Epidemiology/Medical risk factors" ], [ "plos", "/Medicine and health sciences/Medical conditions/Metabolic disorders/Diabetes mellitus" ], [ "plos", "/Physical sciences/Chemistry/Chemical compounds/Organic compounds/Carbohydrates/Monosaccharides/Glucose" ], [ "plos", "/Physical sciences/Chemistry/Organic chemistry/Organic compounds/Carbohydrates/Monosaccharides/Glucose" ], [ "plos", "/Physical sciences/Mathematics/Applied mathematics/Algorithms" ], [ "plos", "/Research and analysis methods/Simulation and modeling/Algorithms" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Learning" ] ], "openalex": "W7128383130", "pdf_url": "https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0001801&type=printable", "pmcid": null, "pmid": null, "published": "2026-11-06", "publisher": "Public Library of Science", "ris": "https://research.draconitic.com/paper/c2zfwhh8c86k0.ris", "sources": [ { "first_seen": "2026-11-07T12:20:24Z", "native_id": "10.1371/journal.pdig.0001801", "source": "plos", "url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001801" } ], "subjects": [ { "field": "biology", "origin": "plos:neuroscience", "subfield": "neuroscience" }, { "field": "medicine", "origin": "plos:endocrinology", "subfield": "endocrinology-metabolism" }, { "field": "medicine", "origin": "plos:anatomy", "subfield": "physiology" } ], "title": "The impact of temporal resampling on the safety of clinical reinforcement learning: A simulated trial", "title_text": "The impact of temporal resampling on the safety of clinical reinforcement learning: A simulated trial", "type": "peer-reviewed", "updated_at": "2026-11-08T12:20:48Z", "url": "https://research.draconitic.com/paper/c2zfwhh8c86k0", "venue": "PLOS Digital Health", "versions": [] }