{ "abstract": "Serial cognitive assessment in multiple sclerosis can help identify patients at higher risk of disability progression, but high dimensionality of reaction-time data makes analysis challenging. We developed a method to convert longitudinal reaction-time data into images and applied deep survival modelling to predict disability progression in relapsing-remitting MS (RRMS). In this cohort study, clinical data were obtained from the MSBase registry and cognitive data from the MSReactor computerised cognitive battery between February 2016 and September 2022, with a median follow-up of 3.2 years. The serial reaction-time data from three tasks (psychomotor function [R], attention [G], and working memory [B]) were converted into multicolour images using RGB encoding. We extracted the key features from the images using a convolutional neural network and combined them with clinical variables in a Transformer-based survival model (MS-TranSurv) to predict time to confirmed disability progression. Discrimination, calibration, and accuracy were assessed using the C-index, integrated Brier score (iBS), and time-dependent area under the receiver operating characteristic (tAUROC). Performance was compared with other machine learning models, including Dynamic DeepHit, Recurrent Deep Survival Machines, a Cox proportional hazards model in traditional statistics using clinical variables only, and a version of MS-TransSurv using mean test values. A total of 746 RRMS patients were included. MS-TranSurv showed slightly higher discrimination compared with benchmark models, with a C-index of 0.61 (95% CI 0.54–0.68) and tAUROC of 0.74 (95% CI 0.63–0.85), as well as comparable calibration, with an iBS of 0.24 (95% CI 0.16–0.32). Using individual test-level data provided slightly better performance than using summary measures. Longitudinal cognitive reaction-time data can be used for survival-based prediction of disability progression in RRMS. This framework supports remote cognitive monitoring and may improve risk stratification and clinical trial enrichment, with potential applicability to other high-dimensional digital biomarkers. Author summary: People with multiple sclerosis often experience changes in thinking speed, attention, and memory, sometimes years before clear physical disability appears. Digital cognitive tests make it possible to measure these changes repeatedly, including at home, but the large amount of data they produce is difficult to analyse and is often simplified in ways that may lose important information. In this study, we developed a new approach to make better use of these data. We transformed repeated reaction-time measurements from simple cognitive tasks into colour-based images that capture patterns of performance over time. We then used a machine learning method designed to analyse changes over time to estimate the risk of future disability progression. By keeping more detailed information from each test, our approach performed better than methods that rely on summary measures. Our findings show that repeated, low-burden cognitive testing can be used to help identify people with multiple sclerosis who are at higher risk of worsening disability. This approach could support earlier clinical review, improve the selection of participants for clinical trials, and may be adapted for use with other types of digital health data in the future.", "abstract_shown": true, "affiliations": [ "Department of Neuroscience, School of Translational Medicine, Monash University, Melbourne, Victoria, Australia", "Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, Victoria, Australia", "AIM for Health Lab, Faculty of IT, Monash University, Melbourne, Victoria, Australia", "Discipline of Pediatric and Child Health, School of Clinical Medicine, University of New South Wales, Sydney, New South Wales, Australia", "Department of Neurology, Alfred Hospital, Melbourne, Victoria, Australia", "Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia", "Department of Neurology, Neuroimmunology Centre, Royal Melbourne Hospital, Melbourne, Victoria, Australia", "CORe, Department of Medicine, University of Melbourne, Melbourne, Victoria, Australia", "Hunter New England Health, New Lambton, New South Wales, Australia", "University of Newcastle, Newcastle, New South Wales, Australia", "Eastern Health Clinical School, Monash University, Box Hill, Victoria, Australia", "Florey Department of Neuroscience and Mental Health, University of Melbourne, Melbourne, Victoria, Australia", "Brain and Mind Centre, University of Sydney, Sydney, New South Wales, Australia", "Sydney Neuroimaging Analysis Centre, Camperdown, New South Wales, Australia", "School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia", "Department of Data Science and AI, Faculty of IT, Monash University, Melbourne, Victoria, Australia", "School of Computing Technologies, RMIT University, Victoria, Australia", "Faculty of IT, Monash-Airdoc Research Center, Monash University, Melbourne, Victoria, Australia" ], "article_type": "Research Article", "arxiv": null, "author_total": 18, "authors": [ { "affs": [ 0 ], "name": "Chao Zhu" }, { "affs": [ 1 ], "name": "Xin Zhang" }, { "affs": [ 2 ], "name": "Anneke van der Walt" }, { "affs": [ 3 ], "name": "Bruce Taylor" }, { "affs": [ 4 ], "name": "Mastura Monif" }, { "affs": [ 5 ], "name": "Tomas Kalincik" }, { "affs": [ 6 ], "name": "Jeanette Lechner-Scott" }, { "affs": [ 7 ], "name": "Katherine Buzzard" }, { "affs": [ 8 ], "name": "Trevor Kilpatrick" }, { "affs": [ 9 ], "name": "Michael Barnett" }, { "affs": [ 10 ], "name": "Zhen Zhou" }, { "affs": [ 11 ], "name": "Vilija Jokubaitis" }, { "affs": [ 12 ], "name": "Melissa Gresle" }, { "affs": [ 13 ], "name": "David Darby" }, { "affs": [ 14 ], "name": "Deval Mehta" }, { "affs": [ 15 ], "name": "Zongyuan Ge" }, { "affs": [ 16 ], "name": "Helmut Butzkueven" }, { "affs": [ 17 ], "name": "Daniel Merlo" } ], "bibtex": "https://research.draconitic.com/paper/ajmfgpfmdbnhs.bib", "cited_by": 0, "doi": "10.1371/journal.pdig.0001722", "fulltext_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001722", "id": "ajmfgpfmdbnhs", "issns": [ "2767-3170" ], "landing_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001722", "language": null, "licence": "CC-BY-4.0", "licence_label": "CC BY 4.0", "licence_source": "journal", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "openalex", "2734|Multiple Sclerosis Research Studies" ], [ "plos", "/Biology and life sciences/Biochemistry/Biomarkers" ], [ "plos", "/Biology and life sciences/Immunology/Clinical immunology/Autoimmune diseases/Multiple sclerosis" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive neuroscience/Reaction time" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognition" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive neuroscience/Reaction time" ], [ "plos", "/Computer and information sciences/Artificial intelligence/Machine learning" ], [ "plos", "/Medicine and health sciences/Clinical medicine/Clinical immunology/Autoimmune diseases/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Diagnostic medicine/Diagnostic radiology/Magnetic resonance imaging" ], [ "plos", "/Medicine and health sciences/Epidemiology/Medical risk factors" ], [ "plos", "/Medicine and health sciences/Immunology/Clinical immunology/Autoimmune diseases/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Medical conditions/Demyelinating disorders/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Medical conditions/Neurodegenerative diseases/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Neurology/Demyelinating disorders/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Neurology/Neurodegenerative diseases/Multiple sclerosis" ], [ "plos", "/Medicine and health sciences/Radiology and imaging/Diagnostic radiology/Magnetic resonance imaging" ], [ "plos", "/Research and analysis methods/Imaging techniques/Diagnostic radiology/Magnetic resonance imaging" ], [ "plos", "/Research and analysis methods/Research design/Cohort studies" ] ], "openalex": "W7214274471", "pdf_url": "https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0001722&type=printable", "pmcid": null, "pmid": null, "published": "2026-09-25", "publisher": "Public Library of Science", "ris": "https://research.draconitic.com/paper/ajmfgpfmdbnhs.ris", "sources": [ { "first_seen": "2026-10-30T12:17:43Z", "native_id": "10.1371/journal.pdig.0001722", "source": "plos", "url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001722" } ], "subjects": [ { "field": "biology", "origin": "plos:immunology", "subfield": "immunology" }, { "field": "biology", "origin": "plos:cognitive neuroscience", "subfield": "neuroscience" }, { "field": "medicine", "origin": "plos:diagnostic radiology", "subfield": "imaging" } ], "title": "Strategies for integrating artificial intelligence and cognitive assessment to predict disability progression in relapsing-remitting multiple sclerosis: A model development study", "title_text": "Strategies for integrating artificial intelligence and cognitive assessment to predict disability progression in relapsing-remitting multiple sclerosis: A model development study", "type": "peer-reviewed", "updated_at": "2026-10-30T12:17:43Z", "url": "https://research.draconitic.com/paper/ajmfgpfmdbnhs", "venue": "PLOS Digital Health", "versions": [] }