A Cross-Dataset Audit of Discrimination and Conformal Prediction Coverage in Keystroke-Based Parkinson's Disease Classi cation
- 1Inner Mongolia University
Abstract
Typing dynamics are being investigated as a digital behavioral marker for Parkinson's disease (PD), but the cross-dataset reliability of typing-based classi ers remains uncertain. We evaluated four public datasets (475 par ticipants; 270 with PD and 205 controls) across 12 transfer directions using split conformal prediction targeting 90% marginal coverage. In all three pri mary directions, coverage was below the 0.90 target and the one-sided tests remained signi cant after Holm correction: 0.598 (95% CI, 0.4830.704) for Tappy[->]MIT, 0.850 (0.7880.899) for MIT[->]Tappy, and 0.732 (0.6840.777) for clinical[->]self-reported transfer. Discrimination and coverage did not con sistently change together: LightGBM had lower AUC than logistic regression in all 12 directions but higher coverage in 11, with larger prediction sets. Class-conditional coverage varied by target cohort: PD-conditional cover age was 0.214 for Tappy[->]MIT, whereas healthy-control coverage was 0.700 for MIT[->]Tappy and 0.627 for clinical[->]self-reported transfer. Recalibration with 9 and 18 labeled target participants raised the lower condence bound above 0.80 in the latter two directions; in Tappy[->]MIT, it remained below 0.80 even with 25 labels. Weighted conformal prediction yielded coverage of 0.9181.000 without target labels but returned the full label set for 86% 100% of participants. Stress tests did not reproduce the observed coverage patterns, and black-box shift estimation did not support pure label shift as the dominant explanation for Tappy[->]MIT. Evaluating typing dynamics as a digital marker for PD requires cross-dataset assessment of discrimination, marginal and class-conditional coverage, and recovery costs. Clinical screen ing utility has not been established.
Related
- Speech markers of depression: cross-sectional discrimination, longitudinal association, and treatment response prediction
- Development and Population-Based Prognostic Utility Assessment of an ICD-10-CM-Based Prediction Model to Identify Left Ventricular Systolic Dysfunction: Discrimination Performance and Population-Based Evaluation in 83,357 Patients with Acute Coronary Syndrome
- Prediction of cognitive impairment in Parkinson’s disease using TabPFN: an integrative analysis of clinical data, longitudinal trajectories, and lipidomics
- The coverage gap in PREVENT: Why cardiovascular risk prediction needs a lab-free complement.
- Validity Gates Expose Limits of Longitudinal Multimodal Parkinson Disease Prediction
BibTeX
@misc{guang2026crossdataset,
title = {{A Cross-Dataset Audit of Discrimination and Conformal Prediction Coverage in Keystroke-Based Parkinson's Disease Classi cation}},
author = {H. Guang and Z. Zhang and J. Qu and J. Ma and J. Wu},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.04.26366890},
url = {https://www.medrxiv.org/content/10.64898/2026.11.04.26366890v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - A Cross-Dataset Audit of Discrimination and Conformal Prediction Coverage in Keystroke-Based Parkinson's Disease Classi cation
AU - H. Guang
AU - Z. Zhang
AU - J. Qu
AU - J. Ma
AU - J. Wu
PY - 2026
DA - 2026/11/04
PB - medRxiv
DO - 10.64898/2026.11.04.26366890
UR - https://www.medrxiv.org/content/10.64898/2026.11.04.26366890v1
AB - Typing dynamics are being investigated as a digital behavioral marker for Parkinson's disease (PD), but the cross-dataset reliability of typing-based classi ers remains uncertain. We evaluated four public datasets (475 par ticipants; 270 with PD and 205 controls) across 12 transfer directions using split conformal prediction targeting 90% marginal coverage. In all three pri mary directions, coverage was below the 0.90 target and the one-sided tests remained signi cant after Holm correction: 0.598 (95% CI, 0.4830.704) for Tappy[->]MIT, 0.850 (0.7880.899) for MIT[->]Tappy, and 0.732 (0.6840.777) for clinical[->]self-reported transfer. Discrimination and coverage did not con sistently change together: LightGBM had lower AUC than logistic regression in all 12 directions but higher coverage in 11, with larger prediction sets. Class-conditional coverage varied by target cohort: PD-conditional cover age was 0.214 for Tappy[->]MIT, whereas healthy-control coverage was 0.700 for MIT[->]Tappy and 0.627 for clinical[->]self-reported transfer. Recalibration with 9 and 18 labeled target participants raised the lower condence bound above 0.80 in the latter two directions; in Tappy[->]MIT, it remained below 0.80 even with 25 labels. Weighted conformal prediction yielded coverage of 0.9181.000 without target labels but returned the full label set for 86% 100% of participants. Stress tests did not reproduce the observed coverage patterns, and black-box shift estimation did not support pure label shift as the dominant explanation for Tappy[->]MIT. Evaluating typing dynamics as a digital marker for PD requires cross-dataset assessment of discrimination, marginal and class-conditional coverage, and recovery costs. Clinical screen ing utility has not been established.
ER -