Foundation-model-based multi-label phenotyping of combined hyperkinetic movement disorders
Abstract
Movement disorders (MDs) frequently co-occur, yet phenomenological and severity assessment shows substantial inter-rater variability. Markerless video could improve reproducibility, but prior work is largely single-symptom, depends on standardized acquisition, and lacks validation and transfer across ages and sites. We combined two foundation models into one frozen backbone: Segment Anything Model 3 (SAM 3) for dense, per-frame markerless segmentation summarized into geometric, contour and grid kinematic signals, and TabICLv2, a tabular foundation model, for in-context multi-label classification of eight hyperkinetic MD phenomenologies. Trained on standardized recordings of 21 adults and 4 controls, it transferred unchanged to two independent datasets, pediatric (n=12) and tremor-dominant adult (n=20), assessed with the CODY-SAMP scale; only the patient-level decision step was recalibrated per site. Under clinician consensus labels, false positives fell to zero in both datasets. Dystonia recovered perfectly (7/7 pediatric; 15/15 adult held-out), chorea fully in children (3/3), and tremor was recovered in adults (11/15) once a tremor-rich cohort made it evaluable, through recalibration alone. Per-region effect-size analysis gave clinically coherent, phenomenology-specific signals and identified myoclonus as the principal failure. Against YOLOv8 sparse keypoints, the dense representation matched under clinician permissive labels (Jaccard 0.63 vs 0.63) and was markedly more robust under clinician-label consensus (0.93 vs 0.76). This frozen foundation-model backbone with light per-site calibration yields transferable, interpretable, conservative multi-label phenotyping of co-occurring hyperkinetic MDs across ages and from standardized to routine video, adding robustness on high-confidence, clinician-agreed labels. Prospective multi-centre validation is required before clinical use.
Related
- Simultaneous hyperkinetic movement disorders phenotyping: a cross-cohort pediatric transfer study using routine videos, markerless pose estimation and a tabular foundation model
- Deep Learning Pose Estimation for Phenotyping of Co‐Occurring Hyperkinetic Movement Disorders
- Translational neuroscience in psychoeducation: Mapping nonmotor circuitry to clinical management in movement disorders
- Special Issue Editorial: Basal Ganglia/Movement Disorders
- Toward Increasing Cost‐Effectiveness of Genetic Testing in Movement Disorders
BibTeX
@misc{cif2026foundationmodelbased,
title = {{Foundation-model-based multi-label phenotyping of combined hyperkinetic movement disorders}},
author = {Laura Cif and Zohra Souei and Diane Demailly and Mayte Castro-Jimenez and Juan Dario Ortigoza-Escobar and Muhammad Mushhood Ur Rehman and Morgan Dornadic and Sophie Huby and Gun-Marie Hariz and Cecile Hubsch and Nathalie Dorison and Eduardo M. Moraud and Jocelyne Bloch and Gabriella Horvath and Olivier Oullier and Xavier Vasques},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.22369},
eprint = {2609.22369},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22369},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Foundation-model-based multi-label phenotyping of combined hyperkinetic movement disorders
AU - Laura Cif
AU - Zohra Souei
AU - Diane Demailly
AU - Mayte Castro-Jimenez
AU - Juan Dario Ortigoza-Escobar
AU - Muhammad Mushhood Ur Rehman
AU - Morgan Dornadic
AU - Sophie Huby
AU - Gun-Marie Hariz
AU - Cecile Hubsch
AU - Nathalie Dorison
AU - Eduardo M. Moraud
AU - Jocelyne Bloch
AU - Gabriella Horvath
AU - Olivier Oullier
AU - Xavier Vasques
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.22369
UR - https://arxiv.org/abs/2609.22369
AB - Movement disorders (MDs) frequently co-occur, yet phenomenological and severity assessment shows substantial inter-rater variability. Markerless video could improve reproducibility, but prior work is largely single-symptom, depends on standardized acquisition, and lacks validation and transfer across ages and sites. We combined two foundation models into one frozen backbone: Segment Anything Model 3 (SAM 3) for dense, per-frame markerless segmentation summarized into geometric, contour and grid kinematic signals, and TabICLv2, a tabular foundation model, for in-context multi-label classification of eight hyperkinetic MD phenomenologies. Trained on standardized recordings of 21 adults and 4 controls, it transferred unchanged to two independent datasets, pediatric (n=12) and tremor-dominant adult (n=20), assessed with the CODY-SAMP scale; only the patient-level decision step was recalibrated per site. Under clinician consensus labels, false positives fell to zero in both datasets. Dystonia recovered perfectly (7/7 pediatric; 15/15 adult held-out), chorea fully in children (3/3), and tremor was recovered in adults (11/15) once a tremor-rich cohort made it evaluable, through recalibration alone. Per-region effect-size analysis gave clinically coherent, phenomenology-specific signals and identified myoclonus as the principal failure. Against YOLOv8 sparse keypoints, the dense representation matched under clinician permissive labels (Jaccard 0.63 vs 0.63) and was markedly more robust under clinician-label consensus (0.93 vs 0.76). This frozen foundation-model backbone with light per-site calibration yields transferable, interpretable, conservative multi-label phenotyping of co-occurring hyperkinetic MDs across ages and from standardized to routine video, adding robustness on high-confidence, clinician-agreed labels. Prospective multi-centre validation is required before clinical use.
ER -