Methodological rigour and risk of bias in the publicly available full-text evidence for permanently approved digital health applications (DiGA): a systematic assessment.
- 1Department of Digital Medicine, Medical School OWL, Bielefeld University, Bielefeld, Germany.
- 2Peter L. Reichertz Institute for Medical Informatics of TU Braunschweig and Hannover Medical School, Hannover Medical School, Hannover, Germany.
Abstract
In Germany, digital health applications (DiGA) require demonstrated positive care effects for permanent approval. No overarching evaluation has yet assessed the methodological quality and risk of bias of studies underlying the 38 DiGA permanently listed as of January 2025. This study systematically evaluates the methodological rigour of these studies using an evaluation framework. Evidence quality was systematically assessed in two parts: a dimension-based assessment developed in prior work, and a framework-based assessment using the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence and the Revised Cochrane Risk-of-Bias Tool for Randomised Trials (RoB2). Of the 38 permanently approved DiGA, only those with manufacturer-published full-text studies were included. All 29 studies (100%) were randomised controlled trials, classified as OCEBM Level 1b. RoB2 rated 27 (93%) at high risk of bias. Most studies (25/29, 86%) used waitlist control groups and patient-reported outcomes (26/29, 90%). Blinding was inadequate in 17 studies (59%). Four studies (4/29, 14%) showed unbalanced dropout (>10% difference) between arms. Ten of the 38 approved DiGA (26%) lacked published or full-text studies; some available studies were German-language only, limiting generalisability. Only six studies (6/29, 21%) had a balanced sex ratio (within 20%). All studies showed medical benefit, but only four (4/29, 14%) demonstrated patient-relevant structural or procedural improvements. Four studies (4/29, 14%) omitted power analysis, and two (7%) did not reach the required sample size for 80% power. Sample sizes ranged from 56 to 1,245. Follow-up exceeded six months in 19 studies (66%). These findings indicate notable gaps in methodological quality and transparency in the public evidence base for permanently approved DiGA, with many studies at high risk of bias per RoB2. This should be considered alongside structural challenges: blinding is often infeasible for digital, psychological, and behavioural interventions, and patient-reported outcomes are frequently the clinically appropriate measure for the indications studied. These findings concern only published study reports and do not reflect the completeness of the underlying Federal Institute for Drugs and Medical Devices (BfArM) regulatory dossiers. Stronger full-text publication requirements and greater methodological transparency could strengthen the evidence base supporting DiGA reimbursement decisions.
Related
- Optimal Sensor Placement for Bridge Structural Health Monitoring: A Systematic Review with Bibliometric Analysis and Methodological Quality Assessment.
- Large language models in rheumatology: applications, methodological quality and clinical translation – a systematic review
- Essential Oils as a Source of Anticancer Molecules: Critical Assessment of Current Evidence and Methodological Limitations—A Systematic Review
- Evidence standards of permanently listed digital health applications in Germany
- Fire dragon cupping therapy for primary dysmenorrhea: a systematic review of available clinical evidence and research gaps
BibTeX
@article{zapp2026methodological,
title = {{Methodological rigour and risk of bias in the publicly available full-text evidence for permanently approved digital health applications (DiGA): a systematic assessment.}},
author = {Sebastian Zapp and Ute von Jan and Urs-Vito Albrecht},
journal = {MHealth},
year = {2026},
month = sep,
doi = {10.21037/mhealth-2026-0038},
url = {https://europepmc.org/article/MED/42999808},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Methodological rigour and risk of bias in the publicly available full-text evidence for permanently approved digital health applications (DiGA): a systematic assessment.
AU - Sebastian Zapp
AU - Ute von Jan
AU - Urs-Vito Albrecht
PY - 2026
DA - 2026/09/30
JO - MHealth
DO - 10.21037/mhealth-2026-0038
UR - https://europepmc.org/article/MED/42999808
AB - In Germany, digital health applications (DiGA) require demonstrated positive care effects for permanent approval. No overarching evaluation has yet assessed the methodological quality and risk of bias of studies underlying the 38 DiGA permanently listed as of January 2025. This study systematically evaluates the methodological rigour of these studies using an evaluation framework. Evidence quality was systematically assessed in two parts: a dimension-based assessment developed in prior work, and a framework-based assessment using the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence and the Revised Cochrane Risk-of-Bias Tool for Randomised Trials (RoB2). Of the 38 permanently approved DiGA, only those with manufacturer-published full-text studies were included. All 29 studies (100%) were randomised controlled trials, classified as OCEBM Level 1b. RoB2 rated 27 (93%) at high risk of bias. Most studies (25/29, 86%) used waitlist control groups and patient-reported outcomes (26/29, 90%). Blinding was inadequate in 17 studies (59%). Four studies (4/29, 14%) showed unbalanced dropout (>10% difference) between arms. Ten of the 38 approved DiGA (26%) lacked published or full-text studies; some available studies were German-language only, limiting generalisability. Only six studies (6/29, 21%) had a balanced sex ratio (within 20%). All studies showed medical benefit, but only four (4/29, 14%) demonstrated patient-relevant structural or procedural improvements. Four studies (4/29, 14%) omitted power analysis, and two (7%) did not reach the required sample size for 80% power. Sample sizes ranged from 56 to 1,245. Follow-up exceeded six months in 19 studies (66%). These findings indicate notable gaps in methodological quality and transparency in the public evidence base for permanently approved DiGA, with many studies at high risk of bias per RoB2. This should be considered alongside structural challenges: blinding is often infeasible for digital, psychological, and behavioural interventions, and patient-reported outcomes are frequently the clinically appropriate measure for the indications studied. These findings concern only published study reports and do not reflect the completeness of the underlying Federal Institute for Drugs and Medical Devices (BfArM) regulatory dossiers. Stronger full-text publication requirements and greater methodological transparency could strengthen the evidence base supporting DiGA reimbursement decisions.
SN - 2306-9740
ER -