Evaluation of AI-assisted summarisation of tertiary clinical genomics reports: results of the QNOMX-VHIR-CPSP-001 Phase 1 study
- 1Qnomx AG
Abstract
BackgroundComprehensive genomic reports in oncology contain complex molecular information that must be translated into concise summaries for treating clinicians. Manual summarisation is time-intensive, and robust evidence supporting AI-assisted summarisation systems remains limited. We evaluated whether AI-generated summaries were non-inferior to manually produced summaries in faithfully representing information from tertiary paediatric cancer genomics reports.
MethodsQNOMX-VHIR-CPSP-001 Phase 1 was a single-site, non-interventional clinical performance study conducted at Vall dHebron Institut de Recerca under ISO 20916:2019. De-identified tertiary paediatric cancer genomics reports were summarised using both the AI-assisted system under evaluation and the standard manual workflow. Three qualified raters assessed both summary types against the source reports using the six-domain, five-point Quality Summary Index (QSI) in a blinded, counterbalanced crossover design with a minimum fourteen-day washout. Co-primary endpoints were composite content and presentation scores. Non-inferiority was assessed using a pre-specified Bayesian hierarchical model with a margin of 0.5 points and confirmed using a frequentist linear mixed model.
ResultsThirty-seven of 38 planned cases contributed 74 paired evaluations. Mean content scores were similar between AI-generated and manual summaries (4.01 vs 3.97), whereas AI-generated summaries achieved higher presentation scores (4.39 vs 4.12). Non-inferiority was demonstrated for both co-primary endpoints: content +0.05 (95% CrI -0.18 to +0.28), presentation +0.28 (+0.10 to +0.47). Frequentist analyses were concordant. Accuracy was the only QSI dimension for which Bayesian non-inferiority was not concluded. Inter-rater reliability was low across all QSI domains.
ConclusionsAI-generated summaries were non-inferior to manually produced summaries for both content and presentation endpoints in tertiary paediatric oncology genomics reports. Presentation scores were higher for AI-generated summaries, whereas content scores were comparable between approaches. Low inter-rater reliability limits confidence in the dimension-level estimates and is the main constraint on future validation studies. Clinical validity and workflow effect were not assessed.
Related
- A pragmatic evaluation of outcome fragility in randomized clinical trials comparing open microdiscectomy and endoscopic-assisted discectomy: A meta-analysis
- Clinical-Epidemiological, Trichoscopic, and Light Microscopic Evaluation of Pediatric Hair and Scalp Disorders: A Cross-Sectional Study from a Tertiary Care Hospital.
- Evaluation of real-time LAMP assay for detection of Entamoeba species in clinical stool samples from a tertiary care center in North India
- Evaluation of Alterations in Serum Uric Acid Levels in Newly Diagnosed Overt Thyroid Dysfunction: A Prospective Observational Study From a Tertiary Care Center
- Evaluation of Therapeutic Drug Monitoring for Vancomycin with Ratio of Area Under the Curve to Minimum Inhibitory Concentration at a Children’s and Women’s Tertiary Hospital
BibTeX
@misc{creeden2026evaluation,
title = {{Evaluation of AI-assisted summarisation of tertiary clinical genomics reports: results of the QNOMX-VHIR-CPSP-001 Phase 1 study}},
author = {J. Creeden and M. Olivecrona and N. Benavent and R. Hladun Alvaro and L. Valero-Arrese and A. Carolina Torres and A. Soriano},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.10.26362656},
url = {https://www.medrxiv.org/content/10.64898/2026.09.10.26362656v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Evaluation of AI-assisted summarisation of tertiary clinical genomics reports: results of the QNOMX-VHIR-CPSP-001 Phase 1 study
AU - J. Creeden
AU - M. Olivecrona
AU - N. Benavent
AU - R. Hladun Alvaro
AU - L. Valero-Arrese
AU - A. Carolina Torres
AU - A. Soriano
PY - 2026
DA - 2026/09/10
PB - medRxiv
DO - 10.64898/2026.09.10.26362656
UR - https://www.medrxiv.org/content/10.64898/2026.09.10.26362656v1
AB - BackgroundComprehensive genomic reports in oncology contain complex molecular information that must be translated into concise summaries for treating clinicians. Manual summarisation is time-intensive, and robust evidence supporting AI-assisted summarisation systems remains limited. We evaluated whether AI-generated summaries were non-inferior to manually produced summaries in faithfully representing information from tertiary paediatric cancer genomics reports. MethodsQNOMX-VHIR-CPSP-001 Phase 1 was a single-site, non-interventional clinical performance study conducted at Vall dHebron Institut de Recerca under ISO 20916:2019. De-identified tertiary paediatric cancer genomics reports were summarised using both the AI-assisted system under evaluation and the standard manual workflow. Three qualified raters assessed both summary types against the source reports using the six-domain, five-point Quality Summary Index (QSI) in a blinded, counterbalanced crossover design with a minimum fourteen-day washout. Co-primary endpoints were composite content and presentation scores. Non-inferiority was assessed using a pre-specified Bayesian hierarchical model with a margin of 0.5 points and confirmed using a frequentist linear mixed model. ResultsThirty-seven of 38 planned cases contributed 74 paired evaluations. Mean content scores were similar between AI-generated and manual summaries (4.01 vs 3.97), whereas AI-generated summaries achieved higher presentation scores (4.39 vs 4.12). Non-inferiority was demonstrated for both co-primary endpoints: content +0.05 (95% CrI -0.18 to +0.28), presentation +0.28 (+0.10 to +0.47). Frequentist analyses were concordant. Accuracy was the only QSI dimension for which Bayesian non-inferiority was not concluded. Inter-rater reliability was low across all QSI domains. ConclusionsAI-generated summaries were non-inferior to manually produced summaries for both content and presentation endpoints in tertiary paediatric oncology genomics reports. Presentation scores were higher for AI-generated summaries, whereas content scores were comparable between approaches. Low inter-rater reliability limits confidence in the dimension-level estimates and is the main constraint on future validation studies. Clinical validity and workflow effect were not assessed.
ER -