A Scientific Claim Stability Framework for Evaluating MRI Morphometry Pipelines
Abstract
MRI morphometry pipelines are often evaluated by measurement agreement or downstream accuracy, yet their scientific value depends on the conclusions they support. Two pipelines can produce similar feature-level results yet support different conclusions about aging, cognitive decline, or neurodegeneration. We introduce Scientific Claim Stability (SCS), a claim-centric evaluation framework that converts region-of-interest (ROI) morphometry tables into auditable claim cards and evaluates their support under changes in dataset, processing pipeline, atlas, feature family, and target. Across 10,370 MRI scans from three datasets and five cohorts, SCS generates 2,783 claim cards and evaluates 89 claim-family pairs. The evaluation distinguishes stable, transferable, and fragile claims. FreeSurfer 8-FastSurfer comparisons show high evidence concordance and low anatomical-support overlap under the primary covariate specification. Acquisition-adjusted analyses show that the magnitude of support disagreement depends on the model specification. Structured review by specialists in geriatrics, neuroscience, and radiology, alongside auxiliary ChatGPT and Gemini reviewers, assesses claim plausibility, clarity, faithfulness, usefulness, and caution. SCS provides a reproducible way to audit the evidential stability of morphometry claims without assuming a universal gold standard.
Related
- From segmentation to morphometry: A post-processed U-Net framework for corneal endothelial cell analysis
- BaseCamp --- An Agentic AI Framework for Automating DNA Sequencing Data Pipelines
- An end-to-end in silico framework for truth-based validation and regression testing of clinical genome sequencing pipelines, from simulated reads to reportable findings
- Evaluating the added value of sequence models in noncoding RNA assays: a baseline-aware framework
- Latent-Factor Stability Is Not Sufficient: Evaluating SABLE-MO in Simulated and Cancer Multi-Omics Data
BibTeX
@misc{can2026scientific,
title = {{A Scientific Claim Stability Framework for Evaluating MRI Morphometry Pipelines}},
author = {Duy-Cat Can and Dang-Duat Tran and Anh-Khang Nguyen and Quoc-Bao Tran and Trung-Hieu Do and Van-Tu Nguyen and Duy-Thanh Vu and Oliver Y. Chén and Hoang-Quynh Le},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.02413},
eprint = {2611.02413},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.02413},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - A Scientific Claim Stability Framework for Evaluating MRI Morphometry Pipelines
AU - Duy-Cat Can
AU - Dang-Duat Tran
AU - Anh-Khang Nguyen
AU - Quoc-Bao Tran
AU - Trung-Hieu Do
AU - Van-Tu Nguyen
AU - Duy-Thanh Vu
AU - Oliver Y. Chén
AU - Hoang-Quynh Le
PY - 2026
DA - 2026/11/02
PB - arXiv
DO - 10.48550/arxiv.2611.02413
UR - https://arxiv.org/abs/2611.02413
AB - MRI morphometry pipelines are often evaluated by measurement agreement or downstream accuracy, yet their scientific value depends on the conclusions they support. Two pipelines can produce similar feature-level results yet support different conclusions about aging, cognitive decline, or neurodegeneration. We introduce Scientific Claim Stability (SCS), a claim-centric evaluation framework that converts region-of-interest (ROI) morphometry tables into auditable claim cards and evaluates their support under changes in dataset, processing pipeline, atlas, feature family, and target. Across 10,370 MRI scans from three datasets and five cohorts, SCS generates 2,783 claim cards and evaluates 89 claim-family pairs. The evaluation distinguishes stable, transferable, and fragile claims. FreeSurfer 8-FastSurfer comparisons show high evidence concordance and low anatomical-support overlap under the primary covariate specification. Acquisition-adjusted analyses show that the magnitude of support disagreement depends on the model specification. Structured review by specialists in geriatrics, neuroscience, and radiology, alongside auxiliary ChatGPT and Gemini reviewers, assesses claim plausibility, clarity, faithfulness, usefulness, and caution. SCS provides a reproducible way to audit the evidential stability of morphometry claims without assuming a universal gold standard.
ER -