{
  "abstract": "Evidence-synthesis teams increasingly depend on machine-learning tools to automate risk-of-bias assessment, but these tools frequently rely on unrunnable, deprecated software stacks. Refactoring legacy tools into modern environments is essential for maintenance, yet introduces a critical risk: silently invalidating published performance metrics. Without rigorous validation, systematic reviewers cannot trust that modernized tools retain their predecessors behavior. We present a five-tier validation framework applied to RCT-Reviewer, an independent ground-up modernization of the widely used RobotReviewer system. To prove preservation, we engineered a compatibility shim to execute the original 2017 code alongside the modernized tool using byte-identical weight files. The framework evaluated predictive validity on a human-labelled benchmark, quantified the cost of dropping obsolete ensemble components, proved bit-exact fidelity, tested parser robustness at scale, and included an external validation arm evaluating agreement on 313 trials against a recent reference standard. A control experiment showing 100% agreement between legacy and modernized implementations on identical text proved that variance versus published values is driven by PDF provenance (open-access versus publisher typeset) rather than codebase refactoring. The modernized tool perfectly reproduced original risk-of-bias judgements in 6,018 comparisons. While the screening classifier experienced a bounded performance reduction (F1 0.969 to 0.925) from removing obsolete components, the models core mathematical validity is preserved. These results establish that reviewers can deploy a maintainable modernization with a clearly bounded performance trade-off, offering a reusable validation framework for clinical NLP modernizations. Future work will explore retraining obsolete neural network components to close the performance gap.",
  "abstract_shown": true,
  "affiliations": [
    "Georgian National University SEU"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 2,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "V. Sahu"
    },
    {
      "affs": [
        0
      ],
      "name": "A. Sahu"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/f4cy44taksgxn.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.09.08.26362546",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.09.08.26362546v1.full",
  "id": "f4cy44taksgxn",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.09.08.26362546v1",
  "language": null,
  "licence": "CC-BY",
  "licence_label": "CC BY",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "health informatics"
    ],
    [
      "openalex",
      "1804|Meta-analysis and systematic reviews"
    ]
  ],
  "openalex": "W7212106736",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.09.08.26362546v1.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-08",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/f4cy44taksgxn.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:18:18Z",
      "native_id": "10.64898/2026.09.08.26362546",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.09.08.26362546v1"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "medrxiv:health informatics",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Preserving a decade of machine-learning validity across a ground-up refactoring: a five-tier automated validation of RCT-Reviewer, an independent modernization of RobotReviewer",
  "title_text": "Preserving a decade of machine-learning validity across a ground-up refactoring: a five-tier automated validation of RCT-Reviewer, an independent modernization of RobotReviewer",
  "type": "preprint",
  "updated_at": "2026-10-30T12:18:18Z",
  "url": "https://research.draconitic.com/paper/f4cy44taksgxn",
  "venue": "medRxiv",
  "versions": []
}