Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models
- 1University of Soutern California
Abstract
Preprints disseminate biomedical research before peer review, but how much those scientific claims change before publication in journals remains uncertain. Here, we matched 72,644 bioRxiv preprints to their peer-reviewed publications by DOI and used a large language model (Claude Sonnet 4.6) to extract one primary and up to two secondary claims from their abstracts. Content change (unchanged, minor, major/substantial) and hedging shift (more cautious, more confident, unchanged) were compared. Four raters independently labelled a 550-pair subset; model-rater agreement (quadratic-weighted Cohen's {kappa} = 0.69) approached agreement among the raters themselves ({kappa} = 0.76). The primary claims were unchanged in 39.8% of abstract pairs, minorly revised in 50.0%, and substantially revised in 10.2%. Claims became more cautious twice as often as more confident (8.4% vs 4.2%). Substantial revisions correlated with longer preprint-to-publication interval. Therefore, among bioRxiv preprints that are later published, central abstract claims remain stable through the preprint-to-publication transition.
Related
- Contextual Biomedical Language Models for Imbalance-Aware Drug-Food Interaction Classification
- A Comparative Benchmark of Biomedical Language Models for Concept Normalization from Real-World Text
- Tracking demographic changes in individual coral colonies through repeated photogrammetric surveys
- Maturational constraints on tracking of temporal attention in infant language acquisition.
- Deep-layer cortical tracking abruptly collapses in the absence of language comprehension
BibTeX
@misc{yin2026tracking,
title = {{Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models}},
author = {H. Yin and W. Ahn and P. M. Forster and R. Rust},
howpublished = {bioRxiv},
year = {2026},
month = jun,
doi = {10.64898/2026.06.30.735556},
url = {https://www.biorxiv.org/content/10.64898/2026.06.30.735556v3},
note = {CC BY},
}
RIS
TY - UNPB
TI - Tracking claim changes from preprint to publication across 72,644 biomedical studies using large language models
AU - H. Yin
AU - W. Ahn
AU - P. M. Forster
AU - R. Rust
PY - 2026
DA - 2026/06/30
PB - bioRxiv
DO - 10.64898/2026.06.30.735556
UR - https://www.biorxiv.org/content/10.64898/2026.06.30.735556v3
AB - Preprints disseminate biomedical research before peer review, but how much those scientific claims change before publication in journals remains uncertain. Here, we matched 72,644 bioRxiv preprints to their peer-reviewed publications by DOI and used a large language model (Claude Sonnet 4.6) to extract one primary and up to two secondary claims from their abstracts. Content change (unchanged, minor, major/substantial) and hedging shift (more cautious, more confident, unchanged) were compared. Four raters independently labelled a 550-pair subset; model-rater agreement (quadratic-weighted Cohen's {kappa} = 0.69) approached agreement among the raters themselves ({kappa} = 0.76). The primary claims were unchanged in 39.8% of abstract pairs, minorly revised in 50.0%, and substantially revised in 10.2%. Claims became more cautious twice as often as more confident (8.4% vs 4.2%). Substantial revisions correlated with longer preprint-to-publication interval. Therefore, among bioRxiv preprints that are later published, central abstract claims remain stable through the preprint-to-publication transition.
ER -