Reporting Quality of Mendelian Randomization Studies Based on STROBE-MR: An Investigation Powered by Large Language Models
- 1University of Bristol
Abstract
2BackgroundMendelian randomization (MR) has been widely used to investigate causal relationships between exposures and outcomes. However, concerns have also been raised about the reporting quality of MR studies. STROBE-MR was developed as a reporting guideline for MR studies, but assessments of compliance with STROBE-MR in published MR studies remain limited and rely largely on manual review. With the rapid development of large language models (LLMs), it is now possible to automate STROBE-MR compliance assessment.
MethodsWe developed a pipeline using OpenAIs GPT-5-mini model for automated assessment of MR reporting quality. Model performance was evaluated on an annotated dataset of 46 eligible MR articles. We applied the pipeline to more than 13,000 MR articles published between 2005 and 2025 to assess compliance with the STROBE-MR guideline and to explore reporting patterns across STROBE-MR items, sections, and time periods. Complete compliance required reporting all applicable components of an item; a sensitivity analysis used a partial compliance criterion requiring at least one applicable component.
ResultsOur LLM-based pipeline agreed with the human consensus on 89.0% of the STROBE-MR assessments. On average, 22 of 43 items (51.2%) met the complete compliance criterion, with substantial variation in reporting rates across STROBE-MR items and sections. The least reported items were found in the "Methods", "Results", and "Other Information" sections. The number of reported items remained stable over time, with slight decreases after 2021. Sensitivity analysis showed that temporal trend patterns differed between complete and partial compliance, particularly in the Methods, Discussion and Other Information sections.
ConclusionsThis study demonstrates the potential of LLMs to automate reporting quality assessment in MR studies. Our results suggest that reporting quality still needs to be improved, and further efforts are required to promote the use of STROBE-MR. Our findings also suggest potential refinements to the STROBE-MR guideline to support more consistent application.
1 Key PointsO_ST_ABSQuestionC_ST_ABSCan large language models (LLMs) accurately assess adherence to the STROBE-MR reporting guideline in Mendelian randomization (MR) studies, and what patterns can we observe in MR reporting?
FindingsThe LLM-based approach achieved good agreement with human consensus (89.0%) and identified substantial variation in reporting quality across STROBE-MR items and sections. No improvement in number of items reported was observed since the introduction of the STROBE-MR guideline.
MeaningLLMs can enable scalable assessment of reporting guideline compliance, providing a practical approach for monitoring reporting quality. Further efforts are needed to promote the use of reporting guidelines for MR studies.
Related
- Stroke and brain white/gray matter alterations : A Mendelian randomization investigation
- Immune Cell-Mediated Interactions Between the Gut Microbiome and Endometriosis: A Mendelian Randomization Investigation
- Abdominal Adiposity and Lacunar Stroke: A Mendelian Randomization Mediation Study
- Genetically Proxied SGLT2 Inhibition and Risk of Epilepsy: A Mediation Mendelian Randomization Study of Circulating Metabolites
- Causal effects of educational attainment on respiratory diseases : A Mendelian randomization study
BibTeX
@misc{zhou2026reporting,
title = {{Reporting Quality of Mendelian Randomization Studies Based on STROBE-MR: An Investigation Powered by Large Language Models}},
author = {J. Zhou and X. Wang and Z. Xu and E. Simpson and J. P. Higgins and T. R. Gaunt and Y. Liu},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.21.26366143},
url = {https://www.medrxiv.org/content/10.64898/2026.10.21.26366143v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Reporting Quality of Mendelian Randomization Studies Based on STROBE-MR: An Investigation Powered by Large Language Models
AU - J. Zhou
AU - X. Wang
AU - Z. Xu
AU - E. Simpson
AU - J. P. Higgins
AU - T. R. Gaunt
AU - Y. Liu
PY - 2026
DA - 2026/10/21
PB - medRxiv
DO - 10.64898/2026.10.21.26366143
UR - https://www.medrxiv.org/content/10.64898/2026.10.21.26366143v1
AB - 2BackgroundMendelian randomization (MR) has been widely used to investigate causal relationships between exposures and outcomes. However, concerns have also been raised about the reporting quality of MR studies. STROBE-MR was developed as a reporting guideline for MR studies, but assessments of compliance with STROBE-MR in published MR studies remain limited and rely largely on manual review. With the rapid development of large language models (LLMs), it is now possible to automate STROBE-MR compliance assessment. MethodsWe developed a pipeline using OpenAIs GPT-5-mini model for automated assessment of MR reporting quality. Model performance was evaluated on an annotated dataset of 46 eligible MR articles. We applied the pipeline to more than 13,000 MR articles published between 2005 and 2025 to assess compliance with the STROBE-MR guideline and to explore reporting patterns across STROBE-MR items, sections, and time periods. Complete compliance required reporting all applicable components of an item; a sensitivity analysis used a partial compliance criterion requiring at least one applicable component. ResultsOur LLM-based pipeline agreed with the human consensus on 89.0% of the STROBE-MR assessments. On average, 22 of 43 items (51.2%) met the complete compliance criterion, with substantial variation in reporting rates across STROBE-MR items and sections. The least reported items were found in the "Methods", "Results", and "Other Information" sections. The number of reported items remained stable over time, with slight decreases after 2021. Sensitivity analysis showed that temporal trend patterns differed between complete and partial compliance, particularly in the Methods, Discussion and Other Information sections. ConclusionsThis study demonstrates the potential of LLMs to automate reporting quality assessment in MR studies. Our results suggest that reporting quality still needs to be improved, and further efforts are required to promote the use of STROBE-MR. Our findings also suggest potential refinements to the STROBE-MR guideline to support more consistent application. 1 Key PointsO_ST_ABSQuestionC_ST_ABSCan large language models (LLMs) accurately assess adherence to the STROBE-MR reporting guideline in Mendelian randomization (MR) studies, and what patterns can we observe in MR reporting? FindingsThe LLM-based approach achieved good agreement with human consensus (89.0%) and identified substantial variation in reporting quality across STROBE-MR items and sections. No improvement in number of items reported was observed since the introduction of the STROBE-MR guideline. MeaningLLMs can enable scalable assessment of reporting guideline compliance, providing a practical approach for monitoring reporting quality. Further efforts are needed to promote the use of reporting guidelines for MR studies.
ER -