Skip to content
DraconiticResearch
Search Sources API
190,839 studies31 sourcesUpdated 28 min ago
Peer-reviewedCC BY 4.0Translational Psychiatry

Counteracting the overestimation of effect size in mass-univariate brain-wide association studies using cross-validation

  • Janik Goltermann1,2,3,4
  • Nils Ralf Winter5,6,4,7
  • Marius Gruber8,9,6,4
  • L. Fisch5,6,4
  • Maike Frederike Richter10,11,12,4
  • Dominik Grotegerd6,4
  • Katharina Dohm6,4
  • Susanne Meinert13,6,14,4
  • Elisabeth Johanna Leehr15,6,4
  • Joscha Böhnlein6,4
  • Anna Kraus6,4
  • Katharina Thiel6,4
24 more authors
  • Alexandra Winter6,4
  • Kira Flinkenflügel6,4
  • Ramona Leenings16,10,17,11,4
  • Carlotta Barkhau18,6,4
  • Jan Ernsting17,19,20,21,4
  • Klaus Berger22,23
  • Heike Minnerup22,23
  • Benjamin Straube24,25
  • Nina Alexander24,25
  • Hamidreza Jamalabadi24,25
  • Frederike Stein24,25
  • Katharina Brosch24,26,25
  • Adrian Wroblewski24,25
  • Florian Thomas‐Odenthal24,25
  • Paula L Usemann24,25
  • Lea Teutenberg24,25
  • Julia‐Katharina Pfarr27,25
  • Andreas Jansen28,29,24,25
  • Igor Nenadić24,25
  • Tilo T. J. Kircher24,25
  • Christian Gaser30,31,32,33,12
  • Nils Opel10,11,34,12,35,4
  • Tim Hahn5,6,4
  • Udo Dannlowski36,37,32,6,4
37 affiliations
  1. 1Department of Psychiatry and Neurosciences, Campus Benjamin Franklin, Charité-Universitätsmedizin Berlin, Berlin, Germany. [email protected]
  2. 2German Center for Mental Health (DZPG), Site Berlin, Potsdam, Germany. [email protected]
  3. 3Institute for Translational Psychiatry, University of Münster, Münster, Germany. [email protected]
  4. 4Institute for Translational Psychiatry, University of Münster, Germany
  5. 5Institute for Machine Learning in Medicine (focus area psychiatry), University of Münster, Münster, Germany
  6. 6Institute for Translational Psychiatry, University of Münster, Münster, Germany
  7. 7Otto Creutzfeldt Center for Cognitive and Behavioral Neuroscience, University of Münster, Germany
  8. 8Cooperative Brain Imaging Center - CoBIC, Goethe University Frankfurt, Frankfurt, Germany
  9. 9Department of Psychiatry, Psychosomatic Medicine, and Psychotherapy, Goethe University Frankfurt, University Hospital, Frankfurt, Germany
  10. 10Department of Psychiatry and Neurosciences, Campus Benjamin Franklin, Charité-Universitätsmedizin Berlin, Berlin, Germany
  11. 11German Center for Mental Health (DZPG), Site Berlin, Potsdam, Germany
  12. 12Department of Psychiatry and Psychotherapy, Jena University Hospital, Germany
  13. 13Institute for Translational Neuroscience, University of Münster, Münster, Germany
  14. 14Institute for Translational Neuroscience, University of Münster, Germany
  15. 15Department of Clinical Psychology and Psychotherapy, Marburg University, Marburg, Germany
  16. 16Department of Neurology, Kepler University Hospital, Johannes Kepler University Linz, Linz, Austria
  17. 17Faculty of Mathematics and Computer Science, University of Münster, Münster, Germany
  18. 18Institute for Biomagnetism and Biosignalanalysis, Universitätsklinikum Münster, Münster, Germany
  19. 19Institute for Geoinformatics, University of Münster, Münster, Germany
  20. 20Faculty of Mathematics and Computer Science, University of Münster, Germany
  21. 21Institute for Geoinformatics, University of Münster, Germany
  22. 22Institute of Epidemiology and Social Medicine, University of Münster, Münster, Germany
  23. 23Institute of Epidemiology and Social Medicine, University of Münster, Germany
  24. 24Department of Psychiatry and Psychotherapy, Marburg University, Marburg, Germany
  25. 25Department of Psychiatry and Psychotherapy, University of Marburg, Germany
  26. 26Northwell Health, New Hyde Park, USA
  27. 27ORIGAMI Neuro Data Science Laboratory, McConnell Brain Imaging Centre, Montreal Neurological Institute, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
  28. 28Center for Mind, Brain and Behavior (CMBB), Universities of Marburg, Gießen and Darmstadt, Marburg, Germany
  29. 29Core-Facility Brainimaging, Faculty of Medicine, Marburg University, Marburg, Germany
  30. 30Department of Neurology, Jena University Hospital, Jena, Germany
  31. 31Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
  32. 32German Center for Mental Health (DZPG), Site Jena Magdeburg Halle, Germany
  33. 33Department of Neurology, Jena University Hospital, Germany
  34. 34Center for Intervention and Research on adaptive and maladaptive brain Circuits underlying mental health (C-I-R-C), Jena-Magdeburg-Halle, Germany
  35. 35German Center for Mental Health (DZPG), Site Jena-Magdeburg-Halle, Germany
  36. 36Center for Intervention and Research on Adaptive and Maladaptive Brain Circuits Underlying Mental Health (C-I-R-C), Site Jena Magdeburg Halle, Germany
  37. 37Department of Psychiatry, Medical School and University Medical Center OWL, Protestant Hospital of the Bethel Foundation, Bielefeld University, Bielefeld, Germany
Open at sourcePDFDOICite

Abstract

Background: Brain-wide association effects are commonly overestimated in psychiatric neuroimaging research, leading to low replicability and generalizability of findings, and ultimately to poor translation to clinical utility. Cross-validation is currently not standardly used in univariate neuroimaging analyses but could pose a potential counteracting measure for this problem.

Methods: We systematically evaluated the benefit of a cross-validation procedure to expose inflated effect size estimates in mass-univariate voxel-based analysis, using body mass index (BMI) as an example predictor for gray matter. A total of n = 3401 MRI scans from a psychiatric cohort were used to investigate the impact of applying cross-validation on effect size estimates across a range of sample sizes and across different localizations with a range of underlying pooled effect sizes.

Results: BMI was negatively associated with gray matter density with a maximum underlying effect size of R ² p = 0.036 (in the full sample of n = 3401). However, without cross-validation, effects were substantially overestimated in small samples, with effect sizes ranging up to R ² p = 0.535. Even with an underlying null-effect, effect size estimates up to R ² p = 0.429 were reached in samples with n = 25. Applying cross-validation revealed that these inflated effects in small samples did not generalize to novel data. A minimum sample size of n = 100 was required for largest brain-wide effects to start generalizing, while samples with n = 400 were needed for smaller effects to generalize. This ‘point of initial generalizability’ was not reached for underlying null effects, even with the maximum available sample of n = 3401.

Conclusions: Results suggest that cross-validation is an effective measure to expose overestimated effects in small samples and thus could help to counteract low replicability and generalizability in brain-wide mass-univariate association studies. Open-source code is provided for practical implementation.

Related

  1. Yield and Yield Components in Soybean Using Genome-Wide Association Studies, GWAS: Current Status, Challenges, and Breeding Prospects
    Aigul Amangeldiyeva, Svetlana Didorenko, Bekzhan Maikotov +3
    Plants (Basel, Switzerland)Peer-reviewedCC BYFull textPlant biologyEurope PMC
  2. Reaction time-related compression underlies overestimation of time-constrained reachability
    Keisuke Tani, Soma Okuuchi, Satoshi Hirose +1
    iSciencePeer-reviewedCC BY 4.0NeuroscienceNeurologyCell PressCrossref
  3. Vpu protects HIV-infected cells from antibody-dependent cellular phagocytosis by counteracting tetherin
    Brian J. Snow, Nida K. Keles, Ahmed Ali +3
    Cell ReportsPeer-reviewedCC BY-NC-ND 4.0MicrobiologyCrossref
  4. Multi-ancestry genome- and transcriptome-wide association studies expand the susceptibility landscape for breast cancer overall and across subtypes
    Zheng Guo, Guochong Jia, Jie Ping +29
    Communications MedicinePeer-reviewedCC BY-NC-ND 4.0Full textGenetics & genomicsCrossref
  5. MaMPK3‐2 Phosphorylating and Stabilizing Transcriptional Repressor MaAS1 by Counteracting MaRNF4‐Mediated Ubiquitination During Banana Fruit Ripening
    Ting‐Ting Luo, Ying‐Ying Yang, Hai‐Ke Tan +6
    Plant biotechnology journalPeer-reviewedCC BYFull textPlant biologyEurope PMC

Cite

BibTeXRIS
BibTeX
@article{goltermann2026counteracting,
  title = {{Counteracting the overestimation of effect size in mass-univariate brain-wide association studies using cross-validation}},
  author = {Janik Goltermann and Nils Ralf Winter and Marius Gruber and L. Fisch and Maike Frederike Richter and Dominik Grotegerd and Katharina Dohm and Susanne Meinert and Elisabeth Johanna Leehr and Joscha Böhnlein and Anna Kraus and Katharina Thiel and Alexandra Winter and Kira Flinkenflügel and Ramona Leenings and Carlotta Barkhau and Jan Ernsting and Klaus Berger and Heike Minnerup and Benjamin Straube and Nina Alexander and Hamidreza Jamalabadi and Frederike Stein and Katharina Brosch and Adrian Wroblewski and Florian Thomas‐Odenthal and Paula L Usemann and Lea Teutenberg and Julia‐Katharina Pfarr and Andreas Jansen and Igor Nenadić and Tilo T. J. Kircher and Christian Gaser and Nils Opel and Tim Hahn and Udo Dannlowski},
  journal = {Translational Psychiatry},
  year = {2026},
  month = nov,
  doi = {10.1038/s41398-026-04529-x},
  url = {https://www.nature.com/articles/s41398-026-04529-x},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Counteracting the overestimation of effect size in mass-univariate brain-wide association studies using cross-validation
AU  - Janik Goltermann
AU  - Nils Ralf Winter
AU  - Marius Gruber
AU  - L. Fisch
AU  - Maike Frederike Richter
AU  - Dominik Grotegerd
AU  - Katharina Dohm
AU  - Susanne Meinert
AU  - Elisabeth Johanna Leehr
AU  - Joscha Böhnlein
AU  - Anna Kraus
AU  - Katharina Thiel
AU  - Alexandra Winter
AU  - Kira Flinkenflügel
AU  - Ramona Leenings
AU  - Carlotta Barkhau
AU  - Jan Ernsting
AU  - Klaus Berger
AU  - Heike Minnerup
AU  - Benjamin Straube
AU  - Nina Alexander
AU  - Hamidreza Jamalabadi
AU  - Frederike Stein
AU  - Katharina Brosch
AU  - Adrian Wroblewski
AU  - Florian Thomas‐Odenthal
AU  - Paula L Usemann
AU  - Lea Teutenberg
AU  - Julia‐Katharina Pfarr
AU  - Andreas Jansen
AU  - Igor Nenadić
AU  - Tilo T. J. Kircher
AU  - Christian Gaser
AU  - Nils Opel
AU  - Tim Hahn
AU  - Udo Dannlowski
PY  - 2026
DA  - 2026/11/04
JO  - Translational Psychiatry
DO  - 10.1038/s41398-026-04529-x
UR  - https://www.nature.com/articles/s41398-026-04529-x
AB  - Background: Brain-wide association effects are commonly overestimated in psychiatric neuroimaging research, leading to low replicability and generalizability of findings, and ultimately to poor translation to clinical utility. Cross-validation is currently not standardly used in univariate neuroimaging analyses but could pose a potential counteracting measure for this problem.  Methods: We systematically evaluated the benefit of a cross-validation procedure to expose inflated effect size estimates in mass-univariate voxel-based analysis, using body mass index (BMI) as an example predictor for gray matter. A total of n = 3401 MRI scans from a psychiatric cohort were used to investigate the impact of applying cross-validation on effect size estimates across a range of sample sizes and across different localizations with a range of underlying pooled effect sizes.  Results: BMI was negatively associated with gray matter density with a maximum underlying effect size of R ² p = 0.036 (in the full sample of n = 3401). However, without cross-validation, effects were substantially overestimated in small samples, with effect sizes ranging up to R ² p = 0.535. Even with an underlying null-effect, effect size estimates up to R ² p = 0.429 were reached in samples with n = 25. Applying cross-validation revealed that these inflated effects in small samples did not generalize to novel data. A minimum sample size of n = 100 was required for largest brain-wide effects to start generalizing, while samples with n = 400 were needed for smaller effects to generalize. This ‘point of initial generalizability’ was not reached for underlying null effects, even with the maximum available sample of n = 3401.  Conclusions: Results suggest that cross-validation is an effective measure to expose overestimated effects in small samples and thus could help to counteract low replicability and generalizability in brain-wide mass-univariate association studies. Open-source code is provided for practical implementation.
SN  - 2158-3188
ER  -