Biobank-scale self-supervised genotype pretraining yields transferable representations for human genetics
- 1Department of Public Laboratory, Key Laboratory of Cancer Prevention and Therapy of Tianjin, Tianjin’s Clinical Research Center for Cancer, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin Medical University , Tianjin 300060 ,
- 2Department of Epidemiology and Biostatistics, Key Laboratory of Molecular Cancer Epidemiology of Tianjin, Key Laboratory of Prevention and Control of Human Major Diseases (Ministry of Education), Tianjin’s Clinical Research Center for Cancer, National Clinical Research Center for Cancer, Tianjin Medical University Cancer Institute and Hospital, Tianjin Medical University , Tianjin 300060 ,
Abstract
Biobank-scale genotyping has created unprecedented cohorts for human genetics, yet the field still lacks scalable methods for learning transferable representations from largely unlabeled individual-level genotype data. Here, we introduce SNPBERT, a self-supervised transformer pretrained on genotypes from 485 923 UK Biobank participants using masked rsID prediction. SNPBERT embeddings recovered known population structure across UK Biobank and three external datasets, performed comparably to principal component analysis, and supported the identification of ancestry-informative marker SNPs. Fine-tuning with an attention-based multiple-instance head generated disease-specific risk scores that provided complementary information beyond conventional polygenic risk scores. The relative increase in Nagelkerke’s R2 ranged from 0.7% to 3.3%, and independent associations were observed for seven of the eight chronic diseases examined. In celiac disease, attention prioritization recapitulated major histocompatibility complex signals while elevating variants in CCHCR1 and NOTCH4 relative to genome-wide association study (GWAS) ranking. These results support biobank-scale genotype pretraining as an early step toward foundation modeling for individual-level genotype data.
Related
- PhysBridge: Response-supervised grid pretraining with physics-constrained local correction for transferable power-system intelligence
- MtReNIRS: A Self-Supervised Foundation Model for Transferable Neurovascular Representations
- BiomECG: Learning Transferable ECG Representations from Small Cohorts
- Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA
- BarcodeMAE+: Rethinking Masked Pretraining and Global Representations for DNA Barcode Foundation Models
BibTeX
@article{sheng2026biobankscale,
title = {{Biobank-scale self-supervised genotype pretraining yields transferable representations for human genetics}},
author = {Chao Sheng and Han Du and Hongru Shen and Hongji Dai and Fangfang Song and Xiangchun Li and Kexin Chen},
journal = {Nucleic Acids Research},
year = {2026},
month = oct,
doi = {10.1093/nar/gkag1059},
url = {https://academic.oup.com/nar/article/doi/10.1093/nar/gkag1059/9004260},
note = {CC BY-NC 4.0},
}
RIS
TY - JOUR
TI - Biobank-scale self-supervised genotype pretraining yields transferable representations for human genetics
AU - Chao Sheng
AU - Han Du
AU - Hongru Shen
AU - Hongji Dai
AU - Fangfang Song
AU - Xiangchun Li
AU - Kexin Chen
PY - 2026
DA - 2026/10/01
JO - Nucleic Acids Research
DO - 10.1093/nar/gkag1059
UR - https://academic.oup.com/nar/article/doi/10.1093/nar/gkag1059/9004260
AB - Biobank-scale genotyping has created unprecedented cohorts for human genetics, yet the field still lacks scalable methods for learning transferable representations from largely unlabeled individual-level genotype data. Here, we introduce SNPBERT, a self-supervised transformer pretrained on genotypes from 485 923 UK Biobank participants using masked rsID prediction. SNPBERT embeddings recovered known population structure across UK Biobank and three external datasets, performed comparably to principal component analysis, and supported the identification of ancestry-informative marker SNPs. Fine-tuning with an attention-based multiple-instance head generated disease-specific risk scores that provided complementary information beyond conventional polygenic risk scores. The relative increase in Nagelkerke’s R2 ranged from 0.7% to 3.3%, and independent associations were observed for seven of the eight chronic diseases examined. In celiac disease, attention prioritization recapitulated major histocompatibility complex signals while elevating variants in CCHCR1 and NOTCH4 relative to genome-wide association study (GWAS) ranking. These results support biobank-scale genotype pretraining as an early step toward foundation modeling for individual-level genotype data.
SN - 0305-1048
SN - 1362-4962
ER -