Reducing double-dipping in single-cell topic models with three-way count splitting
Abstract
Topic models learn gene programs from single-cell RNA sequencing (scRNA-seq) data, and the same counts are then used to test which genes and donors are associated with those programs. Count splitting is the standard remedy for this reuse, but it has not been tested when an encoder reads each cell's counts. On a 104,400-cell lupus cohort with 300 permuted null genes per run, naive tests rejected 36% of null gene-topic pairs at a nominal 5% level, and the usual two-way split rejected 38%, because the encoder sees the tested gene's own held-out count. Encoding topic usage on a separate third of the counts, with beta-binomial thinning, lowered this to 7.7% (6.7% for genes outside the model); for two other models, LDVAE and scETM, the rate was 7.0-7.3% (naive 51-59%). A label-free interferon program shifts in held-out counts in the lupus cohort and in a pre-specified COVID-19 cohort, and donor-level topic usage predicts disease (AUROC 0.96) beyond cell-type proportions (0.86 and 0.58 in the two cohorts). Calibration and prediction did not depend on the model. We do not claim better clustering, prediction or topic stability. Code, figures and result tables are public (MIT license; DOI 10.5281/zenodo.23307193).
Related
- "Double-dipping" in testing for phenotypic differences with scRNA-Seq
- In Vitro Antimicrobial Activity of Chlorhexidine-Curcumin Nanoformulations for Teat Dipping Against Mastitis Pathogens.
- Mammalian spermatogenesis at single-cell resolution downstream of genetic perturbations in meiotic double-strand break repair
- Safety and Efficacy of Single Femoral Vein Puncture Technique for Double Access in Patent Foramen Ovale Closure Under ICE Guidance—A Single Center Experience of the “Two‐In‐One Technique”
- Cross-platform public judgment in a cultural heritage crisis: A topic modeling study of the Nanjing Museum donated-artifact controversy
BibTeX
@misc{qu2026reducing,
title = {{Reducing double-dipping in single-cell topic models with three-way count splitting}},
author = {Zheng Qu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.24130},
eprint = {2610.24130},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24130},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Reducing double-dipping in single-cell topic models with three-way count splitting
AU - Zheng Qu
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2610.24130
UR - https://arxiv.org/abs/2610.24130
AB - Topic models learn gene programs from single-cell RNA sequencing (scRNA-seq) data, and the same counts are then used to test which genes and donors are associated with those programs. Count splitting is the standard remedy for this reuse, but it has not been tested when an encoder reads each cell's counts. On a 104,400-cell lupus cohort with 300 permuted null genes per run, naive tests rejected 36% of null gene-topic pairs at a nominal 5% level, and the usual two-way split rejected 38%, because the encoder sees the tested gene's own held-out count. Encoding topic usage on a separate third of the counts, with beta-binomial thinning, lowered this to 7.7% (6.7% for genes outside the model); for two other models, LDVAE and scETM, the rate was 7.0-7.3% (naive 51-59%). A label-free interferon program shifts in held-out counts in the lupus cohort and in a pre-specified COVID-19 cohort, and donor-level topic usage predicts disease (AUROC 0.96) beyond cell-type proportions (0.86 and 0.58 in the two cohorts). Calibration and prediction did not depend on the model. We do not claim better clustering, prediction or topic stability. Code, figures and result tables are public (MIT license; DOI 10.5281/zenodo.23307193).
ER -