A questionnaire-based model to prioritize additional support during HIV testing encounters among men who have sex with men: temporal and geographic external validation
4 affiliations
- 1Department of Information Management, Nantong Center for Disease Control and Prevention
- 2Office of Comprehensive Affairs, Nantong Center for Disease Control and Prevention
- 3Department of HIV/AIDS and STI Prevention and Control, Nantong Center for Disease Control and Prevention
- 4Department of Medical and Preventive Care Integration, Nantong Center for Disease Control and Prevention
Abstract
Background: HIV sentinel surveillance collects behavioral and service-use information from men who have sex with men (MSM), but these data are mainly used for routine reporting. We assessed whether pre-test questionnaire data could help prioritize testing encounters for optional additional support.
Methods: We conducted a retrospective prediction modeling study using de-identified MSM surveillance records. Data from Nantong collected in 2020–2023 were used for model development, and records from 2024–2025 were used for temporal validation. Surveillance records from an external city in Shandong Province were used for geographic external validation. The final model was a calibrated stacked ensemble combining penalized logistic regression, random forest, and HistGradientBoosting. We evaluated the complete modeling and calibration workflow using nested cross-fitting. Model selection, calibration, and threshold selection used development data only.
Results: The development cohort included 1,585 encounters and 73 confirmed HIV-positive events. Temporal validation included 803 encounters and 29 confirmed HIV-positive events, while geographic external validation included 1,622 encounters and 52 confirmed HIV-positive events. AUROC values were 0.963 (95% CI 0.930–0.990) and 0.974 (95% CI 0.955–0.989) in temporal and geographic validation, respectively. The corresponding AUPRC values were 0.696 (95% CI 0.536–0.880) and 0.746 (95% CI 0.652–0.842). At the locked threshold, sensitivity was 75.9 and 86.5%. Selecting the highest-scoring 10% of encounters captured 86.2 and 88.5% of confirmed events. Calibration slopes were 1.530 and 1.414, respectively. The model overestimated overall risk in geographic validation.
Conclusion: Pre-test questionnaire data supported the ranking of HIV testing encounters in both temporal and geographic validation. The model may help services plan optional counseling, prevention assessment, peer support, or follow-up when service capacity is limited. The scores should not be treated as exact individual probabilities. They should not be used for HIV diagnosis or to restrict access to testing or other standard services. Prospective evaluation and monitoring are needed before routine use.
Related
- A Reproducible Decomposed Large Language Model Pipeline for Structured Abstraction of Multimodal Neuroimaging Reports: Development and Independent Validation in 1,669 Emergency Department Encounters for Dizziness and Vertigo
- Translation and Validation of the 5-Item Dry Eye Questionnaire into Lithuanian
- Validation of the Brief-Cope Questionnaire in a Seropositive Rheumatoid Arthritis Population
- Factor structure and psychometric validation of the antenatal exercise knowledge, attitudes, and practices questionnaire : A secondary analysis of cross-sectional data from Saudi Arabian pregnant women
- Development and Validation of the Food Expectation Questionnaire (FEX‐Q): An Image‐Based Instrument to Assess Food‐Related Perceptions and Symptom Expectations
BibTeX
@article{zhao2026questionnairebased,
title = {{A questionnaire-based model to prioritize additional support during HIV testing encounters among men who have sex with men: temporal and geographic external validation}},
author = {Jian Zhao and Yazhou Yang and Ping Zhu and Shilin Zheng and Xiaowei Xu and Xiaoyi Zhou},
journal = {Frontiers in Public Health},
year = {2026},
month = oct,
doi = {10.3389/fpubh.2026.1944057},
url = {https://www.frontiersin.org/articles/10.3389/fpubh.2026.1944057/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A questionnaire-based model to prioritize additional support during HIV testing encounters among men who have sex with men: temporal and geographic external validation
AU - Jian Zhao
AU - Yazhou Yang
AU - Ping Zhu
AU - Shilin Zheng
AU - Xiaowei Xu
AU - Xiaoyi Zhou
PY - 2026
DA - 2026/10/16
JO - Frontiers in Public Health
DO - 10.3389/fpubh.2026.1944057
UR - https://www.frontiersin.org/articles/10.3389/fpubh.2026.1944057/full
AB - Background: HIV sentinel surveillance collects behavioral and service-use information from men who have sex with men (MSM), but these data are mainly used for routine reporting. We assessed whether pre-test questionnaire data could help prioritize testing encounters for optional additional support. Methods: We conducted a retrospective prediction modeling study using de-identified MSM surveillance records. Data from Nantong collected in 2020–2023 were used for model development, and records from 2024–2025 were used for temporal validation. Surveillance records from an external city in Shandong Province were used for geographic external validation. The final model was a calibrated stacked ensemble combining penalized logistic regression, random forest, and HistGradientBoosting. We evaluated the complete modeling and calibration workflow using nested cross-fitting. Model selection, calibration, and threshold selection used development data only. Results: The development cohort included 1,585 encounters and 73 confirmed HIV-positive events. Temporal validation included 803 encounters and 29 confirmed HIV-positive events, while geographic external validation included 1,622 encounters and 52 confirmed HIV-positive events. AUROC values were 0.963 (95% CI 0.930–0.990) and 0.974 (95% CI 0.955–0.989) in temporal and geographic validation, respectively. The corresponding AUPRC values were 0.696 (95% CI 0.536–0.880) and 0.746 (95% CI 0.652–0.842). At the locked threshold, sensitivity was 75.9 and 86.5%. Selecting the highest-scoring 10% of encounters captured 86.2 and 88.5% of confirmed events. Calibration slopes were 1.530 and 1.414, respectively. The model overestimated overall risk in geographic validation. Conclusion: Pre-test questionnaire data supported the ranking of HIV testing encounters in both temporal and geographic validation. The model may help services plan optional counseling, prevention assessment, peer support, or follow-up when service capacity is limited. The scores should not be treated as exact individual probabilities. They should not be used for HIV diagnosis or to restrict access to testing or other standard services. Prospective evaluation and monitoring are needed before routine use.
SN - 2296-2565
ER -