A decision-tree for trend analyses following major survey redesigns: an analysis using the Canadian Community Health Survey (CCHS)
- 1Public Health Agency of Canada, Health Equity Division, Ottawa, Canada
- 2University of Toronto, Dalla Lana School of Public Health, Toronto, Canada
- 3York University, School of Health Policy and Management, Toronto, Canada
Abstract
Background: Monitoring health inequality trends is essential for assessing progress, stagnation, or regression on the reduction of health inequalities; assessing the impact of certain policies on health inequalities; and identifying emerging areas of concern. Because inequalities can change slowly, long-term data (≥ 15 years) may be required to detect meaningful shifts. However, methodological changes in data sources over time raise concerns about data comparability. Frameworks are needed to guide decisions on whether data should be combined in these instances.
Methods: We used the Canadian Community Health Survey (CCHS) as a case to explore how sensitivity analyses and a decision-tree could help assess the comparability of survey cycles before and after major CCHS redesigns in 2015 and 2022, balancing considerations of statistical and practical significance, and background trends.
Using CCHS cycles 2007–2008 to 2024, we estimated crude prevalence of 20 health indicators across cycles, among adults. We assessed: 1) the relative percent (%) change and 2) absolute percentage-point (PP) change between cycles pre- and post-redesigns (i.e., 2013–2014 vs. 2015–2016, 2019–2021 vs. 2022), including tests of statistical significance. Then, using generalized linear models and their predicted probabilities, we modeled the predicted post-redesign prevalence (for 2015–2016, 2022), based on pre-redesign trends (2007–2014, 2015–2021, respectively). Drawing from literature-based thresholds, the proposed decision-tree was used to flag indicators that met the three criteria described below. These indicators were subjected to additional qualitative investigation. Specifically, the flagged indicators must have demonstrated practically significant change on: A) relative (≥ 10% change) and B) absolute scales (≥ 2.5- or 5-PP change, based on baseline prevalence); and C) statistically significant change (no overlap with the margin of equivalence (95% CI) between observed and predicted post-redesign prevalence).
Results: No indicators met all three criteria post-2015. Post-2022, food insecurity, and self-rated physical and mental health marginally met all three criteria, signaling a need for additional research for long-term reporting.
Conclusions: This study presents a novel and reproducible framework for CCHS-specific analyses that others can refine or adapt. As it remains sensitive to assumptions, caution notes regarding the redesigns continue to be warranted for all CCHS indicators.
Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s12889-026-28851-7.
Related
- Clinical, Behavioral, and Sociodemographic Factors Associated With Self-Reported High Blood Pressure Among Canadian Adults: A Cross-Sectional Analysis of the 2022 Canadian Community Health Survey
- Dynamics of Canadian perceptions of low-dose radiation following science-based educational communication sessions
- mHealth-based lifestyle intervention for hypertension in community-dwelling adults: a cluster-randomized controlled trial with exploratory multi-omics analyses
- Impact of different protein sources on acute muscle protein synthesis following resistance training: a systematic review with standard and dose–response network meta-analyses
- An assisted telemedicine model with digital diagnostics and artificial intelligence-supported clinical decision in hard-to-reach rural communities: a community-based mixed-methods study from Rajasthan, India
BibTeX
@article{blair2026decisiontree,
title = {{A decision-tree for trend analyses following major survey redesigns: an analysis using the Canadian Community Health Survey (CCHS)}},
author = {Alexandra Blair and Matthew E. Perks and Rabina Jahan and Eric Vallières and Le Li and Dolon Chakravartty and Beth Jackson},
journal = {BMC Public Health},
year = {2026},
month = aug,
doi = {10.1186/s12889-026-28851-7},
url = {https://doi.org/10.1186/s12889-026-28851-7},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A decision-tree for trend analyses following major survey redesigns: an analysis using the Canadian Community Health Survey (CCHS)
AU - Alexandra Blair
AU - Matthew E. Perks
AU - Rabina Jahan
AU - Eric Vallières
AU - Le Li
AU - Dolon Chakravartty
AU - Beth Jackson
PY - 2026
DA - 2026/08/01
JO - BMC Public Health
DO - 10.1186/s12889-026-28851-7
UR - https://doi.org/10.1186/s12889-026-28851-7
AB - Background: Monitoring health inequality trends is essential for assessing progress, stagnation, or regression on the reduction of health inequalities; assessing the impact of certain policies on health inequalities; and identifying emerging areas of concern. Because inequalities can change slowly, long-term data (≥ 15 years) may be required to detect meaningful shifts. However, methodological changes in data sources over time raise concerns about data comparability. Frameworks are needed to guide decisions on whether data should be combined in these instances. Methods: We used the Canadian Community Health Survey (CCHS) as a case to explore how sensitivity analyses and a decision-tree could help assess the comparability of survey cycles before and after major CCHS redesigns in 2015 and 2022, balancing considerations of statistical and practical significance, and background trends. Using CCHS cycles 2007–2008 to 2024, we estimated crude prevalence of 20 health indicators across cycles, among adults. We assessed: 1) the relative percent (%) change and 2) absolute percentage-point (PP) change between cycles pre- and post-redesigns (i.e., 2013–2014 vs. 2015–2016, 2019–2021 vs. 2022), including tests of statistical significance. Then, using generalized linear models and their predicted probabilities, we modeled the predicted post-redesign prevalence (for 2015–2016, 2022), based on pre-redesign trends (2007–2014, 2015–2021, respectively). Drawing from literature-based thresholds, the proposed decision-tree was used to flag indicators that met the three criteria described below. These indicators were subjected to additional qualitative investigation. Specifically, the flagged indicators must have demonstrated practically significant change on: A) relative (≥ 10% change) and B) absolute scales (≥ 2.5- or 5-PP change, based on baseline prevalence); and C) statistically significant change (no overlap with the margin of equivalence (95% CI) between observed and predicted post-redesign prevalence). Results: No indicators met all three criteria post-2015. Post-2022, food insecurity, and self-rated physical and mental health marginally met all three criteria, signaling a need for additional research for long-term reporting. Conclusions: This study presents a novel and reproducible framework for CCHS-specific analyses that others can refine or adapt. As it remains sensitive to assumptions, caution notes regarding the redesigns continue to be warranted for all CCHS indicators. Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1186/s12889-026-28851-7.
SN - 1471-2458
ER -