Frequency-Domain Evaluation and Estimation of Semi-Markov Disease Natural History Models
Abstract
Introduction: Natural history models must often be estimated from data that do not directly capture disease transitions. Semi-Markov processes let the risk of progression depend on the time spent in a health state, which Markov models need extra states to represent. Their limitation is computational: discretizing time introduces approximation error, and simulating individuals introduces Monte Carlo error and a non-differentiable calibration objective.
Methods: We show that the model-implied prevalence, incidence and cause-specific mortality of a continuous-time semi-Markov model satisfy convolution equations that become a small linear system when Laplace-transformed. Solving it on a grid of complex frequencies and inverting with an accelerated Fourier series returns the targets without a time grid or Monte Carlo error, and yields exact first- and second-order derivatives for one additional linear solve each. These support Newton-type maximum likelihood estimation, gradient-informed Bayesian sampling, and local identifiability diagnostics. We apply our open-source implementation to a urinary bladder cancer model calibrated to SEER incidence and mortality by age and stage.
Results: The method reproduced an independent implementation to within its accuracy (maximum relative difference 8 × 10 −6 ) and evaluated all targets, their gradient, and their Hessian in 1.3, 2.5, and 9.1 milliseconds; a one-year-cycle cohort approximation deviated by up to 42% and a two-million-person microsimulation by up to 811%. Maximum likelihood and Bayesian estimates agreed, and second-order information showed that the aggregate targets constrain where individuals go from preclinical states far better than how long they stay. Early-stage incidence and mortality showed systematic misfit across age.
Conclusions: The framework supplies existing calibration algorithms with fast, exact model evaluations and derivatives for models whose transition hazards depend on time in state rather than on age.
Highlights: The age-specific prevalence, incidence, and cause-specific mortality implied by a continuous-time semi-Markov model can be computed in the frequency domain without time discretization and without Monte Carlo error.
The same computation returns exact gradients and Hessians of the calibration objective, which enables Newton-type optimization, gradient-informed Bayesian calibration, and local identifiability diagnostics that derivative-free calibration cannot provide.
The framework supplies model evaluations and derivatives to existing calibration algorithms rather than replacing them, as we demonstrate using a urinary bladder cancer model fitted to aggregate SEER data.
Open-source software implements the framework, including competing risks and age-specific background mortality rates.
Related
- LLM-enabled Natural History Study Analysis to Support Rare Disease Research
- Hypertension disease natural history and healthcare utilization when treatment guidelines do not account for race
- The natural history and individualized prediction of liver stiffness-based fibrosis risk in metabolic dysfunction-associated steatotic liver disease
- Eight quick tips for requesting materials from natural history collections
- The Vanishing White Matter registry: a unique source of natural history data
BibTeX
@misc{subiabre2026frequencydomain,
title = {{Frequency-Domain Evaluation and Estimation of Semi-Markov Disease Natural History Models}},
author = {Felipe Subiabre and Jeremy D. Goldhaber-Fiebert and Fernando Alarid-Escudero},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.23.26366343},
url = {https://www.medrxiv.org/content/10.64898/2026.10.23.26366343v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Frequency-Domain Evaluation and Estimation of Semi-Markov Disease Natural History Models
AU - Felipe Subiabre
AU - Jeremy D. Goldhaber-Fiebert
AU - Fernando Alarid-Escudero
PY - 2026
DA - 2026/10/23
PB - medRxiv
DO - 10.64898/2026.10.23.26366343
UR - https://www.medrxiv.org/content/10.64898/2026.10.23.26366343v1
AB - Introduction: Natural history models must often be estimated from data that do not directly capture disease transitions. Semi-Markov processes let the risk of progression depend on the time spent in a health state, which Markov models need extra states to represent. Their limitation is computational: discretizing time introduces approximation error, and simulating individuals introduces Monte Carlo error and a non-differentiable calibration objective. Methods: We show that the model-implied prevalence, incidence and cause-specific mortality of a continuous-time semi-Markov model satisfy convolution equations that become a small linear system when Laplace-transformed. Solving it on a grid of complex frequencies and inverting with an accelerated Fourier series returns the targets without a time grid or Monte Carlo error, and yields exact first- and second-order derivatives for one additional linear solve each. These support Newton-type maximum likelihood estimation, gradient-informed Bayesian sampling, and local identifiability diagnostics. We apply our open-source implementation to a urinary bladder cancer model calibrated to SEER incidence and mortality by age and stage. Results: The method reproduced an independent implementation to within its accuracy (maximum relative difference 8 × 10 −6 ) and evaluated all targets, their gradient, and their Hessian in 1.3, 2.5, and 9.1 milliseconds; a one-year-cycle cohort approximation deviated by up to 42% and a two-million-person microsimulation by up to 811%. Maximum likelihood and Bayesian estimates agreed, and second-order information showed that the aggregate targets constrain where individuals go from preclinical states far better than how long they stay. Early-stage incidence and mortality showed systematic misfit across age. Conclusions: The framework supplies existing calibration algorithms with fast, exact model evaluations and derivatives for models whose transition hazards depend on time in state rather than on age. Highlights: The age-specific prevalence, incidence, and cause-specific mortality implied by a continuous-time semi-Markov model can be computed in the frequency domain without time discretization and without Monte Carlo error. The same computation returns exact gradients and Hessians of the calibration objective, which enables Newton-type optimization, gradient-informed Bayesian calibration, and local identifiability diagnostics that derivative-free calibration cannot provide. The framework supplies model evaluations and derivatives to existing calibration algorithms rather than replacing them, as we demonstrate using a urinary bladder cancer model fitted to aggregate SEER data. Open-source software implements the framework, including competing risks and age-specific background mortality rates.
ER -