Propensity Straight-Through Gradients for Discrete Stochastic Systems
Abstract
We introduce the propensity straight-through (PST) estimator for gradient-based learning through exact simulations of continuous-time Markov chains (CTMCs). These models describe discrete stochastic dynamics across physics, chemistry, and biology, but their categorical event choices prevent ordinary differentiation. PST preserves exact forward trajectories and differentiates, in the backward pass, the normalized reaction propensities the simulator already computes. The affine state update makes this rule exact for the one-step conditional-mean sensitivity. We derive the per-step discrepancy for one-step sensitivities composed along nonlinear trajectories in closed form, show that it vanishes for affine downstream dependence, and bound it quadratically in the stoichiometric jump size. Across reversible dimerization, a genetic oscillator, a 50-task repressilator suite, and experimental patch-clamp recordings, PST recovers parameters to within 0.06-1.7% and converges 2.4-3.0-fold faster than the Gumbel-Softmax straight-through estimator at matched computational cost. At deep-learning scale, PST trains all 203,796 parameters of a stochastic reaction network using exact discrete events from initialization. By differentiating an exact conditional mean, PST eliminates reaction-selection temperature tuning and provides a scalable route to learning through exact stochastic dynamics.
Related
- Decoding plant x environment interactions through spatial gradients in leaf physiology and metabolism
- Impact of Salinity on Rice Production in Vulnerable Southern Coastal Regions of Bangladesh: A Case Study Through a Cobb–Douglas Stochastic Production Function Model
- Assessing the impact of intervention strategies on the multidimensional burden of Mycoplasma hyopneumoniae infection in fattening pigs through a stochastic model
- A Repulsive Bias in Perceived Heading Away from the Straight-Ahead as Determined by Continuous Psychophysics
- Distinct electroencephalography alpha-band rhythms map macroscale brain gradients
BibTeX
@misc{vilar2026propensity,
title = {{Propensity Straight-Through Gradients for Discrete Stochastic Systems}},
author = {Jose M. G. Vilar and Leonor Saiz},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2608.25631},
eprint = {2608.25631},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.25631},
}
RIS
TY - UNPB
TI - Propensity Straight-Through Gradients for Discrete Stochastic Systems
AU - Jose M. G. Vilar
AU - Leonor Saiz
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2608.25631
UR - https://arxiv.org/abs/2608.25631
AB - We introduce the propensity straight-through (PST) estimator for gradient-based learning through exact simulations of continuous-time Markov chains (CTMCs). These models describe discrete stochastic dynamics across physics, chemistry, and biology, but their categorical event choices prevent ordinary differentiation. PST preserves exact forward trajectories and differentiates, in the backward pass, the normalized reaction propensities the simulator already computes. The affine state update makes this rule exact for the one-step conditional-mean sensitivity. We derive the per-step discrepancy for one-step sensitivities composed along nonlinear trajectories in closed form, show that it vanishes for affine downstream dependence, and bound it quadratically in the stoichiometric jump size. Across reversible dimerization, a genetic oscillator, a 50-task repressilator suite, and experimental patch-clamp recordings, PST recovers parameters to within 0.06-1.7% and converges 2.4-3.0-fold faster than the Gumbel-Softmax straight-through estimator at matched computational cost. At deep-learning scale, PST trains all 203,796 parameters of a stochastic reaction network using exact discrete events from initialization. By differentiating an exact conditional mean, PST eliminates reaction-selection temperature tuning and provides a scalable route to learning through exact stochastic dynamics.
ER -