PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise
Abstract
In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.
Related
- A Stochastic Maximum Principle for Partially Observed Jump-Diffusion Systems with State-Dependent Counting-Process Observations
- Bayesian Inference in Partially Observed Critical Systems
- Structural identifiability of partially-observed stochastic processes: from single-particle trajectories to total particle density data
- Defense Filters for Robust Learning of Partially Observed Dynamical Systems
- Deep operator learning for efficient sampling from invariant measures of stochastic differential equations
BibTeX
@misc{petreczky2026pacbayesian,
title = {{PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise}},
author = {Mihaly Petreczky and Mohamad Al Ahdab and John Leth},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.08740},
eprint = {2609.08740},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.08740},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - PAC-Bayesian Bounds for Learning Partially Observed Stochastic Linear Time-Invariant State-Space Systems with Inputs and Sub-Gaussian Noise
AU - Mihaly Petreczky
AU - Mohamad Al Ahdab
AU - John Leth
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2609.08740
UR - https://arxiv.org/abs/2609.08740
AB - In this paper we derive a Probably Approximately Correct (PAC)-Bayesian error bound for partially observed linear time-invariant (LTI) stochastic dynamical systems in state-space form with inputs and sub-Gaussian noise. Such bounds are widespread in machine learning, and they are useful for characterizing the predictive power of models learned from finitely many data points. The bound derived in this paper relates the expectation of prediction errors with the prediction error generated by the model on the data used for learning. In addition, we show that it can also be used to derive bounds for the parameter estimation error. In turn, this allows us to provide finite-sample error bounds for the prediction error and parameter estimation error for a wide class of system identification algorithms. Furthermore, as LTI systems are a sub-class of recurrent neural networks (RNNs), these error bounds could be a first step towards PAC-Bayesian bounds for RNNs.
ER -