Computing the entropy rate of a quantized stationary Gaussian process
Abstract
The entropy rate of a uniformly quantized stationary Gaussian process is the fundamental limit on its lossless compression. It has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small on part of the band, as routine smoothing and filtering produce. Here we present two methods for computing the rate. The first is an analytical approximation, obtained from an exact dithering identity and the Kolmogorov-Szegő formula, in which the quantization noise power acts as a floor on the spectral density. The second is a Monte Carlo estimate of the exact rate: a minimum-phase moving-average representation makes the quantized sequence a hidden Markov process whose optimal particle filter is available in closed form, and the filter's log-likelihood on one long sequence gives the rate. The approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including processes on which the classical formula fails; for coarser steps the estimate should be used instead. At a step of a quarter of the standard deviation, linear predictive coding comes within 2 to 4 percent of the rate, FLAC within 3 to 32 percent, and general-purpose compressors remain 12 to 89 percent above it.
Related
- A Gaussian Process Regression-Enhanced Two-Sweep Parameterized Power Method for Computing Extreme Eigenpairs
- Fundamental Limits of Quantized MIMO ISAC under Gaussian Signaling
- Computing Stationary Equilibria in Measure-Dependent Markov Systems
- Hamiltonian stationary Lagrangian surfaces of constant Gaussian curvature in complex space forms
- Dynamical and stationary non-equilibrium fluctuations for a boundary-driven exclusion process with energy
BibTeX
@misc{magland2026computing,
title = {{Computing the entropy rate of a quantized stationary Gaussian process}},
author = {Jeremy Magland},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.18784},
eprint = {2609.18784},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.18784},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Computing the entropy rate of a quantized stationary Gaussian process
AU - Jeremy Magland
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2609.18784
UR - https://arxiv.org/abs/2609.18784
AB - The entropy rate of a uniformly quantized stationary Gaussian process is the fundamental limit on its lossless compression. It has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small on part of the band, as routine smoothing and filtering produce. Here we present two methods for computing the rate. The first is an analytical approximation, obtained from an exact dithering identity and the Kolmogorov-Szegő formula, in which the quantization noise power acts as a floor on the spectral density. The second is a Monte Carlo estimate of the exact rate: a minimum-phase moving-average representation makes the quantized sequence a hidden Markov process whose optimal particle filter is available in closed form, and the filter's log-likelihood on one long sequence gives the rate. The approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including processes on which the classical formula fails; for coarser steps the estimate should be used instead. At a step of a quarter of the standard deviation, linear predictive coding comes within 2 to 4 percent of the rate, FLAC within 3 to 32 percent, and general-purpose compressors remain 12 to 89 percent above it.
ER -