Universal Estimation of the Fisher Information for Processes with Memory
Abstract
The Fisher information matrix (FIM) determines the accuracy attainable in an estimation problem. For a parametric process with memory, it has a closed form only for narrow classes of models, and it cannot be computed when the model is available only as a simulator. This paper estimates the Fisher information rate of such a process, the per-observation limit of its FIM, from simulator output alone and without knowledge of the memory length. The construction uses the identity between the FIM and the local curvature of the Kullback--Leibler (KL) divergence. Directional divergence rates are estimated by context-tree weighting (CTW), and the matrix is recovered from them by least squares. We establish strong consistency in an iterated limit and a finite-sample bound, uniform over every working depth at least as large as the true memory length, which gives a mean error of order n−1/3. The analysis also delivers a finite-sample rate for CTW-based estimation of the KL divergence rate. Experiments confirm the predicted exponents, show that the accuracy does not degrade as the working depth grows, and use the estimate to plan a sampling budget that attains a prescribed precision. The estimator stays accurate on a hidden-state source that has no finite memory length.
Related
- On the parametric and semiparametric Fisher information matrix for non-zero mean stationary spherical invariant random processes
- Information Lower Bound for Regime Boundary Estimation — Optimal Precision Limit at the n² Scale and Universal Constant c=1/8 in the Fixed-Breakpoint Model
- Quadratic variations of rough generalized Hermite processes: hidden long memory and a universal Gaussian boundary field
- Multi-type Λ-Wright--Fisher processes with polynomial drift: Bernstein duality, long-term behaviour, and simulation of moments
- On the discrete Fisher information
BibTeX
@misc{shi2026universal,
title = {{Universal Estimation of the Fisher Information for Processes with Memory}},
author = {Yibo Shi and Cristian R. Rojas},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.14582},
eprint = {2609.14582},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.14582},
}
RIS
TY - UNPB
TI - Universal Estimation of the Fisher Information for Processes with Memory
AU - Yibo Shi
AU - Cristian R. Rojas
PY - 2026
DA - 2026/09/13
PB - arXiv
DO - 10.48550/arxiv.2609.14582
UR - https://arxiv.org/abs/2609.14582
AB - The Fisher information matrix (FIM) determines the accuracy attainable in an estimation problem. For a parametric process with memory, it has a closed form only for narrow classes of models, and it cannot be computed when the model is available only as a simulator. This paper estimates the Fisher information rate of such a process, the per-observation limit of its FIM, from simulator output alone and without knowledge of the memory length. The construction uses the identity between the FIM and the local curvature of the Kullback--Leibler (KL) divergence. Directional divergence rates are estimated by context-tree weighting (CTW), and the matrix is recovered from them by least squares. We establish strong consistency in an iterated limit and a finite-sample bound, uniform over every working depth at least as large as the true memory length, which gives a mean error of order n^−1/3. The analysis also delivers a finite-sample rate for CTW-based estimation of the KL divergence rate. Experiments confirm the predicted exponents, show that the accuracy does not degrade as the working depth grows, and use the estimate to plan a sampling budget that attains a prescribed precision. The estimator stays accurate on a hidden-state source that has no finite memory length.
ER -