A Poisson Transition and Bias Law for KSG Mutual-Information Estimation
Abstract
We quantify the error of the strict-radius Kraskov--Stögbauer--Grassberger (KSG-1) mutual-information estimator on a Gaussian channel as the sample size n grows and the channel becomes nearly deterministic. For fixed dimension p and neighbor index k, we define the resolution parameter un=ne−In, where In is the true mutual information in nats. When un→ u∈(0,∞), we prove that the estimation error converges in mean square to a computable deterministic bias Bp,k(u). Sampling fluctuations therefore vanish, while an additive bias can persist. We derive this bias from an explicit marked-Poisson law for the neighborhood counts, characterize its small-u asymptotics, and show that it tends to zero as u→∞. The same law describes how often neighborhoods saturate, meaning that they make the largest possible contribution to the estimate. The limiting saturation probability decreases continuously and strictly from one to zero, allowing a Gaussian-model bias correction on fixed critical intervals. Finite-sample experiments show substantial bias even when the true information lies below the estimator's logarithmic ceiling and saturated neighborhoods are rare. Saturation-based correction reduces estimation error in these experiments, though recovering the resolution parameter from the observed saturation fraction can remain difficult when saturation is rare.
Related
- Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance
- Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
- Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation
- A Sharp Mutual Information Bound for Binary Pairs
- Interchanging limits in the mutual information of two intervals separated by one site
BibTeX
@misc{gorle2026poisson,
title = {{A Poisson Transition and Bias Law for KSG Mutual-Information Estimation}},
author = {Abhiram Gorle and Tsachy Weissman},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37941},
eprint = {2610.37941},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37941},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - A Poisson Transition and Bias Law for KSG Mutual-Information Estimation
AU - Abhiram Gorle
AU - Tsachy Weissman
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37941
UR - https://arxiv.org/abs/2610.37941
AB - We quantify the error of the strict-radius Kraskov--Stögbauer--Grassberger (KSG-1) mutual-information estimator on a Gaussian channel as the sample size n grows and the channel becomes nearly deterministic. For fixed dimension p and neighbor index k, we define the resolution parameter u_n=ne^−I_n, where I_n is the true mutual information in nats. When u_n→ u∈(0,∞), we prove that the estimation error converges in mean square to a computable deterministic bias B_p,k(u). Sampling fluctuations therefore vanish, while an additive bias can persist. We derive this bias from an explicit marked-Poisson law for the neighborhood counts, characterize its small-u asymptotics, and show that it tends to zero as u→∞. The same law describes how often neighborhoods saturate, meaning that they make the largest possible contribution to the estimate. The limiting saturation probability decreases continuously and strictly from one to zero, allowing a Gaussian-model bias correction on fixed critical intervals. Finite-sample experiments show substantial bias even when the true information lies below the estimator's logarithmic ceiling and saturated neighborhoods are rare. Saturation-based correction reduces estimation error in these experiments, though recovering the resolution parameter from the observed saturation fraction can remain difficult when saturation is rare.
ER -