Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
Abstract
We study finite sample guarantees for transport based mutual information estimation. Mutual information can be estimated by first learning the conditional and marginal densities and then evaluating their log-density ratio by Monte Carlo. Given training samples, we use triangular transport maps to learn these densities and then compute the mutual information estimate on an independent batch of evaluation samples. For conditional density estimation using triangular transport maps, we establish non-asymptotic guarantees over realizable Hölder classes by viewing the conditional density as an ordinary density with respect to the product of the law of the conditioning variable and Lebesgue measure. This allows standard density estimation techniques to be applied directly to the conditional density estimation. These results yield finite sample guarantees for the resulting mutual information estimator and characterize how a fixed sample budget should be allocated between training and evaluation in order to minimize the mean squared error. Numerical experiments support the predicted training and evaluation error rates and illustrate the resulting sample allocation tradeoff.
Related
- The discontinuous Galerkin method for solving the two-dimensional transport equation with distributed-order memory kernel on triangular meshes
- Estimation of the Label-Noise Transition Matrix with Performance Guarantees via Selective Classification
- Robust dimension-free estimation of simple random tensors: optimal guarantees under heavy tails and adversarial contamination
- Neural Estimation of Summary Statistics for Simulation-Based Inference with Non-Asymptotic Guarantees
- Statistical Guarantees for Covariance Estimation with Adversarial Corruption under Constraints
BibTeX
@misc{li2026mutual,
title = {{Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation}},
author = {Fengyi Li},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.00471},
eprint = {2611.00471},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.00471},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Mutual Information Estimation via Triangular Transport Maps: Finite Sample Guarantees and Sample Allocation
AU - Fengyi Li
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.00471
UR - https://arxiv.org/abs/2611.00471
AB - We study finite sample guarantees for transport based mutual information estimation. Mutual information can be estimated by first learning the conditional and marginal densities and then evaluating their log-density ratio by Monte Carlo. Given training samples, we use triangular transport maps to learn these densities and then compute the mutual information estimate on an independent batch of evaluation samples. For conditional density estimation using triangular transport maps, we establish non-asymptotic guarantees over realizable Hölder classes by viewing the conditional density as an ordinary density with respect to the product of the law of the conditioning variable and Lebesgue measure. This allows standard density estimation techniques to be applied directly to the conditional density estimation. These results yield finite sample guarantees for the resulting mutual information estimator and characterize how a fixed sample budget should be allocated between training and evaluation in order to minimize the mean squared error. Numerical experiments support the predicted training and evaluation error rates and illustrate the resulting sample allocation tradeoff.
ER -