Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance
Abstract
This paper studies the finite-sample bias of plug-in estimators for joint entropy, conditional entropy, and mutual information for finitely supported discrete random variables. Using the univariate reduction framework developed in our companion works, we derive explicit first-order bias formulas. For a pair (X,Y) with support sizes r and s, the biases are respectively -(rs-1)/(2n), -r(s-1)/(2n), and (r-1)(s-1)/(2n). We then propose bias-corrected estimators for the three measures and prove that the uncorrected and corrected versions are asymptotically equivalent, differing by a deterministic term of order O(1/n); consequently, they share the same asymptotic distribution. A simulation study validates the theoretical results, including a dedicated independence study and the relationship with Wilks' theorem. An application to a motor insurance portfolio, where X is the driver's age class and Y the claim severity class, illustrates the improvement brought by the bias correction for risk classification.
Related
- A New Bivariate Excess Wealth Transform Based on Conditional Random Variables
- Design of Privacy-Preserving Estimators Under Prior Information Exposure
- infomeasure-rs: A High-Performance Rust Library for Information-Theoretic Measures and Estimators
- Bivariate Frank Copula: Some More Results on Point Estimation of the Association Parameter from a Bayesian Perspective and Revisiting the Goodness of Fit Tests with an Application to Model Groundwater Data from Dong Thap, Vietnam
- A unifying theory for the evaluation of conditional Akaike information for mixed-effects models
BibTeX
@misc{ba2026biascorrected,
title = {{Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance}},
author = {Amadou Diadie Ba},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.02558},
eprint = {2610.02558},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.02558},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Bias-Corrected Estimators for Joint Entropy, Conditional Entropy, and Mutual Information in the Discrete Bivariate Case: Theory and an Application to Motor Insurance
AU - Amadou Diadie Ba
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.02558
UR - https://arxiv.org/abs/2610.02558
AB - This paper studies the finite-sample bias of plug-in estimators for joint entropy, conditional entropy, and mutual information for finitely supported discrete random variables. Using the univariate reduction framework developed in our companion works, we derive explicit first-order bias formulas. For a pair (X,Y) with support sizes r and s, the biases are respectively -(rs-1)/(2n), -r(s-1)/(2n), and (r-1)(s-1)/(2n). We then propose bias-corrected estimators for the three measures and prove that the uncorrected and corrected versions are asymptotically equivalent, differing by a deterministic term of order O(1/n); consequently, they share the same asymptotic distribution. A simulation study validates the theoretical results, including a dedicated independence study and the relationship with Wilks' theorem. An application to a motor insurance portfolio, where X is the driver's age class and Y the claim severity class, illustrates the improvement brought by the bias correction for risk classification.
ER -