Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator
Abstract
It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper strengthens this classical result by developing several stronger forms of asymptotic normality for the normalized MLE. With additional assumptions on the score, we first establish sub-Gaussian tail bounds and convergence of all moments for the normalized estimation error. We then prove an entropic central limit theorem for a smoothed version of the estimator, showing convergence in relative entropy to the limiting Gaussian law. When the Fisher information of the normalized estimate is bounded, or its density has bounded first derivative, we further show that the smoothing can be removed, yielding entropic normality of the MLE itself. The proofs develop auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.
Related
- Strong Consistency and Asymptotic Normality of the Maximum Likelihood Estimator in r-Largest Generalized Extreme-Value Models
- Finite sample behavior of the maximum likelihood estimator in the Poisson model under Gaussian design
- Uniform Gaussian Approximation for The Quasi-Likelihood Estimator for a Weakly Dependent Nonlinear Time Series Models
- Asymptotic Normality of the Hill Estimator in Linear Preferential Attachment
- Existence and consistency of weighted maximum likelihood estimator for extreme quantile regression
BibTeX
@misc{barnes2026subgaussian,
title = {{Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator}},
author = {Leighton P. Barnes and Alex Dytso},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2605.07107},
eprint = {2605.07107},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.07107},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Sub-Gaussian Concentration and Entropic Normality of the Maximum Likelihood Estimator
AU - Leighton P. Barnes
AU - Alex Dytso
PY - 2026
DA - 2026/08/30
PB - arXiv
DO - 10.48550/arxiv.2605.07107
UR - https://arxiv.org/abs/2605.07107
AB - It is well known that, under standard regularity conditions, the maximum likelihood estimator (MLE) satisfies a central limit theorem and converges in distribution to a Gaussian random variable as the sample size grows. This paper strengthens this classical result by developing several stronger forms of asymptotic normality for the normalized MLE. With additional assumptions on the score, we first establish sub-Gaussian tail bounds and convergence of all moments for the normalized estimation error. We then prove an entropic central limit theorem for a smoothed version of the estimator, showing convergence in relative entropy to the limiting Gaussian law. When the Fisher information of the normalized estimate is bounded, or its density has bounded first derivative, we further show that the smoothing can be removed, yielding entropic normality of the MLE itself. The proofs develop auxiliary tools that may be of independent interest, including exponential consistency bounds, high-moment estimates, and entropy-control arguments for the estimator.
ER -