On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method
Abstract
The Maximum Entropy on the Mean (MEM) method provides a flexible computational framework for solving inverse problems by combining data fidelity with entropy-based regularization. In practice, however, the prior distribution is typically unknown but can be estimated from data, giving rise to the empirical MEM method. We establish a parametric convergence rate of O(n−1/2) in expectation for empirical MEM, improving upon the previously established O(n−1/4) guarantee by King-Roskamp et al. (2026). Our proof is based on a novel stability analysis of the primal and dual optimization problems under perturbations of the underlying probability measure, relying only on foundational tools from convex analysis and probability. We further show that the MEM dual problem admits a reformulation as an expected risk minimization problem, thereby placing MEM within the modern framework of stochastic optimization and enabling scalable stochastic gradient algorithms for large-scale inverse problems. Together, these results place empirical MEM as a statistically and computationally efficient methodology for data-driven inverse problems.
Related
- Concentration of empirical entropy and transfer entropy for non-regular chains with unbounded memory
- New insights into the NLP-Id bound for maximum-entropy sampling
- Types of statistical arguments in empirical research in psychology
- Statistical-computational trade-offs for recursive adaptive partitioning estimators
- Online tensor learning: Computational and statistical trade-offs, adaptivity and optimal regret
BibTeX
@misc{kingroskamp2026computational,
title = {{On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method}},
author = {Matthew King-Roskamp and Gabriel Rioux and Rustum Choksi and Tim Hoheisel},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.27705},
eprint = {2608.27705},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.27705},
}
RIS
TY - UNPB
TI - On the Computational and Statistical Efficiency of the Empirical Maximum Entropy on the Mean Method
AU - Matthew King-Roskamp
AU - Gabriel Rioux
AU - Rustum Choksi
AU - Tim Hoheisel
PY - 2026
DA - 2026/08/27
PB - arXiv
DO - 10.48550/arxiv.2608.27705
UR - https://arxiv.org/abs/2608.27705
AB - The Maximum Entropy on the Mean (MEM) method provides a flexible computational framework for solving inverse problems by combining data fidelity with entropy-based regularization. In practice, however, the prior distribution is typically unknown but can be estimated from data, giving rise to the empirical MEM method. We establish a parametric convergence rate of O(n^−1/2) in expectation for empirical MEM, improving upon the previously established O(n^−1/4) guarantee by King-Roskamp et al. (2026). Our proof is based on a novel stability analysis of the primal and dual optimization problems under perturbations of the underlying probability measure, relying only on foundational tools from convex analysis and probability. We further show that the MEM dual problem admits a reformulation as an expected risk minimization problem, thereby placing MEM within the modern framework of stochastic optimization and enabling scalable stochastic gradient algorithms for large-scale inverse problems. Together, these results place empirical MEM as a statistically and computationally efficient methodology for data-driven inverse problems.
ER -