Skip to content
DraconiticResearch
Search Sources API
190,332 studies31 sourcesUpdated 12 min ago
PreprintCC BY 4.0HAL

On Minimum Divergence for Distribution Reconstruction

  • Justin Bec
  • Thomas Le Corre
  • Aurélien Garivier
Open at sourcePDFDOICite

Abstract

We study the possibility of generalizing maximum-entropy reconstruction, which approximately recovers a probability distribution from finitely many (generalized) moments by minimizing the Kullback--Leibler (KL) divergence to a reference measure. The reconstruction lies in an exponential family and is also the best KL approximation of the unknown distribution within that family: moment matching and information projection coincide. Replacing KL by a smooth f-divergence, the reconstruction still exists in an associated family, but the best approximation property does not. Specifically, a family is called entropy-adapted if the projection of any distribution matches its moments. We show that, up to equivalence, KL is the only C2 f-divergence that is entropy-adapted, whatever the choice of the generalized moments. We illustrate the consequences of this failure numerically.

Related

  1. Adaptive LASSO Penalized Minimum Density Power Divergence Estimation through Least Squares Approximation: Application to Bone Mineral Density Data from the SWAN Study
    Udita Goswami, Shuvashree Mondal
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. Variance Bounds for Minimum Perfect Matching Problem by Random Walk on Multinomial Distribution
    Shi Feng
    arXivPreprintarXiv licenceFull textProbabilityarXiv
  3. Divergence-Minimizing Distribution Matching for Parallel Channels and Channels with Memory
    Francesca Diedolo, Gerhard Kramer
    arXivPreprintarXiv licenceFull textGeneral mathematicsarXiv
  4. Distribution-Dependent Improvements of McDiarmid's Inequality via Rényi Divergence
    Amedeo Roberto Esposito, Adrien Vandenbroucque, Michael Gastpar
    arXivPreprintCC BY 4.0Full textProbabilityarXiv
  5. Beyond the Kullback--Leibler divergence: the likelihood-ratio distribution as a detectability measure
    Amirhossein Samandar, Mikel Martin Barandiaran, Linn Htat Lu +13
    HALPreprintCC BY 4.0Full textCosmologyInstrumentation & methodsHALarXiv

Cite

BibTeXRIS
BibTeX
@misc{bec2026minimum,
  title = {{On Minimum Divergence for Distribution Reconstruction}},
  author = {Justin Bec and Thomas Le Corre and Aurélien Garivier},
  howpublished = {HAL},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.21872},
  eprint = {2610.21872},
  archivePrefix = {arXiv},
  url = {https://hal.science/hal-05802905v1},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - On Minimum Divergence for Distribution Reconstruction
AU  - Justin Bec
AU  - Thomas Le Corre
AU  - Aurélien Garivier
PY  - 2026
DA  - 2026/10/15
PB  - HAL
DO  - 10.48550/arxiv.2610.21872
UR  - https://hal.science/hal-05802905v1
AB  - We study the possibility of generalizing maximum-entropy reconstruction, which approximately recovers a probability distribution from finitely many (generalized) moments by minimizing the Kullback--Leibler (KL) divergence to a reference measure. The reconstruction lies in an exponential family and is also the best KL approximation of the unknown distribution within that family: moment matching and information projection coincide. Replacing KL by a smooth f-divergence, the reconstruction still exists in an associated family, but the best approximation property does not. Specifically, a family is called entropy-adapted if the projection of any distribution matches its moments. We show that, up to equivalence, KL is the only C^2 f-divergence that is entropy-adapted, whatever the choice of the generalized moments. We illustrate the consequences of this failure numerically.
ER  -