Skip to content
DraconiticResearch
Search Sources API
203,948 studies31 sourcesUpdated 48 min ago
PreprintCC BY 4.0arXiv

Estimating Implicit Expected Log Density Ratios

  • Avi Amalanshu
  • Chandler Squires
  • Yizhou Lu
  • Pradeep Ravikumar
Open at sourcePDFDOICite

Abstract

When exact probabilities cannot be computed, \emph{density ratio estimation (DRE)} is a key subroutine in the wide range of applications where density ratios appear, from experimental design to generative modelling and reinforcement learning. In this paper, we introduce the concept of an \emph{expected log density ratio (ELDR)}: log density ratios that are integrated under some third, possibly distinct measure. We show many problems with implicit densities where DRE is naïvely used are actually ELDR estimation tasks, and prove fundamental limitations on finite-sample ELDR estimation. Based on these results, we argue that the typical DRE-based approaches are unsuitable for ELDR estimation, and provide some foundational motivation towards bespoke ELDR estimators. To demonstrate, we propose a simple modification to some DRE methods that allows samples convergent to the integral distribution to appear in training data. We show through benchmarks that this can improve ELDR estimation error.

Related

  1. Estimating the expected output of wide random MLPs more efficiently than sampling
    Wilson Wu, Victor Lecomte, Michael Winer +3
    arXivPreprintCC BY 4.0Full textStatisticsCondensed matterarXiv
  2. Estimating Implicit Regularization in Deep Learning
    Joseph H. Rudoler, Kevin Tan, Giles Hooker +1
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  3. Estimating Marginal Odds Ratios for Cluster Randomized Trials: The Cluster-Level GLM.
    Jennifer A Thompson, Brennan C Kahan, Andrew Copas
    Statistics in medicinePeer-reviewedCC BYFull textStatisticsEurope PMC
  4. Minimax estimation of the expected conditional covariance under bounds on the covariate density
    Seyoung Park
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  5. Density of length ratios on Teichmüller space
    Nhat Minh Doan, Hanh Vo
    arXivPreprintarXiv licenceFull textGeometry & topologyarXiv

Cite

BibTeXRIS
BibTeX
@misc{amalanshu2026estimating,
  title = {{Estimating Implicit Expected Log Density Ratios}},
  author = {Avi Amalanshu and Chandler Squires and Yizhou Lu and Pradeep Ravikumar},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.38151},
  eprint = {2610.38151},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38151},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - Estimating Implicit Expected Log Density Ratios
AU  - Avi Amalanshu
AU  - Chandler Squires
AU  - Yizhou Lu
AU  - Pradeep Ravikumar
PY  - 2026
DA  - 2026/10/28
PB  - arXiv
DO  - 10.48550/arxiv.2610.38151
UR  - https://arxiv.org/abs/2610.38151
AB  - When exact probabilities cannot be computed, \emph{density ratio estimation (DRE)} is a key subroutine in the wide range of applications where density ratios appear, from experimental design to generative modelling and reinforcement learning. In this paper, we introduce the concept of an \emph{expected log density ratio (ELDR)}: log density ratios that are integrated under some third, possibly distinct measure. We show many problems with implicit densities where DRE is naïvely used are actually ELDR estimation tasks, and prove fundamental limitations on finite-sample ELDR estimation. Based on these results, we argue that the typical DRE-based approaches are unsuitable for ELDR estimation, and provide some foundational motivation towards bespoke ELDR estimators. To demonstrate, we propose a simple modification to some DRE methods that allows samples convergent to the integral distribution to appear in training data. We show through benchmarks that this can improve ELDR estimation error.
ER  -