A Unifying View of Distributionally Robust Optimization for Probability Smoothing via Maximum Entropy
Abstract
The probability smoothing problem seeks estimators that assign positive probability to all categories, including those unobserved in a finite sample. In our previous work, we established that the estimator derived from distributionally robust optimization (DRO) with a q-norm ambiguity set (q-DRO) satisfies the axioms of Positivity, Symmetry, and Order Preservation, while relaxing the rigid Ratio Preservation axiom characterizing Laplace smoothing. In this paper, we observe that Symmetry itself imposes a strong constraint: it forces all unobserved categories to receive identical probability regardless of their relation to observed categories. Motivated by this observation, we study DRO with an optimal transport ambiguity set (OT-DRO), which incorporates inter-category dissimilarities through a cost matrix and thereby enables structure-respecting smoothing. We derive a tractable dual reformulation of OT-DRO as an empirical cross-entropy minimization with a Lipschitz-type penalty on the log-probability vector, and we identify an explicit threshold on the robustness radius at which the optimal estimator reduces to the uniform distribution, paralleling the threshold established for q-DRO. We then show, through Sion's minimax theorem, that DRO for probability smoothing over any compact convex ambiguity set that contains a strictly positive distribution is equivalent to entropy maximization over that set. This unifying perspective explains the parallel threshold structure in q-DRO and OT-DRO.
Related
- On Over-Conservatism Under Low-Probability Extreme Events: A KL-Regularized Wasserstein Distributionally Robust Planning Method
- Data-driven joint optimization of maintenance and spare parts provisioning: A distributionally robust approach
- Conformal-DRO: Distributionally Robust Optimization with Conformalized Ambiguity Set
- On tractability, complexity, and mixed-integer convex programming representability of distributionally favorable optimization
- Calibrating Ambiguity Set via Diagnostic Transport for Distributionally Robust Optimization
BibTeX
@misc{kurihara2026unifying,
title = {{A Unifying View of Distributionally Robust Optimization for Probability Smoothing via Maximum Entropy}},
author = {Kota Kurihara and Yuya Yamakawa and Yoichi Izunaga},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.25684},
eprint = {2610.25684},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.25684},
}
RIS
TY - UNPB
TI - A Unifying View of Distributionally Robust Optimization for Probability Smoothing via Maximum Entropy
AU - Kota Kurihara
AU - Yuya Yamakawa
AU - Yoichi Izunaga
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.25684
UR - https://arxiv.org/abs/2610.25684
AB - The probability smoothing problem seeks estimators that assign positive probability to all categories, including those unobserved in a finite sample. In our previous work, we established that the estimator derived from distributionally robust optimization (DRO) with a q-norm ambiguity set (q-DRO) satisfies the axioms of Positivity, Symmetry, and Order Preservation, while relaxing the rigid Ratio Preservation axiom characterizing Laplace smoothing. In this paper, we observe that Symmetry itself imposes a strong constraint: it forces all unobserved categories to receive identical probability regardless of their relation to observed categories. Motivated by this observation, we study DRO with an optimal transport ambiguity set (OT-DRO), which incorporates inter-category dissimilarities through a cost matrix and thereby enables structure-respecting smoothing. We derive a tractable dual reformulation of OT-DRO as an empirical cross-entropy minimization with a Lipschitz-type penalty on the log-probability vector, and we identify an explicit threshold on the robustness radius at which the optimal estimator reduces to the uniform distribution, paralleling the threshold established for q-DRO. We then show, through Sion's minimax theorem, that DRO for probability smoothing over any compact convex ambiguity set that contains a strictly positive distribution is equivalent to entropy maximization over that set. This unifying perspective explains the parallel threshold structure in q-DRO and OT-DRO.
ER -