Hierarchical Utility Calibration for Structured Multiclass Decisions
Abstract
In multiclass probabilistic prediction, Utility Calibration (UC), which focuses auditing on specified utilities, has recently received attention as a way to guarantee downstream decisions while controlling computational and sample requirements. At the same time, some multiclass problems have meaningful label hierarchies that play important roles in medicine and image classification, yet how UC evaluates utility within a hierarchy remains insufficiently understood. We show that the difference between realized utility and predicted mean utility admits an exact decomposition into a sum of contributions from the internal nodes of the label tree. This decomposition shows that positive and negative contributions from different nodes can cancel, and that even when UC is small, the utility errors remaining in parts of the hierarchy need not be small. To address this problem, we propose Hierarchical Utility Calibration (HUC), which evaluates each node contribution before summation while retaining the same target utility, subgroup, and predicted-utility interval. We further provide finite-sample evaluation over all predicted-utility intervals and propose HUC-Boost, which updates only violated internal nodes, with theoretical guarantees for both.
Related
- Divide et Calibra: Multiclass Local Calibration via Vector Quantization
- Calibration and Risk-Budgeted Decisions with System One Models
- Structured Neural SDEs for Functional Calibration
- A Movie Rating Prediction Method Based on Structured Prior Integration and Target-Aware User Interest with Ridge-Based Calibration
- Hierarchical Gaussian Processes for Survey Weighting: Local Equivalent Weights in Structured Populations
BibTeX
@misc{futami2026hierarchical,
title = {{Hierarchical Utility Calibration for Structured Multiclass Decisions}},
author = {Futoshi Futami and Jerry Huang and Ichiro Takeuchi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.36532},
eprint = {2609.36532},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.36532},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Hierarchical Utility Calibration for Structured Multiclass Decisions
AU - Futoshi Futami
AU - Jerry Huang
AU - Ichiro Takeuchi
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.36532
UR - https://arxiv.org/abs/2609.36532
AB - In multiclass probabilistic prediction, Utility Calibration (UC), which focuses auditing on specified utilities, has recently received attention as a way to guarantee downstream decisions while controlling computational and sample requirements. At the same time, some multiclass problems have meaningful label hierarchies that play important roles in medicine and image classification, yet how UC evaluates utility within a hierarchy remains insufficiently understood. We show that the difference between realized utility and predicted mean utility admits an exact decomposition into a sum of contributions from the internal nodes of the label tree. This decomposition shows that positive and negative contributions from different nodes can cancel, and that even when UC is small, the utility errors remaining in parts of the hierarchy need not be small. To address this problem, we propose Hierarchical Utility Calibration (HUC), which evaluates each node contribution before summation while retaining the same target utility, subgroup, and predicted-utility interval. We further provide finite-sample evaluation over all predicted-utility intervals and propose HUC-Boost, which updates only violated internal nodes, with theoretical guarantees for both.
ER -