Unified Approach for Weakly Supervised Multicalibration
Abstract
Multicalibration requires predicted scores to agree with label probabilities across rich families of subgroups and score-dependent tests, but existing methods require clean input-label pairs for evaluation and post-processing. This assumption fails in weakly supervised learning (WSL) regimes -- including positive-unlabeled, unlabeled-unlabeled, and positive-confidence learning -- where clean labels are costly or unavailable even though reliable uncertainty estimates may be crucial. We address this gap by developing estimators of multicalibration error and post-hoc correction methods for WSL settings in which clean input-label pairs are unavailable. We propose a unified framework for estimating and correcting multicalibration under weak supervision by combining contamination-matrix risk rewrites with witness-based calibration constraints, yielding corrected multicalibration moments with finite-sample guarantees. We further propose weak-label multicalibration boost (WLMC), a generic post-hoc recalibration algorithm under weak supervision. Finally, we conduct experiments across multiple weak-supervision settings to evaluate multicalibration behavior and offer empirical insight into uncertainty estimation under weak supervision.
Related
- From Weakly Supervised Segmentation to Diagnostic Decisions: An Annotation-Economics Review for Medical Imaging
- Progressive Reliability-Guided Refinement with Cross-Modal Consistency and Iterative Correction for Weakly Supervised PET/CT Tumor Segmentation
- Weakly Einstein and weakly η-Einstein structures on almost contact metric three-manifolds
- Nonconstant weakly harmonic maps with constant trace
- The Douglas--Rachford algorithm for normal cones converges only weakly
BibTeX
@misc{futami2026unified,
title = {{Unified Approach for Weakly Supervised Multicalibration}},
author = {Futoshi Futami and Takashi Ishida},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.09857},
eprint = {2605.09857},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.09857},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Unified Approach for Weakly Supervised Multicalibration
AU - Futoshi Futami
AU - Takashi Ishida
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2605.09857
UR - https://arxiv.org/abs/2605.09857
AB - Multicalibration requires predicted scores to agree with label probabilities across rich families of subgroups and score-dependent tests, but existing methods require clean input-label pairs for evaluation and post-processing. This assumption fails in weakly supervised learning (WSL) regimes -- including positive-unlabeled, unlabeled-unlabeled, and positive-confidence learning -- where clean labels are costly or unavailable even though reliable uncertainty estimates may be crucial. We address this gap by developing estimators of multicalibration error and post-hoc correction methods for WSL settings in which clean input-label pairs are unavailable. We propose a unified framework for estimating and correcting multicalibration under weak supervision by combining contamination-matrix risk rewrites with witness-based calibration constraints, yielding corrected multicalibration moments with finite-sample guarantees. We further propose weak-label multicalibration boost (WLMC), a generic post-hoc recalibration algorithm under weak supervision. Finally, we conduct experiments across multiple weak-supervision settings to evaluate multicalibration behavior and offer empirical insight into uncertainty estimation under weak supervision.
ER -