How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage
Abstract
Conformal prediction provides distribution-free finite-sample marginal coverage, but post-hoc calibration data may be too scarce to learn how uncertainty varies across inputs. Meanwhile, abundant covariates can often be labeled cheaply by domain models or general-purpose language models. We study whether these synthetic labels can improve conditional coverage when only a small trusted sample is available. Building on score-quantile regression, we introduce prediction-powered quantile learning: a synthetic-labeled pool estimates pinball risk, paired trusted and synthetic outcomes correct its bias, and an independent trusted split performs final conformalization. Profiling pinball risk over scalar corrections reveals that population conditional-coverage error is its functional gradient; the corresponding Hessian removes global shifts and weights remaining shape error by boundary density. Composing this geometry with prediction-powered learning yields a three-resource expansion and a benefit--cost rule for synthetic power. Across eight regression benchmarks, synthetic-powered quantile learning substantially improves downstream conditional coverage while preserving marginal validity and producing more compact prediction sets. A human-rating study finds similar gains from external LLM labels and exposes a quality--quantity--cost tradeoff.
Related
- Conformal Prediction and Conditional Coverage for Tabular Foundation Models
- Sharp training-conditional coverage for conformal prediction under covariate shift
- Graph-Localized Coverage for Conformal Prediction on Inductive Graphs
- Metric Conformal Prediction via the Conditional Expected Local Radius
- Radiomics-Based Thyroid Nodule Classification with Label-Conditional Conformal Prediction
BibTeX
@misc{chen2026synthetic,
title = {{How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage}},
author = {Qianyi Chen and Bo Li},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.33482},
eprint = {2609.33482},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.33482},
}
RIS
TY - UNPB
TI - How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage
AU - Qianyi Chen
AU - Bo Li
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.33482
UR - https://arxiv.org/abs/2609.33482
AB - Conformal prediction provides distribution-free finite-sample marginal coverage, but post-hoc calibration data may be too scarce to learn how uncertainty varies across inputs. Meanwhile, abundant covariates can often be labeled cheaply by domain models or general-purpose language models. We study whether these synthetic labels can improve conditional coverage when only a small trusted sample is available. Building on score-quantile regression, we introduce prediction-powered quantile learning: a synthetic-labeled pool estimates pinball risk, paired trusted and synthetic outcomes correct its bias, and an independent trusted split performs final conformalization. Profiling pinball risk over scalar corrections reveals that population conditional-coverage error is its functional gradient; the corresponding Hessian removes global shifts and weights remaining shape error by boundary density. Composing this geometry with prediction-powered learning yields a three-resource expansion and a benefit--cost rule for synthetic power. Across eight regression benchmarks, synthetic-powered quantile learning substantially improves downstream conditional coverage while preserving marginal validity and producing more compact prediction sets. A human-rating study finds similar gains from external LLM labels and exposes a quality--quantity--cost tradeoff.
ER -