Pattern-Calibrated Multimodal Prediction under Blockwise Missingness
Abstract
Blockwise missingness in multimodal data is usually treated as an incomplete-input problem. We instead focus on prediction for a prespecified observed-modality pattern, where the observed modality set determines the information on which the prediction rule can condition. A procedure that imputes missing modalities, zero-fills unobserved modalities, or trains a single pooled predictor may borrow information across patterns, but it can also mix pattern-specific prediction rules. We propose Multimodal Overlap-aware Shared-specific Alignment and Inter-pattern Calibration (MOSAIC), a pattern-calibrated framework for borrowing across missingness patterns without collapsing their prediction rules. MOSAIC learns shared and modality-specific representations, uses the available representations that overlap with the target pattern to fit a first-stage predictor, and then estimates the calibration gap from target-pattern data. We establish non-asymptotic bounds that decompose the error into overlap effective sample size, calibration gap, and representation-learning error, clarifying when cross-pattern borrowing improves over local fitting and when the improvement is controlled by rule mismatch or representation-learning error. Simulations examine representation recovery and target-pattern correction, and applications to ICU mortality prediction, emotion recognition, and glaucoma classification show gains when target-pattern samples are limited or pattern-specific rules differ.
Related
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Missingness-Aware Conformal Prediction Under Cross-Hospital Distribution Shift
- Covariance-Calibrated Prediction-Powered Inference
- Moment-Calibrated Conformal Prediction with Side Information
- Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk
BibTeX
@misc{yu2026patterncalibrated,
title = {{Pattern-Calibrated Multimodal Prediction under Blockwise Missingness}},
author = {Junhan Yu and Kejian Zhang and Doudou Zhou and Guojun Zhu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.01821},
eprint = {2607.01821},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.01821},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Pattern-Calibrated Multimodal Prediction under Blockwise Missingness
AU - Junhan Yu
AU - Kejian Zhang
AU - Doudou Zhou
AU - Guojun Zhu
PY - 2026
DA - 2026/09/02
PB - arXiv
DO - 10.48550/arxiv.2607.01821
UR - https://arxiv.org/abs/2607.01821
AB - Blockwise missingness in multimodal data is usually treated as an incomplete-input problem. We instead focus on prediction for a prespecified observed-modality pattern, where the observed modality set determines the information on which the prediction rule can condition. A procedure that imputes missing modalities, zero-fills unobserved modalities, or trains a single pooled predictor may borrow information across patterns, but it can also mix pattern-specific prediction rules. We propose Multimodal Overlap-aware Shared-specific Alignment and Inter-pattern Calibration (MOSAIC), a pattern-calibrated framework for borrowing across missingness patterns without collapsing their prediction rules. MOSAIC learns shared and modality-specific representations, uses the available representations that overlap with the target pattern to fit a first-stage predictor, and then estimates the calibration gap from target-pattern data. We establish non-asymptotic bounds that decompose the error into overlap effective sample size, calibration gap, and representation-learning error, clarifying when cross-pattern borrowing improves over local fitting and when the improvement is controlled by rule mismatch or representation-learning error. Simulations examine representation recovery and target-pattern correction, and applications to ICU mortality prediction, emotion recognition, and glaucoma classification show gains when target-pattern samples are limited or pattern-specific rules differ.
ER -