A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity
Abstract
Missing data, measurement error, and population heterogeneity are pervasive challenges in analyzing data arising from modern observational studies and machine learning applications. Although these problems frequently coexist and interact, they are often treated separately in existing works. We propose a unified probabilistic framework that jointly addresses these issues utilizing deep latent variable representation. The proposed method integrates a novel hierarchical tree-routed variational autoencoder with pattern-aware latent representations and calibration-based denoising. The framework accommodates missing data mechanisms, including MCAR, MAR, and MNAR, while simultaneously learning subgroup-specific and globally shared latent structure. The introduced reconvergent routing mechanism enables selective parameters to be shared across related subpopulations, which offers flexibility as well as improved statistical efficiency. Simulation studies demonstrate substantial improvements over existing deep generative imputation approaches under complex heterogeneous missingness and measurement-error settings. The proposed framework provides a principled approach for learning from noisy and incomplete data in modern healthcare and other high-dimensional applications.
Related
- Missing data and cluster graphs: cluster-level missingness vs variable-level missingness
- The price of a weak shadow variable: inference under nonignorable missingness as relevance vanishes
- Observation-Specific Uncertainty Modeling for Latent-Variable Regression in High-Dimensional Analytical Data
- BGM-IV: AI-Powered Bayesian Generative Modeling for Instrumental Variable Regression with High-Dimensional Covariates
- Memory-Augmented Phase-Field Modeling: Onsager Formulation, Internal-Variable Lifting, and Structure-Preserving Computation
BibTeX
@misc{charvadeh2026deep,
title = {{A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity}},
author = {Yasin Khadem Charvadeh and Grace Y. Yi and Mithat Gönen and Pouya Faroughi},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.30040},
eprint = {2608.30040},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.30040},
}
RIS
TY - UNPB
TI - A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity
AU - Yasin Khadem Charvadeh
AU - Grace Y. Yi
AU - Mithat Gönen
AU - Pouya Faroughi
PY - 2026
DA - 2026/08/30
PB - arXiv
DO - 10.48550/arxiv.2608.30040
UR - https://arxiv.org/abs/2608.30040
AB - Missing data, measurement error, and population heterogeneity are pervasive challenges in analyzing data arising from modern observational studies and machine learning applications. Although these problems frequently coexist and interact, they are often treated separately in existing works. We propose a unified probabilistic framework that jointly addresses these issues utilizing deep latent variable representation. The proposed method integrates a novel hierarchical tree-routed variational autoencoder with pattern-aware latent representations and calibration-based denoising. The framework accommodates missing data mechanisms, including MCAR, MAR, and MNAR, while simultaneously learning subgroup-specific and globally shared latent structure. The introduced reconvergent routing mechanism enables selective parameters to be shared across related subpopulations, which offers flexibility as well as improved statistical efficiency. Simulation studies demonstrate substantial improvements over existing deep generative imputation approaches under complex heterogeneous missingness and measurement-error settings. The proposed framework provides a principled approach for learning from noisy and incomplete data in modern healthcare and other high-dimensional applications.
ER -