Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation
Abstract
Reliable generalization in conditional latent variable models requires understanding both identifiability and extrapolation: how observed variation across attributes determines latent structure, and how that structure determines distributions at unseen attributes. However, existing identifiability and extrapolation guarantees are largely model-specific, with separate analyses in nonlinear ICA, causal representation learning, perturbation modeling, and related conditional latent variable models. We introduce concept modulation models (CMMs), an attribute-indexed class of conditional generative models with structure A→ Λ→ C→ X, where attributes select modulators, modulators induce latent concept laws, and concepts generate observed features. CMMs lift transition-based identifiability to conditional settings by showing that feature agreement on observed attributes induces a latent concept transition constrained by the CMM class. We express these constraints through attribute potentials, log-density ratios between attribute-conditioned concept laws, separating the generic lifting step from model-specific rigidity arguments. The same potentials control extrapolation: agreement at unseen attributes holds exactly when the transported attribute-potential identities extend to those attributes. This yields algebraic extrapolation criteria, identifies the common potential-based proof objects behind several existing identifiability and extrapolation results, and, when combined with the model-specific rigidity arguments in those works, recovers their stated conclusions.
Related
- α-TCAV: A Unified Framework for Testing with Concept Activation Vectors
- Quantitative Γ-Convergence: A Variational Extrapolation Framework
- A Unified Space-Time Variational Framework for Wave Equations on Differential Forms
- Centered Weak Discrete Riemannian Gradients: A Unified Framework for Riemannian Optimization
- A unified variational framework for augmented Lagrangian preconditioning of saddle point systems: applications to mixed-dimensional and fictitious domain methods
BibTeX
@misc{yi2026concept,
title = {{Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation}},
author = {Soheun Yi and Yizhou Lu and Chandler Squires and Pradeep Ravikumar},
howpublished = {arXiv},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2606.18509},
eprint = {2606.18509},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.18509},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation
AU - Soheun Yi
AU - Yizhou Lu
AU - Chandler Squires
AU - Pradeep Ravikumar
PY - 2026
DA - 2026/06/16
PB - arXiv
DO - 10.48550/arxiv.2606.18509
UR - https://arxiv.org/abs/2606.18509
AB - Reliable generalization in conditional latent variable models requires understanding both identifiability and extrapolation: how observed variation across attributes determines latent structure, and how that structure determines distributions at unseen attributes. However, existing identifiability and extrapolation guarantees are largely model-specific, with separate analyses in nonlinear ICA, causal representation learning, perturbation modeling, and related conditional latent variable models. We introduce concept modulation models (CMMs), an attribute-indexed class of conditional generative models with structure A→ Λ→ C→ X, where attributes select modulators, modulators induce latent concept laws, and concepts generate observed features. CMMs lift transition-based identifiability to conditional settings by showing that feature agreement on observed attributes induces a latent concept transition constrained by the CMM class. We express these constraints through attribute potentials, log-density ratios between attribute-conditioned concept laws, separating the generic lifting step from model-specific rigidity arguments. The same potentials control extrapolation: agreement at unseen attributes holds exactly when the transported attribute-potential identities extend to those attributes. This yields algebraic extrapolation criteria, identifies the common potential-based proof objects behind several existing identifiability and extrapolation results, and, when combined with the model-specific rigidity arguments in those works, recovers their stated conclusions.
ER -