An Input Skip Can Reparameterize a VAE Without Improving the Generator
Abstract
An input skip can reduce reconstruction error while leaving a model's distribution of new samples unchanged. For a Gaussian variational autoencoder (VAE) with constant input skip 0≤α<1, a model without a skip can be written in skip form by multiplying its decoder output by b=1−α and its noise variance by b2. The distribution of observations and latent variables and its exact generative scores stay the same, while reconstruction mean squared error (MSE) is multiplied by b2. Separate training can produce different generators because optimization depends on the parameter coordinates. In a Gaussian experiment with known density, changing skip and observation variance together reduces MSE but worsens energy score and the estimate of negative log density from importance sampling. On seven annual Deep-ocean Assessment and Reporting of Tsunamis (DART) records, energy score differences at the same observation variance have both signs on four original records and favor the skip on three records from 2018. Probabilistic principal component analysis (PPCA) has lower mean energy scores than every neural DART fit. With the encoder and decoder kept the same, estimating observation variance from training reconstruction error can also worsen energy score. Assessing generation requires more than reconstruction error.
Related
- Input-data perturbation in groundwater level prediction: a controlled experiment on accuracy–reliability trade-offs among data augmentation and Bayesian-type uncertainty estimation methods
- Bayesian Inversion Beyond the Generator's Range: Reducing Non-physical Constraints from Generative Neural Networks
- Exploring the role of input data on hail nowcast skill using spatiotemporal neural networks
- Water–fertilizer coupling balances yield and the input-related carbon footprint in double-cropping greenhouse Hami melon
- Multi-temporal Canopy Height Modeling using Deep Learning and Sentinel-2: The Influence of Training Data and Input Feature Representations
BibTeX
@misc{lee2026input,
title = {{An Input Skip Can Reparameterize a VAE Without Improving the Generator}},
author = {Kevin Lee},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37456},
eprint = {2610.37456},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37456},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - An Input Skip Can Reparameterize a VAE Without Improving the Generator
AU - Kevin Lee
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.37456
UR - https://arxiv.org/abs/2610.37456
AB - An input skip can reduce reconstruction error while leaving a model's distribution of new samples unchanged. For a Gaussian variational autoencoder (VAE) with constant input skip 0≤α<1, a model without a skip can be written in skip form by multiplying its decoder output by b=1−α and its noise variance by b^2. The distribution of observations and latent variables and its exact generative scores stay the same, while reconstruction mean squared error (MSE) is multiplied by b^2. Separate training can produce different generators because optimization depends on the parameter coordinates. In a Gaussian experiment with known density, changing skip and observation variance together reduces MSE but worsens energy score and the estimate of negative log density from importance sampling. On seven annual Deep-ocean Assessment and Reporting of Tsunamis (DART) records, energy score differences at the same observation variance have both signs on four original records and favor the skip on three records from 2018. Probabilistic principal component analysis (PPCA) has lower mean energy scores than every neural DART fit. With the encoder and decoder kept the same, estimating observation variance from training reconstruction error can also worsen energy score. Assessing generation requires more than reconstruction error.
ER -