From Good Starts to Optimal Inference: Generalized Latent Factor Models with Missingness and Implicit Regularization
Abstract
Generalized latent factor models provide a flexible framework for analyzing high-dimensional non-Gaussian data, but principled estimation and uncertainty quantification under missingness remain substantially less developed. We develop a theory that connects a computationally tractable nonconvex procedure directly to statistical inference for nonlinear latent factor models with exponential-family links and partially observed entries. Our procedure combines a link-aware double-SVD initialization, a unilateral refinement that achieves rowwise consistency, and vanilla gradient descent. We show that the refined initializer enters a region of incoherence and contraction and that gradient descent remains in this region through implicit regularization, contracting rapidly down to the statistical estimation error without explicit incoherence or balancing regularization. Our central result is a uniform rowwise linear approximation for the actual output of gradient descent that isolates the leading score fluctuations from higher-order estimation and optimization errors. These expansions yield asymptotically valid individual and Gaussian multiplier-bootstrap simultaneous inference for latent factors, together with simultaneous confidence bands for missing-entry means, without requiring an additional debiasing step. The resulting estimation rate matches a restricted-class minimax lower bound up to logarithmic factors, while the theory accommodates severe missingness, weak low-rank signals, and diminishing local curvature. Simulations support the theoretical findings, and an application to large language model evaluation illustrates uncertainty-aware estimation and ranking of latent model capabilities.
Related
- Strong convergence of a slowly damped primal-dual dynamical system with general time-scaling, Tikhonov regularization and implicit Hessian damping
- Quasi-randomization-based inference for multivariate Mann-Whitney effects under random missingness
- The price of a weak shadow variable: inference under nonignorable missingness as relevance vanishes
- Convergence rates of mixed primal-dual dynamical systems with implicit Hessian damping and Tikhonov regularization
- Longitudinal causal inference under informative missingness using the self-censoring model
BibTeX
@misc{huang2026from,
title = {{From Good Starts to Optimal Inference: Generalized Latent Factor Models with Missingness and Implicit Regularization}},
author = {Chengzhu Huang and Yuqi Gu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.11740},
eprint = {2609.11740},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.11740},
}
RIS
TY - UNPB
TI - From Good Starts to Optimal Inference: Generalized Latent Factor Models with Missingness and Implicit Regularization
AU - Chengzhu Huang
AU - Yuqi Gu
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2609.11740
UR - https://arxiv.org/abs/2609.11740
AB - Generalized latent factor models provide a flexible framework for analyzing high-dimensional non-Gaussian data, but principled estimation and uncertainty quantification under missingness remain substantially less developed. We develop a theory that connects a computationally tractable nonconvex procedure directly to statistical inference for nonlinear latent factor models with exponential-family links and partially observed entries. Our procedure combines a link-aware double-SVD initialization, a unilateral refinement that achieves rowwise consistency, and vanilla gradient descent. We show that the refined initializer enters a region of incoherence and contraction and that gradient descent remains in this region through implicit regularization, contracting rapidly down to the statistical estimation error without explicit incoherence or balancing regularization. Our central result is a uniform rowwise linear approximation for the actual output of gradient descent that isolates the leading score fluctuations from higher-order estimation and optimization errors. These expansions yield asymptotically valid individual and Gaussian multiplier-bootstrap simultaneous inference for latent factors, together with simultaneous confidence bands for missing-entry means, without requiring an additional debiasing step. The resulting estimation rate matches a restricted-class minimax lower bound up to logarithmic factors, while the theory accommodates severe missingness, weak low-rank signals, and diminishing local curvature. Simulations support the theoretical findings, and an application to large language model evaluation illustrates uncertainty-aware estimation and ranking of latent model capabilities.
ER -