A Latent-Space Statistical Learning Framework for Semicontinuous Outcomes: Regularized Deviance and Boundary-Stabilized Optimization
Abstract
Semicontinuous outcomes characterized by an exact zero mass and heavy-tailed continuous distributions challenge empirical risk modeling and machine learning. Compound Poisson-Gamma Tweedie processes impose a rigid mean-variance coupling that distorts zero-mass probabilities under extreme tail dispersion, flattening continuous density and reducing extreme-risk discrimination. Furthermore, heuristic post-hoc target capping introduces systematic bias while leaving loss gradients vulnerable to tail leverage. We present a unified, statistically coherent likelihood framework for semicontinuous modeling under arbitrary zero-inflation and explicit upper boundary constraints. Optimization and likelihood evaluations operate strictly in a latent space defined by the power-scaling mapping yi* = min{(yi/s)λ, U*}, where s ∈ (0, U) anchors scale and λ∈ (0,1) regularizes tail curvature. The architecture partitions the response into three regimes: an unconstrained Bernoulli hurdle, an analytical continuous Gamma body, and an upper boundary point mass governed by incomplete Gamma survival ratios. Rather than an arbitrary truncation, the boundary functions as an intrinsic tail accumulator bounding score and Hessian dynamics. We derive the closed-form deviance objective, score vectors, and block-diagonal Hessian matrix, proving that parameter spaces are information-orthogonal to enable concurrent classification and regression updates. Asymptotic boundary analysis demonstrates that score and curvature scale linearly in the deep tail, establishing a quadratic restoring force that prevents gradient collapse. The framework breaks the Tweedie parameter rigidity paradox while maintaining structural fidelity across both zero occurrence and extreme severity differentiation.
Related
- Stabilized Higher-Order Influence Functions: Statistical Theory of a Class of Bilinear Forms
- Stabilized SQP Method for Convex Piecewise Linear-Quadratic Composite Optimization Problems
- A Stochastic Proximal-Gradient Method for Regularized Generally Constrained Continuous Optimization
- CRANDALL-PAZY RIGIDITY FOR LOWER SEMICONTINUOUS CONVEX FUNCTIONS: A BALANCED MEDIAN-FLOW ARGUMENT
- A sequential regularized piecewise affine algorithm for nonconvex nonsmooth multicomposite optimization in RNN training
BibTeX
@misc{wang2026latentspace,
title = {{A Latent-Space Statistical Learning Framework for Semicontinuous Outcomes: Regularized Deviance and Boundary-Stabilized Optimization}},
author = {Jianping Philip Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2608.26286},
eprint = {2608.26286},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.26286},
}
RIS
TY - UNPB
TI - A Latent-Space Statistical Learning Framework for Semicontinuous Outcomes: Regularized Deviance and Boundary-Stabilized Optimization
AU - Jianping Philip Wang
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2608.26286
UR - https://arxiv.org/abs/2608.26286
AB - Semicontinuous outcomes characterized by an exact zero mass and heavy-tailed continuous distributions challenge empirical risk modeling and machine learning. Compound Poisson-Gamma Tweedie processes impose a rigid mean-variance coupling that distorts zero-mass probabilities under extreme tail dispersion, flattening continuous density and reducing extreme-risk discrimination. Furthermore, heuristic post-hoc target capping introduces systematic bias while leaving loss gradients vulnerable to tail leverage. We present a unified, statistically coherent likelihood framework for semicontinuous modeling under arbitrary zero-inflation and explicit upper boundary constraints. Optimization and likelihood evaluations operate strictly in a latent space defined by the power-scaling mapping y_i^* = min{(y_i/s)^λ, U^*}, where s ∈ (0, U) anchors scale and λ∈ (0,1) regularizes tail curvature. The architecture partitions the response into three regimes: an unconstrained Bernoulli hurdle, an analytical continuous Gamma body, and an upper boundary point mass governed by incomplete Gamma survival ratios. Rather than an arbitrary truncation, the boundary functions as an intrinsic tail accumulator bounding score and Hessian dynamics. We derive the closed-form deviance objective, score vectors, and block-diagonal Hessian matrix, proving that parameter spaces are information-orthogonal to enable concurrent classification and regression updates. Asymptotic boundary analysis demonstrates that score and curvature scale linearly in the deep tail, establishing a quadratic restoring force that prevents gradient collapse. The framework breaks the Tweedie parameter rigidity paradox while maintaining structural fidelity across both zero occurrence and extreme severity differentiation.
ER -