Zero-Complement Extrapolation: Parameter Recovery and Classification in Severely Imbalanced Logistic Regression under Rank-Deficient Regimes
Abstract
Parameter estimation and classification in binary logistic regression deteriorate severely when the number of minority class observations (n1) is smaller than the feature dimension (p). Under such rank-deficient rarity (n1 < p), maximum likelihood estimators suffer from quasi-complete separation, penalized likelihood methods exhibit severe intercept bias, and convex minority oversampling techniques such as SMOTE collapse into an (n1 − 1)-dimensional affine subspace. In this paper, we propose a novel \textit{Self-Calibrated Zero-Complement} estimator that shifts the paradigm from minority interpolation to majority fringe extrapolation. Grounded in Stein's Lemma, we prove that the conditional expectation shift of the abundant majority class (Y=0) induced by logistic truncation uniquely identifies the true hyperplane normal β after Mahalanobis whitening. We further establish that extrapolating synthetic minority instances outward from the high-quantile density fringe of the majority distribution along the conjugate direction preserves full-rank covariance regularity (rank = p). To accommodate both semi-supervised environments (with an unlabeled background mean) and strictly supervised subsamples, we introduce a hybrid estimator incorporating the intrinsic third-moment skewness of the majority class alongside a location-shift-invariant zero-boundary intercept calibration. Extensive Monte Carlo simulations and real-world benchmark applications (Wisconsin Diagnostic Breast Cancer and Banknote Authentication) under extreme rarity (n1 ∈ {3, 5}) demonstrate that our approach prevents parameter divergence, recovers the oracle hyperplane with high fidelity, and substantially outperforms Firth's regression and SMOTE in F1-score and estimation stability.
Related
- Extrapolation of extreme covariates in generalized additive regression using extreme value theory
- Density-Ratio Rescoring for Imbalanced Classification Using Raking Duals and Classifier Scores
- Class-Conditioned Diffusion Augmentation and Explainable In-Context Ensemble Learning for Imbalanced Maintenance Action Classification in Aircraft Systems
- Local data augmentation for imbalanced binary classification
- Hierarchical Majority Decomposition for Imbalanced Classification: A Recursive Binary Majority-Class Tree with Persistent Minority Comparison
BibTeX
@misc{park2026zerocomplement,
title = {{Zero-Complement Extrapolation: Parameter Recovery and Classification in Severely Imbalanced Logistic Regression under Rank-Deficient Regimes}},
author = {Chongsun Park},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.39194},
eprint = {2610.39194},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.39194},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Zero-Complement Extrapolation: Parameter Recovery and Classification in Severely Imbalanced Logistic Regression under Rank-Deficient Regimes
AU - Chongsun Park
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.39194
UR - https://arxiv.org/abs/2610.39194
AB - Parameter estimation and classification in binary logistic regression deteriorate severely when the number of minority class observations (n_1) is smaller than the feature dimension (p). Under such rank-deficient rarity (n_1 < p), maximum likelihood estimators suffer from quasi-complete separation, penalized likelihood methods exhibit severe intercept bias, and convex minority oversampling techniques such as SMOTE collapse into an (n_1 − 1)-dimensional affine subspace. In this paper, we propose a novel \textit{Self-Calibrated Zero-Complement} estimator that shifts the paradigm from minority interpolation to majority fringe extrapolation. Grounded in Stein's Lemma, we prove that the conditional expectation shift of the abundant majority class (Y=0) induced by logistic truncation uniquely identifies the true hyperplane normal β after Mahalanobis whitening. We further establish that extrapolating synthetic minority instances outward from the high-quantile density fringe of the majority distribution along the conjugate direction preserves full-rank covariance regularity (rank = p). To accommodate both semi-supervised environments (with an unlabeled background mean) and strictly supervised subsamples, we introduce a hybrid estimator incorporating the intrinsic third-moment skewness of the majority class alongside a location-shift-invariant zero-boundary intercept calibration. Extensive Monte Carlo simulations and real-world benchmark applications (Wisconsin Diagnostic Breast Cancer and Banknote Authentication) under extreme rarity (n_1 ∈ {3, 5}) demonstrate that our approach prevents parameter divergence, recovers the oracle hyperplane with high fidelity, and substantially outperforms Firth's regression and SMOTE in F1-score and estimation stability.
ER -