Oracle high-dimensional M-estimation using smooth reparameterization for sparsity
Abstract
This paper establishes a unified non-linear regularization framework for high-dimensional M-estimation, encompassing both linear regression and Cox's proportional hazards models. Rather than relying on traditional additive non-convex penalties, the proposed paradigm embeds sparsity directly into the parameter mapping using a smooth (C2) component-wise ReParametrization map for Sparsity (RePS). This geometric formulation dynamically adapts to local parameter scales, successfully suppressing high-dimensional noise and recovering unbiased oracle asymptotic normality under the large-sample limit. Through the Primal-Dual Witness method, we establish a unified oracle equivalence for both model classes under a liberated micro-penalty scaling regime λ≪ n−1/2, effectively accommodating model-specific structures such as the shift-invariance in survival analysis. Extensive Monte Carlo simulations demonstrate that the RePS framework consistently achieves near-perfect false-positive control and optimal 95\% confidence interval coverage, outperforming conventional non-convex regularizers (SCAD and MCP) which suffer from severe variance inflation and under-coverage due to post-selection noise leakage.
Related
- Sparsity Saves Compute, Not Samples: Oracle Inequalities for Dense and Sparse Mixture of Experts
- Smooth Tensor Multi-View Clustering on Well Logs via Hypergraph Topology and Concurrent Group Sparsity
- Near-Optimal Acceleration for Smooth ℓp / ℓq Nondual Convex First-Order Oracle Optimization
- Matching Higher-Order Oracle Complexity for Smooth Monotone Variational Inequalities
- On the Optimal Linear Minimization Oracle Complexity of Active-Hull Methods for Minimizing Convex Hölder Smooth Functions
BibTeX
@misc{nishiyama2026oracle,
title = {{Oracle high-dimensional $M$-estimation using smooth reparameterization for sparsity}},
author = {Yoichi Nishiyama},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.19558},
eprint = {2609.19558},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.19558},
}
RIS
TY - UNPB
TI - Oracle high-dimensional M-estimation using smooth reparameterization for sparsity
AU - Yoichi Nishiyama
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2609.19558
UR - https://arxiv.org/abs/2609.19558
AB - This paper establishes a unified non-linear regularization framework for high-dimensional M-estimation, encompassing both linear regression and Cox's proportional hazards models. Rather than relying on traditional additive non-convex penalties, the proposed paradigm embeds sparsity directly into the parameter mapping using a smooth (C^2) component-wise ReParametrization map for Sparsity (RePS). This geometric formulation dynamically adapts to local parameter scales, successfully suppressing high-dimensional noise and recovering unbiased oracle asymptotic normality under the large-sample limit. Through the Primal-Dual Witness method, we establish a unified oracle equivalence for both model classes under a liberated micro-penalty scaling regime λ≪ n^−1/2, effectively accommodating model-specific structures such as the shift-invariance in survival analysis. Extensive Monte Carlo simulations demonstrate that the RePS framework consistently achieves near-perfect false-positive control and optimal 95\% confidence interval coverage, outperforming conventional non-convex regularizers (SCAD and MCP) which suffer from severe variance inflation and under-coverage due to post-selection noise leakage.
ER -