Benign Overfitting under Heterogeneous Input Fusion
Abstract
Benign overfitting is extensively studied when learning from a single high-dimensional input, but its behavior under heterogeneous input fusion remains largely unexplored. We study this question for minimum-norm linear interpolation under a heterogeneous Gaussian design, comparing two statistically dependent input blocks with their fusion while holding the underlying population task fixed. For regression, we identify a full-spectrum covariance certificate whose asymptotic status is independent of the cutoff threshold and prove that it is preserved by every positive-semidefinite joint covariance consistent with the two marginals. This protection is sharp, yet it does not extend to all benign regression problems: outside the certified regime, two benign marginals can have a harmful fusion. For one-sparse Gaussian classification, benignity in the regular regime is characterized by the balance between surviving predictive signal and nuisance contamination. Fusion can move these two quantities in opposite directions, and within this model class every marginal-to-joint benign/non-benign pattern is attainable. We further show that the same fused input can have qualitatively different effects on regression and classification. These results establish that benign overfitting under heterogeneous fusion is determined by the joint signal and spectral geometry created by input interaction, rather than by marginal benignity alone.
Related
- Benign Overfitting Property of the Minimum ℓq -norm Interpolant Estimator in Regression Problem via Feature Space Decomposition
- Benign Overfitting for General Norms and Distributions
- Global minimax risk and acquisition laws for heterogeneous information fusion
- Systematic modelling of a large-scale, historicised, well-detailed building land use, with the fusion of massive and heterogeneous datasets
- THGFM: Dual-Branch Temporal Heterogeneous Graph Fusion Model
BibTeX
@misc{liu2026benign,
title = {{Benign Overfitting under Heterogeneous Input Fusion}},
author = {Houzhen Liu and Xiaobo Xia},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.09340},
eprint = {2610.09340},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.09340},
}
RIS
TY - UNPB
TI - Benign Overfitting under Heterogeneous Input Fusion
AU - Houzhen Liu
AU - Xiaobo Xia
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.09340
UR - https://arxiv.org/abs/2610.09340
AB - Benign overfitting is extensively studied when learning from a single high-dimensional input, but its behavior under heterogeneous input fusion remains largely unexplored. We study this question for minimum-norm linear interpolation under a heterogeneous Gaussian design, comparing two statistically dependent input blocks with their fusion while holding the underlying population task fixed. For regression, we identify a full-spectrum covariance certificate whose asymptotic status is independent of the cutoff threshold and prove that it is preserved by every positive-semidefinite joint covariance consistent with the two marginals. This protection is sharp, yet it does not extend to all benign regression problems: outside the certified regime, two benign marginals can have a harmful fusion. For one-sparse Gaussian classification, benignity in the regular regime is characterized by the balance between surviving predictive signal and nuisance contamination. Fusion can move these two quantities in opposite directions, and within this model class every marginal-to-joint benign/non-benign pattern is attainable. We further show that the same fused input can have qualitatively different effects on regression and classification. These results establish that benign overfitting under heterogeneous fusion is determined by the joint signal and spectral geometry created by input interaction, rather than by marginal benignity alone.
ER -