Score-Preserving Continual Lasso under Task Shift
Abstract
Continual learning (CL) aims to learn from sequential tasks while mitigating catastrophic forgetting when historical data are no longer accessible. We study continual learning for high-dimensional sparse linear regression, where ℓ1 regularization creates a difficulty absent from continual ridge regression: that recursively reusing an ℓ1-penalized estimator can propagate earlier shrinkage and accumulate regularization bias across tasks. We propose score-preserving continual Lasso (SP-CL), which separates sparse estimation from state transmission by preserving the historical squared-loss score. Under squared loss, its cumulative estimator exactly recovers a full-history Lasso minimizer. When task coefficients coincide, it inherits the full-history sparse rate. For related task-specific coefficients, SP-CL uses a regularized current-task correction that leaves the transmitted state unchanged and attains the offline source-assisted rate under an oracle related-task condition. We also derive Overall/BWT/FWT decompositions that distinguish historical information loss, task conflict, and current-task specialization, and validate the resulting regimes in simulation.
Related
- ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
- Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
- On a Class of Non-Local Elliptic-Transport Systems in Score-Guided Probability Flow on SE(3): Score-Singularity Resolution, Space-Time ADN Shift, and Markov Mean-Field Games
- Reliable Replay through Spatial Coherence in Online Continual Learning
- Why Constrained Updates Still Forget: A Functional-Path Analysis of Continual SFT
BibTeX
@misc{guo2026scorepreserving,
title = {{Score-Preserving Continual Lasso under Task Shift}},
author = {Jingyi Guo and Runze Li and Lingzhou Xue},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.32708},
eprint = {2610.32708},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.32708},
}
RIS
TY - UNPB
TI - Score-Preserving Continual Lasso under Task Shift
AU - Jingyi Guo
AU - Runze Li
AU - Lingzhou Xue
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2610.32708
UR - https://arxiv.org/abs/2610.32708
AB - Continual learning (CL) aims to learn from sequential tasks while mitigating catastrophic forgetting when historical data are no longer accessible. We study continual learning for high-dimensional sparse linear regression, where ℓ_1 regularization creates a difficulty absent from continual ridge regression: that recursively reusing an ℓ_1-penalized estimator can propagate earlier shrinkage and accumulate regularization bias across tasks. We propose score-preserving continual Lasso (SP-CL), which separates sparse estimation from state transmission by preserving the historical squared-loss score. Under squared loss, its cumulative estimator exactly recovers a full-history Lasso minimizer. When task coefficients coincide, it inherits the full-history sparse rate. For related task-specific coefficients, SP-CL uses a regularized current-task correction that leaves the transmitted state unchanged and attains the offline source-assisted rate under an oracle related-task condition. We also derive Overall/BWT/FWT decompositions that distinguish historical information loss, task conflict, and current-task specialization, and validate the resulting regimes in simulation.
ER -