Series-wise Cross-validation for Global Time Series Forecasting
Abstract
We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning. The prevailing CV strategy in forecasting is to partition data across time with the last part of each series used for testing. With global models trained on collections of time series, however, a fundamentally different approach becomes possible: partitioning across series and time. We formalize series-wise CV for global forecasting models and benchmark several instantiations, including K-fold CV, bootstrapping, and Monte Carlo CV, on 11 datasets with 10 neural network architectures. We evaluate CV procedures along five dimensions: performance estimation, overoptimism rate, model selection accuracy, regret, and downstream forecasting performance. We withhold a random fraction of series from training so that they appear only in the test set. This protocol simulates the emergence of new series not seen during training, a common scenario in global forecasting. Under this protocol, our results reveal a nuanced trade-off: series-wise CV yields better downstream forecasting accuracy by guiding model selection toward configurations that generalize across both time and series, while time-wise CV retains a slight advantage for estimating out-of-sample error.
Related
- Hierarchical Time Series Forecasting with Operator-Valued Kernels
- Coherent Hierarchical Forecasting for Proportion and Discrete Time Series
- Informed forecasting: Leveraging auxiliary knowledge to boost large language model performance on time series forecasting
- Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting
- Mixed-Frequency Time Series Forecasting via Depth-Separable Neural Networks
BibTeX
@misc{cerqueira2026serieswise,
title = {{Series-wise Cross-validation for Global Time Series Forecasting}},
author = {Vitor Cerqueira and Mariana Oliveira and Luis Roque},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.08252},
eprint = {2611.08252},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.08252},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Series-wise Cross-validation for Global Time Series Forecasting
AU - Vitor Cerqueira
AU - Mariana Oliveira
AU - Luis Roque
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.08252
UR - https://arxiv.org/abs/2611.08252
AB - We consider the problem of cross-validation (CV) in time series forecasting, with a focus on global forecasting models based on deep learning. The prevailing CV strategy in forecasting is to partition data across time with the last part of each series used for testing. With global models trained on collections of time series, however, a fundamentally different approach becomes possible: partitioning across series and time. We formalize series-wise CV for global forecasting models and benchmark several instantiations, including K-fold CV, bootstrapping, and Monte Carlo CV, on 11 datasets with 10 neural network architectures. We evaluate CV procedures along five dimensions: performance estimation, overoptimism rate, model selection accuracy, regret, and downstream forecasting performance. We withhold a random fraction of series from training so that they appear only in the test set. This protocol simulates the emergence of new series not seen during training, a common scenario in global forecasting. Under this protocol, our results reveal a nuanced trade-off: series-wise CV yields better downstream forecasting accuracy by guiding model selection toward configurations that generalize across both time and series, while time-wise CV retains a slight advantage for estimating out-of-sample error.
ER -