Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting
Abstract
Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies how predictive structure, head count, training sample size, and long-run dependence enter the selection criterion.
Related
- BitterBuster: A virtual world for understanding naturalistic patch foraging
- Understanding axial attention in TSFMs through single location regression
- Convergent Stochastic Training of Multi-Headed Attention and Understanding LoRA
- Morpheus-2.0: Multiscale Probabilistic Forecasting with Dual-Axis Attention
- Choosing Stimuli Wisely by Using Individual Differences: A Step-by-Step Tutorial for the One-Stimulus-Per-Factor Principle in Vision Science and Optometric Assessment
BibTeX
@misc{chen2026patch,
title = {{Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting}},
author = {Junyu Chen and Likai Chen and Weining Wang},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.08160},
eprint = {2611.08160},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.08160},
}
RIS
TY - UNPB
TI - Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting
AU - Junyu Chen
AU - Likai Chen
AU - Weining Wang
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.08160
UR - https://arxiv.org/abs/2611.08160
AB - Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies how predictive structure, head count, training sample size, and long-run dependence enter the selection criterion.
ER -