Hierarchical Empirical-Bayes Forecasting of Sparse Counts with Exposure: An Application to Airline Demand and Premium-Cabin Capacity Sharing
Abstract
Sparse count forecasting arises when events are rare, exposure varies across observational units, and short histories contain many zeros. Using thin-market airline demand as a motivating application, this paper develops a hierarchical exposure-adjusted Poisson-Gamma (HEPG) framework for probabilistic forecasting. Conditional counts follow a Poisson model with an exposure offset and pooled calendar effects, while unit specific baseline intensities receive a Gamma prior whose hyperparameters are estimated empirically across the population. The resulting posterior is Gamma and the posterior predictive distribution is negative binomial, providing closed-form partial pooling and explicit propagation of parameter uncertainty. This is especially useful for zero-history or low-exposure units, where independent maximum-likelihood estimates are unstable or collapse to zero. An adaptive extension, A-HEPG, adds a moment-matched negative-binomial variance correction when residual extra-Poisson variation remains. In synthetic experiments with 400 heterogeneous markets, HEPG improves average negative log predictive probability relative to complete pooling and independent route-level Poisson estimation. At a mean latent rate of 0.15 requests per exposure unit, the score is 0.525 for HEPG, compared with 0.585 for pooled Poisson and 0.731 for route-level Poisson maximum likelihood. Under strong overdispersion, A-HEPG reduces the score from 0.552 to 0.529. A downstream cabin-capacity example links the predictive distribution to a seat-protection rule based on predictive survival probabilities and fare ratios. The study emphasizes coherent probabilistic inference, partial pooling, and diagnostic model escalation for sparse applied count data.
Related
- Hierarchical Bayes meets hierarchical forecasting: A flexible framework for level-focused forecasts
- Hierarchical Time Series Forecasting with Operator-Valued Kernels
- Coherent Hierarchical Forecasting for Proportion and Discrete Time Series
- Probabilistic Forecasting of Electric Vehicle Charging in a Time-varying Hierarchical Infrastructure
- Spatiotemporal Joint Power Forecasting for Hydro-Wind-Solar Complementary Systems via a Hierarchical Complementary-Prior Regularized MSTD-HGFormer
BibTeX
@misc{gudavadze2026hierarchical,
title = {{Hierarchical Empirical-Bayes Forecasting of Sparse Counts with Exposure: An Application to Airline Demand and Premium-Cabin Capacity Sharing}},
author = {Irakli Gudavadze},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.14530},
eprint = {2611.14530},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.14530},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Hierarchical Empirical-Bayes Forecasting of Sparse Counts with Exposure: An Application to Airline Demand and Premium-Cabin Capacity Sharing
AU - Irakli Gudavadze
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.14530
UR - https://arxiv.org/abs/2611.14530
AB - Sparse count forecasting arises when events are rare, exposure varies across observational units, and short histories contain many zeros. Using thin-market airline demand as a motivating application, this paper develops a hierarchical exposure-adjusted Poisson-Gamma (HEPG) framework for probabilistic forecasting. Conditional counts follow a Poisson model with an exposure offset and pooled calendar effects, while unit specific baseline intensities receive a Gamma prior whose hyperparameters are estimated empirically across the population. The resulting posterior is Gamma and the posterior predictive distribution is negative binomial, providing closed-form partial pooling and explicit propagation of parameter uncertainty. This is especially useful for zero-history or low-exposure units, where independent maximum-likelihood estimates are unstable or collapse to zero. An adaptive extension, A-HEPG, adds a moment-matched negative-binomial variance correction when residual extra-Poisson variation remains. In synthetic experiments with 400 heterogeneous markets, HEPG improves average negative log predictive probability relative to complete pooling and independent route-level Poisson estimation. At a mean latent rate of 0.15 requests per exposure unit, the score is 0.525 for HEPG, compared with 0.585 for pooled Poisson and 0.731 for route-level Poisson maximum likelihood. Under strong overdispersion, A-HEPG reduces the score from 0.552 to 0.529. A downstream cabin-capacity example links the predictive distribution to a seat-protection rule based on predictive survival probabilities and fare ratios. The study emphasizes coherent probabilistic inference, partial pooling, and diagnostic model escalation for sparse applied count data.
ER -