The influence of ensemble size and composition on the performance of combined real-time COVID-19 forecasts
- 1Institute of Statistics, Karlsruhe Institute of Technology, Karlsruhe, Germany
- 2Centre for Mathematical Modelling of Infectious Diseases, London School of Hygiene & Tropical Medicine, London, United Kingdom
- 3Department of Infectious Disease Epidemiology and Dynamics, London School of Hygiene & Tropical Medicine, London, United Kingdom
Abstract
During infectious disease outbreaks, short-term forecasts can play an important role for both decision makers and the general public. While previous research has shown that combining individual forecasts into an ensemble improves accuracy and consistency, practical guidance for organisers of multi-model prediction platforms on how to construct an ensemble has been scarce. In particular, it is not entirely clear how ensemble performance relates to the size of the underlying model pool, a relevant question when relying on voluntary contributions from modelling teams that face competing priorities. Furthermore, the exact composition of an ensemble forecast may influence its performance. Ensembles can either include all models equally or, alternatively, discriminate based on past performance or other characteristics. Using data from the European COVID-19 Forecast Hub we investigated these questions, with the aim of offering practical guidance to organisers of multi-model prediction platforms during infectious disease outbreaks. We found that including more models both improved and stabilised aggregate ensemble performance, while selecting for better component models did not yield any particular advantage. Diversity among models, whether measured numerically or qualitatively, did not have a clear impact on ensemble performance. These results suggest that for those soliciting contributions to collaborative ensembles there are more obvious gains to be made from increasing participation to moderate levels than from optimising component models. Author summary: Infectious disease outbreaks such as the recent COVID-19 pandemic present uncertain situations for governments, health agencies and the general public. Short-term forecasts of key indicators, such as case numbers and deaths from the disease, can help inform expectations about how an outbreak is likely to develop in the near future. These forecasts are often produced using computer models that differ in their underlying assumptions and use of available data. One common approach to improve prediction quality is to collect forecasts from several such models and combine them into a so-called ensemble forecast. However, for organisers of these forecasting efforts it is often unclear how to best construct such an ensemble, in particular how many individual forecasts to collect or whether some models’ forecasts should be selected over others. Using forecasts submitted to the European COVID-19 Forecast Hub, we examined how ensemble performance depends on the number and selection of contributing model forecasts. We found that ensembles generally became more accurate and more stable as additional models’ forecasts were included. In contrast, selectively including only models with better past performance did not consistently improve results. We also found no clear evidence that greater diversity among modelling approaches led to better ensemble forecasts. Our findings suggest that ensemble forecasting efforts benefit more from encouraging broad participation to obtain a moderate number of forecasts than from selecting individual models’ forecasts.
Related
- Statistical versus machine learning-based spatial interpolation of post-processed ensemble weather forecasts
- Combined Influence of Membrane Profiling and Wing-Shaped Spacers on Ion Transport: A Two-Dimensional Potentiodynamic Study
- Match forecasts in UEFA club competitions: Elo ratings versus Transfermarkt valuations
- Target alignment, dilution and forecast selection when cross-sectional forecasts share a common target
- From trends to optimized forecasts: Quantifying the skill and advancing the utility of dynamics-based early warnings for tipping events
BibTeX
@article{becker2026influence,
title = {{The influence of ensemble size and composition on the performance of combined real-time COVID-19 forecasts}},
author = {Friederike Becker and Katharine Sherratt and Nikos I. Bosse and Sebastian Funk},
journal = {PLOS Computational Biology},
year = {2026},
month = nov,
doi = {10.1371/journal.pcbi.1014908},
url = {https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1014908},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - The influence of ensemble size and composition on the performance of combined real-time COVID-19 forecasts
AU - Friederike Becker
AU - Katharine Sherratt
AU - Nikos I. Bosse
AU - Sebastian Funk
PY - 2026
DA - 2026/11/03
JO - PLOS Computational Biology
DO - 10.1371/journal.pcbi.1014908
UR - https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1014908
AB - During infectious disease outbreaks, short-term forecasts can play an important role for both decision makers and the general public. While previous research has shown that combining individual forecasts into an ensemble improves accuracy and consistency, practical guidance for organisers of multi-model prediction platforms on how to construct an ensemble has been scarce. In particular, it is not entirely clear how ensemble performance relates to the size of the underlying model pool, a relevant question when relying on voluntary contributions from modelling teams that face competing priorities. Furthermore, the exact composition of an ensemble forecast may influence its performance. Ensembles can either include all models equally or, alternatively, discriminate based on past performance or other characteristics. Using data from the European COVID-19 Forecast Hub we investigated these questions, with the aim of offering practical guidance to organisers of multi-model prediction platforms during infectious disease outbreaks. We found that including more models both improved and stabilised aggregate ensemble performance, while selecting for better component models did not yield any particular advantage. Diversity among models, whether measured numerically or qualitatively, did not have a clear impact on ensemble performance. These results suggest that for those soliciting contributions to collaborative ensembles there are more obvious gains to be made from increasing participation to moderate levels than from optimising component models. Author summary: Infectious disease outbreaks such as the recent COVID-19 pandemic present uncertain situations for governments, health agencies and the general public. Short-term forecasts of key indicators, such as case numbers and deaths from the disease, can help inform expectations about how an outbreak is likely to develop in the near future. These forecasts are often produced using computer models that differ in their underlying assumptions and use of available data. One common approach to improve prediction quality is to collect forecasts from several such models and combine them into a so-called ensemble forecast. However, for organisers of these forecasting efforts it is often unclear how to best construct such an ensemble, in particular how many individual forecasts to collect or whether some models’ forecasts should be selected over others. Using forecasts submitted to the European COVID-19 Forecast Hub, we examined how ensemble performance depends on the number and selection of contributing model forecasts. We found that ensembles generally became more accurate and more stable as additional models’ forecasts were included. In contrast, selectively including only models with better past performance did not consistently improve results. We also found no clear evidence that greater diversity among modelling approaches led to better ensemble forecasts. Our findings suggest that ensemble forecasting efforts benefit more from encouraging broad participation to obtain a moderate number of forecasts than from selecting individual models’ forecasts.
SN - 1553-7358
ER -