Multi-model forecasts in the context of the Mpox outbreak in multiple countries (July 28th, 2022 through January 26th, 2023)
In late July, public health officials noted an unprecedented surge in Mpox cases in non-endemic cases around the World. In response, our team began producing weekly forecasts for the most heavily afflicted areas. As the case levels have significantly decreased, evaluating model performance is essential to advance the growing field of epidemic forecasting. We obtained reported Mpox case data from the CDC and OWID teams through the week of 1/26/2023 to produce retrospective weekly forecasts (e.g., 1-week, 2-week, 3-week, and 4-week) for study areas using auto-regressive moving average (ARIMA), general additive model (GAM), simple linear regression (SLR), spatial-wave, and ensemble n-sub-epidemic modeling frameworks. Model performance was then compared via MSE, MAE, WIS, and 95% PI coverage metrics. The spatial-wave modeling framework performed superior across most locations and forecasting horizons in average MSE, MAE, and WIS compared to the other included frameworks. It was followed closely in success by the n-sub-epidemic top-ranked, weighted, and un-weighted ensemble (2) models. Regarding average 95% PI coverage, the n-sub-epidemic unweighted ensemble (2) model performed best across all forecasting horizons for most locations. However, there was more widespread success noted across all modeling frameworks, with many locations seeing multiple models performing equally well in terms of average 95% PI coverage. Model performance tended to increase as we entered the declining phase of the outbreak. Overall, the spatial-wave and ensemble n-sub-epidemic frameworks outperformed other established models (e.g., ARIMA, SLR, GAM). Similar to past performance under different scenarios (e.g., COVID-19), the success seen with both frameworks highlights the continued utility of the models for short-term forecasting epidemic outbreaks.