NovDemo-2: Ensemble Forecasting

[akruel01]

Scenario 2: Ensemble Forecasting

Please redo the Forecasting Challenge from the 12-month milestone for a single time point (this is the same time point that was part of the 18-month program evaluation as well). Data for the following time period should already be readily available and processed, from the 18-month evaluation.

Time: July 19th, 2021, during the upswing of the Covid wave caused by the arrival of the Delta variant. Vaccines were available at this time.

• Location: New York state
• Task: Take a single model, calibrate it using historical data prior to the given date, and predict cases, hospitalizations, and deaths over a 4-week timeframe starting from the given date. Do the same thing with an ensemble of at least 4 structurally different models (e.g. one model could include hospitalizations, another could focus on wastewater, one could be stratified, etc.). Each of the 4 models should have at least one compartment not present in
  the others. You can compare your own outputs against other forecasters who submitted their results in the Covid-19 Forecast Hub for this time period, using the following metrics:
• Weighted interval score (WIS): “Can be interpreted as a generalization of the absolute error to probabilistic forecasts and allows for a decomposition into a measure of sharpness [spread] and penalties for over- and under-prediction.” A lower score indicates better performance.
• Absolute error: is the absolute value of the difference between the actual value and the point forecast. The point forecast of a model when not provided explicitly is taken to be the 50% quantile of the forecast distribution. For example scripts MITRE will use to evaluate the forecasts, please refer to this Github link