Forecast scores are metrics that quantify how well probabilistic or point forecasts match the observed outcomes
MAE
Measures the average magnitude of the errors in the predictions. Lower values mean predictions are closer on average to actual data
MSE
Measures the average of the squared differences between forecast and observation. Lower values mean fewer and/or smaller large errors in predictions
CRPS
Measures how well the predicted cumulative distribution function (CDF) matches the observed outcome. Lower values indicate the forecast distribution better matches the observed outcome
Log Score
Measures the log-probability assigned to the observed value by the forecast distribution. Higher log-probabilities mean the forecast assigned more probability to the actual observation
Interval Score
Evaluates prediction intervals (e.g., 90% confidence intervals) of the forecast. Lower values mean narrower prediction intervals that still capture the true value appropriately