Historical skill · Dec 2026
How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.
Takeaway — Dec 2026
Historical skill is uncertain: cross-validated 1993–2016 RPSS is +0.002, and its interval (−0.044 to +0.051) includes no improvement over climatology. The exploratory 2017–2024 result (+0.043) is encouraging but those years were inspected during method selection.
Stronger historical evidence (interval above zero): ONDJ +0.133, Oct +0.091.
| Target | Cross-validated 1993–2016 (main) | In words | Exploratory 2017–2024 |
|---|---|---|---|
| Dec | +0.002 (−0.044 to +0.051) | Small estimated improvement; skill uncertain | +0.043 (−0.020 to +0.098) |
| ONDJ | +0.133 (+0.032 to +0.245) | Moderate improvement over climatology (interval above zero) | +0.280 (+0.034 to +0.575) |
| Oct | +0.091 (+0.025 to +0.159) | Modest improvement over climatology (interval above zero) | +0.103 (+0.005 to +0.241) |
| Nov | +0.077 (−0.030 to +0.182) | Small estimated improvement; skill uncertain | +0.193 (−0.005 to +0.438) |
| Jan | +0.002 (−0.013 to +0.016) | Small estimated improvement; skill uncertain | −0.007 (−0.035 to +0.021) |
RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.
Technical detail: years better, p-values and blend weights
| Target | Years better (cross-validated) | p (one target / Holm) | Years better (exploratory) | Blend weight λ |
|---|---|---|---|---|
| ONDJ | 18 of 24 | 0.010 / 0.040 | 6 of 8 | 0.18 |
| Oct | 19 of 24 | 0.007 / 0.037 | 6 of 8 | 0.41 |
| Nov | 15 of 24 | 0.086 / 0.257 | 5 of 8 | 0.32 |
| Dec | 9 of 24 | 0.471 / 0.799 | 5 of 8 | 0.70 |
| Jan | 13 of 24 | 0.400 / 0.799 | 4 of 8 | 0.90 |
Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: ONDJ, Oct remain significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).
Historical verification explorer
The evidence behind the scores above, for Dec 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 9 of 24 years. Weakest years: 1998 (−0.210), 2005 (−0.145), 2001 (−0.136).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 69 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 20%–30% | 27% | 34% | 1,891 |
| below normal | 30%–40% | 35% | 35% | 7,468 |
| below normal | 40%–50% | 43% | 36% | 4,417 |
| below normal | 50%–60% | 51% | 27% | 11 |
| near normal | 10%–20% | 20% | 20% | 130 |
| near normal | 20%–30% | 24% | 30% | 10,611 |
| near normal | 30%–40% | 34% | 34% | 2,572 |
| near normal | 40%–50% | 44% | 42% | 473 |
| near normal | 50%–60% | 50% | 100% | 1 |
| above normal | 20%–30% | 28% | 13% | 841 |
| above normal | 30%–40% | 35% | 33% | 9,371 |
| above normal | 40%–50% | 43% | 43% | 3,509 |
| above normal | 50%–60% | 52% | 3% | 66 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.2303 | 0.2271 | −0.014 |
| near normal | 0.2155 | 0.2133 | −0.011 |
| above normal | 0.2217 | 0.2258 | +0.018 |
Rainfall amounts
Historical rainfall-amount diagnostics (CRPS, CRPSS and amount errors) have not been generated for Dec 2026 in this area and period; this needs a separate cross-validated amount evaluation. Single-season amount scores are in the Verification section.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 5 of 8 years. Weakest years: 2023 (−0.122), 2022 (−0.024), 2018 (−0.008).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 50 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 20%–30% | 27% | 40% | 769 |
| below normal | 30%–40% | 36% | 49% | 2,388 |
| below normal | 40%–50% | 42% | 63% | 1,443 |
| near normal | 20%–30% | 26% | 21% | 3,084 |
| near normal | 30%–40% | 33% | 33% | 1,341 |
| near normal | 40%–50% | 44% | 39% | 175 |
| above normal | 20%–30% | 29% | 9% | 487 |
| above normal | 30%–40% | 34% | 20% | 3,283 |
| above normal | 40%–50% | 43% | 44% | 830 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.2681 | 0.2837 | +0.055 |
| near normal | 0.1842 | 0.1945 | +0.053 |
| above normal | 0.1840 | 0.1886 | +0.024 |
Rainfall amounts
Historical rainfall-amount diagnostics (CRPS, CRPSS and amount errors) have not been generated for Dec 2026 in this area and period; this needs a separate cross-validated amount evaluation. Single-season amount scores are in the Verification section.
Historical performance by rainfall region
Probability skill by rainfall region (full regional matrix)
| Region | ONDJ | Oct | Nov | Dec | Jan |
|---|---|---|---|---|---|
| All Ethiopia | +0.039 (−0.038 to +0.123) 14/24 yrs | +0.013 (−0.038 to +0.064) 15/24 yrs | +0.040 (−0.032 to +0.117) 14/24 yrs | +0.003 (−0.026 to +0.031) 10/24 yrs | +0.001 (−0.010 to +0.010) 12/24 yrs |
| R0 arid / marginal | −0.059 (−0.207 to +0.071) 14/24 yrs | −0.045 (−0.122 to +0.029) 10/24 yrs | −0.095 (−0.213 to +0.022) 11/24 yrs | +0.017 (−0.018 to +0.054) 12/24 yrs | −0.002 (−0.010 to +0.006) 14/24 yrs |
| R1 western unimodal | −0.003 (−0.089 to +0.081) 14/24 yrs | −0.029 (−0.100 to +0.038) 14/24 yrs | +0.044 (−0.003 to +0.095) 17/24 yrs | −0.003 (−0.038 to +0.025) 14/24 yrs | +0.001 (−0.010 to +0.011) 14/24 yrs |
| R2 Belg–Kiremt | −0.033 (−0.121 to +0.063) 7/24 yrs | −0.044 (−0.107 to +0.018) 9/24 yrs | +0.009 (−0.062 to +0.090) 12/24 yrs | +0.007 (−0.021 to +0.035) 12/24 yrs | 0.000 (−0.011 to +0.009) 13/24 yrs |
| R3 Gu–Deyr | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
| ONDJ R3 (Deyr) rainfall domain | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.