Historical skill · ONDJ 2026/27
How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.
Takeaway — ONDJ 2026/27
Cross-validated 1993–2016 skill is +0.133 (interval +0.032 to +0.245, above zero): a moderate improvement over climatology. The exploratory 2017–2024 result (+0.280) is encouraging but those years were inspected during method selection.
Stronger historical evidence (interval above zero): Oct +0.091.
| Target | Cross-validated 1993–2016 (main) | In words | Exploratory 2017–2024 |
|---|---|---|---|
| ONDJ | +0.133 (+0.032 to +0.245) | Moderate improvement over climatology (interval above zero) | +0.280 (+0.034 to +0.575) |
| Oct | +0.091 (+0.025 to +0.159) | Modest improvement over climatology (interval above zero) | +0.103 (+0.005 to +0.241) |
| Nov | +0.077 (−0.030 to +0.182) | Small estimated improvement; skill uncertain | +0.193 (−0.005 to +0.438) |
| Dec | +0.002 (−0.044 to +0.051) | Small estimated improvement; skill uncertain | +0.043 (−0.020 to +0.098) |
| Jan | +0.002 (−0.013 to +0.016) | Small estimated improvement; skill uncertain | −0.007 (−0.035 to +0.021) |
RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.
Technical detail: years better, p-values and blend weights
| Target | Years better (cross-validated) | p (one target / Holm) | Years better (exploratory) | Blend weight λ |
|---|---|---|---|---|
| ONDJ | 18 of 24 | 0.010 / 0.040 | 6 of 8 | 0.18 |
| Oct | 19 of 24 | 0.007 / 0.037 | 6 of 8 | 0.41 |
| Nov | 15 of 24 | 0.086 / 0.257 | 5 of 8 | 0.32 |
| Dec | 9 of 24 | 0.471 / 0.799 | 5 of 8 | 0.70 |
| Jan | 13 of 24 | 0.400 / 0.799 | 4 of 8 | 0.90 |
Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: ONDJ, Oct remain significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).
Historical verification explorer
The evidence behind the scores above, for ONDJ 2026/27 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 18 of 24 years. Weakest years: 2000 (−0.351), 2007 (−0.194), 2008 (−0.127).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 69 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 0%–10% | 9% | 3% | 173 |
| below normal | 10%–20% | 15% | 9% | 2,125 |
| below normal | 20%–30% | 26% | 24% | 3,829 |
| below normal | 30%–40% | 36% | 38% | 3,788 |
| below normal | 40%–50% | 45% | 52% | 2,682 |
| below normal | 50%–60% | 54% | 70% | 1,099 |
| below normal | 60%–70% | 63% | 59% | 104 |
| near normal | 0%–10% | 9% | 21% | 67 |
| near normal | 10%–20% | 17% | 19% | 2,434 |
| near normal | 20%–30% | 26% | 26% | 4,610 |
| near normal | 30%–40% | 34% | 35% | 4,043 |
| near normal | 40%–50% | 44% | 41% | 2,118 |
| near normal | 50%–60% | 53% | 41% | 488 |
| near normal | 60%–70% | 63% | 40% | 35 |
| near normal | 70%–80% | 72% | 40% | 5 |
| above normal | 0%–10% | 7% | 0% | 10 |
| above normal | 10%–20% | 16% | 13% | 1,118 |
| above normal | 20%–30% | 26% | 23% | 4,388 |
| above normal | 30%–40% | 36% | 30% | 3,654 |
| above normal | 40%–50% | 44% | 37% | 2,057 |
| above normal | 50%–60% | 54% | 60% | 1,429 |
| above normal | 60%–70% | 65% | 83% | 756 |
| above normal | 70%–80% | 74% | 85% | 388 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.1981 | 0.2266 | +0.126 |
| near normal | 0.2063 | 0.2119 | +0.026 |
| above normal | 0.1951 | 0.2270 | +0.141 |
Rainfall amounts
| Rainfall amount skill (CRPSS) | +0.178 |
| CRPS, corrected / climatology | 41.5 / 50.5 mm |
| Average rainfall error, corrected (raw model) | −25.5 mm (−41.1 mm) |
| RMSE, corrected / climatology | 77.2 / 99.8 mm |
Performance by year
Probability skill (RPSS) of each year; better than climatology in 6 of 8 years. Weakest years: 2024 (−0.170), 2017 (−0.052), 2022 (+0.001).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 50 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 0%–10% | 9% | 0% | 157 |
| below normal | 10%–20% | 14% | 1% | 972 |
| below normal | 20%–30% | 26% | 29% | 397 |
| below normal | 30%–40% | 36% | 45% | 1,359 |
| below normal | 40%–50% | 44% | 47% | 1,343 |
| below normal | 50%–60% | 54% | 48% | 343 |
| below normal | 60%–70% | 61% | 69% | 29 |
| near normal | 0%–10% | 9% | 0% | 10 |
| near normal | 10%–20% | 17% | 11% | 960 |
| near normal | 20%–30% | 25% | 30% | 1,642 |
| near normal | 30%–40% | 34% | 39% | 1,403 |
| near normal | 40%–50% | 44% | 58% | 483 |
| near normal | 50%–60% | 54% | 80% | 86 |
| near normal | 60%–70% | 63% | 94% | 16 |
| above normal | 10%–20% | 17% | 11% | 510 |
| above normal | 20%–30% | 25% | 9% | 1,474 |
| above normal | 30%–40% | 34% | 16% | 1,257 |
| above normal | 40%–50% | 44% | 25% | 223 |
| above normal | 50%–60% | 56% | 89% | 189 |
| above normal | 60%–70% | 66% | 98% | 594 |
| above normal | 70%–80% | 72% | 100% | 353 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.1897 | 0.2226 | +0.147 |
| near normal | 0.2018 | 0.2210 | +0.087 |
| above normal | 0.1314 | 0.2231 | +0.411 |
Rainfall amounts
| Rainfall amount skill (CRPSS) | +0.178 |
| CRPS, corrected / climatology | 41.5 / 50.5 mm |
| Average rainfall error, corrected (raw model) | −25.5 mm (−41.1 mm) |
| RMSE, corrected / climatology | 77.2 / 99.8 mm |
Historical performance by rainfall region
Probability skill by rainfall region (full regional matrix)
| Region | ONDJ | Oct | Nov | Dec | Jan |
|---|---|---|---|---|---|
| All Ethiopia | +0.039 (−0.038 to +0.123) 14/24 yrs | +0.013 (−0.038 to +0.064) 15/24 yrs | +0.040 (−0.032 to +0.117) 14/24 yrs | +0.003 (−0.026 to +0.031) 10/24 yrs | +0.001 (−0.010 to +0.010) 12/24 yrs |
| R0 arid / marginal | −0.059 (−0.207 to +0.071) 14/24 yrs | −0.045 (−0.122 to +0.029) 10/24 yrs | −0.095 (−0.213 to +0.022) 11/24 yrs | +0.017 (−0.018 to +0.054) 12/24 yrs | −0.002 (−0.010 to +0.006) 14/24 yrs |
| R1 western unimodal | −0.003 (−0.089 to +0.081) 14/24 yrs | −0.029 (−0.100 to +0.038) 14/24 yrs | +0.044 (−0.003 to +0.095) 17/24 yrs | −0.003 (−0.038 to +0.025) 14/24 yrs | +0.001 (−0.010 to +0.011) 14/24 yrs |
| R2 Belg–Kiremt | −0.033 (−0.121 to +0.063) 7/24 yrs | −0.044 (−0.107 to +0.018) 9/24 yrs | +0.009 (−0.062 to +0.090) 12/24 yrs | +0.007 (−0.021 to +0.035) 12/24 yrs | 0.000 (−0.011 to +0.009) 13/24 yrs |
| R3 Gu–Deyr | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
| ONDJ R3 (Deyr) rainfall domain | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.