Historical skill · JJAS 2026
How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.
Takeaway — JJAS 2026
Cross-validated 1993–2016 skill is +0.044 (interval +0.002 to +0.084, above zero): a modest improvement over climatology. The exploratory 2017–2025 result (+0.032) is encouraging but those years were inspected during method selection. The 2026 verification (+0.446) is one season and does not override the historical uncertainty.
Stronger historical evidence (interval above zero): Jul +0.031.
| Target | Cross-validated 1993–2016 (main) | In words | Exploratory 2017–2025 |
|---|---|---|---|
| JJAS | +0.044 (+0.002 to +0.084) | Modest improvement over climatology (interval above zero) | +0.032 (+0.010 to +0.052) |
| Jun | −0.001 (−0.035 to +0.027) | No demonstrated improvement over climatology | +0.015 (−0.007 to +0.038) |
| Jul | +0.031 (+0.004 to +0.058) | Modest improvement over climatology (interval above zero) | +0.027 (−0.021 to +0.079) |
| Aug | +0.011 (−0.006 to +0.027) | Small estimated improvement; skill uncertain | +0.031 (+0.012 to +0.054) |
| Sep | +0.006 (−0.012 to +0.024) | Small estimated improvement; skill uncertain | +0.006 (−0.018 to +0.030) |
RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.
Technical detail: years better, p-values and blend weights
| Target | Years better (cross-validated) | p (one target / Holm) | Years better (exploratory) | Blend weight λ |
|---|---|---|---|---|
| JJAS | 17 of 24 | 0.029 / 0.115 | 7 of 9 | 0.59 |
| Jun | 15 of 24 | 0.510 / 0.537 | 6 of 9 | 0.83 |
| Jul | 19 of 24 | 0.019 / 0.094 | 5 of 9 | 0.60 |
| Aug | 16 of 24 | 0.103 / 0.308 | 7 of 9 | 0.58 |
| Sep | 14 of 24 | 0.268 / 0.537 | 4 of 9 | 0.81 |
Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: no target remains significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).
Historical verification explorer
The evidence behind the scores above, for JJAS 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 17 of 24 years. Weakest years: 2004 (−0.223), 2016 (−0.157), 1998 (−0.092).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 100 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 10%–20% | 20% | 65% | 86 |
| below normal | 20%–30% | 26% | 23% | 7,444 |
| below normal | 30%–40% | 34% | 34% | 8,236 |
| below normal | 40%–50% | 44% | 53% | 3,087 |
| below normal | 50%–60% | 55% | 63% | 1,063 |
| below normal | 60%–70% | 61% | 70% | 76 |
| near normal | 10%–20% | 19% | 22% | 398 |
| near normal | 20%–30% | 27% | 25% | 7,742 |
| near normal | 30%–40% | 34% | 34% | 10,790 |
| near normal | 40%–50% | 42% | 42% | 1,056 |
| near normal | 50%–60% | 51% | 33% | 6 |
| above normal | 20%–30% | 26% | 22% | 5,921 |
| above normal | 30%–40% | 35% | 39% | 9,334 |
| above normal | 40%–50% | 44% | 43% | 3,995 |
| above normal | 50%–60% | 53% | 42% | 713 |
| above normal | 60%–70% | 61% | 28% | 29 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.2138 | 0.2268 | +0.057 |
| near normal | 0.2090 | 0.2121 | +0.015 |
| above normal | 0.2198 | 0.2266 | +0.030 |
Rainfall amounts
| Rainfall amount skill (CRPSS) | −0.018 |
| CRPS, corrected / climatology | 46.1 / 45.3 mm |
| Average rainfall error, corrected (raw model) | −13.6 mm (−5.2 mm) |
| RMSE, corrected / climatology | 96.0 / 93.1 mm |
Performance by year
Probability skill (RPSS) of each year; better than climatology in 7 of 9 years. Weakest years: 2017 (−0.024), 2018 (−0.021), 2021 (+0.013).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 50 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 20%–30% | 26% | 17% | 3,025 |
| below normal | 30%–40% | 34% | 18% | 3,754 |
| below normal | 40%–50% | 43% | 36% | 717 |
| below normal | 50%–60% | 52% | 100% | 1 |
| near normal | 20%–30% | 27% | 22% | 1,828 |
| near normal | 30%–40% | 34% | 32% | 5,210 |
| near normal | 40%–50% | 42% | 46% | 459 |
| above normal | 20%–30% | 27% | 43% | 2,205 |
| above normal | 30%–40% | 34% | 50% | 3,528 |
| above normal | 40%–50% | 45% | 60% | 1,361 |
| above normal | 50%–60% | 54% | 55% | 403 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.1715 | 0.1771 | +0.032 |
| near normal | 0.2088 | 0.2123 | +0.016 |
| above normal | 0.2685 | 0.2773 | +0.032 |
Rainfall amounts
| Rainfall amount skill (CRPSS) | −0.018 |
| CRPS, corrected / climatology | 46.1 / 45.3 mm |
| Average rainfall error, corrected (raw model) | −13.6 mm (−5.2 mm) |
| RMSE, corrected / climatology | 96.0 / 93.1 mm |
Historical performance by rainfall region
Probability skill by rainfall region (full regional matrix)
| Region | JJAS | Jun | Jul | Aug | Sep |
|---|---|---|---|---|---|
| All Ethiopia | +0.027 (−0.006 to +0.058) 18/24 yrs | 0.000 (−0.025 to +0.019) 15/24 yrs | +0.014 (−0.016 to +0.042) 17/24 yrs | +0.012 (−0.006 to +0.030) 12/24 yrs | +0.003 (−0.012 to +0.018) 14/24 yrs |
| R0 arid / marginal | +0.066 (−0.003 to +0.133) 15/24 yrs | −0.001 (−0.034 to +0.027) 13/24 yrs | +0.004 (−0.070 to +0.078) 14/24 yrs | +0.042 (+0.002 to +0.081) 17/24 yrs | +0.003 (−0.036 to +0.037) 14/24 yrs |
| R1 western unimodal | +0.031 (−0.004 to +0.066) 15/24 yrs | −0.007 (−0.038 to +0.020) 14/24 yrs | +0.030 (+0.004 to +0.057) 16/24 yrs | −0.002 (−0.017 to +0.012) 15/24 yrs | +0.006 (−0.014 to +0.026) 14/24 yrs |
| R2 Belg–Kiremt | +0.055 (−0.006 to +0.115) 17/24 yrs | +0.006 (−0.036 to +0.037) 14/24 yrs | +0.033 (−0.008 to +0.074) 17/24 yrs | +0.024 (+0.002 to +0.046) 14/24 yrs | +0.006 (−0.015 to +0.029) 14/24 yrs |
| R3 Gu–Deyr | −0.001 (−0.037 to +0.034) 14/24 yrs | 0.000 (−0.020 to +0.017) 14/24 yrs | −0.028 (−0.090 to +0.027) 12/24 yrs | +0.007 (−0.020 to +0.035) 13/24 yrs | −0.001 (−0.028 to +0.023) 13/24 yrs |
| JJAS R1+R2 rainfall domain | +0.044 (+0.002 to +0.084) 17/24 yrs | −0.001 (−0.035 to +0.027) 15/24 yrs | +0.031 (+0.004 to +0.058) 19/24 yrs | +0.011 (−0.006 to +0.027) 16/24 yrs | +0.006 (−0.012 to +0.024) 14/24 yrs |
Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.