Historical skill · Nov 2026
How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.
Takeaway — Nov 2026
Historical skill is uncertain: cross-validated 1993–2016 RPSS is +0.077, and its interval (−0.030 to +0.182) includes no improvement over climatology. The exploratory 2017–2024 result (+0.193) is encouraging but those years were inspected during method selection.
Stronger historical evidence (interval above zero): ONDJ +0.133, Oct +0.091.
| Target | Cross-validated 1993–2016 (main) | In words | Exploratory 2017–2024 |
|---|---|---|---|
| Nov | +0.077 (−0.030 to +0.182) | Small estimated improvement; skill uncertain | +0.193 (−0.005 to +0.438) |
| ONDJ | +0.133 (+0.032 to +0.245) | Moderate improvement over climatology (interval above zero) | +0.280 (+0.034 to +0.575) |
| Oct | +0.091 (+0.025 to +0.159) | Modest improvement over climatology (interval above zero) | +0.103 (+0.005 to +0.241) |
| Dec | +0.002 (−0.044 to +0.051) | Small estimated improvement; skill uncertain | +0.043 (−0.020 to +0.098) |
| Jan | +0.002 (−0.013 to +0.016) | Small estimated improvement; skill uncertain | −0.007 (−0.035 to +0.021) |
RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.
Technical detail: years better, p-values and blend weights
| Target | Years better (cross-validated) | p (one target / Holm) | Years better (exploratory) | Blend weight λ |
|---|---|---|---|---|
| ONDJ | 18 of 24 | 0.010 / 0.040 | 6 of 8 | 0.18 |
| Oct | 19 of 24 | 0.007 / 0.037 | 6 of 8 | 0.41 |
| Nov | 15 of 24 | 0.086 / 0.257 | 5 of 8 | 0.32 |
| Dec | 9 of 24 | 0.471 / 0.799 | 5 of 8 | 0.70 |
| Jan | 13 of 24 | 0.400 / 0.799 | 4 of 8 | 0.90 |
Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: ONDJ, Oct remain significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).
Historical verification explorer
The evidence behind the scores above, for Nov 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 15 of 24 years. Weakest years: 1993 (−0.510), 2009 (−0.277), 2010 (−0.269).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 68 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 0%–10% | 9% | 100% | 4 |
| below normal | 10%–20% | 18% | 25% | 1,275 |
| below normal | 20%–30% | 25% | 32% | 3,954 |
| below normal | 30%–40% | 34% | 28% | 5,360 |
| below normal | 40%–50% | 44% | 47% | 2,539 |
| below normal | 50%–60% | 53% | 84% | 509 |
| below normal | 60%–70% | 61% | 96% | 27 |
| near normal | 0%–10% | 9% | 33% | 6 |
| near normal | 10%–20% | 17% | 17% | 1,271 |
| near normal | 20%–30% | 25% | 25% | 6,065 |
| near normal | 30%–40% | 35% | 37% | 4,538 |
| near normal | 40%–50% | 44% | 45% | 1,400 |
| near normal | 50%–60% | 54% | 52% | 332 |
| near normal | 60%–70% | 63% | 49% | 55 |
| near normal | 70%–80% | 71% | 100% | 1 |
| above normal | 10%–20% | 19% | 4% | 328 |
| above normal | 20%–30% | 25% | 11% | 2,947 |
| above normal | 30%–40% | 34% | 30% | 5,359 |
| above normal | 40%–50% | 44% | 49% | 3,275 |
| above normal | 50%–60% | 54% | 65% | 1,558 |
| above normal | 60%–70% | 62% | 75% | 201 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.2197 | 0.2267 | +0.031 |
| near normal | 0.2042 | 0.2132 | +0.042 |
| above normal | 0.1981 | 0.2259 | +0.123 |
Rainfall amounts
Historical rainfall-amount diagnostics (CRPS, CRPSS and amount errors) have not been generated for Nov 2026 in this area and period; this needs a separate cross-validated amount evaluation. Single-season amount scores are in the Verification section.
Performance by year
Probability skill (RPSS) of each year; better than climatology in 5 of 8 years. Weakest years: 2024 (−0.141), 2020 (−0.058), 2022 (−0.024).
Reliability
Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 50 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.
Reliability table (forecast bins, observed frequency, counts)
| Category | Bin | Mean forecast | Observed frequency | Cell-years |
|---|---|---|---|---|
| below normal | 10%–20% | 16% | 0% | 666 |
| below normal | 20%–30% | 25% | 16% | 1,051 |
| below normal | 30%–40% | 35% | 43% | 1,984 |
| below normal | 40%–50% | 44% | 41% | 761 |
| below normal | 50%–60% | 52% | 88% | 82 |
| near normal | 10%–20% | 17% | 8% | 385 |
| near normal | 20%–30% | 25% | 26% | 2,304 |
| near normal | 30%–40% | 34% | 41% | 1,546 |
| near normal | 40%–50% | 44% | 52% | 268 |
| near normal | 50%–60% | 53% | 85% | 41 |
| above normal | 10%–20% | 18% | 23% | 39 |
| above normal | 20%–30% | 27% | 13% | 1,053 |
| above normal | 30%–40% | 35% | 19% | 1,877 |
| above normal | 40%–50% | 44% | 40% | 604 |
| above normal | 50%–60% | 55% | 95% | 555 |
| above normal | 60%–70% | 64% | 100% | 409 |
| above normal | 70%–80% | 71% | 100% | 7 |
How strong are the signals?
Share of cell-years by the leading tercile probability.
| Category | Forecast | Climatology | Brier skill |
|---|---|---|---|
| below normal | 0.1896 | 0.2146 | +0.117 |
| near normal | 0.2042 | 0.2175 | +0.061 |
| above normal | 0.1731 | 0.2346 | +0.262 |
Rainfall amounts
Historical rainfall-amount diagnostics (CRPS, CRPSS and amount errors) have not been generated for Nov 2026 in this area and period; this needs a separate cross-validated amount evaluation. Single-season amount scores are in the Verification section.
Historical performance by rainfall region
Probability skill by rainfall region (full regional matrix)
| Region | ONDJ | Oct | Nov | Dec | Jan |
|---|---|---|---|---|---|
| All Ethiopia | +0.039 (−0.038 to +0.123) 14/24 yrs | +0.013 (−0.038 to +0.064) 15/24 yrs | +0.040 (−0.032 to +0.117) 14/24 yrs | +0.003 (−0.026 to +0.031) 10/24 yrs | +0.001 (−0.010 to +0.010) 12/24 yrs |
| R0 arid / marginal | −0.059 (−0.207 to +0.071) 14/24 yrs | −0.045 (−0.122 to +0.029) 10/24 yrs | −0.095 (−0.213 to +0.022) 11/24 yrs | +0.017 (−0.018 to +0.054) 12/24 yrs | −0.002 (−0.010 to +0.006) 14/24 yrs |
| R1 western unimodal | −0.003 (−0.089 to +0.081) 14/24 yrs | −0.029 (−0.100 to +0.038) 14/24 yrs | +0.044 (−0.003 to +0.095) 17/24 yrs | −0.003 (−0.038 to +0.025) 14/24 yrs | +0.001 (−0.010 to +0.011) 14/24 yrs |
| R2 Belg–Kiremt | −0.033 (−0.121 to +0.063) 7/24 yrs | −0.044 (−0.107 to +0.018) 9/24 yrs | +0.009 (−0.062 to +0.090) 12/24 yrs | +0.007 (−0.021 to +0.035) 12/24 yrs | 0.000 (−0.011 to +0.009) 13/24 yrs |
| R3 Gu–Deyr | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
| ONDJ R3 (Deyr) rainfall domain | +0.133 (+0.032 to +0.245) 18/24 yrs | +0.091 (+0.025 to +0.159) 19/24 yrs | +0.077 (−0.030 to +0.182) 15/24 yrs | +0.002 (−0.044 to +0.051) 9/24 yrs | +0.002 (−0.013 to +0.016) 13/24 yrs |
Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.