Historical skill · Nov 2026 · ONDJ 2026/27 — September initialization · ONDJ R3 (Deyr) rainfall domain

Historical skill · Nov 2026

How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.

Takeaway — Nov 2026

Historical skill is uncertain: cross-validated 1993–2016 RPSS is +0.077, and its interval (−0.030 to +0.182) includes no improvement over climatology. The exploratory 2017–2024 result (+0.193) is encouraging but those years were inspected during method selection.

Stronger historical evidence (interval above zero): ONDJ +0.133, Oct +0.091.

ONDJ 2026/27 (September initialization), ONDJ R3 (Deyr) rainfall domain: probability skill (RPSS) of the final method against climatology, with whole-year 95% intervals
TargetCross-validated 1993–2016 (main)In wordsExploratory 2017–2024
Nov+0.077 (−0.030 to +0.182)Small estimated improvement; skill uncertain+0.193 (−0.005 to +0.438)
ONDJ+0.133 (+0.032 to +0.245)Moderate improvement over climatology (interval above zero)+0.280 (+0.034 to +0.575)
Oct+0.091 (+0.025 to +0.159)Modest improvement over climatology (interval above zero)+0.103 (+0.005 to +0.241)
Dec+0.002 (−0.044 to +0.051)Small estimated improvement; skill uncertain+0.043 (−0.020 to +0.098)
Jan+0.002 (−0.013 to +0.016)Small estimated improvement; skill uncertain−0.007 (−0.035 to +0.021)

RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.

Technical detail: years better, p-values and blend weights
TargetYears better (cross-validated)p (one target / Holm)Years better (exploratory)Blend weight λ
ONDJ18 of 240.010 / 0.0406 of 80.18
Oct19 of 240.007 / 0.0376 of 80.41
Nov15 of 240.086 / 0.2575 of 80.32
Dec9 of 240.471 / 0.7995 of 80.70
Jan13 of 240.400 / 0.7994 of 80.90

Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: ONDJ, Oct remain significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).

Historical verification explorer

The evidence behind the scores above, for Nov 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.

Evaluation
Cross-validated (nested leave-one-year-out), 1993–2016
Benchmark
Climatology: observed tercile frequencies of the training years (about one-third each)
Area
ONDJ R3 (Deyr) rainfall domain; probabilities on 99% of it on average
Sample
24 years, 13,668 cell-years. Cells within a year are spatially correlated, so the independent sample is closer to one value per year.

Performance by year

Probability skill (RPSS) of each year; better than climatology in 15 of 24 years. Weakest years: 1993 (−0.510), 2009 (−0.277), 2010 (−0.269).

Better than climatologyWorse
−0.60−0.40−0.200.00+0.20+0.40+0.601993: RPSS −0.510 (forecast RPS 0.807, climatology 0.535; 572 cells)199319931994: RPSS +0.273 (forecast RPS 0.361, climatology 0.497; 568 cells)1995: RPSS +0.034 (forecast RPS 0.449, climatology 0.465; 572 cells)1996: RPSS +0.107 (forecast RPS 0.369, climatology 0.413; 572 cells)1997: RPSS +0.539 (forecast RPS 0.252, climatology 0.546; 568 cells)199719971998: RPSS +0.178 (forecast RPS 0.392, climatology 0.477; 572 cells)1999: RPSS +0.370 (forecast RPS 0.297, climatology 0.472; 571 cells)2000: RPSS +0.075 (forecast RPS 0.353, climatology 0.382; 568 cells)2001: RPSS +0.234 (forecast RPS 0.360, climatology 0.471; 572 cells)20012002: RPSS −0.067 (forecast RPS 0.390, climatology 0.366; 568 cells)2003: RPSS +0.040 (forecast RPS 0.340, climatology 0.355; 568 cells)2004: RPSS −0.029 (forecast RPS 0.539, climatology 0.524; 568 cells)2005: RPSS +0.130 (forecast RPS 0.369, climatology 0.424; 571 cells)20052006: RPSS +0.181 (forecast RPS 0.389, climatology 0.475; 568 cells)2007: RPSS −0.042 (forecast RPS 0.422, climatology 0.405; 568 cells)2008: RPSS −0.032 (forecast RPS 0.444, climatology 0.431; 571 cells)2009: RPSS −0.277 (forecast RPS 0.514, climatology 0.403; 570 cells)20092010: RPSS −0.269 (forecast RPS 0.693, climatology 0.546; 572 cells)2011: RPSS +0.162 (forecast RPS 0.447, climatology 0.533; 568 cells)2012: RPSS −0.120 (forecast RPS 0.375, climatology 0.335; 568 cells)2013: RPSS +0.156 (forecast RPS 0.458, climatology 0.543; 568 cells)20132014: RPSS +0.073 (forecast RPS 0.332, climatology 0.358; 568 cells)2015: RPSS +0.493 (forecast RPS 0.256, climatology 0.505; 568 cells)2016: RPSS −0.039 (forecast RPS 0.419, climatology 0.404; 569 cells)

Reliability

below normalnear normalabove normal
0.00.00.20.20.40.40.60.60.80.81.01.0below normal: forecast 9%, observed 100% (4 cell-years, too few to judge)below normal: forecast 18%, observed 25% (1275 cell-years)below normal: forecast 25%, observed 32% (3954 cell-years)below normal: forecast 34%, observed 28% (5360 cell-years)below normal: forecast 44%, observed 47% (2539 cell-years)below normal: forecast 53%, observed 84% (509 cell-years)below normal: forecast 61%, observed 96% (27 cell-years, too few to judge)near normal: forecast 9%, observed 33% (6 cell-years, too few to judge)near normal: forecast 17%, observed 17% (1271 cell-years)near normal: forecast 25%, observed 25% (6065 cell-years)near normal: forecast 35%, observed 37% (4538 cell-years)near normal: forecast 44%, observed 45% (1400 cell-years)near normal: forecast 54%, observed 52% (332 cell-years)near normal: forecast 63%, observed 49% (55 cell-years, too few to judge)near normal: forecast 71%, observed 100% (1 cell-years, too few to judge)above normal: forecast 19%, observed 4% (328 cell-years)above normal: forecast 25%, observed 11% (2947 cell-years)above normal: forecast 34%, observed 30% (5359 cell-years)above normal: forecast 44%, observed 49% (3275 cell-years)above normal: forecast 54%, observed 65% (1558 cell-years)above normal: forecast 62%, observed 75% (201 cell-years)Forecast probability

Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 68 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.

Reliability table (forecast bins, observed frequency, counts)
CategoryBinMean forecastObserved frequencyCell-years
below normal0%–10%9%100%4
below normal10%–20%18%25%1,275
below normal20%–30%25%32%3,954
below normal30%–40%34%28%5,360
below normal40%–50%44%47%2,539
below normal50%–60%53%84%509
below normal60%–70%61%96%27
near normal0%–10%9%33%6
near normal10%–20%17%17%1,271
near normal20%–30%25%25%6,065
near normal30%–40%35%37%4,538
near normal40%–50%44%45%1,400
near normal50%–60%54%52%332
near normal60%–70%63%49%55
near normal70%–80%71%100%1
above normal10%–20%19%4%328
above normal20%–30%25%11%2,947
above normal30%–40%34%30%5,359
above normal40%–50%44%49%3,275
above normal50%–60%54%65%1,558
above normal60%–70%62%75%201

How strong are the signals?

Share of cell-years by the leading tercile probability.

weak (<40%)
29%
moderate (40–50%)
51%
strong (50–60%)
18%
very strong (≥60%)
2%
Category Brier scores (lower is better) and Brier skill against climatology
CategoryForecastClimatologyBrier skill
below normal0.21970.2267+0.031
near normal0.20420.2132+0.042
above normal0.19810.2259+0.123

Rainfall amounts

Historical rainfall-amount diagnostics (CRPS, CRPSS and amount errors) have not been generated for Nov 2026 in this area and period; this needs a separate cross-validated amount evaluation. Single-season amount scores are in the Verification section.

Historical performance by rainfall region

Probability skill by rainfall region (full regional matrix)
ONDJ 2026/27: probability skill (RPSS) by rainfall region, cross-validated 1993–2016, with whole-year 95% intervals and years better than climatology
RegionONDJOctNovDecJan
All Ethiopia+0.039 (−0.038 to +0.123) 14/24 yrs+0.013 (−0.038 to +0.064) 15/24 yrs+0.040 (−0.032 to +0.117) 14/24 yrs+0.003 (−0.026 to +0.031) 10/24 yrs+0.001 (−0.010 to +0.010) 12/24 yrs
R0 arid / marginal−0.059 (−0.207 to +0.071) 14/24 yrs−0.045 (−0.122 to +0.029) 10/24 yrs−0.095 (−0.213 to +0.022) 11/24 yrs+0.017 (−0.018 to +0.054) 12/24 yrs−0.002 (−0.010 to +0.006) 14/24 yrs
R1 western unimodal−0.003 (−0.089 to +0.081) 14/24 yrs−0.029 (−0.100 to +0.038) 14/24 yrs+0.044 (−0.003 to +0.095) 17/24 yrs−0.003 (−0.038 to +0.025) 14/24 yrs+0.001 (−0.010 to +0.011) 14/24 yrs
R2 Belg–Kiremt−0.033 (−0.121 to +0.063) 7/24 yrs−0.044 (−0.107 to +0.018) 9/24 yrs+0.009 (−0.062 to +0.090) 12/24 yrs+0.007 (−0.021 to +0.035) 12/24 yrs0.000 (−0.011 to +0.009) 13/24 yrs
R3 Gu–Deyr+0.133 (+0.032 to +0.245) 18/24 yrs+0.091 (+0.025 to +0.159) 19/24 yrs+0.077 (−0.030 to +0.182) 15/24 yrs+0.002 (−0.044 to +0.051) 9/24 yrs+0.002 (−0.013 to +0.016) 13/24 yrs
ONDJ R3 (Deyr) rainfall domain+0.133 (+0.032 to +0.245) 18/24 yrs+0.091 (+0.025 to +0.159) 19/24 yrs+0.077 (−0.030 to +0.182) 15/24 yrs+0.002 (−0.044 to +0.051) 9/24 yrs+0.002 (−0.013 to +0.016) 13/24 yrs

Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.