Historical skill · FMAM 2026 · FMAM 2026 — February initialization · FMAM rainfall contribution domain

Historical skill · FMAM 2026

How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.

Takeaway — FMAM 2026

Historical skill is uncertain: cross-validated 1993–2016 RPSS is +0.017, and its interval (−0.038 to +0.067) includes no improvement over climatology. The exploratory 2017–2025 result (+0.061) is encouraging but those years were inspected during method selection. The 2026 verification (+0.006) is one season and does not override the historical uncertainty.

Stronger historical evidence (interval above zero): Feb +0.137.

FMAM 2026 (February initialization), FMAM rainfall contribution domain: probability skill (RPSS) of the final method against climatology, with whole-year 95% intervals
TargetCross-validated 1993–2016 (main)In wordsExploratory 2017–2025
FMAM+0.017 (−0.038 to +0.067)Small estimated improvement; skill uncertain+0.061 (+0.029 to +0.095)
Feb+0.137 (+0.050 to +0.234)Moderate improvement over climatology (interval above zero)+0.049 (−0.017 to +0.128)
Mar+0.011 (−0.020 to +0.042)Small estimated improvement; skill uncertain+0.041 (+0.007 to +0.080)
Apr+0.011 (−0.032 to +0.048)Small estimated improvement; skill uncertain+0.015 (−0.009 to +0.042)
May+0.007 (−0.051 to +0.050)Small estimated improvement; skill uncertain+0.017 (−0.033 to +0.057)

RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.

Technical detail: years better, p-values and blend weights
TargetYears better (cross-validated)p (one target / Holm)Years better (exploratory)Blend weight λ
FMAM15 of 240.276 / 1.0008 of 90.40
Feb16 of 240.002 / 0.0125 of 90.50
Mar15 of 240.251 / 1.0007 of 90.73
Apr16 of 240.305 / 1.0005 of 90.58
May17 of 240.434 / 1.0008 of 90.55

Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: Feb remains significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).

Historical verification explorer

The evidence behind the scores above, for FMAM 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.

Evaluation
Cross-validated (nested leave-one-year-out), 1993–2016
Benchmark
Climatology: observed tercile frequencies of the training years (about one-third each)
Area
FMAM rainfall contribution domain; probabilities on 100% of it on average
Sample
24 years, 26,880 cell-years. Cells within a year are spatially correlated, so the independent sample is closer to one value per year.

Performance by year

Probability skill (RPSS) of each year; better than climatology in 15 of 24 years. Weakest years: 1996 (−0.289), 2006 (−0.221), 2011 (−0.173).

Better than climatologyWorse
−0.30−0.20−0.100.00+0.10+0.20+0.301993: RPSS +0.197 (forecast RPS 0.361, climatology 0.450; 1120 cells)19931994: RPSS −0.051 (forecast RPS 0.435, climatology 0.414; 1120 cells)1995: RPSS +0.087 (forecast RPS 0.432, climatology 0.473; 1120 cells)1996: RPSS −0.289 (forecast RPS 0.626, climatology 0.486; 1120 cells)19961997: RPSS +0.115 (forecast RPS 0.380, climatology 0.429; 1120 cells)19971998: RPSS +0.029 (forecast RPS 0.422, climatology 0.435; 1120 cells)1999: RPSS −0.072 (forecast RPS 0.515, climatology 0.480; 1120 cells)2000: RPSS +0.199 (forecast RPS 0.392, climatology 0.489; 1120 cells)2001: RPSS +0.040 (forecast RPS 0.399, climatology 0.416; 1120 cells)20012002: RPSS +0.038 (forecast RPS 0.434, climatology 0.451; 1120 cells)2003: RPSS +0.031 (forecast RPS 0.397, climatology 0.409; 1120 cells)2004: RPSS −0.109 (forecast RPS 0.519, climatology 0.468; 1120 cells)2005: RPSS −0.030 (forecast RPS 0.473, climatology 0.460; 1120 cells)20052006: RPSS −0.221 (forecast RPS 0.513, climatology 0.420; 1120 cells)2007: RPSS +0.031 (forecast RPS 0.409, climatology 0.422; 1120 cells)2008: RPSS +0.130 (forecast RPS 0.448, climatology 0.514; 1120 cells)2009: RPSS +0.159 (forecast RPS 0.410, climatology 0.488; 1120 cells)20092010: RPSS +0.222 (forecast RPS 0.391, climatology 0.502; 1120 cells)20102011: RPSS −0.173 (forecast RPS 0.515, climatology 0.439; 1120 cells)2012: RPSS +0.050 (forecast RPS 0.408, climatology 0.429; 1120 cells)2013: RPSS −0.094 (forecast RPS 0.505, climatology 0.462; 1120 cells)20132014: RPSS +0.045 (forecast RPS 0.419, climatology 0.439; 1120 cells)2015: RPSS +0.025 (forecast RPS 0.443, climatology 0.454; 1120 cells)2016: RPSS −0.007 (forecast RPS 0.441, climatology 0.438; 1120 cells)

Reliability

below normalnear normalabove normal
0.00.00.20.20.40.40.60.60.80.81.01.0below normal: forecast 18%, observed 27% (613 cell-years)below normal: forecast 26%, observed 26% (8774 cell-years)below normal: forecast 35%, observed 38% (11145 cell-years)below normal: forecast 44%, observed 44% (5434 cell-years)below normal: forecast 53%, observed 33% (906 cell-years)below normal: forecast 61%, observed 37% (8 cell-years, too few to judge)near normal: forecast 18%, observed 21% (784 cell-years)near normal: forecast 26%, observed 28% (11765 cell-years)near normal: forecast 34%, observed 33% (12019 cell-years)near normal: forecast 43%, observed 39% (2235 cell-years)near normal: forecast 52%, observed 51% (74 cell-years, too few to judge)near normal: forecast 64%, observed 0% (3 cell-years, too few to judge)above normal: forecast 19%, observed 53% (432 cell-years)above normal: forecast 26%, observed 29% (8049 cell-years)above normal: forecast 35%, observed 29% (11932 cell-years)above normal: forecast 44%, observed 49% (5164 cell-years)above normal: forecast 54%, observed 50% (933 cell-years)above normal: forecast 62%, observed 53% (370 cell-years)Forecast probability

Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 134 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.

Reliability table (forecast bins, observed frequency, counts)
CategoryBinMean forecastObserved frequencyCell-years
below normal10%–20%18%27%613
below normal20%–30%26%26%8,774
below normal30%–40%35%38%11,145
below normal40%–50%44%44%5,434
below normal50%–60%53%33%906
below normal60%–70%61%37%8
near normal10%–20%18%21%784
near normal20%–30%26%28%11,765
near normal30%–40%34%33%12,019
near normal40%–50%43%39%2,235
near normal50%–60%52%51%74
near normal60%–70%64%0%3
above normal10%–20%19%53%432
above normal20%–30%26%29%8,049
above normal30%–40%35%29%11,932
above normal40%–50%44%49%5,164
above normal50%–60%54%50%933
above normal60%–70%62%53%370

How strong are the signals?

Share of cell-years by the leading tercile probability.

weak (<40%)
44%
moderate (40–50%)
48%
strong (50–60%)
7%
very strong (≥60%)
1%
Category Brier scores (lower is better) and Brier skill against climatology
CategoryForecastClimatologyBrier skill
below normal0.22310.2266+0.016
near normal0.21160.2128+0.006
above normal0.22220.2262+0.018

Rainfall amounts

Rainfall amounts, 2017–2025 (exploratory; parameters fitted on 1993–2016) — All Ethiopia (amount diagnostics are computed for the whole country)
Rainfall amount skill (CRPSS)+0.071
CRPS, corrected / climatology54.5 / 58.6 mm
Average rainfall error, corrected (raw model)−28.0 mm (−102.8 mm)
RMSE, corrected / climatology99.9 / 106.1 mm

Historical performance by rainfall region

Probability skill by rainfall region (full regional matrix)
FMAM 2026: probability skill (RPSS) by rainfall region, cross-validated 1993–2016, with whole-year 95% intervals and years better than climatology
RegionFMAMFebMarAprMay
All Ethiopia+0.021 (−0.031 to +0.069) 15/24 yrs+0.126 (+0.043 to +0.217) 18/24 yrs+0.011 (−0.017 to +0.038) 15/24 yrs+0.009 (−0.034 to +0.044) 15/24 yrs+0.008 (−0.042 to +0.047) 16/24 yrs
R0 arid / marginal+0.023 (−0.072 to +0.103) 15/24 yrs+0.057 (−0.081 to +0.206) 13/24 yrs−0.005 (−0.058 to +0.044) 15/24 yrs−0.028 (−0.102 to +0.038) 13/24 yrs−0.015 (−0.079 to +0.040) 13/24 yrs
R1 western unimodal+0.019 (−0.035 to +0.067) 18/24 yrs+0.156 (+0.068 to +0.248) 18/24 yrs+0.009 (−0.018 to +0.034) 15/24 yrs+0.016 (−0.024 to +0.047) 17/24 yrs+0.009 (−0.033 to +0.046) 15/24 yrs
R2 Belg–Kiremt+0.037 (−0.050 to +0.108) 17/24 yrs+0.166 (+0.076 to +0.264) 20/24 yrs+0.023 (−0.017 to +0.058) 16/24 yrs−0.001 (−0.063 to +0.052) 14/24 yrs+0.009 (−0.051 to +0.059) 16/24 yrs
R3 Gu–Deyr+0.012 (−0.042 to +0.064) 15/24 yrs+0.082 (−0.006 to +0.180) 14/24 yrs+0.006 (−0.030 to +0.041) 14/24 yrs+0.017 (−0.034 to +0.064) 16/24 yrs+0.009 (−0.066 to +0.062) 19/24 yrs
FMAM rainfall contribution domain+0.017 (−0.038 to +0.067) 15/24 yrs+0.137 (+0.050 to +0.234) 16/24 yrs+0.011 (−0.020 to +0.042) 15/24 yrs+0.011 (−0.032 to +0.048) 16/24 yrs+0.007 (−0.051 to +0.050) 17/24 yrs
FMAM main-season rainfall domain (Belg / Gu / Ganna, >=40% of annual)+0.013 (−0.040 to +0.066) 16/24 yrs+0.080 (−0.005 to +0.176) 14/24 yrs+0.007 (−0.029 to +0.042) 15/24 yrs+0.013 (−0.041 to +0.060) 17/24 yrs+0.009 (−0.066 to +0.062) 18/24 yrs

Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.