Historical skill · FMAM 2026 · FMAM 2026 — January initialization · FMAM rainfall contribution domain

Historical skill · FMAM 2026

How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.

Takeaway — FMAM 2026

Historical skill is uncertain: cross-validated 1993–2016 RPSS is +0.036, and its interval (−0.003 to +0.071) includes no improvement over climatology. The exploratory 2017–2025 result (+0.014) is encouraging but those years were inspected during method selection. The 2026 verification (−0.054) is one season and does not override the historical uncertainty.

Stronger historical evidence (interval above zero): May +0.035.

FMAM 2026 (January initialization), FMAM rainfall contribution domain: probability skill (RPSS) of the final method against climatology, with whole-year 95% intervals
TargetCross-validated 1993–2016 (main)In wordsExploratory 2017–2025
FMAM+0.036 (−0.003 to +0.071)Small estimated improvement; skill uncertain+0.014 (−0.048 to +0.080)
Feb+0.026 (−0.006 to +0.059)Small estimated improvement; skill uncertain+0.025 (−0.003 to +0.057)
Mar+0.001 (−0.012 to +0.012)Small estimated improvement; skill uncertain+0.011 (−0.004 to +0.024)
Apr−0.002 (−0.020 to +0.013)No demonstrated improvement over climatology+0.001 (−0.027 to +0.028)
May+0.035 (+0.009 to +0.060)Modest improvement over climatology (interval above zero)−0.006 (−0.089 to +0.051)

RPSS +0.05 means a 5% lower ranked probability score than climatology. An interval that includes zero means the gain is not established.

Technical detail: years better, p-values and blend weights
TargetYears better (cross-validated)p (one target / Holm)Years better (exploratory)Blend weight λ
FMAM18 of 240.043 / 0.1713 of 90.34
Feb16 of 240.067 / 0.2027 of 90.76
Mar12 of 240.451 / 0.9015 of 90.86
Apr12 of 240.564 / 0.9015 of 90.76
May19 of 240.009 / 0.0446 of 90.59

Each interval and one-target p-value describes that target alone. Because 5 targets are examined, the Holm-adjusted p is the stricter test: May remains significant after adjustment. λ is the weight given to climatology in the blend (higher = closer to climatology).

Historical verification explorer

The evidence behind the scores above, for FMAM 2026 in this area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.

Evaluation
Cross-validated (nested leave-one-year-out), 1993–2016
Benchmark
Climatology: observed tercile frequencies of the training years (about one-third each)
Area
FMAM rainfall contribution domain; probabilities on 100% of it on average
Sample
24 years, 26,880 cell-years. Cells within a year are spatially correlated, so the independent sample is closer to one value per year.

Performance by year

Probability skill (RPSS) of each year; better than climatology in 18 of 24 years. Weakest years: 2006 (−0.308), 2003 (−0.070), 2001 (−0.063).

Better than climatologyWorse
−0.40−0.200.00+0.20+0.401993: RPSS +0.149 (forecast RPS 0.383, climatology 0.450; 1120 cells)19931994: RPSS +0.017 (forecast RPS 0.407, climatology 0.414; 1120 cells)1995: RPSS +0.153 (forecast RPS 0.401, climatology 0.473; 1120 cells)1996: RPSS +0.074 (forecast RPS 0.450, climatology 0.486; 1120 cells)1997: RPSS +0.096 (forecast RPS 0.388, climatology 0.429; 1120 cells)19971998: RPSS +0.053 (forecast RPS 0.412, climatology 0.435; 1120 cells)1999: RPSS +0.114 (forecast RPS 0.425, climatology 0.480; 1120 cells)2000: RPSS −0.015 (forecast RPS 0.496, climatology 0.489; 1120 cells)2001: RPSS −0.063 (forecast RPS 0.442, climatology 0.416; 1120 cells)20012002: RPSS +0.004 (forecast RPS 0.450, climatology 0.451; 1120 cells)2003: RPSS −0.070 (forecast RPS 0.438, climatology 0.409; 1120 cells)2004: RPSS +0.083 (forecast RPS 0.429, climatology 0.468; 1120 cells)2005: RPSS +0.045 (forecast RPS 0.439, climatology 0.460; 1120 cells)20052006: RPSS −0.308 (forecast RPS 0.549, climatology 0.420; 1120 cells)20062007: RPSS +0.034 (forecast RPS 0.408, climatology 0.422; 1120 cells)2008: RPSS +0.072 (forecast RPS 0.477, climatology 0.514; 1120 cells)2009: RPSS +0.006 (forecast RPS 0.485, climatology 0.488; 1120 cells)20092010: RPSS +0.203 (forecast RPS 0.400, climatology 0.502; 1120 cells)20102011: RPSS −0.012 (forecast RPS 0.444, climatology 0.439; 1120 cells)2012: RPSS +0.098 (forecast RPS 0.387, climatology 0.429; 1120 cells)2013: RPSS −0.016 (forecast RPS 0.469, climatology 0.462; 1120 cells)20132014: RPSS +0.014 (forecast RPS 0.433, climatology 0.439; 1120 cells)2015: RPSS +0.021 (forecast RPS 0.444, climatology 0.454; 1120 cells)2016: RPSS +0.033 (forecast RPS 0.423, climatology 0.438; 1120 cells)

Reliability

below normalnear normalabove normal
0.00.00.20.20.40.40.60.60.80.81.01.0below normal: forecast 9%, observed 24% (21 cell-years, too few to judge)below normal: forecast 18%, observed 21% (1478 cell-years)below normal: forecast 26%, observed 26% (7352 cell-years)below normal: forecast 35%, observed 37% (12048 cell-years)below normal: forecast 44%, observed 44% (5017 cell-years)below normal: forecast 54%, observed 43% (842 cell-years)below normal: forecast 63%, observed 39% (120 cell-years, too few to judge)below normal: forecast 71%, observed 100% (2 cell-years, too few to judge)near normal: forecast 9%, observed 21% (43 cell-years, too few to judge)near normal: forecast 16%, observed 23% (1896 cell-years)near normal: forecast 25%, observed 27% (9970 cell-years)near normal: forecast 35%, observed 33% (10263 cell-years)near normal: forecast 44%, observed 38% (3999 cell-years)near normal: forecast 53%, observed 36% (659 cell-years)near normal: forecast 63%, observed 38% (50 cell-years, too few to judge)above normal: forecast 17%, observed 26% (1328 cell-years)above normal: forecast 26%, observed 22% (7252 cell-years)above normal: forecast 34%, observed 34% (11130 cell-years)above normal: forecast 44%, observed 49% (5586 cell-years)above normal: forecast 54%, observed 57% (1380 cell-years)above normal: forecast 64%, observed 46% (190 cell-years)above normal: forecast 72%, observed 50% (14 cell-years, too few to judge)Forecast probability

Points on the dashed diagonal are reliable: events forecast with probability p happen p of the time. Points flatter than the diagonal mean over-confident probabilities. Open points have fewer than 134 cell-years (0.5% of the sample, at least 50) and are not joined. Hover a point for its count.

Reliability table (forecast bins, observed frequency, counts)
CategoryBinMean forecastObserved frequencyCell-years
below normal0%–10%9%24%21
below normal10%–20%18%21%1,478
below normal20%–30%26%26%7,352
below normal30%–40%35%37%12,048
below normal40%–50%44%44%5,017
below normal50%–60%54%43%842
below normal60%–70%63%39%120
below normal70%–80%71%100%2
near normal0%–10%9%21%43
near normal10%–20%16%23%1,896
near normal20%–30%25%27%9,970
near normal30%–40%35%33%10,263
near normal40%–50%44%38%3,999
near normal50%–60%53%36%659
near normal60%–70%63%38%50
above normal10%–20%17%26%1,328
above normal20%–30%26%22%7,252
above normal30%–40%34%34%11,130
above normal40%–50%44%49%5,586
above normal50%–60%54%57%1,380
above normal60%–70%64%46%190
above normal70%–80%72%50%14

How strong are the signals?

Share of cell-years by the leading tercile probability.

weak (<40%)
35%
moderate (40–50%)
53%
strong (50–60%)
11%
very strong (≥60%)
1%
Category Brier scores (lower is better) and Brier skill against climatology
CategoryForecastClimatologyBrier skill
below normal0.22060.2266+0.027
near normal0.21280.21280.000
above normal0.21610.2262+0.045

Rainfall amounts

Rainfall amounts, 2017–2025 (exploratory; parameters fitted on 1993–2016) — All Ethiopia (amount diagnostics are computed for the whole country)
Rainfall amount skill (CRPSS)+0.016
CRPS, corrected / climatology57.7 / 58.6 mm
Average rainfall error, corrected (raw model)−27.1 mm (−113.4 mm)
RMSE, corrected / climatology104.5 / 106.1 mm

Historical performance by rainfall region

Probability skill by rainfall region (full regional matrix)
FMAM 2026: probability skill (RPSS) by rainfall region, cross-validated 1993–2016, with whole-year 95% intervals and years better than climatology
RegionFMAMFebMarAprMay
All Ethiopia+0.032 (−0.003 to +0.067) 18/24 yrs+0.027 (−0.002 to +0.055) 16/24 yrs+0.002 (−0.009 to +0.012) 13/24 yrs+0.001 (−0.014 to +0.014) 15/24 yrs+0.028 (+0.005 to +0.051) 17/24 yrs
R0 arid / marginal+0.051 (−0.012 to +0.113) 14/24 yrs+0.026 (−0.023 to +0.076) 14/24 yrs+0.002 (−0.024 to +0.025) 15/24 yrs−0.019 (−0.050 to +0.007) 11/24 yrs+0.033 (0.000 to +0.066) 15/24 yrs
R1 western unimodal−0.003 (−0.053 to +0.043) 13/24 yrs+0.040 (+0.016 to +0.062) 18/24 yrs+0.003 (−0.008 to +0.013) 14/24 yrs+0.013 (−0.004 to +0.030) 15/24 yrs−0.014 (−0.041 to +0.014) 10/24 yrs
R2 Belg–Kiremt+0.061 (+0.011 to +0.113) 17/24 yrs+0.039 (+0.010 to +0.066) 18/24 yrs+0.008 (−0.001 to +0.017) 16/24 yrs−0.008 (−0.036 to +0.014) 14/24 yrs+0.045 (+0.014 to +0.075) 19/24 yrs
R3 Gu–Deyr+0.036 (−0.012 to +0.078) 18/24 yrs+0.008 (−0.032 to +0.048) 13/24 yrs−0.003 (−0.022 to +0.015) 10/24 yrs+0.002 (−0.018 to +0.020) 14/24 yrs+0.047 (+0.004 to +0.088) 19/24 yrs
FMAM rainfall contribution domain+0.036 (−0.003 to +0.071) 18/24 yrs+0.026 (−0.006 to +0.059) 16/24 yrs+0.001 (−0.012 to +0.012) 12/24 yrs−0.002 (−0.020 to +0.013) 12/24 yrs+0.035 (+0.009 to +0.060) 19/24 yrs
FMAM main-season rainfall domain (Belg / Gu / Ganna, >=40% of annual)+0.041 (−0.008 to +0.083) 17/24 yrs+0.006 (−0.034 to +0.047) 13/24 yrs−0.002 (−0.021 to +0.014) 10/24 yrs+0.001 (−0.021 to +0.020) 13/24 yrs+0.050 (+0.007 to +0.092) 20/24 yrs

Positive values mean lower RPS than climatology. Intervals resample whole years; regions are the fixed 1993–2025 rainfall regimes, and the last rows are this cycle's rainfall domains. Regional results are noisier than national ones.