Research

Which methods are better

29 methods have been compared with the approach the platform publishes, for 4 seasons and their months (20 targets), over All Ethiopia: 12 bias corrections, 8 probability calibrations, 6 rainfall-distribution models, 1 two-stage calibration and 2 statistical downscaling methods. Each was tested on its own against the published approach. So far no method is recommended in place of the published approach; the published forecasts keep it.

How a method is judged

1. Confirmed better (historical evidence): better over 1993–2016 when each year is forecast from the other years, significant after correcting for the number of methods and targets tested (Benjamini–Hochberg-adjusted p < 0.05, which controls the share of false findings; Holm until 17 October 2026), and not worse over the test years from 2017 on. A confirmed method becomes a candidate.
2. Validation: a candidate is then scored on independent seasons as they are observed. It is recommended after 3 seasons that are not worse than the published approach, and not supported if worse.
3. One change at a time: each method was tested against the published approach, not combined with another; the published forecasts change only after a recommendation and a decision.

Which method to use in each season

A candidate is confirmed on the historical evidence and waiting for validation seasons. The evidence for each target is on Method selection.

Findings

• Tercile probabilities. Confirmed better for at least one target: Square-root-space Mean-Variance Scaling (FMAM (Feb) Feb). No probability calibration, rainfall-distribution model, two-stage calibration or statistical downscaling method is confirmed better for any target. Most consistent: Square-root-space Mean-Variance Scaling, better on average for 14 of 20 targets over 1993–2016 and 16 of 20 over 2017+.
• Rainfall amounts. Confirmed better for at least one target: Square-root-space Mean-Variance Scaling (5), Gamma Quantile Mapping (4), Variance Scaling (unbounded) (4), Principal component regression (PCR) (3), EMOS, censored shifted gamma (2), Power Transformation (2), Regularized quantile regression (1), Leung–Dirichlet calibration (1), Canonical correlation analysis (CCA) (1). Most consistent: Square-root-space Mean-Variance Scaling, better on average for 20 of 20 targets over 1993–2016 and 18 of 20 over 2017+.

Comparison: tercile probabilities

Against the published MVS + climatology blend (RPS). Counts are targets; “Better in 2026” counts the targets of the independent 2026 seasons observed so far.

Comparison: rainfall amounts

Against the published MVS-corrected ensemble (CRPS). Counts are targets; “Better in 2026” counts the targets of the independent 2026 seasons observed so far.

More statistical power

The same out-of-sample scores, no refitting, read three ways that give the tests more power:

1. 33 years: each target's 1993–2016 leave-one-year-out years joined with its test years (2017 onward, fitted on 1993–2016), so every year 1993–2025 counts, scored by a model that did not see it.
2. Pooled: for each method and year, its skill difference from the published approach averaged over all its targets of all seasons (as a share of the climatology score, so targets of different rainfall add up); one test over the years.
3. Two corrections for testing many methods: Holm (controls the chance of any false finding) and Benjamini–Hochberg (controls the share of false findings; less conservative).

The selection rule above applies Benjamini–Hochberg within each benchmark family (it used Holm until 17 October 2026; changed by decision after these analyses were seen). The pooled and 33-year tests below are sensitivity analyses, not part of the rule.

Tercile probabilities

Significantly better than the published approach when pooled: Holm none; Benjamini–Hochberg none.

Rainfall amounts

Significantly better than the published approach when pooled: Holm Regularized quantile regression, Square-root-space Mean-Variance Scaling, Gamma Quantile Mapping, Variance Scaling (unbounded); Benjamini–Hochberg Regularized quantile regression, Square-root-space Mean-Variance Scaling, EMOS, censored shifted gamma, Gamma Quantile Mapping, Variance Scaling (unbounded).

Pooled skill gain: mean over years of the method's improvement over the published approach, as a share of the climatology score (+1% = 1% of the climatology score better). Years better: years where the pooled difference favours the method.

Research pages

• Method selection
  The approach the evidence supports for each season, target and product, with the evidence for each change.
• Bias correction
  Thirteen rainfall-amount corrections against the operational Mean-Variance Scaling.
• Probability calibration
  Probability-stage methods and rainfall-distribution models (Phases 1–3) against the published probabilities.
• Amount and range shadow
  EMOS medians and 10–90% ranges next to the MVS ensemble, scored season by season.
• Statistical downscaling
  Model output statistics (CCA, PCR), the CPT approach of ICPAC and IRI, against the published pipeline.
• Leung–Dirichlet calibration
  Member-wise Leung correction and identity-anchored Dirichlet, each stage against the published pipeline.