Seasonal Rainfall Outlook Platform
Interpretation · Bias correction · January initialization · FMAM and its months

Interpretation · January initialization · FMAM and its months

Interpretation — national aggregate (All Ethiopia)

  1. All tested corrections improve national FMAM rainfall-amount CRPSS over raw ECMWF in 1993–2016 (raw -0.696; corrected +0.009 to +0.029). Performance varies for monthly tercile probabilities: 0 method–month combinations score below the raw model on RPSS.
  2. Accounting for the whole search. With Holm adjustment across all 60 method–target comparisons, the amount (CRPS) screening passes are none, and the tercile (RPS) passes are Square-root-space Mean-Variance Scaling (Feb, Mar), Power Transformation (Feb, Mar), Empirical Quantile Mapping (Feb) and Gamma Quantile Mapping (Feb, Mar). Within-method passes that do not survive this check: Empirical Quantile Mapping Mar, Gamma Quantile Mapping Mar, Square-root-space Mean-Variance Scaling Feb, Square-root-space Mean-Variance Scaling Mar, Variance Scaling (unbounded) Feb and Variance Scaling (unbounded) Mar.
  3. Empirical Quantile Mapping reproduces the CHIRPS distribution and, with the dry policy, its dry share (totals ≤ 1 mm: Feb 13.0% vs CHIRPS 12.9%); it passes the screening for Feb and Mar, but its FMAM amount skill is lower than Mean-Variance Scaling (+0.011 vs +0.025). A possible explanation is sampling noise in an empirical quantile function built from 23 CHIRPS values per fold; the results do not isolate it.
  4. Scaled Distribution Mapping follows the published algorithm (Switanek et al. 2017, steps 1–7; checked numerically against an independent implementation of the steps). Where the model is drier than CHIRPS it cannot add rain: in Feb 51% of model members exceed 1 mm against 87% of CHIRPS years, and SDM, by design, keeps the model’s number of rain members, so its corrected dry share is 50% (CHIRPS 13%).
  5. Delta Change has a high FMAM CRPSS in 1993–2016 (+0.029) but the gain does not pass the screening (Holm p = 1.00), it is lower in 2017–2025 (+0.010 vs +0.016), its ensemble is too wide (spread–error 1.04) and its tercile skill is low. It is the CHIRPS climatology rescaled by the forecast’s mean: historical scenarios, not corrected members.
  6. The methods that use the forecast’s own distribution (Detrended Quantile Mapping, Quantile Delta Mapping, Scaled Distribution Mapping, CDF-transform) score lower on FMAM terciles (+0.019 to +0.024 vs +0.034) and none passes the season screening. A possible explanation is that the distribution of a single forecast’s 25–51 members is a noisy estimate of the “future” distribution these methods were designed for; the results do not isolate the cause. Where model quantiles are dry, QDM falls back to the CHIRPS quantile (members affected: Feb 49% and Mar 20%), so its quantile changes are only partly preserved.
  7. May remains hard: every method has negative amount skill over 2017–2025.
  8. No tested method demonstrates a statistically significant improvement over Mean-Variance Scaling for the published tercile probabilities under this validation protocol, although several have slightly higher point estimates. Choosing the method each year from training-only scores, the probability procedure (SEL-P) scores FMAM blended RPSS +0.035 against +0.032, confirmed for none. Amounts: the amount procedure (SEL-A) scores CRPSS +0.020 against +0.025, confirmed for Feb; fixed methods confirmed against the family of all declared methods and targets: none.
  9. The climatology blend narrows the differences. Across the methods, FMAM RPSS (1993–2016) spans 0.014 after the blend against 0.016 for the unblended screening counts. A better amount correction is trusted more by the blend: in Feb the fitted weight on climatology is 0.35 with Square-root-space Mean-Variance Scaling against 0.73 with Mean-Variance Scaling.
  10. Choosing by training scores can backfire. For FMAM, SEL-A chose Delta Change from the 1993–2016 scores; over 2017–2025 it has a worse mean CRPS (Δ +0.345 mm, 95% interval -0.929 to +1.631 mm; better than Mean-Variance Scaling in 5 of 9 years, with the largest losses in 2020 and 2021); for Feb, SEL-P chose CDF-transform from the 1993–2016 scores; over 2017–2025 it has a worse mean RPS (Δ +0.020, 95% interval -0.013 to +0.052; better than Mean-Variance Scaling in 3 of 9 years, with the largest losses in 2021 and 2018); for Apr, SEL-P chose Local Intensity Scaling from the 1993–2016 scores; over 2017–2025 it has a worse mean RPS (Δ +0.001, 95% interval -0.001 to +0.004; better than Mean-Variance Scaling in 5 of 9 years, with the largest losses in 2020 and 2021); for May, SEL-P chose Square-root-space Mean-Variance Scaling from the 1993–2016 scores; over 2017–2025 it has a worse mean RPS (Δ +0.007, 95% interval -0.004 to +0.020; better than Mean-Variance Scaling in 3 of 9 years, with the largest losses in 2021 and 2020); for FMAM, SEL-P chose Local Intensity Scaling from the 1993–2016 scores; over 2017–2025 it has a worse mean RPS (Δ +0.001, 95% interval -0.006 to +0.005; better than Mean-Variance Scaling in 2 of 9 years, with the largest losses in 2021 and 2018). These choices fail the no-harm rule; where the interval includes 0 they do not establish a significant deterioration. A method that ranks first on the training years is not, on that alone, a replacement.
  11. No cell of All Ethiopia has exactly 0 mm in every training year, for any target, at 0.25° with CHIRPS. The all-zero guard (a separate MVS variant, outside the declared family) therefore never acts and equals Mean-Variance Scaling: the non-zero totals that Mean-Variance Scaling gives where every training year is 0 mm (its spread ratio cannot fall below 0.5) do not arise in these data. The guard remains a documented option; a finite history of zero rainfall does not make rain impossible.

3 of the 14 interpretation statements do not hold for this cycle’s results and are not shown (each is checked against the numbers before it is displayed).

Caveats

  • Screening skill is from 1993–2016 leave-one-year-out (clean) and 2017–2025 (exploratory: these years were seen while the operational method was chosen).
  • Screening tercile scores use smoothed member counts without the climatology blend; the confirmation scores the blended product.
  • Two multiplicity families are shown: within one method across its targets (✓) and across all methods and targets (✓✓). The confirmation’s selection-aware procedures include the method choice in the evaluation itself.
  • Scores are area-weighted over many correlated cells but tested year by year, so each year counts once.
  • The ordinary CRPS scores the delivered ensemble; the fair CRPS is a diagnostic under a sampling interpretation that DC’s rescaled years and methods tied to the forecast mean or ranks only approximate.
  • Results are national aggregates unless a table says otherwise; a method can rank differently in a region (see the regional tables).

← SummaryData and methods →