Archived release 2026-10-14.2 (2026-10-14): the page as published then (code c99305a). Current release · Change log
Ethiopia Seasonal Rainfall
Research reconstruction · not an official EMI or ICPAC forecast

Seasonal rainfall outlook

Research reconstruction · not an official forecast

Observations: CHIRPS v3.0 since 11 October 2026. Every cycle is calibrated and verified with CHIRPS v3.0. The 2026 forecasts shown here were recalibrated with v3.0 after their outcomes were known, so they are retrospective reconstructions, not forecasts issued in advance. The versions calibrated with CHIRPS v2.0 are in the release archive (release 2026-10-10.2); the v2.0 vs v3.0 comparison is in the Methods section.

Average local probability

Rainfall amount

Forecast signal

Historical skill

Signal strength describes how far this forecast's probabilities depart from climatology. Historical skill describes how forecasts made the same way scored in past years. A strong signal is not a confidence rating.

All targets in this cycle

Official outlook comparison

Cycle status and record

Maps

Open full-size Download PNG

How to read this map

Grey, white and blank areas

Light grey: outside the selected area. Dark grey: inside the area but without values, for example cells where reference-period rainfall is too small for tercile categories, or (percent maps) below 10 mm. White on tercile maps: no category reaches 40%, or two categories tie. Sea cells have no CHIRPS data.

Smoothing and resolution. Maps are drawn on a 12× finer display grid with light Gaussian smoothing (σ = 0.6 grid cells); all numbers on this page use the original 0.25° grid. The model's own resolution is about 1°: each model cell covers 4×4 grid cells, so detail within a 1° box comes from local CHIRPS rainfall statistics used in the amount correction, not from the model.

Verification

Frozen forecasts compared with official CHIRPS v3.0 observations once a target period is complete. Each result is one season or month, so it shows what happened this time, not long-term reliability.

Historical performance

How forecasts made with the same method scored in past years against climatology (the observed frequency of each tercile in the training years, about one-third each). The main number is the cross-validated 1993–2016 score: each year is forecast with parameters fitted without that year. Later years are exploratory: they were seen while the method was chosen.

Historical verification explorer

The evidence behind the scores above, for the selected cycle, target and area. Choose the evaluation period; all diagnostics use the blended probabilities against climatology.

Historical performance by rainfall region

Bias correction

How the model’s rainfall amounts are corrected toward CHIRPS, and how alternative bias-correction methods compare with the operational one: the data, every method’s definition, the steps, the screening and confirmation results, and what they mean.

Each cycle has its own benchmark (initialization, season and months).

September initialization · ONDJ and its months

Research benchmark. The published forecasts still use Mean-Variance Scaling (operational). This section compares 12 other rainfall-amount bias-correction methods with it, member by member, for every grid cell and target: a screening of the amount correction alone, then a confirmation through the full probability pipeline that also accounts for how a method is chosen.

Suggested method

The platform suggests a bias-correction method for every target and product from the historical evidence (the confirmation against CHIRPS, counting every method tried) and the independent validation seasons scored since (the shadow and validation runs), by a rule fixed before those seasons are scored. A suggestion is advice for a decision: the published forecast always uses Mean-Variance Scaling until a change is decided. 1 of 10 target–product combinations have a candidate.

The rule
  • Candidate: A method is a candidate only if the confirmation (CHIRPS, outer leave-one-year-out over the training years, full pipeline) finds it better than the operational method with Holm-adjusted p < 0.05 across every declared method and target, and not worse over the test years. Variants outside the declared family are not candidates.
  • Choice: Among candidates, the method the selection-aware procedure chooses from all training years (training-only information); if that method is not a candidate, the candidate with the best training-period score. Other candidates are listed as alternatives: a confirmation establishes improvement against the operational method, not a ranking among candidates.
  • Validation: Independent validation seasons are forecast years scored after the benchmark (bias_validation.py). A candidate is 'recommended' once at least minimum_seasons have been scored for it and the operational method, and its mean score over them is not worse; with fewer seasons it stays a 'candidate'; with enough seasons and a worse mean it is 'not supported by validation'. Minimum: 3 seasons.
Suggested methods (national aggregate)
TargetProductSuggested methodStatusEvidence
OctPublished tercile probabilitiesMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Oct on the historical evidence.
OctRainfall amountsMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Oct on the historical evidence.
NovPublished tercile probabilitiesMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Nov on the historical evidence.
NovRainfall amountsMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Nov on the historical evidence.
DecPublished tercile probabilitiesMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Dec on the historical evidence.
DecRainfall amountsMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Dec on the historical evidence.
JanPublished tercile probabilitiesMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for Jan on the historical evidence.
JanRainfall amountsSquare-root-space Mean-Variance ScalingCandidate — validation 0 of 3 seasonsSquare-root-space Mean-Variance Scaling is confirmed better than Mean-Variance Scaling for Jan (CRPS (mm) difference -0.4787 over the training years, Holm p 0.0209; -0.6902 over the test years). Independent validation: 0 of 3 season(s). Also confirmed: Gamma Quantile Mapping.
ONDJPublished tercile probabilitiesMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for ONDJ on the historical evidence.
ONDJRainfall amountsMean-Variance ScalingKeep the operational methodNo method is confirmed better than Mean-Variance Scaling for ONDJ on the historical evidence.

Summary — national aggregate (All Ethiopia)

  • All tested corrections improve national ONDJ rainfall-amount CRPSS over raw ECMWF in 1993–2016 (raw -0.180; corrected +0.053 to +0.093). Performance varies for monthly tercile probabilities: 3 method–month combinations score below the raw model on RPSS (Detrended Quantile Mapping in Dec, Scaled Distribution Mapping in Dec and CDF-transform in Dec).
  • Accounting for the whole search. With Holm adjustment across all 60 method–target comparisons, the amount (CRPS) screening passes are Square-root-space Mean-Variance Scaling (Jan) and Gamma Quantile Mapping (Jan), and the tercile (RPS) passes are Square-root-space Mean-Variance Scaling (Nov, Jan), Power Transformation (Jan) and Gamma Quantile Mapping (Jan). Within-method passes that do not survive this check: Empirical Quantile Mapping Jan, Gamma Quantile Mapping ONDJ, Square-root-space Mean-Variance Scaling Dec, Square-root-space Mean-Variance Scaling ONDJ and Variance Scaling (unbounded) Jan.

What the evidence supports

  • Published tercile probabilities: no tested method demonstrates a statistically significant improvement over Mean-Variance Scaling under this validation protocol (several have slightly higher point estimates); the selection-aware procedure is confirmed for none.
  • Rainfall amounts: confirmed against Mean-Variance Scaling (family-wide, full pipeline): Square-root-space Mean-Variance Scaling for Jan; Gamma Quantile Mapping for Jan. The selection-aware procedure is confirmed for Jan. These comparisons establish improvement against the reference, not that one candidate is better than another.
  • The operational method stays Mean-Variance Scaling until a change is decided by initialization, target and product, and recorded in the docs, on this evidence; confirmed improvements limited to some months may justify a per-target choice.
  • If amounts and probabilities come to use different correction paths, the product metadata must record both methods so that every output can be traced.
  • The same screening and confirmation run for the other cycles with one configuration file each (config/bias_correction/).

Data

Data used by the benchmark
ItemDetails
ForecastsECMWF SEAS5 (system 51), initialized on 1 September; 25 members 1993–2016, 51 from 2017; remapped from 1° to the 0.25° CHIRPS grid by conservative remapping (each 1° cell fills a 4 × 4 block). Johnson et al. 2019
ObservationsCHIRPS v3.0 monthly totals, averaged from 0.05° to 0.25°
TargetsOct, Nov, Dec, Jan and the ONDJ season; each corrected separately
ValuesOne total per member, grid cell and year (mm), the same inputs as the operational tool (data/processed/init09_<target>/)
Observed years1993–2024 (seasons starting in these years)
Training / test years1993–2016 (clean evidence) / 2017–2024 (exploratory)
Cells1,484 grid cells of Ethiopia (0.25°); a cell is corrected only if CHIRPS is complete in every training year
Regions scoredAll Ethiopia (1,484 cells) — primary (national aggregate); ONDJ R3 (Deyr) domain (575 cells)
Score coverage (amounts / probabilities, share of the national area)Oct 100% / 100%; Nov 100% / 100%; Dec 100% / 93%; Jan 100% / 77%; ONDJ 100% / 100%. Tercile scores need two distinct tercile boundaries defining three categories, a CHIRPS standard deviation of at least 1 mm and a lower boundary above 0 mm, so dry months have fewer probability cells.
Dry totals≤ 1 mm in a month or season (not a dry day); exactly 0 mm is reported separately
WeightingEvery member is used; each training year has total weight 1/Y (each member of year y weighs 1/(Y·My)), so years with 25 and 51 members count equally. Each observed year weighs 1/Y.
Runconfig/bias_correction/sep_ondj.json; 2026-10-11 20:38 UTC; commit bed63b906c; scoring version 3

Methods

Each method is fitted per grid cell and target from the training years only and applied to each member of the scored year. Methods that keep a “change signal” (DC, DQM, QDM, SDM, CDF-t) were designed for climate change, where a model’s reference period is compared with a future period; here the “future” is the forecast itself: the target year’s ensemble against the model climatology.

How overlapping names were resolved
  • Delta Change appeared twice in the requested list; it is one method.
  • Mean–Variance Scaling is the operational method, with the project’s 0.5–2 ratio limits. Variance Scaling uses the same algebra and the same fallback (ratio 1 where the model spread is below 1 mm) without the limits, so the comparison isolates the limits.
  • Delta Change returns the CHIRPS training years rescaled by the forecast signal (ensemble mean / model climatology): historical scenarios, not corrected members. Applied to each member instead it would be identical to Linear Scaling.
  • Square-root-space MVS (a mean–variance scaling of √x) is distinct from Power Transformation (a·xb with b fitted to the coefficient of variation).
  • The raw model is scored as a baseline, not screened.
Dry totals and tied values

A total at or below 1 mm in the month or season is “dry” (not a dry day). For the quantile-mapping methods (EQM, Gamma QM, DQM, QDM) the dry shares of the model climatology and of CHIRPS are estimated with the equal-year weights; a forecast’s dry members are spread evenly over the model’s dry probability by rank (deterministic stratified tie-breaking, with ties resolved by member index), and a probability within the CHIRPS dry share maps to 0 mm, also after any extrapolation of the upper tail. Without this, tied zeros share one probability and the corrected dry share does not follow CHIRPS. Tests cover repeated zeros, all-dry samples and very few wet values; the dry-month table under Results compares the shares. CDF-t, LOCI and the scaling methods have no occurrence adjustment; SDM uses its own rain-member rule (Switanek et al. 2017).

12 methods, the operational reference and the raw baseline
MethodFamilyDefinition (per cell and target)Member-wiseUses the forecast ensemble’s own distributionParametersReferences
Raw ECMWF (no correction)
raw		baselinex' = x (the uncorrected model; baseline, not a candidate)yesno—Johnson et al. 2019
Linear Scaling
ls		scalingx' = x * mu_obs / mu_model (multiplicative mean correction; ratio 1 where mu_model < 0.1 mm)yesno{"min_model_mean": 0.1}Lenderink et al. 2007, Teutschbein & Seibert 2012
Mean-Variance Scaling (operational)
mvs		scalingx' = max(0, mu_obs + r (x - mu_model)), r = clip(sd_obs/sd_model, 0.5, 2); r = 1 if sd_model < 1 mmyesno{"ratio_bounds": [0.5, 2.0], "min_model_sd": 1.0}Torralba et al. 2017, Manzanas et al. 2019
Variance Scaling (unbounded)
vs		scalingSame algebra and the same r = 1 fallback (sd_model < 1 mm) as MVS, without the 0.5-2 ratio limitsyesno{"ratio_bounds": null, "min_model_sd": 1.0}Teutschbein & Seibert 2012
Square-root-space Mean-Variance Scaling
sqrt_mvs		scalingMVS applied to sqrt(x) (r limits 0.5-2; r = 1 if the sqrt-space sd < 0.1), clipped at 0 and squared back (docs/36)yesno{"ratio_bounds": [0.5, 2.0], "min_model_sd": 0.1, "space": "sqrt"}Manzanas et al. 2019
Power Transformation
pt		scalingx' = a x^b: b (0.1-3) matches the coefficient of variation where reachable (bound hits reported), a the meanyesno{"b_bounds": [0.1, 3.0]}Leander & Buishand 2007, Teutschbein & Seibert 2012
Delta Change
dc		change-preservingTraining-year CHIRPS totals times a = forecast ensemble mean / model mean (a in 0.2-5): historical scenarios, not corrected membersno (one value per training year)yes{"ratio_bounds": [0.2, 5.0], "min_model_mean": 0.1}Hay et al. 2000, Teutschbein & Seibert 2012
Local Intensity Scaling
loci		scalingModel threshold with the observed frequency of totals above 1 mm, then intensity scaling above it (designed for daily rainfall; here monthly or seasonal totals)yesno{"wet_threshold": 1.0}Schmidli et al. 2006
Empirical Quantile Mapping
eqm		quantile mappingx' = Q_obs(F_model(x)) with the dry policy; constant-ratio extrapolation outside the training rangeyesno{"extrapolation": "constant_ratio", "wet_threshold": 1.0}Boé et al. 2007, Themeßl et al. 2011, Gudmundsson et al. 2012
Gamma Quantile Mapping
gqm		quantile mappingx' = G_obs^-1(G_model(x)), G = dry mass (<= 1 mm) + gamma by weighted maximum likelihood, with the dry policy; linear scaling where fewer than 3 wet valuesyesno{"wet_threshold": 1.0}Piani et al. 2010
Detrended Quantile Mapping
dqm		change-preservinga * EQM(x / a), a = forecast ensemble mean / model mean (0.2-5): removes and reapplies the bounded ratio; exact preservation of the final mean anomaly is not guaranteedyesyes{"extrapolation": "constant_ratio", "ratio_bounds": [0.2, 5.0], "min_model_mean": 0.1, "wet_threshold": 1.0}Cannon et al. 2015
Quantile Delta Mapping
qdm		change-preservingx' = Q_obs(tau) * x / Q_model(tau), tau = rank of x in the forecast ensemble; change capped at 5 and replaced by Q_obs(tau) where Q_model(tau) <= 1 mm (both reported), so preservation is partial thereyesyes{"max_ratio": 5.0, "wet_threshold": 1.0}Cannon et al. 2015
Scaled Distribution Mapping
sdm		change-preservingSwitanek et al. (2017) precipitation steps 1-7: gamma fits, recurrence intervals interpolated to the forecast's length, scaled CDF, rain-member count; model climatology with equal-year weightsyesyes{"wet_threshold": 1.0}Switanek et al. 2017
CDF-transform
cdft		change-preservingF_obs,f = F_obs(F_model^-1(F_forecast)); x' = F_obs,f^-1(F_forecast(x)); implementation choices: model and forecast first scaled to the CHIRPS mean, values stay within the scaled forecast rangeyesyes{"min_model_mean": 0.1}Michelangeli et al. 2009, Vrac et al. 2012
MVS with all-zero guard (variant)
mvs_zero_guard		variantOperational MVS, but corrected totals are 0 mm where every training year has exactly 0 mm (a documented modelling choice, not a physical rule)yesno{"ratio_bounds": [0.5, 2.0], "min_model_sd": 1.0, "zero_guard": true}Torralba et al. 2017, Manzanas et al. 2019

Steps

  1. 1

    Load

    Per-member totals of every target and year, and CHIRPS for 1993–2024.

    bias_benchmark.py · run_calibration.load_inputs
  2. 2

    Fit

    For each cell and target, the method’s statistics from the training years only, with equal-year weights.

    bias_methods.METHODS[key].fit
  3. 3

    Apply

    Correct every member of the scored year individually (member-wise).

    bias_methods.METHODS[key].apply
  4. 4

    Score

    Ensemble CRPS (and fair CRPS), ensemble-mean errors, spread–error, rank histograms, tercile RPS, dry shares and distribution ratios, per cell; then area-weighted per year.

    bias_benchmark.score_cells · summarise
  5. 5

    Screen

    Each method against Mean-Variance Scaling per target: one-sided sign-flip test on whole years, Holm within the method and across all methods, no harm in 2017–2024.

    bias_benchmark.make_gates
  6. 6

    Confirm

    Every method through the full probability pipeline (blend weight fitted inside the folds), and selection-aware procedures that choose the method each year from training-only scores.

    bias_confirmation.py
Evaluation designs (no amount method has a parameter tuned on scores)
DesignFitsScoredRole
Screening: trainingleave-one-year-out within 1993–2016each 1993–2016 yearclean evidence for the screening
Screening: test1993–20162017–2024exploratory (these years were seen before)
Screening: fullleave-one-year-out within 1993–2024each yearper-cell maps and distribution checks (descriptive)
Confirmationouter leave-one-year-out within 1993–2016; inside each fold the blend weight (inner leave-one-year-out) and, for SEL-P/SEL-A, the method choiceeach 1993–2016 yearfull pipeline; accounts for the method choice
Confirmation: test1993–2016 (fits, blend weight and method choice)2017–2024exploratory
Forecast1993–20242026 membersmember-by-member comparison
Scores
ScoreWhat it measuresReference
CRPS / CRPSSError of the corrected ensemble’s empirical distribution, as delivered, in mm; skill against the CHIRPS training climatology (the training years as an ensemble). Primary amount score.
Fair CRPS / CRPSSThe same under the interpretation that the members are a sample of an underlying distribution; reported as a diagnostic and screened separately. The justification assumes exchangeable members, which DC’s rescaled years and methods tied to the forecast mean or ranks only approximate.Ferro 2014
Ensemble meanBias, MAE, RMSE and correlation over years
Spread–error ratio√(mean ensemble variance·(K+1)/K) / RMSE of the ensemble mean; 1 = spread matches the error, < 1 too narrow
Rank histogram, reliability indexPosition of CHIRPS within the ensemble (10 bins; each rank interval shared among the bins it overlaps); index Σ|fi − 1/10|, 0 = flatDelle Monache et al. 2006
Tercile RPS / RPSS (screening)α = 0.5 smoothed member counts against the training CHIRPS terciles, without the blend: isolates the amount correction
Blended RPSS, Brier skill, reliability (confirmation)The published probability product: counts, smoothing and the climatology blend with its weight fitted inside the folds; category Brier skill for below and above normal; reliability tables
Dry shares and distribution ratiosPooled out-of-fold corrected members (equal-year weights) against CHIRPS of the same years: shares at exactly 0 mm and at ≤ 1 mm; mean, standard deviation, 10th and 90th percentile ratios (area-weighted medians)

Screening results

Region for the result tables

Tables for: All Ethiopia. ✓ = screening pass within the method (Holm across its targets); ✓✓ = also passes with Holm across all methods and targets. A pass is screening evidence, not an adoption decision. Bold: the operational method. Skill is against the CHIRPS training climatology (0 = no better than climatology).

ONDJ season at a glance — All Ethiopia
MethodAmount CRPSSTercile RPSS (no blend)Diagnostics, leave-one-year-out 1993–2016
1993–20162017–20241993–20162017–2024Mean bias (mm)MAE (mm)CorrelationSpread–errorReliability indexMembers ≤ 1 mm
Raw ECMWF (no correction)-0.180-0.116-0.149-0.109-14.869.60.430.540.370.0%
Linear Scaling+0.070+0.131+0.025+0.058-0.157.40.310.780.160.0%
Mean-Variance Scaling (operational)+0.070+0.178+0.035+0.065-0.557.60.270.960.091.8%
Variance Scaling (unbounded)+0.063+0.182+0.036+0.064-0.557.80.261.010.102.3%
Square-root-space Mean-Variance Scaling+0.080 ✓+0.189+0.038+0.073-0.657.40.280.940.070.1%
Power Transformation+0.071+0.188+0.036+0.065-0.957.90.261.010.050.0%
Delta Change+0.093+0.114+0.028+0.011-0.157.40.311.160.050.0%
Local Intensity Scaling+0.070+0.131+0.024+0.058-0.257.40.300.770.170.0%
Empirical Quantile Mapping+0.053+0.168+0.025+0.049-1.157.80.250.970.100.0%
Gamma Quantile Mapping+0.082 ✓+0.192+0.039+0.076-0.557.30.280.930.080.0%
Detrended Quantile Mapping+0.072+0.115+0.010+0.024-1.757.00.320.980.080.0%
Quantile Delta Mapping+0.075+0.118+0.009+0.020-0.157.40.311.040.090.0%
Scaled Distribution Mapping+0.061+0.108-0.005+0.010-5.256.90.300.860.180.4%
CDF-transform+0.074+0.118+0.011+0.029-3.856.80.320.830.120.0%
MVS with all-zero guard (variant)+0.070+0.178+0.035+0.065-0.557.60.270.960.091.8%
Amount skill (CRPSS) by target, leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)-0.203-0.033-0.118-0.551-0.180
Linear Scaling-0.016+0.073-0.008-0.132+0.070
Mean-Variance Scaling (operational)-0.002+0.084+0.028-0.048+0.070
Variance Scaling (unbounded)-0.004+0.082+0.031-0.007 ✓+0.063
Square-root-space Mean-Variance Scaling+0.006+0.089+0.037 ✓+0.034 ✓✓+0.080 ✓
Power Transformation+0.001+0.082+0.020-0.025+0.071
Delta Change+0.021+0.085+0.026-0.047+0.093
Local Intensity Scaling-0.016+0.079+0.010-0.104+0.070
Empirical Quantile Mapping-0.012+0.075+0.033+0.010+0.053
Gamma Quantile Mapping+0.007+0.088+0.030+0.015 ✓✓+0.082 ✓
Detrended Quantile Mapping+0.005+0.075-0.001-0.065+0.072
Quantile Delta Mapping+0.010+0.076+0.014-0.027+0.075
Scaled Distribution Mapping-0.002+0.068-0.021-0.216+0.061
CDF-transform+0.002+0.073-0.019-0.108+0.074
MVS with all-zero guard (variant)-0.002+0.084+0.028-0.048+0.070
Tercile skill (RPSS of smoothed counts, no blend) by target, 1993–2016
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)-0.165-0.043-0.175-0.516-0.149
Linear Scaling-0.033+0.035-0.080-0.144+0.025
Mean-Variance Scaling (operational)-0.009+0.044-0.068-0.213+0.035
Variance Scaling (unbounded)-0.009+0.045-0.060-0.176 ✓+0.036
Square-root-space Mean-Variance Scaling-0.005+0.060 ✓✓-0.030 ✓-0.017 ✓✓+0.038
Power Transformation-0.013+0.044-0.049-0.046 ✓✓+0.036
Delta Change-0.003+0.044-0.154-0.214+0.028
Local Intensity Scaling-0.028+0.047-0.109-0.153+0.024
Empirical Quantile Mapping-0.023+0.046-0.065-0.010 ✓+0.025
Gamma Quantile Mapping-0.006+0.052-0.111-0.038 ✓✓+0.039
Detrended Quantile Mapping-0.020+0.031-0.189-0.211+0.010
Quantile Delta Mapping-0.015+0.020-0.116-0.074+0.009
Scaled Distribution Mapping-0.025+0.018-0.192-0.321-0.005
CDF-transform-0.026+0.010-0.241-0.254+0.011
MVS with all-zero guard (variant)-0.009+0.044-0.068-0.213+0.035
More scores by target: test years, fair CRPS, bias, correlation, spread, reliability, coverage
Amount skill (CRPSS), fit 1993–2016, scored 2017–2024
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)-0.120+0.031-0.183-0.573-0.116
Linear Scaling+0.054+0.125-0.114-0.182+0.131
Mean-Variance Scaling (operational)+0.059+0.158-0.033-0.078+0.178
Variance Scaling (unbounded)+0.056+0.159-0.032+0.007+0.182
Square-root-space Mean-Variance Scaling+0.060+0.162-0.026+0.093+0.189
Power Transformation+0.049+0.165-0.062+0.007+0.188
Delta Change+0.028+0.117-0.019-0.030+0.114
Local Intensity Scaling+0.052+0.134-0.079-0.115+0.131
Empirical Quantile Mapping+0.033+0.158-0.015+0.105+0.168
Gamma Quantile Mapping+0.059+0.165-0.037+0.053+0.192
Detrended Quantile Mapping+0.034+0.116-0.042-0.063+0.115
Quantile Delta Mapping+0.036+0.123-0.043+0.032+0.118
Scaled Distribution Mapping+0.025+0.103-0.055-0.184+0.108
CDF-transform+0.030+0.120-0.097-0.151+0.118
MVS with all-zero guard (variant)+0.059+0.158-0.033-0.078+0.178
Amount skill with the fair CRPS (diagnostic), 1993–2016
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)-0.226-0.048-0.136-0.599-0.205
Linear Scaling-0.031+0.068-0.007-0.119+0.060
Mean-Variance Scaling (operational)-0.009+0.082+0.023-0.066+0.066
Variance Scaling (unbounded)-0.011+0.080+0.026-0.019 ✓✓+0.060
Square-root-space Mean-Variance Scaling-0.002+0.085+0.032 ✓+0.030 ✓✓+0.075 ✓
Power Transformation-0.006+0.079+0.016-0.029+0.067
Delta Change+0.022+0.089+0.027-0.049+0.097
Local Intensity Scaling-0.031+0.075+0.009-0.099+0.059
Empirical Quantile Mapping-0.020+0.071+0.027+0.004 ✓+0.046
Gamma Quantile Mapping-0.001+0.083+0.024+0.008 ✓✓+0.076
Detrended Quantile Mapping-0.002+0.072-0.007-0.073+0.066
Quantile Delta Mapping+0.005+0.074+0.010-0.030+0.071
Scaled Distribution Mapping-0.013+0.059-0.035-0.240+0.052
CDF-transform-0.009+0.067-0.026-0.117+0.065
MVS with all-zero guard (variant)-0.009+0.082+0.023-0.066+0.066
Tercile skill (RPSS, no blend), fit 1993–2016, scored 2017–2024
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)-0.058-0.054-0.084-0.257-0.109
Linear Scaling+0.062+0.006-0.028-0.011+0.058
Mean-Variance Scaling (operational)+0.012+0.041-0.049-0.300+0.065
Variance Scaling (unbounded)+0.008+0.044-0.042-0.220+0.064
Square-root-space Mean-Variance Scaling+0.018+0.058-0.033+0.052+0.073
Power Transformation+0.007+0.050-0.026+0.013+0.065
Delta Change-0.023-0.026-0.061-0.053+0.011
Local Intensity Scaling+0.037-0.001-0.036-0.018+0.058
Empirical Quantile Mapping-0.041+0.022-0.014+0.092+0.049
Gamma Quantile Mapping-0.002+0.050-0.136-0.012+0.076
Detrended Quantile Mapping-0.028-0.021-0.086-0.051+0.024
Quantile Delta Mapping-0.018-0.012-0.026+0.083+0.020
Scaled Distribution Mapping-0.033-0.068-0.088-0.127+0.010
CDF-transform-0.014-0.009-0.119-0.027+0.029
MVS with all-zero guard (variant)+0.012+0.041-0.049-0.300+0.065
Ensemble-mean bias (mm), 1993–2016
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)+4.4-5.4-4.8-9.0-14.8
Linear Scaling-0.0-0.1+0.0+0.1-0.1
Mean-Variance Scaling (operational)+0.4+0.2+0.1-0.0-0.5
Variance Scaling (unbounded)+0.4+0.3+0.1-0.0-0.5
Square-root-space Mean-Variance Scaling-0.1-0.1+0.0+0.0-0.6
Power Transformation-0.1-0.1+0.0-0.0-0.9
Delta Change-0.0-0.1+0.0+0.1-0.1
Local Intensity Scaling-0.0-0.1-0.3-0.9-0.2
Empirical Quantile Mapping-0.2-0.1-0.0-0.1-1.1
Gamma Quantile Mapping-0.1-0.1-0.1-0.1-0.5
Detrended Quantile Mapping-0.2-0.1+0.1+0.4-1.7
Quantile Delta Mapping+0.0+0.3+0.4+0.6-0.1
Scaled Distribution Mapping-4.5-3.9-3.4-5.3-5.2
CDF-transform-1.8-0.6+0.4+1.2-3.8
MVS with all-zero guard (variant)+0.4+0.2+0.1-0.0-0.5
Correlation of the ensemble mean with CHIRPS over years, 1993–2016
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)+0.23+0.38+0.16+0.18+0.43
Linear Scaling+0.07+0.24+0.03+0.11+0.31
Mean-Variance Scaling (operational)+0.08+0.23+0.02+0.01+0.27
Variance Scaling (unbounded)+0.08+0.23+0.01-0.03+0.26
Square-root-space Mean-Variance Scaling+0.09+0.23+0.01+0.05+0.28
Power Transformation+0.08+0.23+0.02+0.02+0.26
Delta Change+0.07+0.24+0.03+0.08+0.31
Local Intensity Scaling+0.07+0.24+0.03+0.08+0.30
Empirical Quantile Mapping+0.07+0.22+0.00-0.01+0.25
Gamma Quantile Mapping+0.09+0.22+0.00-0.04+0.28
Detrended Quantile Mapping+0.09+0.24+0.02+0.11+0.32
Quantile Delta Mapping+0.07+0.24+0.02+0.05+0.31
Scaled Distribution Mapping+0.06+0.25+0.06+0.11+0.30
CDF-transform+0.06+0.24+0.04+0.16+0.32
MVS with all-zero guard (variant)+0.08+0.23+0.02+0.01+0.27
Spread–error ratio, 1993–2016 (1 = spread matches the error)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)0.610.680.630.390.54
Linear Scaling0.720.941.151.810.78
Mean-Variance Scaling (operational)0.901.010.960.830.96
Variance Scaling (unbounded)0.931.021.001.001.01
Square-root-space Mean-Variance Scaling0.900.970.951.070.94
Power Transformation0.951.041.010.981.01
Delta Change1.041.151.181.161.16
Local Intensity Scaling0.720.951.101.590.77
Empirical Quantile Mapping0.931.020.990.980.97
Gamma Quantile Mapping0.900.960.940.980.93
Detrended Quantile Mapping0.961.041.041.060.98
Quantile Delta Mapping0.991.071.081.201.04
Scaled Distribution Mapping0.810.870.810.750.86
CDF-transform0.830.961.031.130.83
MVS with all-zero guard (variant)0.901.010.960.830.96
Rank-histogram reliability index, 1993–2016 (0 = flat)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)0.450.140.381.020.37
Linear Scaling0.380.130.380.660.16
Mean-Variance Scaling (operational)0.100.110.220.680.09
Variance Scaling (unbounded)0.100.110.210.690.10
Square-root-space Mean-Variance Scaling0.150.150.260.110.07
Power Transformation0.180.110.280.260.05
Delta Change0.050.130.420.500.05
Local Intensity Scaling0.330.130.400.650.17
Empirical Quantile Mapping0.090.070.250.230.10
Gamma Quantile Mapping0.130.130.230.420.08
Detrended Quantile Mapping0.080.100.450.520.08
Quantile Delta Mapping0.090.080.330.270.09
Scaled Distribution Mapping0.170.150.530.780.18
CDF-transform0.220.190.530.590.12
MVS with all-zero guard (variant)0.100.110.220.680.09
Score coverage, mean per year, 1993–2016 (share of the region’s area)
TargetAmount cellsAmount areaProbability cellsProbability area
Oct1484100%1478100%
Nov1484100%1478100%
Dec1484100%137893%
Jan1484100%114977%
ONDJ1484100%1484100%
Dry months: shares of totals at exactly 0 mm and at ≤ 1 mm, CHIRPS and corrected
Area-weighted mean share over cells, pooled leave-one-year-out 1993–2024
Exactly 0 mm≤ 1 mm
Oct: CHIRPS0.4%2.3%
Raw ECMWF (no correction)0.0%0.8%
Linear Scaling0.0%0.9%
Mean-Variance Scaling (operational)4.3%5.0%
Variance Scaling (unbounded)4.4%5.2%
Square-root-space Mean-Variance Scaling0.9%2.7%
Power Transformation0.0%1.5%
Delta Change0.4%2.4%
Local Intensity Scaling2.3%2.3%
Empirical Quantile Mapping2.3%2.4%
Gamma Quantile Mapping2.5%3.2%
Detrended Quantile Mapping2.2%2.4%
Quantile Delta Mapping2.3%2.6%
Scaled Distribution Mapping3.6%3.7%
CDF-transform0.0%2.0%
MVS with all-zero guard (variant)4.3%5.0%
Nov: CHIRPS0.7%5.2%
Raw ECMWF (no correction)0.5%8.5%
Linear Scaling0.5%7.7%
Mean-Variance Scaling (operational)7.6%9.4%
Variance Scaling (unbounded)8.2%10.0%
Square-root-space Mean-Variance Scaling2.2%6.3%
Power Transformation0.5%6.5%
Delta Change0.7%5.7%
Local Intensity Scaling5.5%5.5%
Empirical Quantile Mapping5.2%5.3%
Gamma Quantile Mapping5.5%7.9%
Detrended Quantile Mapping4.9%5.6%
Quantile Delta Mapping5.2%6.1%
Scaled Distribution Mapping12.4%12.8%
CDF-transform0.3%4.9%
MVS with all-zero guard (variant)7.6%9.4%
Dec: CHIRPS0.6%18.9%
Raw ECMWF (no correction)6.5%32.5%
Linear Scaling6.5%32.2%
Mean-Variance Scaling (operational)12.0%22.2%
Variance Scaling (unbounded)8.9%21.3%
Square-root-space Mean-Variance Scaling2.7%19.6%
Power Transformation6.5%26.1%
Delta Change0.6%20.4%
Local Intensity Scaling21.5%21.5%
Empirical Quantile Mapping18.9%19.0%
Gamma Quantile Mapping18.2%23.4%
Detrended Quantile Mapping18.6%21.4%
Quantile Delta Mapping18.9%20.7%
Scaled Distribution Mapping36.0%39.8%
CDF-transform0.3%20.0%
MVS with all-zero guard (variant)12.0%22.2%
Jan: CHIRPS1.0%16.6%
Raw ECMWF (no correction)20.0%55.3%
Linear Scaling20.0%43.1%
Mean-Variance Scaling (operational)4.6%16.8%
Variance Scaling (unbounded)1.5%16.1%
Square-root-space Mean-Variance Scaling0.8%17.2%
Power Transformation20.0%27.2%
Delta Change1.0%20.6%
Local Intensity Scaling28.1%28.1%
Empirical Quantile Mapping16.6%16.7%
Gamma Quantile Mapping14.9%18.7%
Detrended Quantile Mapping16.5%21.6%
Quantile Delta Mapping16.6%17.2%
Scaled Distribution Mapping48.1%56.4%
CDF-transform0.2%19.3%
MVS with all-zero guard (variant)4.6%16.8%
ONDJ: CHIRPS0.0%0.0%
Raw ECMWF (no correction)0.0%0.0%
Linear Scaling0.0%0.0%
Mean-Variance Scaling (operational)1.5%1.6%
Variance Scaling (unbounded)2.0%2.1%
Square-root-space Mean-Variance Scaling0.0%0.1%
Power Transformation0.0%0.0%
Delta Change0.0%0.0%
Local Intensity Scaling0.0%0.0%
Empirical Quantile Mapping0.0%0.0%
Gamma Quantile Mapping0.0%0.0%
Detrended Quantile Mapping0.0%0.0%
Quantile Delta Mapping0.0%0.0%
Scaled Distribution Mapping0.7%0.7%
CDF-transform0.0%0.0%
MVS with all-zero guard (variant)1.5%1.6%
Distribution of the corrected members (mean, spread, 10th and 90th percentiles)
Mean of the corrected members / CHIRPS (area-weighted median of cell ratios, 1993–2024; 1 = same)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)0.980.790.820.340.86
Linear Scaling1.001.001.001.011.00
Mean-Variance Scaling (operational)1.001.001.001.001.00
Variance Scaling (unbounded)1.001.001.001.001.00
Square-root-space Mean-Variance Scaling1.001.001.001.001.00
Power Transformation1.001.001.001.001.00
Delta Change1.001.001.001.001.00
Local Intensity Scaling1.001.000.970.901.00
Empirical Quantile Mapping1.001.001.001.001.00
Gamma Quantile Mapping1.001.001.001.001.00
Detrended Quantile Mapping1.001.001.001.020.99
Quantile Delta Mapping1.001.011.031.041.00
Scaled Distribution Mapping0.940.880.720.510.97
CDF-transform0.990.991.041.110.98
MVS with all-zero guard (variant)1.001.001.001.001.00
Standard deviation of the corrected members / CHIRPS (area-weighted median of cell ratios, 1993–2024; 1 = same)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)0.790.771.050.960.69
Linear Scaling0.810.941.262.500.84
Mean-Variance Scaling (operational)1.000.991.011.020.99
Variance Scaling (unbounded)1.000.991.011.020.99
Square-root-space Mean-Variance Scaling0.980.940.931.110.94
Power Transformation1.001.001.011.001.00
Delta Change1.061.091.171.341.07
Local Intensity Scaling0.810.961.161.700.83
Empirical Quantile Mapping1.000.991.001.000.98
Gamma Quantile Mapping0.980.910.880.910.92
Detrended Quantile Mapping0.991.001.091.290.96
Quantile Delta Mapping1.011.031.161.370.99
Scaled Distribution Mapping0.880.850.870.970.86
CDF-transform0.930.981.131.530.89
MVS with all-zero guard (variant)1.000.991.011.020.99
10th percentile of the corrected members / CHIRPS (area-weighted median of cell ratios, 1993–2024; 1 = same)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)1.270.520.060.000.93
Linear Scaling1.250.730.070.001.03
Mean-Variance Scaling (operational)0.880.610.621.160.86
Variance Scaling (unbounded)0.850.580.711.160.82
Square-root-space Mean-Variance Scaling0.950.750.570.860.91
Power Transformation1.000.640.240.000.89
Delta Change0.920.860.710.620.91
Local Intensity Scaling1.200.650.260.001.04
Empirical Quantile Mapping0.960.970.980.990.97
Gamma Quantile Mapping0.970.780.590.700.94
Detrended Quantile Mapping0.950.870.690.610.96
Quantile Delta Mapping0.970.890.760.930.97
Scaled Distribution Mapping0.970.690.000.000.98
CDF-transform1.010.870.610.580.98
MVS with all-zero guard (variant)0.880.610.621.160.86
90th percentile of the corrected members / CHIRPS (area-weighted median of cell ratios, 1993–2024; 1 = same)
MethodOctNovDecJanONDJ
Raw ECMWF (no correction)0.920.811.100.470.81
Linear Scaling0.961.011.311.420.98
Mean-Variance Scaling (operational)1.031.011.110.921.04
Variance Scaling (unbounded)1.031.011.070.921.05
Square-root-space Mean-Variance Scaling1.030.991.091.061.02
Power Transformation1.031.011.121.111.04
Delta Change1.061.051.071.121.06
Local Intensity Scaling0.961.011.161.080.98
Empirical Quantile Mapping1.031.041.071.041.03
Gamma Quantile Mapping1.030.981.000.971.02
Detrended Quantile Mapping1.031.031.081.131.02
Quantile Delta Mapping1.041.041.111.081.04
Scaled Distribution Mapping0.960.890.830.710.97
CDF-transform1.011.031.151.301.01
MVS with all-zero guard (variant)1.031.011.110.921.04
Screening statistics for every method, target and score (within-method and family-wide Holm)

Screening rule (docs/36 rule applied as a screen): a method passes for a target and score if, on the leave-one-year-out 1993-2016 per-year scores, its mean improvement over mvs has a one-sided sign-flip p-value below 0.05 after Holm adjustment, and its mean score over 2017-2024 (fits on 1993-2016) is not worse. 'screen_pass': Holm across the 5 targets of one method. 'screen_pass_family': Holm across every method and target of the benchmark for that score. A screening pass is not an operational adoption decision (see the confirmation run).

Screening statistics, CRPS (method minus Mean-Variance Scaling; negative favours the method)
MethodTargetΔ CRPS 1993–2016 (mm)95% intervalYears betterp (one-sided)Holm p (method)Holm p (all 60)Δ 2017–2024Screening
Linear ScalingOct+0.41-0.38 to +1.2811/240.81771.00001.0000+0.16—
Linear ScalingNov+0.22+0.02 to +0.406/240.98051.00001.0000+0.77—
Linear ScalingDec+0.27+0.12 to +0.417/240.99841.00001.0000+0.38—
Linear ScalingJan+0.49-0.08 to +1.009/240.94391.00001.0000+0.42—
Linear ScalingONDJ-0.01-1.28 to +1.2314/240.49181.00001.0000+2.36—
Variance Scaling (unbounded)Oct+0.07-0.04 to +0.178/240.89311.00001.0000+0.08—
Variance Scaling (unbounded)Nov+0.05+0.02 to +0.087/240.99731.00001.0000-0.02—
Variance Scaling (unbounded)Dec-0.02-0.06 to +0.0115/240.11100.44401.0000-0.00—
Variance Scaling (unbounded)Jan-0.24-0.37 to -0.1116/240.00120.00620.0722-0.34✓
Variance Scaling (unbounded)ONDJ+0.34+0.04 to +0.677/240.97891.00001.0000-0.19—
Square-root-space Mean-Variance ScalingOct-0.23-0.48 to +0.0315/240.04880.09761.0000-0.01—
Square-root-space Mean-Variance ScalingNov-0.09-0.22 to +0.0516/240.09550.09761.0000-0.10—
Square-root-space Mean-Variance ScalingDec-0.07-0.11 to -0.0220/240.00480.01910.2722-0.03✓
Square-root-space Mean-Variance ScalingJan-0.48-0.72 to -0.2518/240.00040.00180.0209-0.69✓✓
Square-root-space Mean-Variance ScalingONDJ-0.43-0.75 to -0.1318/240.00650.01940.3615-0.57✓
Power TransformationOct-0.09-0.51 to +0.3515/240.34691.00001.0000+0.30—
Power TransformationNov+0.05-0.13 to +0.2414/240.68411.00001.0000-0.16—
Power TransformationDec+0.06-0.03 to +0.146/240.91871.00001.0000+0.14—
Power TransformationJan-0.13-0.36 to +0.1012/240.13170.65871.0000-0.34—
Power TransformationONDJ-0.03-0.67 to +0.6515/240.46631.00001.0000-0.50—
Delta ChangeOct-0.68-1.60 to +0.1814/240.08110.37921.0000+0.94—
Delta ChangeNov-0.02-0.57 to +0.5011/240.47771.00001.0000+0.95—
Delta ChangeDec+0.02-0.17 to +0.2311/240.57281.00001.0000-0.07—
Delta ChangeJan-0.01-0.47 to +0.4110/240.48921.00001.0000-0.19—
Delta ChangeONDJ-1.04-2.38 to +0.3416/240.07580.37921.0000+3.22—
Local Intensity ScalingOct+0.42-0.39 to +1.2911/240.81661.00001.0000+0.23—
Local Intensity ScalingNov+0.10-0.07 to +0.269/240.86161.00001.0000+0.57—
Local Intensity ScalingDec+0.14-0.00 to +0.276/240.96461.00001.0000+0.22—
Local Intensity ScalingJan+0.32-0.20 to +0.829/240.87491.00001.0000+0.15—
Local Intensity ScalingONDJ+0.01-1.30 to +1.2914/240.50701.00001.0000+2.39—
Empirical Quantile MappingOct+0.32-0.20 to +0.8311/240.87821.00001.0000+0.81—
Empirical Quantile MappingNov+0.19-0.17 to +0.5713/240.82691.00001.0000-0.01—
Empirical Quantile MappingDec-0.03-0.18 to +0.1211/240.33581.00001.0000-0.09—
Empirical Quantile MappingJan-0.34-0.61 to -0.0813/240.01170.05830.6294-0.74—
Empirical Quantile MappingONDJ+0.78-0.35 to +2.0614/240.86651.00001.0000+0.49—
Gamma Quantile MappingOct-0.26-0.59 to +0.0615/240.06540.19611.0000-0.01—
Gamma Quantile MappingNov-0.06-0.23 to +0.1113/240.23700.47411.0000-0.16—
Gamma Quantile MappingDec-0.02-0.06 to +0.0315/240.27690.47411.0000+0.02—
Gamma Quantile MappingJan-0.36-0.54 to -0.2019/240.00010.00070.0084-0.53✓✓
Gamma Quantile MappingONDJ-0.52-0.92 to -0.1418/240.00740.02970.4087-0.71✓
Detrended Quantile MappingOct-0.20-0.89 to +0.4615/240.28441.00001.0000+0.75—
Detrended Quantile MappingNov+0.18-0.16 to +0.5412/240.83601.00001.0000+0.98—
Detrended Quantile MappingDec+0.22+0.04 to +0.418/240.98551.00001.0000+0.04—
Detrended Quantile MappingJan+0.10-0.52 to +0.649/240.61341.00001.0000-0.06—
Detrended Quantile MappingONDJ-0.09-1.12 to +1.0215/240.44021.00001.0000+3.18—
Quantile Delta MappingOct-0.36-1.09 to +0.3515/240.17960.89801.0000+0.70—
Quantile Delta MappingNov+0.15-0.14 to +0.4611/240.83141.00001.0000+0.81—
Quantile Delta MappingDec+0.11-0.08 to +0.329/240.80631.00001.0000+0.05—
Quantile Delta MappingJan-0.12-0.63 to +0.3510/240.32741.00001.0000-0.44—
Quantile Delta MappingONDJ-0.22-1.08 to +0.7416/240.33461.00001.0000+3.06—
Scaled Distribution MappingOct+0.00-1.17 to +1.2814/240.49891.00001.0000+1.03—
Scaled Distribution MappingNov+0.32-0.34 to +1.0714/240.79861.00001.0000+1.29—
Scaled Distribution MappingDec+0.36-0.12 to +0.8410/240.91651.00001.0000+0.10—
Scaled Distribution MappingJan+0.98+0.21 to +1.7311/240.98871.00001.0000+0.43—
Scaled Distribution MappingONDJ+0.40-1.01 to +2.0215/240.67681.00001.0000+3.53—
CDF-transformOct-0.10-0.87 to +0.6814/240.40511.00001.0000+0.87—
CDF-transformNov+0.23-0.06 to +0.5411/240.91631.00001.0000+0.89—
CDF-transformDec+0.35+0.13 to +0.596/240.99841.00001.0000+0.30—
CDF-transformJan+0.35-0.35 to +0.966/240.83471.00001.0000+0.30—
CDF-transformONDJ-0.18-1.16 to +0.8716/240.37361.00001.0000+3.04—
MVS with all-zero guard (variant)Oct+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Nov+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Dec+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Jan+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)ONDJ+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
Screening statistics, fair CRPS (method minus Mean-Variance Scaling; negative favours the method)
MethodTargetΔ fair CRPS 1993–2016 (mm)95% intervalYears betterp (one-sided)Holm p (method)Holm p (all 60)Δ 2017–2024Screening
Linear ScalingOct+0.63-0.16 to +1.489/240.91871.00001.0000+0.26—
Linear ScalingNov+0.25+0.06 to +0.435/240.99061.00001.0000+0.79—
Linear ScalingDec+0.21+0.06 to +0.357/240.99121.00001.0000+0.36—
Linear ScalingJan+0.30-0.28 to +0.829/240.83561.00001.0000+0.34—
Linear ScalingONDJ+0.27-0.98 to +1.4913/240.65831.00001.0000+2.51—
Variance Scaling (unbounded)Oct+0.04-0.06 to +0.149/240.78811.00001.0000+0.07—
Variance Scaling (unbounded)Nov+0.04+0.01 to +0.079/240.99111.00001.0000-0.02—
Variance Scaling (unbounded)Dec-0.03-0.06 to +0.0115/240.07660.30641.0000-0.01—
Variance Scaling (unbounded)Jan-0.26-0.39 to -0.1418/240.00060.00290.0339-0.35✓✓
Variance Scaling (unbounded)ONDJ+0.27-0.02 to +0.607/240.95041.00001.0000-0.22—
Square-root-space Mean-Variance ScalingOct-0.21-0.46 to +0.0515/240.06890.13781.0000-0.00—
Square-root-space Mean-Variance ScalingNov-0.07-0.20 to +0.0716/240.16590.16591.0000-0.09—
Square-root-space Mean-Variance ScalingDec-0.07-0.11 to -0.0220/240.00550.02190.3063-0.03✓
Square-root-space Mean-Variance ScalingJan-0.54-0.78 to -0.3120/240.00010.00030.0029-0.72✓✓
Square-root-space Mean-Variance ScalingONDJ-0.37-0.69 to -0.0618/240.01630.04890.8962-0.54✓
Power TransformationOct-0.09-0.51 to +0.3515/240.35151.00001.0000+0.30—
Power TransformationNov+0.04-0.13 to +0.2314/240.66621.00001.0000-0.17—
Power TransformationDec+0.04-0.04 to +0.127/240.84351.00001.0000+0.13—
Power TransformationJan-0.20-0.43 to +0.0214/240.04680.23421.0000-0.38—
Power TransformationONDJ-0.04-0.67 to +0.6115/240.44551.00001.0000-0.51—
Delta ChangeOct-0.88-1.82 to -0.0214/240.03630.18071.0000+0.31—
Delta ChangeNov-0.13-0.69 to +0.3912/240.32660.97971.0000+0.52—
Delta ChangeDec-0.03-0.21 to +0.1713/240.38550.97971.0000-0.22—
Delta ChangeJan-0.09-0.56 to +0.3311/240.35140.97971.0000-0.33—
Delta ChangeONDJ-1.34-2.71 to +0.0616/240.03610.18071.0000+2.26—
Local Intensity ScalingOct+0.63-0.17 to +1.5011/240.91861.00001.0000+0.33—
Local Intensity ScalingNov+0.13-0.04 to +0.299/240.92661.00001.0000+0.58—
Local Intensity ScalingDec+0.10-0.04 to +0.237/240.90371.00001.0000+0.20—
Local Intensity ScalingJan+0.18-0.35 to +0.6811/240.74121.00001.0000+0.09—
Local Intensity ScalingONDJ+0.31-0.99 to +1.5613/240.67241.00001.0000+2.54—
Empirical Quantile MappingOct+0.33-0.20 to +0.8411/240.88361.00001.0000+0.82—
Empirical Quantile MappingNov+0.20-0.16 to +0.5813/240.84351.00001.0000-0.00—
Empirical Quantile MappingDec-0.03-0.17 to +0.1212/240.34401.00001.0000-0.09—
Empirical Quantile MappingJan-0.39-0.66 to -0.1314/240.00470.02370.2702-0.76✓
Empirical Quantile MappingONDJ+0.87-0.25 to +2.1612/240.89501.00001.0000+0.53—
Gamma Quantile MappingOct-0.24-0.57 to +0.0914/240.09110.27331.0000+0.01—
Gamma Quantile MappingNov-0.03-0.20 to +0.1412/240.35670.71331.0000-0.14—
Gamma Quantile MappingDec-0.01-0.05 to +0.0414/240.38420.71331.0000+0.02—
Gamma Quantile MappingJan-0.41-0.58 to -0.2521/240.00000.00010.0009-0.55✓✓
Gamma Quantile MappingONDJ-0.44-0.85 to -0.0518/240.02220.08891.0000-0.67—
Detrended Quantile MappingOct-0.21-0.90 to +0.4515/240.27421.00001.0000+0.75—
Detrended Quantile MappingNov+0.18-0.16 to +0.5412/240.83781.00001.0000+0.98—
Detrended Quantile MappingDec+0.21+0.04 to +0.408/240.98341.00001.0000+0.04—
Detrended Quantile MappingJan+0.04-0.59 to +0.589/240.54301.00001.0000-0.08—
Detrended Quantile MappingONDJ-0.01-1.05 to +1.1115/240.49331.00001.0000+3.22—
Quantile Delta MappingOct-0.40-1.14 to +0.3215/240.15340.76711.0000+0.70—
Quantile Delta MappingNov+0.14-0.15 to +0.4511/240.81091.00001.0000+0.81—
Quantile Delta MappingDec+0.09-0.10 to +0.3111/240.76251.00001.0000+0.05—
Quantile Delta MappingJan-0.20-0.72 to +0.2711/240.23260.93051.0000-0.47—
Quantile Delta MappingONDJ-0.22-1.09 to +0.7416/240.33230.99691.0000+3.08—
Scaled Distribution MappingOct+0.11-1.06 to +1.3913/240.56431.00001.0000+1.08—
Scaled Distribution MappingNov+0.43-0.24 to +1.1712/240.86331.00001.0000+1.33—
Scaled Distribution MappingDec+0.41-0.07 to +0.899/240.94051.00001.0000+0.12—
Scaled Distribution MappingJan+0.97+0.20 to +1.7211/240.98811.00001.0000+0.42—
Scaled Distribution MappingONDJ+0.62-0.78 to +2.2315/240.76201.00001.0000+3.62—
CDF-transformOct-0.01-0.78 to +0.7614/240.48781.00001.0000+0.90—
CDF-transformNov+0.27-0.03 to +0.5911/240.94331.00001.0000+0.90—
CDF-transformDec+0.34+0.13 to +0.597/240.99841.00001.0000+0.30—
CDF-transformJan+0.28-0.42 to +0.897/240.78091.00001.0000+0.27—
CDF-transformONDJ+0.04-0.93 to +1.1015/240.52771.00001.0000+3.13—
MVS with all-zero guard (variant)Oct+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Nov+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Dec+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)Jan+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
MVS with all-zero guard (variant)ONDJ+0.00+0.00 to +0.000/241.00001.0000— (variant)+0.00—
Screening statistics, RPS (no blend) (method minus Mean-Variance Scaling; negative favours the method)
MethodTargetΔ RPS (no blend) 1993–201695% intervalYears betterp (one-sided)Holm p (method)Holm p (all 60)Δ 2017–2024Screening
Linear ScalingOct+0.0109-0.0080 to +0.02899/240.86671.00001.0000-0.0219—
Linear ScalingNov+0.0043-0.0098 to +0.018310/240.71641.00001.0000+0.0150—
Linear ScalingDec+0.0056-0.0219 to +0.03189/240.65461.00001.0000-0.0096—
Linear ScalingJan-0.0315-0.1398 to +0.073110/240.29261.00001.0000-0.1296—
Linear ScalingONDJ+0.0048-0.0056 to +0.015612/240.80091.00001.0000+0.0033—
Variance Scaling (unbounded)Oct+0.0001-0.0013 to +0.00139/240.52940.93371.0000+0.0014—
Variance Scaling (unbounded)Nov-0.0006-0.0015 to +0.000313/240.10500.41991.0000-0.0013—
Variance Scaling (unbounded)Dec-0.0035-0.0087 to +0.001815/240.10940.41991.0000-0.0030—
Variance Scaling (unbounded)Jan-0.0168-0.0268 to -0.006616/240.00290.01430.1568-0.0356✓
Variance Scaling (unbounded)ONDJ-0.0001-0.0022 to +0.00198/240.46680.93371.0000+0.0005—
Square-root-space Mean-Variance ScalingOct-0.0015-0.0046 to +0.001512/240.17270.27181.0000-0.0028—
Square-root-space Mean-Variance ScalingNov-0.0071-0.0106 to -0.003919/240.00020.00090.0128-0.0077✓✓
Square-root-space Mean-Variance ScalingDec-0.0170-0.0313 to -0.003915/240.01210.03630.6531-0.0070✓
Square-root-space Mean-Variance ScalingJan-0.0890-0.1345 to -0.045918/240.00020.00080.0097-0.1581✓✓
Square-root-space Mean-Variance ScalingONDJ-0.0014-0.0037 to +0.001013/240.13590.27181.0000-0.0036—
Power TransformationOct+0.0022-0.0045 to +0.00869/240.73831.00001.0000+0.0020—
Power TransformationNov-0.0002-0.0047 to +0.004411/240.46971.00001.0000-0.0043—
Power TransformationDec-0.0086-0.0192 to +0.001314/240.06370.25461.0000-0.0106—
Power TransformationJan-0.0756-0.1179 to -0.035917/240.00040.00220.0254-0.1404✓✓
Power TransformationONDJ-0.0001-0.0080 to +0.008013/240.48711.00001.0000+0.0000—
Delta ChangeOct-0.0027-0.0215 to +0.015412/240.39061.00001.0000+0.0149—
Delta ChangeNov-0.0000-0.0222 to +0.021010/240.49791.00001.0000+0.0294—
Delta ChangeDec+0.0389+0.0008 to +0.07918/240.96611.00001.0000+0.0056—
Delta ChangeJan+0.0006-0.0759 to +0.07108/240.50581.00001.0000-0.1111—
Delta ChangeONDJ+0.0032-0.0143 to +0.020010/240.63661.00001.0000+0.0245—
Local Intensity ScalingOct+0.0088-0.0045 to +0.02159/240.89411.00001.0000-0.0110—
Local Intensity ScalingNov-0.0015-0.0125 to +0.009210/240.39791.00001.0000+0.0185—
Local Intensity ScalingDec+0.0185-0.0221 to +0.05699/240.80741.00001.0000-0.0057—
Local Intensity ScalingJan-0.0273-0.1362 to +0.079810/240.31911.00001.0000-0.1266—
Local Intensity ScalingONDJ+0.0050-0.0058 to +0.016212/240.80441.00001.0000+0.0033—
Empirical Quantile MappingOct+0.0065-0.0061 to +0.019212/240.83361.00001.0000+0.0226—
Empirical Quantile MappingNov-0.0007-0.0126 to +0.010611/240.45021.00001.0000+0.0080—
Empirical Quantile MappingDec-0.0011-0.0290 to +0.026510/240.47041.00001.0000-0.0160—
Empirical Quantile MappingJan-0.0920-0.1479 to -0.039316/240.00220.01090.1215-0.1762✓
Empirical Quantile MappingONDJ+0.0047-0.0048 to +0.014612/240.81961.00001.0000+0.0072—
Gamma Quantile MappingOct-0.0010-0.0071 to +0.004913/240.36970.73951.0000+0.0057—
Gamma Quantile MappingNov-0.0035-0.0107 to +0.003011/240.17150.68591.0000-0.0041—
Gamma Quantile MappingDec+0.0195+0.0047 to +0.03518/240.98940.98941.0000+0.0402—
Gamma Quantile MappingJan-0.0794-0.1078 to -0.052322/240.00000.00000.0000-0.1294✓✓
Gamma Quantile MappingONDJ-0.0014-0.0045 to +0.001713/240.19260.68591.0000-0.0049—
Detrended Quantile MappingOct+0.0050-0.0072 to +0.016912/240.78301.00001.0000+0.0171—
Detrended Quantile MappingNov+0.0060-0.0094 to +0.02149/240.76981.00001.0000+0.0271—
Detrended Quantile MappingDec+0.0548+0.0168 to +0.09176/240.99491.00001.0000+0.0172—
Detrended Quantile MappingJan-0.0011-0.0787 to +0.07058/240.48851.00001.0000-0.1118—
Detrended Quantile MappingONDJ+0.0113+0.0026 to +0.02049/240.98741.00001.0000+0.0187—
Quantile Delta MappingOct+0.0027-0.0099 to +0.015112/240.65961.00001.0000+0.0129—
Quantile Delta MappingNov+0.0109-0.0027 to +0.02427/240.93201.00001.0000+0.0231—
Quantile Delta MappingDec+0.0218-0.0128 to +0.05599/240.88121.00001.0000-0.0104—
Quantile Delta MappingJan-0.0632-0.1330 to +0.003812/240.04870.24341.0000-0.1719—
Quantile Delta MappingONDJ+0.0121+0.0034 to +0.02097/240.99311.00001.0000+0.0208—
Scaled Distribution MappingOct+0.0075-0.0090 to +0.024113/240.80351.00001.0000+0.0193—
Scaled Distribution MappingNov+0.0116-0.0137 to +0.03719/240.80781.00001.0000+0.0481—
Scaled Distribution MappingDec+0.0563-0.0121 to +0.12238/240.93861.00001.0000+0.0181—
Scaled Distribution MappingJan+0.0489-0.0850 to +0.18029/240.75431.00001.0000-0.0774—
Scaled Distribution MappingONDJ+0.0185+0.0070 to +0.03018/240.99731.00001.0000+0.0250—
CDF-transformOct+0.0078-0.0023 to +0.01798/240.92351.00001.0000+0.0110—
CDF-transformNov+0.0155+0.0032 to +0.02796/240.98831.00001.0000+0.0220—
CDF-transformDec+0.0786+0.0382 to +0.12167/240.99951.00001.0000+0.0325—
CDF-transformJan+0.0188-0.0635 to +0.09318/240.67451.00001.0000-0.1226—
CDF-transformONDJ+0.0110+0.0038 to +0.01836/240.99591.00001.0000+0.0165—
MVS with all-zero guard (variant)Oct+0.0000+0.0000 to +0.00000/241.00001.0000— (variant)+0.0000—
MVS with all-zero guard (variant)Nov+0.0000+0.0000 to +0.00000/241.00001.0000— (variant)+0.0000—
MVS with all-zero guard (variant)Dec+0.0000+0.0000 to +0.00000/241.00001.0000— (variant)+0.0000—
MVS with all-zero guard (variant)Jan+0.0000+0.0000 to +0.00000/241.00001.0000— (variant)+0.0000—
MVS with all-zero guard (variant)ONDJ+0.0000+0.0000 to +0.00000/241.00001.0000— (variant)+0.0000—
How often limits, caps and fallbacks act
How often limits, caps and fallbacks act, leave-one-year-out 1993–2016 (mean over folds)
MethodTargetActivated (share of the corrected area, or of members / cells for events)
Linear ScalingDecmean fallback: 0.4% of the area
Linear ScalingJanmean fallback: 2.8% of the area
Mean-Variance Scaling (operational)Octratio lower limit: 0.9% of the area; ratio upper limit: 11.8% of the area; clipped at zero: 6.5%
Mean-Variance Scaling (operational)Novratio lower limit: 5.2% of the area; ratio upper limit: 9.4% of the area; clipped at zero: 8.0%
Mean-Variance Scaling (operational)Decsd fallback: 1.3% of the area; ratio lower limit: 17.3% of the area; ratio upper limit: 5.8% of the area; clipped at zero: 12.2%
Mean-Variance Scaling (operational)Jansd fallback: 3.9% of the area; ratio lower limit: 25.8% of the area; ratio upper limit: 28.2% of the area; clipped at zero: 4.5%
Mean-Variance Scaling (operational)ONDJratio lower limit: 2.2% of the area; ratio upper limit: 22.3% of the area; clipped at zero: 1.7%
Variance Scaling (unbounded)Octclipped at zero: 6.8%
Variance Scaling (unbounded)Novclipped at zero: 8.2%
Variance Scaling (unbounded)Decsd fallback: 1.3% of the area; clipped at zero: 8.9%
Variance Scaling (unbounded)Jansd fallback: 3.9% of the area; clipped at zero: 1.8%
Variance Scaling (unbounded)ONDJclipped at zero: 2.2%
Square-root-space Mean-Variance ScalingOctratio upper limit: 0.4% of the area; clipped at zero: 1.5%
Square-root-space Mean-Variance ScalingNovratio lower limit: 0.3% of the area; clipped at zero: 2.3%
Square-root-space Mean-Variance ScalingDecratio lower limit: 14.0% of the area; clipped at zero: 2.9%
Square-root-space Mean-Variance ScalingJanratio lower limit: 30.9% of the area; ratio upper limit: 0.3% of the area; clipped at zero: 1.0%
Square-root-space Mean-Variance ScalingONDJratio lower limit: 0.0% of the area; ratio upper limit: 1.8% of the area; clipped at zero: 0.0%
Power TransformationDecb at bound: 1.7% of the area
Power TransformationJanb at bound: 23.3% of the area
Delta ChangeDecmean fallback: 0.4% of the area; ratio capped: 0.1%
Delta ChangeJanmean fallback: 2.8% of the area; ratio capped: 1.1%
Local Intensity ScalingOctalways wet: 91.6% of the area
Local Intensity ScalingNovalways wet: 78.9% of the area
Local Intensity ScalingDecalways wet: 73.6% of the area; never wet: 2.9% of the area
Local Intensity ScalingJanalways wet: 80.2% of the area; never wet: 9.2% of the area
Local Intensity ScalingONDJalways wet: 100.0% of the area
Empirical Quantile MappingOctmodel has dry: 25.2% of the area; obs has dry: 8.4% of the area; extrapolated: 0.3%
Empirical Quantile MappingNovmodel has dry: 86.0% of the area; obs has dry: 21.1% of the area; extrapolated: 0.2%
Empirical Quantile MappingDecmodel has dry: 98.9% of the area; obs has dry: 26.4% of the area; extrapolated: 0.2%
Empirical Quantile MappingJanmodel has dry: 100.0% of the area; obs has dry: 19.8% of the area; extrapolated: 0.2%
Empirical Quantile MappingONDJmodel has dry: 1.6% of the area; extrapolated: 0.4%
Gamma Quantile MappingOctmodel has dry: 25.2% of the area; obs has dry: 8.4% of the area
Gamma Quantile MappingNovmodel has dry: 86.0% of the area; obs has dry: 21.1% of the area
Gamma Quantile MappingDecgamma fallback: 2.7% of the area; model has dry: 98.9% of the area; obs has dry: 26.4% of the area
Gamma Quantile MappingJangamma fallback: 4.5% of the area; model has dry: 100.0% of the area; obs has dry: 19.8% of the area
Gamma Quantile MappingONDJmodel has dry: 1.6% of the area
Detrended Quantile MappingOctmodel has dry: 25.2% of the area; obs has dry: 8.4% of the area; extrapolated: 0.2%
Detrended Quantile MappingNovmodel has dry: 86.0% of the area; obs has dry: 21.1% of the area; extrapolated: 0.1%
Detrended Quantile MappingDecmodel has dry: 98.9% of the area; obs has dry: 26.4% of the area; ratio capped: 0.1%; extrapolated: 0.1%
Detrended Quantile MappingJanmodel has dry: 100.0% of the area; obs has dry: 19.8% of the area; ratio capped: 1.1%; extrapolated: 0.1%
Detrended Quantile MappingONDJmodel has dry: 1.6% of the area; extrapolated: 0.2%
Quantile Delta MappingOctmodel has dry: 25.2% of the area; obs has dry: 8.4% of the area; low denominator: 0.6%; ratio capped: 0.0%
Quantile Delta MappingNovmodel has dry: 86.0% of the area; obs has dry: 21.1% of the area; low denominator: 7.9%; ratio capped: 0.3%
Quantile Delta MappingDecmodel has dry: 98.9% of the area; obs has dry: 26.4% of the area; low denominator: 31.5%; ratio capped: 0.4%
Quantile Delta MappingJanmodel has dry: 100.0% of the area; obs has dry: 19.8% of the area; low denominator: 54.4%; ratio capped: 0.2%
Quantile Delta MappingONDJmodel has dry: 1.6% of the area; ratio capped: 0.0%
Scaled Distribution MappingDecgamma fallback: 5.5% of the area; linear scaling fallback: 5.9%
Scaled Distribution MappingJangamma fallback: 13.7% of the area; linear scaling fallback: 23.4%
CDF-transformOctat forecast range edge: 13.9%
CDF-transformNovat forecast range edge: 7.1%
CDF-transformDecmean fallback: 0.4% of the area; at forecast range edge: 2.2%
CDF-transformJanmean fallback: 2.8% of the area; at forecast range edge: 0.6%
CDF-transformONDJat forecast range edge: 9.5%
MVS with all-zero guard (variant)Octratio lower limit: 0.9% of the area; ratio upper limit: 11.8% of the area; clipped at zero: 6.5%
MVS with all-zero guard (variant)Novratio lower limit: 5.2% of the area; ratio upper limit: 9.4% of the area; clipped at zero: 8.0%
MVS with all-zero guard (variant)Decsd fallback: 1.3% of the area; ratio lower limit: 17.3% of the area; ratio upper limit: 5.8% of the area; clipped at zero: 12.2%
MVS with all-zero guard (variant)Jansd fallback: 3.9% of the area; ratio lower limit: 25.8% of the area; ratio upper limit: 28.2% of the area; clipped at zero: 4.5%
MVS with all-zero guard (variant)ONDJratio lower limit: 2.2% of the area; ratio upper limit: 22.3% of the area; clipped at zero: 1.7%
Where the amount differences come from: cells grouped by their training-year climate

Each cell is grouped by its CHIRPS totals in the fold’s training years only: every year exactly 0 mm; every year ≤ 1 mm (not all 0); some years dry and some wet; never dry. Contributions are area-weighted CRPS differences from Mean-Variance Scaling (mm; negative favours the method) and add up to the national difference. Leave-one-year-out 1993–2016, All Ethiopia.

Oct: contributions to the CRPS difference from Mean-Variance Scaling (mm)
MethodTotal Δ CRPSall 0 mm
0.0% of area		all ≤ 1 mm
0.0% of area		mixed
8.4% of area		always wet
91.6% of area
Linear Scaling+0.415+0.000+0.000+0.034+0.381
Variance Scaling (unbounded)+0.067+0.000+0.000+0.000+0.067
Square-root-space Mean-Variance Scaling-0.227+0.000+0.000-0.014-0.213
Power Transformation-0.087+0.000+0.000+0.005-0.092
Delta Change-0.677+0.000+0.000-0.044-0.634
Local Intensity Scaling+0.417+0.000+0.000-0.010+0.427
Empirical Quantile Mapping+0.318+0.000+0.000+0.021+0.298
Gamma Quantile Mapping-0.263+0.000+0.000-0.010-0.253
Detrended Quantile Mapping-0.204+0.000+0.000-0.022-0.182
Quantile Delta Mapping-0.357+0.000+0.000-0.037-0.320
Scaled Distribution Mapping+0.001+0.000+0.000-0.023+0.024
CDF-transform-0.099+0.000+0.000-0.016-0.083
MVS with all-zero guard (variant)+0.000+0.000+0.000+0.000+0.000
Nov: contributions to the CRPS difference from Mean-Variance Scaling (mm)
MethodTotal Δ CRPSall 0 mm
0.0% of area		all ≤ 1 mm
0.0% of area		mixed
21.1% of area		always wet
78.9% of area
Linear Scaling+0.221+0.000+0.000+0.088+0.133
Variance Scaling (unbounded)+0.046+0.000+0.000+0.002+0.044
Square-root-space Mean-Variance Scaling-0.093+0.000+0.000-0.020-0.073
Power Transformation+0.047+0.000+0.000+0.026+0.021
Delta Change-0.016+0.000+0.000+0.061-0.077
Local Intensity Scaling+0.095+0.000+0.000-0.012+0.107
Empirical Quantile Mapping+0.186+0.000+0.000+0.039+0.146
Gamma Quantile Mapping-0.065+0.000+0.000-0.009-0.055
Detrended Quantile Mapping+0.182+0.000+0.000+0.069+0.113
Quantile Delta Mapping+0.154+0.000+0.000+0.062+0.091
Scaled Distribution Mapping+0.323+0.000+0.000+0.068+0.254
CDF-transform+0.228+0.000+0.000+0.069+0.159
MVS with all-zero guard (variant)+0.000+0.000+0.000+0.000+0.000
Dec: contributions to the CRPS difference from Mean-Variance Scaling (mm)
MethodTotal Δ CRPSall 0 mm
0.0% of area		all ≤ 1 mm
2.9% of area		mixed
23.6% of area		always wet
73.6% of area
Linear Scaling+0.266+0.000-0.001+0.002+0.264
Variance Scaling (unbounded)-0.022+0.000-0.001-0.004-0.017
Square-root-space Mean-Variance Scaling-0.065+0.000-0.001-0.014-0.051
Power Transformation+0.063+0.000-0.001-0.013+0.077
Delta Change+0.019+0.000-0.001-0.028+0.048
Local Intensity Scaling+0.136+0.000+0.001+0.005+0.130
Empirical Quantile Mapping-0.033+0.000+0.001+0.011-0.044
Gamma Quantile Mapping-0.015+0.000+0.001+0.010-0.026
Detrended Quantile Mapping+0.218+0.000+0.001+0.018+0.199
Quantile Delta Mapping+0.105+0.000+0.001+0.007+0.097
Scaled Distribution Mapping+0.360+0.000-0.001+0.006+0.354
CDF-transform+0.346+0.000-0.001-0.006+0.353
MVS with all-zero guard (variant)+0.000+0.000+0.000+0.000+0.000
Jan: contributions to the CRPS difference from Mean-Variance Scaling (mm)
MethodTotal Δ CRPSall 0 mm
0.0% of area		all ≤ 1 mm
9.2% of area		mixed
10.6% of area		always wet
80.2% of area
Linear Scaling+0.487+0.000+0.003+0.003+0.480
Variance Scaling (unbounded)-0.240+0.000-0.005-0.006-0.229
Square-root-space Mean-Variance Scaling-0.479+0.000-0.003-0.018-0.458
Power Transformation-0.134+0.000+0.001-0.004-0.131
Delta Change-0.005+0.000+0.001-0.006-0.001
Local Intensity Scaling+0.325+0.000+0.010+0.011+0.303
Empirical Quantile Mapping-0.337+0.000+0.010+0.011-0.359
Gamma Quantile Mapping-0.365+0.000+0.010+0.003-0.378
Detrended Quantile Mapping+0.100+0.000+0.010+0.011+0.079
Quantile Delta Mapping-0.121+0.000+0.010+0.012-0.144
Scaled Distribution Mapping+0.977+0.000+0.003+0.027+0.947
CDF-transform+0.350+0.000+0.004+0.003+0.343
MVS with all-zero guard (variant)+0.000+0.000+0.000+0.000+0.000
ONDJ: contributions to the CRPS difference from Mean-Variance Scaling (mm)
MethodTotal Δ CRPSall 0 mm
0.0% of area		all ≤ 1 mm
0.0% of area		mixed
0.0% of area		always wet
100.0% of area
Linear Scaling-0.012+0.000+0.000+0.000-0.012
Variance Scaling (unbounded)+0.336+0.000+0.000+0.000+0.336
Square-root-space Mean-Variance Scaling-0.432+0.000+0.000+0.000-0.432
Power Transformation-0.028+0.000+0.000+0.000-0.028
Delta Change-1.044+0.000+0.000+0.000-1.044
Local Intensity Scaling+0.014+0.000+0.000+0.000+0.014
Empirical Quantile Mapping+0.779+0.000+0.000+0.000+0.779
Gamma Quantile Mapping-0.524+0.000+0.000+0.000-0.524
Detrended Quantile Mapping-0.086+0.000+0.000+0.000-0.086
Quantile Delta Mapping-0.218+0.000+0.000+0.000-0.218
Scaled Distribution Mapping+0.402+0.000+0.000+0.000+0.402
CDF-transform-0.179+0.000+0.000+0.000-0.179
MVS with all-zero guard (variant)+0.000+0.000+0.000+0.000+0.000

Confirmation: full pipeline and method selection

The screening isolates the amount correction. The confirmation evaluates what the site publishes: each method’s corrected members go through the full probability pipeline (tercile counts, smoothing and the climatology blend), with every fit and the blend weight inside the cross-validation folds. Two selection-aware procedures then choose a method for each left-out year using only the other training years — SEL-P by the lowest inner blended RPS (probabilities), SEL-A by the lowest inner CRPS (amounts) — so their scores include the effect of choosing among the 14 declared methods. ✓ = better than Mean-Variance Scaling with Holm-adjusted p < 0.05 (methods: across all methods and targets; SEL-P/SEL-A: across the targets) and not worse over 2017–2024. Scope: All Ethiopia.

ONDJ season through the full pipeline (outer leave-one-year-out 1993–2016; test years exploratory)
MethodBlended RPSS 1993–2016Blended RPSS 2017–2024BSS below 1993–2016BSS above 1993–2016Amount CRPSS 1993–2016Amount CRPSS 2017–2024Mean blend weight λ
Mean-Variance Scaling (operational)+0.039+0.075+0.028+0.051+0.070+0.1780.20
Linear Scaling+0.037+0.071+0.018+0.056+0.070+0.1310.28
Variance Scaling (unbounded)+0.039+0.074+0.029+0.050+0.063+0.1820.21
Square-root-space Mean-Variance Scaling+0.040+0.080+0.030+0.050+0.080+0.1890.18
Power Transformation+0.038+0.075+0.028+0.049+0.071+0.1880.19
Delta Change+0.026+0.029+0.016+0.035+0.093+0.1140.23
Local Intensity Scaling+0.037+0.071+0.018+0.056+0.070+0.1310.29
Empirical Quantile Mapping+0.028+0.062+0.016+0.040+0.053+0.1680.22
Gamma Quantile Mapping+0.041+0.082+0.031+0.050+0.082+0.1920.18
Detrended Quantile Mapping+0.020+0.045+0.007+0.032+0.072+0.1150.32
Quantile Delta Mapping+0.018+0.041+0.003+0.033+0.075+0.1180.31
Scaled Distribution Mapping+0.015+0.040-0.000+0.030+0.061+0.1080.38
CDF-transform+0.022+0.049+0.009+0.036+0.074+0.1180.32
MVS with all-zero guard (variant)+0.039+0.075+0.028+0.051+0.070+0.1780.20
SEL-P+0.040+0.082+0.030+0.050+0.081+0.192—
SEL-A+0.026+0.029+0.016+0.035+0.093+0.114—
Blended tercile RPSS by target, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Mean-Variance Scaling (operational)+0.013+0.040+0.003+0.001+0.039
Linear Scaling+0.013+0.038+0.006-0.000+0.037
Variance Scaling (unbounded)+0.013+0.041+0.004+0.001+0.039
Square-root-space Mean-Variance Scaling+0.014+0.050+0.007+0.007+0.040
Power Transformation+0.012+0.039+0.005+0.003+0.038
Delta Change+0.004+0.035-0.000-0.006+0.026
Local Intensity Scaling+0.012+0.043+0.006+0.001+0.037
Empirical Quantile Mapping+0.002+0.034+0.005-0.003+0.028
Gamma Quantile Mapping+0.014+0.044+0.006+0.005+0.041
Detrended Quantile Mapping+0.003+0.028+0.001-0.006+0.020
Quantile Delta Mapping+0.004+0.021+0.006-0.005+0.018
Scaled Distribution Mapping+0.004+0.028+0.006-0.002+0.015
CDF-transform+0.005+0.023-0.001-0.003+0.022
MVS with all-zero guard (variant)+0.013+0.040+0.003+0.001+0.039
SEL-P+0.014+0.050+0.007+0.007+0.040
SEL-A+0.004+0.050+0.002+0.007+0.026
Amount CRPSS by target in the confirmation design, and the selection-aware procedures
Amount CRPSS by target, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Mean-Variance Scaling (operational)-0.002+0.084+0.028-0.048+0.070
Linear Scaling-0.016+0.073-0.008-0.132+0.070
Variance Scaling (unbounded)-0.004+0.082+0.031-0.007+0.063
Square-root-space Mean-Variance Scaling+0.006+0.089+0.037+0.034 ✓+0.080
Power Transformation+0.001+0.082+0.020-0.025+0.071
Delta Change+0.021+0.085+0.026-0.047+0.093
Local Intensity Scaling-0.016+0.079+0.010-0.104+0.070
Empirical Quantile Mapping-0.012+0.075+0.033+0.010+0.053
Gamma Quantile Mapping+0.007+0.088+0.030+0.015 ✓+0.082
Detrended Quantile Mapping+0.005+0.075-0.001-0.065+0.072
Quantile Delta Mapping+0.010+0.076+0.014-0.027+0.075
Scaled Distribution Mapping-0.002+0.068-0.021-0.216+0.061
CDF-transform+0.002+0.073-0.019-0.108+0.074
MVS with all-zero guard (variant)-0.002+0.084+0.028-0.048+0.070
SEL-P+0.007+0.089+0.037+0.034+0.081
SEL-A+0.021+0.089+0.034+0.034 ✓+0.093
Selection-aware procedures against the operational method (Δ: RPS for SEL-P, CRPS for SEL-A; negative favours the procedure)
ProcedureTargetMethods chosen (1993–2016, years)Chosen for 2017–2024Δ score vs MVSHolm p (targets)Δ 2017–2024Confirmed
SEL-POctGamma Quantile Mapping 23, Square-root-space Mean-Variance Scaling 1Gamma Quantile Mapping-0.00071.0000+0.0025no
SEL-PNovSquare-root-space Mean-Variance Scaling 24Square-root-space Mean-Variance Scaling-0.00430.7338-0.0037no
SEL-PDecSquare-root-space Mean-Variance Scaling 24Square-root-space Mean-Variance Scaling-0.00211.0000+0.0009no
SEL-PJanSquare-root-space Mean-Variance Scaling 24Square-root-space Mean-Variance Scaling-0.00281.0000-0.0212no
SEL-PONDJGamma Quantile Mapping 18, Square-root-space Mean-Variance Scaling 6Gamma Quantile Mapping-0.00031.0000-0.0033no
SEL-AOctDelta Change 24Delta Change-0.67720.2275+0.9372no
SEL-ANovSquare-root-space Mean-Variance Scaling 24Square-root-space Mean-Variance Scaling-0.09310.2275-0.1049no
SEL-ADecSquare-root-space Mean-Variance Scaling 23, Empirical Quantile Mapping 1Square-root-space Mean-Variance Scaling-0.04240.2027-0.0334no
SEL-AJanSquare-root-space Mean-Variance Scaling 24Square-root-space Mean-Variance Scaling-0.47870.0018-0.6902yes
SEL-AONDJDelta Change 24Delta Change-1.04350.2275+3.2167no

MVS ratio-limit study

The operational correction limits the spread ratio r = σobs/σmodel to 0.5–2. This study runs the same screening and confirmation for that correction with wider limits (1/3–3, 1/4–4) and none, all with the same fallback (r = 1 where the model spread is below 1 mm), to see whether the limits are worth changing. ✓ = better than the operational limits with Holm-adjusted p < 0.05 across the 3 alternatives × 5 targets and not worse over the test years. Run at commit fb7425f.

Confirmed better than the operational limits — published (blended) probabilities: none; rainfall amounts: MVS, ratio limits 1/3-3 (Jan), MVS, ratio limits 1/4-4 (Jan) and Variance Scaling (unbounded) (Jan).

Blended tercile RPSS through the full pipeline, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Mean-Variance Scaling (operational, 0.5–2)+0.013+0.040+0.003+0.001+0.039
MVS, ratio limits 1/3-3+0.013+0.041+0.004+0.001+0.039
MVS, ratio limits 1/4-4+0.013+0.041+0.004+0.001+0.039
Variance Scaling (unbounded)+0.013+0.041+0.004+0.001+0.039
Amount CRPSS, leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Mean-Variance Scaling (operational, 0.5–2)-0.002+0.084+0.028-0.048+0.070
MVS, ratio limits 1/3-3-0.004+0.082+0.031-0.013 ✓+0.063
MVS, ratio limits 1/4-4-0.004+0.082+0.031-0.008 ✓+0.063
Variance Scaling (unbounded)-0.004+0.082+0.031-0.007 ✓+0.063
Where the operational limits act
Share of the corrected area, mean over the leave-one-year-out folds, 1993–2016
TargetLower limit active (r would be < 0.5)Upper limit active (r would be > 2)Spread–error, operationalSpread–error, no limits
Oct1%12%0.900.93
Nov5%9%1.011.02
Dec17%6%0.961.00
Jan26%28%0.831.00
ONDJ2%22%0.961.01

Figures

Figures show the national aggregate (All Ethiopia) or per-cell maps of the whole country, whatever region is selected for the tables.

Open full size

Interpretation — national aggregate (All Ethiopia)

  1. All tested corrections improve national ONDJ rainfall-amount CRPSS over raw ECMWF in 1993–2016 (raw -0.180; corrected +0.053 to +0.093). Performance varies for monthly tercile probabilities: 3 method–month combinations score below the raw model on RPSS (Detrended Quantile Mapping in Dec, Scaled Distribution Mapping in Dec and CDF-transform in Dec).
  2. Accounting for the whole search. With Holm adjustment across all 60 method–target comparisons, the amount (CRPS) screening passes are Square-root-space Mean-Variance Scaling (Jan) and Gamma Quantile Mapping (Jan), and the tercile (RPS) passes are Square-root-space Mean-Variance Scaling (Nov, Jan), Power Transformation (Jan) and Gamma Quantile Mapping (Jan). Within-method passes that do not survive this check: Empirical Quantile Mapping Jan, Gamma Quantile Mapping ONDJ, Square-root-space Mean-Variance Scaling Dec, Square-root-space Mean-Variance Scaling ONDJ and Variance Scaling (unbounded) Jan.
  3. Square-root-space Mean-Variance Scaling is at least as good as Mean-Variance Scaling in the national aggregate for every tested target and period, on amounts and unblended terciles (ONDJ CRPSS +0.080 vs +0.070 in 1993–2016, +0.189 vs +0.178 in 2017–2024). Its ONDJ amount screening pass does not survive the family-wide correction. Members clipped to 0 mm in the months: 4.5%–12.2% with Mean-Variance Scaling, 1.0%–2.9% with this method. This need not hold in each region; see the tables for the ONDJ R3 (Deyr) domain.
  4. The 0.5–2 limits of Mean-Variance Scaling matter most where the raw model is far too narrow (Jan). With the same 1 mm fallback in both, removing the limits is the only difference: there the upper limit is active over 28% of the area and Mean-Variance Scaling stays too narrow (spread–error 0.83), while the unbounded form reaches 1.00.
  5. Empirical Quantile Mapping reproduces the CHIRPS distribution and, with the dry policy, its dry share (totals ≤ 1 mm: Dec 19.0% vs CHIRPS 18.9%; Jan 16.7% vs CHIRPS 16.6%); it passes the screening for Jan, but its ONDJ amount skill is lower than Mean-Variance Scaling (+0.053 vs +0.070). A possible explanation is sampling noise in an empirical quantile function built from 23 CHIRPS values per fold; the results do not isolate it.
  6. Scaled Distribution Mapping follows the published algorithm (Switanek et al. 2017, steps 1–7; checked numerically against an independent implementation of the steps). Where the model is drier than CHIRPS it cannot add rain: in Jan 46% of model members exceed 1 mm against 83% of CHIRPS years, and SDM, by design, keeps the model’s number of rain members, so its corrected dry share is 55% (CHIRPS 17%).
  7. Delta Change has a high ONDJ CRPSS in 1993–2016 (+0.093) but the gain does not pass the screening (Holm p = 0.38), it is lower in 2017–2024 (+0.114 vs +0.178), its ensemble is too wide (spread–error 1.16) and its tercile skill is low. It is the CHIRPS climatology rescaled by the forecast’s mean: historical scenarios, not corrected members.
  8. The methods that use the forecast’s own distribution (Detrended Quantile Mapping, Quantile Delta Mapping, Scaled Distribution Mapping, CDF-transform) score lower on ONDJ terciles (-0.005 to +0.011 vs +0.035) and none passes the season screening. A possible explanation is that the distribution of a single forecast’s 25–51 members is a noisy estimate of the “future” distribution these methods were designed for; the results do not isolate the cause. Where model quantiles are dry, QDM falls back to the CHIRPS quantile (members affected: Dec 32% and Jan 54%), so its quantile changes are only partly preserved.
  9. Linear Scaling and Local Intensity Scaling correct the mean only (LOCI nearly so for totals that are always wet), so the ensemble stays too narrow (spread–error 0.78 and 0.77 for ONDJ).
  10. Dec remains hard: every method has negative amount skill over 2017–2024.
  11. No tested method demonstrates a statistically significant improvement over Mean-Variance Scaling for the published tercile probabilities under this validation protocol, although several have slightly higher point estimates. Choosing the method each year from training-only scores, the probability procedure (SEL-P) scores ONDJ blended RPSS +0.040 against +0.039, confirmed for none. Amounts: the amount procedure (SEL-A) scores CRPSS +0.093 against +0.070, confirmed for Jan; fixed methods confirmed against the family of all declared methods and targets: Square-root-space Mean-Variance Scaling (Jan) and Gamma Quantile Mapping (Jan).
  12. The climatology blend narrows the differences. Across the methods, ONDJ RPSS (1993–2016) spans 0.025 after the blend against 0.044 for the unblended screening counts. A better amount correction is trusted more by the blend: in Jan the fitted weight on climatology is 0.60 with Square-root-space Mean-Variance Scaling against 0.91 with Mean-Variance Scaling.
  13. Choosing by training scores can backfire. For Oct, SEL-A chose Delta Change from the 1993–2016 scores; over 2017–2024 it has a worse mean CRPS (Δ +0.937 mm, 95% interval -0.229 to +2.405 mm; better than Mean-Variance Scaling in 4 of 8 years, with the largest losses in 2019 and 2023); for ONDJ, SEL-A chose Delta Change from the 1993–2016 scores; over 2017–2024 it has a worse mean CRPS (Δ +3.217 mm, 95% interval -1.217 to +9.206 mm; better than Mean-Variance Scaling in 5 of 8 years, with the largest losses in 2019 and 2023); for Oct, SEL-P chose Gamma Quantile Mapping from the 1993–2016 scores; over 2017–2024 it has a worse mean RPS (Δ +0.003, 95% interval -0.001 to +0.006; better than Mean-Variance Scaling in 4 of 8 years, with the largest losses in 2024 and 2019); for Dec, SEL-P chose Square-root-space Mean-Variance Scaling from the 1993–2016 scores; over 2017–2024 it has a worse mean RPS (Δ +0.001, 95% interval -0.012 to +0.012; better than Mean-Variance Scaling in 2 of 8 years, with the largest losses in 2023 and 2021). These choices fail the no-harm rule; where the interval includes 0 they do not establish a significant deterioration. A method that ranks first on the training years is not, on that alone, a replacement.
  14. Where the confirmed amount gains come from (climate groups from each fold’s training years; contributions add up to the national difference from Mean-Variance Scaling): Square-root-space Mean-Variance Scaling in Jan (Δ CRPS -0.479 mm): all ≤ 1 mm -0.003 mm (1%; 9% of the area), mixed -0.018 mm (4%; 11% of the area), always wet -0.458 mm (96%; 80% of the area); Gamma Quantile Mapping in Jan (Δ CRPS -0.365 mm): all ≤ 1 mm +0.010 mm (-3%; 9% of the area), mixed +0.003 mm (-1%; 11% of the area), always wet -0.378 mm (104%; 80% of the area).
  15. No cell of All Ethiopia has exactly 0 mm in every training year, for any target, at 0.25° with CHIRPS. The all-zero guard (a separate MVS variant, outside the declared family) therefore never acts and equals Mean-Variance Scaling: the non-zero totals that Mean-Variance Scaling gives where every training year is 0 mm (its spread ratio cannot fall below 0.5) do not arise in these data. The guard remains a documented option; a finite history of zero rainfall does not make rain impossible.
Caveats
  • Screening skill is from 1993–2016 leave-one-year-out (clean) and 2017–2024 (exploratory: these years were seen while the operational method was chosen).
  • Screening tercile scores use smoothed member counts without the climatology blend; the confirmation scores the blended product.
  • Two multiplicity families are shown: within one method across its targets (✓) and across all methods and targets (✓✓). The confirmation’s selection-aware procedures include the method choice in the evaluation itself.
  • Scores are area-weighted over many correlated cells but tested year by year, so each year counts once.
  • The ordinary CRPS scores the delivered ensemble; the fair CRPS is a diagnostic under a sampling interpretation that DC’s rescaled years and methods tied to the forecast mean or ranks only approximate.
  • Results are national aggregates unless a table says otherwise; a method can rank differently in a region (switch the tables to the regional view).

Downloads, provenance and code

Run provenance
Run provenance (full hashes in benchmark.json and manifest.json)
ItemFingerprint
Commitbed63b906c77cfb3450514d7034ec5f68d067bc0
Configurationconfig/bias_correction/sep_ondj.json sha256 3c5c2ef194f52971…
Project configurationconfig/ondj/project.json sha256 e9ad8cfb30b21a73…
Maskscells: 57ad8861c598abb9…; ethiopia: 57ad8861c598abb9…; ondj_domain: 336c572d69af3c71…
InputsOct: 65 files, 0d2561be8800f597…; Nov: 65 files, 73d9d400dfa5457a…; Dec: 65 files, df5a657046e479f4…; Jan: 65 files, fc51a4c4480bdc0b…; ONDJ: 65 files, a8ba572412dc96ae…
Codebias_methods.py 03b2fdd8a645…; bias_benchmark.py b2cbc2834937…; bias_confirmation.py f2fd9f9c0a73…; significance.py 3143761d5d20…; calibration_core.py f0635cc563e6…; compare_calibration.py 9aa137c9a026…; local_blend.py ada5c571fb77…; run_calibration.py 1f34e8fc92fd…; run_monthly.py dc13dfb54b78…; cycle.py 00e076fc93dc…; common.py 72b83f5c5300…
Scoring version3
SoftwarePython 3.11.9, NumPy 1.26.4
References
  1. Boé, J., Terray, L., Habets, F., & Martin, E. (2007). Statistical and dynamical downscaling of the Seine basin climate for hydro-meteorological studies. International Journal of Climatology, 27, 1643–1655. doi:10.1002/joc.1602
  2. Cannon, A. J., Sobie, S. R., & Murdock, T. Q. (2015). Bias correction of GCM precipitation by quantile mapping: how well do methods preserve changes in quantiles and extremes? Journal of Climate, 28, 6938–6959. doi:10.1175/JCLI-D-14-00754.1
  3. Delle Monache, L., Hacker, J. P., Zhou, Y., Deng, X., & Stull, R. B. (2006). Probabilistic aspects of meteorological and ozone regional ensemble forecasts. Journal of Geophysical Research: Atmospheres, 111, D24307. doi:10.1029/2005JD006917
  4. Ferro, C. A. T. (2014). Fair scores for ensemble forecasts. Quarterly Journal of the Royal Meteorological Society, 140, 1917–1923. doi:10.1002/qj.2270
  5. Gudmundsson, L., Bremnes, J. B., Haugen, J. E., & Engen-Skaugen, T. (2012). Technical Note: Downscaling RCM precipitation to the station scale using statistical transformations — a comparison of methods. Hydrology and Earth System Sciences, 16, 3383–3390. doi:10.5194/hess-16-3383-2012
  6. Hay, L. E., Wilby, R. L., & Leavesley, G. H. (2000). A comparison of delta change and downscaled GCM scenarios for three mountainous basins in the United States. JAWRA Journal of the American Water Resources Association, 36, 387–397. doi:10.1111/j.1752-1688.2000.tb04276.x
  7. Johnson, S. J., Stockdale, T. N., Ferranti, L., et al. (2019). SEAS5: the new ECMWF seasonal forecast system. Geoscientific Model Development, 12, 1087–1117. doi:10.5194/gmd-12-1087-2019
  8. Leander, R., & Buishand, T. A. (2007). Resampling of regional climate model output for the simulation of extreme river flows. Journal of Hydrology, 332, 487–496. doi:10.1016/j.jhydrol.2006.08.006
  9. Lenderink, G., Buishand, A., & van Deursen, W. (2007). Estimates of future discharges of the river Rhine using two scenario methodologies: direct versus delta approach. Hydrology and Earth System Sciences, 11, 1145–1159. doi:10.5194/hess-11-1145-2007
  10. Manzanas, R., Gutiérrez, J. M., Bhend, J., et al. (2019). Bias adjustment and ensemble recalibration methods for seasonal forecasting: a comprehensive intercomparison using the C3S dataset. Climate Dynamics, 53, 1287–1305. doi:10.1007/s00382-019-04640-4
  11. Michelangeli, P.-A., Vrac, M., & Loukos, H. (2009). Probabilistic downscaling approaches: application to wind cumulative distribution functions. Geophysical Research Letters, 36, L11708. doi:10.1029/2009GL038401
  12. Piani, C., Weedon, G. P., Best, M., et al. (2010). Statistical bias correction of global simulated daily precipitation and temperature for the application of hydrological models. Journal of Hydrology, 395, 199–215. doi:10.1016/j.jhydrol.2010.10.024
  13. Schmidli, J., Frei, C., & Vidale, P. L. (2006). Downscaling from GCM precipitation: a benchmark for dynamical and statistical downscaling methods. International Journal of Climatology, 26, 679–689. doi:10.1002/joc.1287
  14. Switanek, M. B., Troch, P. A., Castro, C. L., et al. (2017). Scaled distribution mapping: a bias correction method that preserves raw climate model projected changes. Hydrology and Earth System Sciences, 21, 2649–2666. doi:10.5194/hess-21-2649-2017
  15. Teutschbein, C., & Seibert, J. (2012). Bias correction of regional climate model simulations for hydrological climate-change impact studies: review and evaluation of different methods. Journal of Hydrology, 456–457, 12–29. doi:10.1016/j.jhydrol.2012.05.052
  16. Themeßl, M. J., Gobiet, A., & Leuprecht, A. (2011). Empirical-statistical downscaling and error correction of daily precipitation from regional climate models. International Journal of Climatology, 31, 1530–1544. doi:10.1002/joc.2168
  17. Torralba, V., Doblas-Reyes, F. J., MacLeod, D., Christel, I., & Davis, M. (2017). Seasonal climate prediction: a new source of information for the management of wind energy resources. Journal of Applied Meteorology and Climatology, 56, 1231–1247. doi:10.1175/JAMC-D-16-0204.1
  18. Vrac, M., Drobinski, P., Merlo, A., et al. (2012). Dynamical and statistical downscaling of the French Mediterranean climate: uncertainty assessment. Natural Hazards and Earth System Sciences, 12, 2769–2784. doi:10.5194/nhess-12-2769-2012

DOIs checked against Crossref, 11 October 2026.

Probability and rainfall-distribution calibration

Can a different probability method improve the tercile probabilities users receive? The published pipeline is MVS-corrected members → smoothed member counts → climatology blend (the incumbent). Phase 1 keeps the amount correction fixed (Pathway A) and replaces only the probability stage: every method starts from the same smoothed, unblended probabilities, except the logistic models, which use declared ensemble predictors (standardized ensemble mean and log spread ratio). Every parameter — the blend weight, the temperature, the Dirichlet map, the logistic coefficients — is fitted only on inner out-of-fold years within each left-out year, and a selection-aware procedure (SEL) chooses the method for each year by inner cross-validation. Later phases add binary-calibration extensions and full rainfall-distribution models (Pathway B). The published forecasts are unchanged.

Implementation plan (the order is a plan, not a ranking of performance)
PhaseIdentifierMethodFamilyOutputStatus
1pseudocountPseudocount smoothingbaseline / stabilizationtercile probabilitiesimplemented, results below
1climatology_blendClimatology blending (incumbent)baseline / shrinkagetercile probabilitiesimplemented, results below
1temperatureTemperature scalingtercile calibrationtercile probabilitiesimplemented, results below
1dirichletRegularized Dirichlet calibrationtercile calibrationtercile probabilitiesimplemented, results below
1multinomial_logisticRegularized multinomial logistic regressiontercile predictiontercile probabilitiesimplemented, results below
1ordinal_logisticOrdinal (cumulative-logit) logistic regressiontercile predictiontercile probabilitiesimplemented, results below
2platt_ovrPlatt (sigmoid) calibration, one-vs-rest + normalizationbinary calibrationtercile probabilitiesimplemented, results below
2beta_ovrBeta calibration, one-vs-rest + normalizationbinary calibrationtercile probabilitiesimplemented, results below
2isotonic_ovrIsotonic regression, one-vs-rest + normalizationbinary calibrationtercile probabilitiesimplemented, results below
2emos_csgEMOS, censored shifted gammarainfall distributionCDF, quantiles, tercilesimplemented, results below
2extended_logisticExtended logistic regressionrainfall distributionCDF, tercilesimplemented, results below
2quantile_regressionRegularized quantile regressionrainfall quantilesquantiles (CDF by interpolation), tercilesimplemented, results below
3bjpBayesian joint probabilityrainfall distributionCDF, quantiles, tercilesplanned
3bma_precipitationBayesian model averaging (precipitation)rainfall distributionCDF, quantiles, tercilesplanned
3quantile_regression_forestQuantile regression forestrainfall quantilesconditional distribution, tercilesplanned
Definitions kept distinct
  • Pseudocount smoothing pk = (nk + α)/(M + 3α) stabilizes count probabilities; the climatology blend p = (1 − λ)·psmooth + λ·pclim learns how much forecast information to keep. Both are kept as references.
  • Dirichlet calibration (a regularized linear map of the log probabilities, then softmax) is not pseudocount smoothing.
  • Dirichlet uses only the log probabilities; multinomial logistic uses only the ensemble predictors, so they are different models. Ordinal logistic uses the same predictors with ordered cutpoints (below → near → above).
  • All methods are pooled over the grid cells of a target with equal weight per year; a method that cannot be fitted falls back to the incumbent and the fallback is counted.

September initialization · ONDJ and its months

Confirmed better than the incumbent climatology blend for the published tercile probabilities (RPS): none. ✓ = better than the incumbent with Holm-adjusted p < 0.05 across every method and target (SEL: across the targets) and not worse over 2017–2024. Amount correction fixed: MVS. National aggregate (All Ethiopia). Run at commit 0849f44.

Tercile RPSS, outer leave-one-year-out 1993–2016 (bold: incumbent)
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.013+0.040+0.003+0.001+0.039
Pseudocount smoothing (no blend)-0.009+0.044-0.068-0.213+0.035
Temperature scaling+0.016+0.027+0.004-0.031+0.043
Regularized Dirichlet calibration+0.010+0.050+0.010+0.007+0.037
Regularized multinomial logistic regression-0.003+0.062+0.008-0.016+0.034
Ordinal (cumulative-logit) logistic regression-0.003+0.063+0.007-0.014+0.037
SEL (method chosen each year by inner cross-validation)+0.016+0.051+0.002+0.001+0.043
Tercile RPSS, fit 1993–2016, scored 2017–2024 (exploratory)
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.027+0.053+0.015-0.005+0.075
Pseudocount smoothing (no blend)+0.012+0.041-0.049-0.300+0.065
Temperature scaling+0.027+0.042+0.007-0.098+0.078
Regularized Dirichlet calibration+0.012+0.015+0.010+0.057+0.069
Regularized multinomial logistic regression-0.005+0.060+0.027+0.031+0.064
Ordinal (cumulative-logit) logistic regression-0.004+0.048+0.023+0.032+0.063
SEL (method chosen each year by inner cross-validation)+0.027+0.060+0.015-0.005+0.078
Log loss, Brier skill, tests, fitted parameters and the selection
Log-loss skill against climatology, 1993–2016
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.015+0.021-0.001+0.001+0.026
Pseudocount smoothing (no blend)-0.008-0.010-0.197-0.529+0.018
Temperature scaling+0.015+0.017-0.003-0.039+0.029
Regularized Dirichlet calibration+0.013+0.034+0.008+0.004+0.025
Regularized multinomial logistic regression-0.002+0.040+0.007-0.011+0.020
Ordinal (cumulative-logit) logistic regression-0.002+0.040+0.003-0.008+0.024
SEL (method chosen each year by inner cross-validation)+0.015+0.030-0.005+0.001+0.029
Brier skill, below normal, 1993–2016
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.013+0.003-0.013-0.002+0.028
Pseudocount smoothing (no blend)-0.011-0.013-0.138-0.331+0.022
Temperature scaling+0.015+0.004-0.014-0.069+0.031
Regularized Dirichlet calibration+0.013+0.024-0.003+0.002+0.026
Regularized multinomial logistic regression-0.004+0.033-0.008-0.011+0.022
Ordinal (cumulative-logit) logistic regression-0.003+0.034-0.010-0.012+0.024
SEL (method chosen each year by inner cross-validation)+0.015+0.020-0.010-0.002+0.031
Brier skill, above normal, 1993–2016
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.013+0.078+0.019+0.003+0.051
Pseudocount smoothing (no blend)-0.006+0.101+0.003-0.095+0.049
Temperature scaling+0.017+0.050+0.023+0.007+0.054
Regularized Dirichlet calibration+0.007+0.076+0.024+0.011+0.048
Regularized multinomial logistic regression-0.003+0.092+0.023-0.020+0.045
Ordinal (cumulative-logit) logistic regression-0.002+0.091+0.023-0.016+0.050
SEL (method chosen each year by inner cross-validation)+0.017+0.082+0.014+0.003+0.054
Each method against the incumbent (negative Δ favours the method)
MethodTargetΔ RPS 1993–2016Years betterHolm pΔ RPS 2017–2024Years betterConfirmed
Pseudocount smoothing (no blend)Oct+0.009712/241.0000+0.00665/8no
Pseudocount smoothing (no blend)Nov-0.001711/241.0000+0.00572/8no
Pseudocount smoothing (no blend)Dec+0.03195/241.0000+0.02962/8no
Pseudocount smoothing (no blend)Jan+0.09686/241.0000+0.13271/8no
Pseudocount smoothing (no blend)ONDJ+0.001810/241.0000+0.00453/8no
Temperature scalingOct-0.001311/241.0000-0.00002/8no
Temperature scalingNov+0.005912/241.0000+0.00493/8no
Temperature scalingDec-0.000715/241.0000+0.00383/8no
Temperature scalingJan+0.014511/241.0000+0.04194/8no
Temperature scalingONDJ-0.001414/241.0000-0.00165/8no
Regularized Dirichlet calibrationOct+0.001311/241.0000+0.00632/8no
Regularized Dirichlet calibrationNov-0.004414/241.0000+0.01712/8no
Regularized Dirichlet calibrationDec-0.003415/241.0000+0.00273/8no
Regularized Dirichlet calibrationJan-0.002713/241.0000-0.02755/8no
Regularized Dirichlet calibrationONDJ+0.001114/241.0000+0.00253/8no
Regularized multinomial logistic regressionOct+0.00727/241.0000+0.01390/8no
Regularized multinomial logistic regressionNov-0.010116/241.0000-0.00314/8no
Regularized multinomial logistic regressionDec-0.002314/241.0000-0.00556/8no
Regularized multinomial logistic regressionJan+0.00756/241.0000-0.01584/8no
Regularized multinomial logistic regressionONDJ+0.002611/241.0000+0.00513/8no
Ordinal (cumulative-logit) logistic regressionOct+0.00697/241.0000+0.01320/8no
Ordinal (cumulative-logit) logistic regressionNov-0.010216/241.0000+0.00244/8no
Ordinal (cumulative-logit) logistic regressionDec-0.001716/241.0000-0.00364/8no
Ordinal (cumulative-logit) logistic regressionJan+0.00677/241.0000-0.01634/8no
Ordinal (cumulative-logit) logistic regressionONDJ+0.001111/241.0000+0.00524/8no
SELOct-0.001311/241.0000 (targets)-0.00002/8no
SELNov-0.004914/241.0000 (targets)-0.00314/8no
SELDec+0.00039/241.0000 (targets)+0.00000/8no
SELJan+0.00000/241.0000 (targets)+0.00000/8no
SELONDJ-0.001414/241.0000 (targets)-0.00165/8no
Fitted parameters (fit on 1993–2016) and the selection-aware choices
TargetBlend weight λTemperature TOrdinal cutpointsOrdinal slopes (mean, log spread)SEL choices (1993–2016 folds)Fallbacks
Oct0.452.32-0.60, +0.70+0.99, -0.18Temperature scaling 240
Nov0.312.73-0.63, +0.74+1.93, -0.18Regularized multinomial logistic regression 12, Ordinal (cumulative-logit) logistic regression 9, Pseudocount smoothing (no blend) 2, Climatology blend (incumbent) 10
Dec0.704.00-0.63, +0.69+0.97, +0.03Temperature scaling 12, Climatology blend (incumbent) 11, Regularized multinomial logistic regression 10
Jan0.904.00-0.61, +0.67+0.56, -0.04Climatology blend (incumbent) 240
ONDJ0.211.52-0.68, +0.69+1.39, -0.04Temperature scaling 240
Probability calibration skill summary
RPSS and log-loss skill by method and target.
Reliability by method
Reliability of each method for the season (below and above normal).

Phase 2 · binary calibrators (one-vs-rest, normalized)

Platt, beta and isotonic calibration of each category against the rest, divided by their sum inside the pipeline, against the incumbent (their own family: Holm across these methods and the targets). Confirmed better: none.

Tercile RPSS, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.013+0.040+0.003+0.001+0.039
Platt (sigmoid) calibration, one-vs-rest + normalization+0.015+0.049+0.013+0.006+0.042
Beta calibration, one-vs-rest + normalization+0.013+0.050+0.010+0.008+0.039
Isotonic regression, one-vs-rest + normalization+0.009+0.049+0.008-0.039+0.039
SEL (chosen each year)+0.013+0.044+0.002+0.002+0.039
Test years
Tercile RPSS, fit 1993–2016, scored 2017–2024 (exploratory)
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.027+0.053+0.015-0.005+0.075
Platt (sigmoid) calibration, one-vs-rest + normalization+0.004+0.020+0.018+0.021+0.065
Beta calibration, one-vs-rest + normalization+0.011+0.012+0.017+0.035+0.067
Isotonic regression, one-vs-rest + normalization+0.013+0.017+0.011-0.303+0.070
SEL (chosen each year)+0.027+0.053+0.015-0.005+0.075

Phase 2 · rainfall-distribution models (Pathway B)

Each model predicts a full rainfall distribution from the raw ensemble (pooled over cells in units relative to each cell's climatology); terciles at the training CHIRPS boundaries, quantiles and intervals come from it. References, in the same folds: the published probabilities (tercile RPS) and the MVS-corrected ensemble (CRPS, quantiles). ✓ = better than the reference with Holm-adjusted p < 0.05 across the models and targets, and not worse over 2017–2024. Confirmed better — tercile probabilities: none; rainfall amounts (CRPS): none.

Tercile RPSS, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Published probabilities (incumbent)+0.013+0.040+0.003+0.001+0.039
EMOS, censored shifted gamma+0.003+0.037-0.113-0.060+0.029
Heteroscedastic extended logistic regression-0.000+0.032-0.076-0.040+0.030
Regularized quantile regression-0.010+0.017-0.143-0.064+0.019
Amount CRPSS against the CHIRPS climatology, 1993–2016
MethodOctNovDecJanONDJ
MVS-corrected ensemble-0.002+0.084+0.029-0.040+0.070
EMOS, censored shifted gamma+0.016+0.071+0.015-0.029+0.049
Heteroscedastic extended logistic regression-0.011+0.058-0.017-0.023+0.062
Regularized quantile regression-0.010+0.056+0.033-0.036+0.032
Interval coverage, median pinball loss, test years and the model definitions
Share of observations inside the 10–90% interval, 1993–2016 (80% is calibrated)
MethodOctNovDecJanONDJ
MVS-corrected ensemble72%75%76%46%75%
EMOS, censored shifted gamma77%78%84%80%79%
Heteroscedastic extended logistic regression90%89%92%89%83%
Regularized quantile regression77%77%79%77%76%
Pinball loss of the median (mm), 1993–2016 (lower is better)
MethodOctNovDecJanONDJ
MVS-corrected ensemble21.0012.025.055.4128.80
EMOS, censored shifted gamma20.7412.265.125.6429.29
Heteroscedastic extended logistic regression21.0112.455.205.5929.04
Regularized quantile regression21.1012.285.105.5229.28
Fit 1993–2016, scored 2017–2024 (exploratory)
MethodOctNovDecJanONDJ
Published probabilities / MVS ensemble+0.027 / +0.059+0.053 / +0.158+0.016 / -0.033-0.004 / -0.066+0.075 / +0.178
EMOS, censored shifted gamma (RPSS / CRPSS)-0.011 / +0.018+0.044 / +0.132-0.180 / -0.035-0.074 / -0.033+0.060 / +0.146
Heteroscedastic extended logistic regression (RPSS / CRPSS)+0.036 / +0.021+0.003 / +0.135-0.097 / -0.116-0.030 / -0.031+0.043 / +0.117
Regularized quantile regression (RPSS / CRPSS)+0.004 / +0.021-0.031 / +0.134-0.191 / +0.007-0.037 / +0.014+0.012 / +0.134
Models
ModelEstimatesPredictorsFittingOutputs
EMOS, censored shifted gammarainfall distribution with a probability of zeroensemble mean m' and mean absolute difference md' (relative)maximum likelihood, pooled over cellsCDF, quantiles, tercile probabilities
Heteroscedastic extended logistic regressioncoherent CDF across all thresholdssqrt of the ensemble mean m', log spread s' (relative)log loss over each cell’s CHIRPS deciles, pooledCDF, quantiles, tercile probabilities
Regularized quantile regressionconditional quantiles at 19 levels (5–95%), CDF by interpolationensemble mean m' and spread s' (relative)pinball loss per level (Huber-smoothed), L2 1e-4, pooledCDF, quantiles, tercile probabilities
Distribution models summary
Tercile RPSS and amount CRPSS of the distribution models against the published pipeline.

Full write-up: docs/48_PROBABILITY_CALIBRATION.md. Code: scripts/prob_calibration.py, scripts/prob_benchmark.py.

References
  1. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 1321–1330. proceedings.mlr.press/v70/guo17a.html
  2. Kull, M., Perello-Nieto, M., Kängsepp, M., Silva Filho, T., Song, H., & Flach, P. (2019). Beyond temperature scaling: obtaining well-calibrated multiclass probabilities with Dirichlet calibration. Advances in Neural Information Processing Systems 32 (NeurIPS 2019). arxiv.org/abs/1910.12656
  3. McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society, Series B, 42, 109–127. doi.org/10.1111/j.2517-6161.1980.tb01109.x
  4. Rajagopalan, B., Lall, U., & Zebiak, S. E. (2002). Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Monthly Weather Review, 130, 1792–1811. doi.org/10.1175/1520-0493(2002)130<1792:CCFTRA>2.0.CO;2
  5. Wilks, D. S. (2019). Statistical Methods in the Atmospheric Sciences (4th ed.). Elsevier.
  6. Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers (pp. 61–74). MIT Press.
  7. Kull, M., Silva Filho, T., & Flach, P. (2017). Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers. Proceedings of AISTATS 2017, PMLR 54. proceedings.mlr.press/v54/kull17a.html
  8. Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. Proceedings of the 8th ACM SIGKDD, 694–699. doi.org/10.1145/775047.775151
  9. Scheuerer, M., & Hamill, T. M. (2015). Statistical postprocessing of ensemble precipitation forecasts by fitting censored, shifted gamma distributions. Monthly Weather Review, 143, 4578–4596. doi.org/10.1175/MWR-D-15-0061.1
  10. Wilks, D. S. (2009). Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteorological Applications, 16, 361–368. doi.org/10.1002/met.134
  11. Messner, J. W., Mayr, G. J., Zeileis, A., & Wilks, D. S. (2014). Heteroscedastic extended logistic regression for postprocessing of ensemble guidance. Monthly Weather Review, 142, 448–456. doi.org/10.1175/MWR-D-13-00271.1
  12. Koenker, R., & Bassett, G. (1978). Regression quantiles. Econometrica, 46, 33–50. doi.org/10.2307/1913643

Rainfall climatology and domains

Where does a season's rain matter? Before a forecast domain is chosen, this review computes the observed climatology (seasonal total, annual total and the season's share), compares it with EMI's published long-term maps, and tests every combination of rainfall and annual-share cutoffs. Cutoffs are chosen for what the domain means climatologically, never to improve a forecast score.

Methods & data

In brief

Each cycle starts from the ECMWF SEAS5 seasonal ensemble issued on the 1st of the initialization month. For every 0.25° grid cell in Ethiopia the forecast rainfall is corrected toward the CHIRPS observed mean and variability, then turned into tercile probabilities (below, near or above normal relative to the cycle's CHIRPS reference period). The probabilities are finally blended with climatology using one weight per target, chosen to give the best past scores. Climatology assigns approximately one-third probability to each tercile; the calculations use the observed category frequencies of the training reference period.

Probabilities and amounts are produced separately: the probability skill (RPSS) on this site refers to the blended probabilities, and the rainfall amount skill (CRPSS) to the amount-corrected ensemble.

The raw benchmark is the uncorrected model: raw ECMWF members counted against each cell's CHIRPS tercile limits, with no correction, smoothing or blend. For May-initialized JJAS over 2017–2025 it scores RPSS −0.44, far below climatology, which is why calibration is needed.

Data

Data sources
SourceDetails
ECMWF SEAS5Copernicus CDS seasonal-original-single-levels, system 51, total precipitation, 1993 to the forecast year; 25 members to 2016, 51 from 2017; 1° grid
CHIRPS v3.0monthly rainfall, official 0.05° Africa product averaged onto the 0.25° grid (5 × 5 cells); archive from 1993; verification months from the official monthly files, checked against the archive where they overlap. Funk et al. (2026), The Climate Hazards Center Infrared Precipitation with Stations, Version 3, Scientific Data 13, 718, doi:10.1038/s41597-026-07096-4.
AreasAll Ethiopia (1,484 cells) and, per season, the rainfall domains offered in the area selector — FMAM: FMAM rainfall contribution domain; FMAM main-season rainfall domain (Belg / Gu / Ganna, >=40% of annual) · JJAS: JJAS R1+R2 rainfall domain · ONDJ: ONDJ R3 (Deyr) rainfall domain. Each definition is shown under the selector and in the rainfall-domain review. The domains are for display and summaries only; calibration uses every cell.
CHIRPS v2.0 vs v3.0 (why and what changed, 11 October 2026)

CHIRPS v3.0 (Funk et al. 2026) replaced v2.0 for calibration and verification. Over Ethiopia, 1993–2025, v3.0 is about 15% wetter (consistent with its gauge-undercatch correction) while each season's share of annual rainfall and the ranking of years agree closely, so tercile limits move up. The same method was refitted on v3.0: the decision gates give the same outcome, and skill is similar overall. Each version is scored against its own observations.

Climatology, 1993–2025
SeasonMean rainfall v2.0 → v3.0National series correlationSame tercile category
FMAM249 → 279 mm (+12%)0.97878%
JJAS441 → 518 mm (+17%)0.97371%
ONDJ141 → 163 mm (+16%)0.98478%
Calibration with each version
CycleCross-validated RPSS 1993–2016, v2.0 → v3.0Rainfall-amount CRPSS (development run), v2.0 → v3.0
May · JJAS+0.046 → +0.027+0.017 → -0.018
January · FMAM+0.027 → +0.032-0.001 → +0.016
February · FMAM+0.022 → +0.021+0.053 → +0.071
September · ONDJ+0.020 → +0.039+0.181 → +0.178
CHIRPS v2.0 and v3.0 climatology over Ethiopia
Annual and seasonal mean rainfall, CHIRPS v2.0 and v3.0, and the difference.
Workflow (eight scripted stages)
  1. 1

    Download

    ECMWF SEAS5 (system 51) daily accumulated precipitation from the Copernicus CDS for the initialization month, 1993 to the forecast year.

    download_seasonal_forecasts_daily_c3s.py
  2. 2

    Inspect

    Inventory every year: initialization, lead times, member counts and accumulation increments (GRIB packing tolerance 0.2 mm).

    inspect_inputs.py
  3. 3

    Prepare

    De-accumulate to daily totals, sum the season and each month; CHIRPS on the same dates.

    prepare_seasonal.py · run_monthly.py
  4. 4

    Regrid

    Each 1° model cell is copied into its 4×4 block of 0.25° CHIRPS cells (no interpolation).

    regrid_seasonal.py
  5. 5

    Calibrate

    Per-cell amount correction, smoothed tercile counts and one climatology-blend weight per target.

    final_shared_blend.py
  6. 6

    Evaluate

    Nested cross-validation 1993–2016; exploratory fixed-fit scores for later years; whole-year significance tests.

    local_blend.py · decision_gates.py
  7. 7

    Deliver

    Maps, summaries and an offline gallery; forecasts frozen with SHA-256 hashes before observations are used.

    run_operational.py · build_season_products.py
  8. 8

    Verify

    Official CHIRPS v3.0 monthly files, checked against the archive where they overlap, scored per month and season.

    prepare_verification_2026.py · verify_frozen_2026.py

Every stage is a script in scripts/; documents in docs/ record each step, issue and decision (runbook: docs/38; per-season guides: docs/40).

Equations
  1. Amount correction per cell, equal weight per year: x̂ = max(0, μobs + r·(x − μmodel)), r = clip(σobs/σmodel, 0.5, 2)
  2. Tercile probabilities from the corrected members against CHIRPS tercile limits, smoothed: (n + 0.5)/(M + 1.5)
  3. Climatology blend: pfinal = (1 − λ)·pmodel + λ·pclimatology, with one λ per target fitted on leave-one-year-out RPS.
    pclimatology is, for each grid cell, the observed frequency of each tercile in the training reference years (about one-third each; it differs slightly because of sample size, category limits and ties).
  4. Skill score: RPSS = 1 − RPSforecast / RPSclimatology; RPSS +0.05 means a 5% lower ranked probability score than climatology. CRPSS is the same for rainfall amounts.
Method and references

Rainfall amounts are corrected cell by cell with mean–variance scaling (also called simple bias adjustment or variance scaling; Teutschbein & Seibert 2012; Torralba et al. 2017; Manzanas et al. 2019): each member is shifted to the CHIRPS mean and its anomaly rescaled by the ratio of CHIRPS to model standard deviation (equations above). It corrects the mean and spread, not the shape of the distribution; it is not quantile mapping. The model mean and spread are pooled over all members and years, with equal weight per year (25 members to 2016, 51 from 2017); training years 1993–2025.

Probabilities are the share of corrected members in each CHIRPS tercile (ensemble relative frequency; Wilks 2019), smoothed so no category is 0 or 1, then blended with climatology. Weighting a model against climatology follows Rajagopalan et al. (2002) and Robertson et al. (2004); here one weight per target is fitted by minimising the ranked probability score of leave-one-year-out probabilities.

Scores: ranked probability score (Epstein 1969) and continuous ranked probability score (Hersbach 2000), as skill scores against climatology. Forecasts: ECMWF SEAS5 (Johnson et al. 2019); observations: CHIRPS v3.0 (Funk et al. 2026), v2.0 before 11 October 2026 (Funk et al. 2015).

Choices of this project, not taken from a single paper: the 0.5–2 limits on the spread ratio, mean-only correction where the model spread is below 1 mm, clipping at 0 mm (its weakness in dry months, issue I-08), equal weighting of years, the count smoothing and one blend weight shared by all cells. A square-root-space amount correction passed the gate for amounts and is recommended for the next cycle (docs/36). Details: docs/46. DOIs checked on Crossref, 11 October 2026.

  1. Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. Journal of Applied Meteorology, 8(6), 985–987. doi:10.1175/1520-0450(1969)008<0985:ASSFPF>2.0.CO;2
  2. Funk, C., et al. (2015). The climate hazards infrared precipitation with stations — a new environmental record for monitoring extremes. Scientific Data, 2, 150066. doi:10.1038/sdata.2015.66
  3. Funk, C., et al. (2026). The Climate Hazards Center Infrared Precipitation with Stations, Version 3. Scientific Data, 13, 718. doi:10.1038/s41597-026-07096-4
  4. Hersbach, H. (2000). Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting, 15(5), 559–570. doi:10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2
  5. Johnson, S. J., et al. (2019). SEAS5: the new ECMWF seasonal forecast system. Geoscientific Model Development, 12, 1087–1117. doi:10.5194/gmd-12-1087-2019
  6. Manzanas, R., et al. (2019). Bias adjustment and ensemble recalibration methods for seasonal forecasting: a comprehensive intercomparison using the C3S dataset. Climate Dynamics, 53, 1287–1305. doi:10.1007/s00382-019-04640-4
  7. Rajagopalan, B., Lall, U., & Zebiak, S. E. (2002). Categorical climate forecasts through regularization and optimal combination of multiple GCM ensembles. Monthly Weather Review, 130(7), 1792–1811. doi:10.1175/1520-0493(2002)130<1792:CCFTRA>2.0.CO;2
  8. Robertson, A. W., Lall, U., Zebiak, S. E., & Goddard, L. (2004). Improved combination of multiple atmospheric GCM ensembles for seasonal prediction. Monthly Weather Review, 132(12), 2732–2744. doi:10.1175/MWR2818.1
  9. Teutschbein, C., & Seibert, J. (2012). Bias correction of regional climate model simulations for hydrological climate-change impact studies: review and evaluation of different methods. Journal of Hydrology, 456–457, 12–29. doi:10.1016/j.jhydrol.2012.05.052
  10. Torralba, V., et al. (2017). Seasonal climate prediction: a new source of information for the management of wind energy resources. Journal of Applied Meteorology and Climatology, 56(5), 1231–1247. doi:10.1175/JAMC-D-16-0204.1
  11. Wilks, D. S. (2019). Statistical Methods in the Atmospheric Sciences (4th ed.). Elsevier.
How the method was chosen

A change is adopted only if it improves the cross-validated 1993–2016 scores (Holm-adjusted p < 0.05 over the five JJAS targets) and does not worsen 2017–2025. Also evaluated and not adopted: regularized Dirichlet recalibration and per-cell or regime-specific blend weights.

Method changes tested with whole-year significance gates (May-initialized JJAS targets; negative = better)
Candidate changeScore change 1993–2016Score change 2017–2025Decision
Fit the blend weight on Ethiopia only+0.0003 to +0.0009−0.0043 to +0.0001Not adopted
Square-root amount correction (probabilities)−0.0003 to +0.0008−0.0018 to −0.0002Not adopted
Remove the climatology blend+0.0194 to +0.0508−0.0115 to +0.0310Not adopted
Plain climatology−0.0002 to +0.0121+0.0038 to +0.0197Not adopted
Square-root amount correction (amounts, CRPS in mm)−0.13 to −0.06lower in all targetsPasses for all targets; candidate for the next cycle
Ensemble size and spread
Ensemble-size check, May-initialized JJAS targets, 2017–2025 (lower RPS is better)
TargetRPS, 51 membersRPS, 25-member subsets (mean)Spread / error (hindcast → 2017–2025)
JJAS0.44040.44200.68 → 0.75
Jun0.45980.46010.62 → 0.63
Jul0.44490.44670.71 → 0.76
Aug0.44050.44150.74 → 0.73
Sep0.45780.45870.75 → 0.76

Calibration parameters are fitted mostly on 25-member hindcasts and applied to 51-member forecasts. In 2017–2025 the 51-member forecasts scored about the same as, or slightly better than, 25-member subsets, so the transfer did not degrade scores in the years tested; this does not show it is neutral in every year or region. The corrected ensemble is under-dispersive (spread smaller than error), which the climatology blend partly offsets. Monthly and seasonal outlooks are calibrated separately: for JJAS 2026 over all Ethiopia the corrected monthly means sum to 432 mm against 430 mm for the season (+1.7 mm).

Quality control
  • Every forecast file is inventoried; re-downloaded years were bit-identical, confirming SEAS5 system 51.
  • Small negative daily increments (≤ 0.15 mm) are GRIB packing rounding; tolerated up to 0.2 mm and clipped, and the seasonal sum is checked.
  • CHIRPS sea cells (223) are always missing; no partial season enters any fit.
  • Verification observations are compared with the archive in the overlap year (tolerance 0.0001 mm/day) before use.
Limitations
  • Few evaluation years and single verification seasons give wide uncertainty; regional and monthly results are noisy.
  • The later evaluation years were inspected during method development; the cleanest evidence is the cross-validated 1993–2016 score.
  • The 0.25° maps repeat 1° model information; they add no dynamical detail.
  • All cycles are research reconstructions produced after their initialization dates, not official EMI or ICPAC forecasts.
Reproduce 
git clone https://github.com/YonSci/calibrated_seasonal_rainfall.git
python -m venv .venv && .venv\Scripts\activate && pip install -r requirements.txt
set CALIBRATION_CYCLE=config\cycles\sep_2026_ondj.json
python scripts\run_operational.py --config config\cycles\sep_2026_ondj.json --workflow all
python scripts\build_site.py

Raw ECMWF and CHIRPS data are not stored in the repository. Step-by-step guides: docs/40; project status: docs/36.

Downloads

Complete package

One ZIP file with the PDF bulletin, summary CSV, metadata, README, every map (forecast and verification), verification reports and the forecast NetCDF files for the selected cycle.

Release archive and change log: earlier published versions and what changed.

Individual maps: use Download PNG in the map viewer. Code, documentation and notebooks: GitHub repository.