Seasonal Rainfall Outlook Platform
Phase 2 · Probability calibration · September initialization · ONDJ and its months

Phase 2 · September initialization · ONDJ and its months

Binary calibrators (one-vs-rest, normalized)

Platt, beta and isotonic calibration of each category against the rest, divided by their sum inside the pipeline, against the incumbent (their own family: Holm across these methods and the targets). Confirmed better: none.

Tercile RPSS, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.013+0.040+0.003+0.001+0.039
Platt (sigmoid) calibration, one-vs-rest + normalization+0.015+0.049+0.013+0.006+0.042
Beta calibration, one-vs-rest + normalization+0.013+0.050+0.010+0.008+0.039
Isotonic regression, one-vs-rest + normalization+0.009+0.049+0.008-0.039+0.039
SEL (chosen each year)+0.013+0.044+0.002+0.002+0.039
Tercile RPSS, fit 1993–2016, scored 2017–2024 (exploratory)
MethodOctNovDecJanONDJ
Climatology blend (incumbent)+0.027+0.053+0.015-0.005+0.075
Platt (sigmoid) calibration, one-vs-rest + normalization+0.004+0.020+0.018+0.021+0.065
Beta calibration, one-vs-rest + normalization+0.011+0.012+0.017+0.035+0.067
Isotonic regression, one-vs-rest + normalization+0.013+0.017+0.011-0.303+0.070
SEL (chosen each year)+0.027+0.053+0.015-0.005+0.075

Rainfall-distribution models (Pathway B)

Each model predicts a full rainfall distribution from the raw ensemble (pooled over cells in units relative to each cell's climatology); terciles at the training CHIRPS boundaries, quantiles and intervals come from it. References, in the same folds: the published probabilities (tercile RPS) and the MVS-corrected ensemble (CRPS, quantiles). ✓ = better than the reference with Holm-adjusted p < 0.05 across the models and targets, and not worse over 2017–2024. Confirmed better — tercile probabilities: none; rainfall amounts (CRPS): none.

Tercile probabilities

Tercile RPSS, outer leave-one-year-out 1993–2016
MethodOctNovDecJanONDJ
Published probabilities (incumbent)+0.013+0.040+0.003+0.001+0.039
EMOS, censored shifted gamma+0.003+0.037-0.113-0.060+0.029
Heteroscedastic extended logistic regression-0.000+0.032-0.076-0.040+0.030
Regularized quantile regression-0.010+0.017-0.143-0.064+0.019

Rainfall amounts

Amount CRPSS against the CHIRPS climatology, 1993–2016
MethodOctNovDecJanONDJ
MVS-corrected ensemble-0.002+0.084+0.029-0.040+0.070
EMOS, censored shifted gamma+0.016+0.071+0.015-0.029+0.049
Heteroscedastic extended logistic regression-0.011+0.058-0.017-0.023+0.062
Regularized quantile regression-0.010+0.056+0.033-0.036+0.032

Intervals and quantiles

Share of observations inside the 10–90% interval, 1993–2016 (80% is calibrated)
MethodOctNovDecJanONDJ
MVS-corrected ensemble72%75%76%46%75%
EMOS, censored shifted gamma77%78%84%80%79%
Heteroscedastic extended logistic regression90%89%92%89%83%
Regularized quantile regression77%77%79%77%76%
Pinball loss of the median (mm), 1993–2016 (lower is better)
MethodOctNovDecJanONDJ
MVS-corrected ensemble21.0012.025.055.4128.80
EMOS, censored shifted gamma20.7412.265.125.6429.29
Heteroscedastic extended logistic regression21.0112.455.205.5929.04
Regularized quantile regression21.1012.285.105.5229.28

Test years

Fit 1993–2016, scored 2017–2024 (exploratory)
MethodOctNovDecJanONDJ
Published probabilities / MVS ensemble+0.027 / +0.059+0.053 / +0.158+0.016 / -0.033-0.004 / -0.066+0.075 / +0.178
EMOS, censored shifted gamma (RPSS / CRPSS)-0.011 / +0.018+0.044 / +0.132-0.180 / -0.035-0.074 / -0.033+0.060 / +0.146
Heteroscedastic extended logistic regression (RPSS / CRPSS)+0.036 / +0.021+0.003 / +0.135-0.097 / -0.116-0.030 / -0.031+0.043 / +0.117
Regularized quantile regression (RPSS / CRPSS)+0.004 / +0.021-0.031 / +0.134-0.191 / +0.007-0.037 / +0.014+0.012 / +0.134

Fits and model definitions

  • EMOS, censored shifted gamma: converged in every fold.
  • Heteroscedastic extended logistic regression: converged in every fold.
  • Regularized quantile regression: converged in every fold; quantile rearrangement needed for at most 0.9% of cells.
Models
ModelEstimatesPredictorsFittingOutputs
EMOS, censored shifted gammarainfall distribution with a probability of zeroensemble mean m' and mean absolute difference md' (relative)maximum likelihood, pooled over cellsCDF, quantiles, tercile probabilities
Heteroscedastic extended logistic regressioncoherent CDF across all thresholdssqrt of the ensemble mean m', log spread s' (relative)log loss over each cell’s CHIRPS deciles, pooledCDF, quantiles, tercile probabilities
Regularized quantile regressionconditional quantiles at 19 levels (5–95%), CDF by interpolationensemble mean m' and spread s' (relative)pinball loss per level (Huber-smoothed), L2 1e-4, pooledCDF, quantiles, tercile probabilities
Tercile RPSS and amount CRPSS of the Phase 2 distribution models against the published pipeline.
Tercile RPSS and amount CRPSS of the Phase 2 distribution models against the published pipeline.

← Phase 1Phase 3 →