Phase 3 · Probability calibration · January initialization · FMAM and its months
Phase 3 · January initialization · FMAM and its months
Rainfall-distribution models (Pathway B)
Each model predicts a full rainfall distribution from the raw ensemble (pooled over cells in units relative to each cell's
climatology); terciles at the training CHIRPS boundaries, quantiles and intervals come from it. References, in the same folds:
the published probabilities (tercile RPS) and the MVS-corrected ensemble (CRPS, quantiles). ✓ = better than the reference with
Holm-adjusted p < 0.05 across the models and targets, and not worse over 2017–2025. Confirmed better — tercile probabilities:
none; rainfall amounts (CRPS): none.
Tercile probabilities
Tercile RPSS, outer leave-one-year-out 1993–2016
Method
Feb
Mar
Apr
May
FMAM
Published probabilities (incumbent)
+0.033
+0.002
+0.001
+0.028
+0.032
Bayesian joint probability
+0.026
-0.088
-0.005
+0.019
+0.030
Bayesian model averaging for precipitation
+0.014
-0.037
-0.010
+0.030
+0.044
Quantile regression forest
+0.028
-0.107
-0.087
-0.027
-0.025
Rainfall amounts
Amount CRPSS against the CHIRPS climatology, 1993–2016
Method
Feb
Mar
Apr
May
FMAM
MVS-corrected ensemble
+0.074
-0.044
-0.016
+0.007
+0.025
Bayesian joint probability
+0.085
-0.048
-0.018
-0.016
+0.019
Bayesian model averaging for precipitation
+0.084
-0.014
-0.011
+0.008
+0.044
Quantile regression forest
+0.018
-0.100
-0.071
-0.033
-0.022
Intervals and quantiles
Share of observations inside the 10–90% interval, 1993–2016 (80% is calibrated)
Method
Feb
Mar
Apr
May
FMAM
MVS-corrected ensemble
56%
54%
68%
69%
72%
Bayesian joint probability
83%
82%
81%
81%
82%
Bayesian model averaging for precipitation
82%
81%
82%
84%
80%
Quantile regression forest
74%
71%
72%
73%
72%
Pinball loss of the median (mm), 1993–2016 (lower is better)
Method
Feb
Mar
Apr
May
FMAM
MVS-corrected ensemble
5.09
14.09
20.47
20.44
33.73
Bayesian joint probability
5.09
14.77
20.24
20.56
33.97
Bayesian model averaging for precipitation
5.14
14.12
20.53
20.59
33.60
Quantile regression forest
5.50
15.36
21.74
21.36
35.85
Test years
Fit 1993–2016, scored 2017–2025 (exploratory)
Method
Feb
Mar
Apr
May
FMAM
Published probabilities / MVS ensemble
+0.020 / +0.002
+0.008 / +0.005
+0.002 / -0.006
+0.004 / -0.007
+0.017 / +0.016
Bayesian joint probability (RPSS / CRPSS)
-0.029 / +0.023
-0.037 / -0.034
-0.006 / -0.008
-0.006 / -0.056
+0.013 / +0.015
Bayesian model averaging for precipitation (RPSS / CRPSS)
-0.025 / +0.029
+0.013 / +0.023
-0.041 / -0.017
-0.051 / -0.042
+0.012 / +0.023
Quantile regression forest (RPSS / CRPSS)
-0.034 / -0.016
-0.007 / +0.004
-0.052 / -0.031
-0.086 / -0.049
-0.006 / +0.017
Fits and model definitions
Bayesian joint probability: did not converge in every fold.
Bayesian model averaging for precipitation: converged in every fold.
Quantile regression forest: converged in every fold.
Models
Model
Estimates
Predictors
Fitting
Outputs
Bayesian joint probability
rainfall distribution (log-sinh normal, censored at zero, Student-t predictive)
ensemble mean m' (relative)
censored maximum likelihood of the transforms and the joint normal; parameter uncertainty integrated with the number of training years as the effective sample
CDF, quantiles, tercile probabilities
Bayesian model averaging for precipitation
mixture over members: probability of zero and a gamma for the cube root of the amount
each member's amount f' (relative), equal weights for exchangeable members
maximum likelihood of the mixture, pooled
CDF, quantiles, tercile probabilities
Quantile regression forest
conditional distribution as leaf-weighted training amounts; quantiles at 39 levels (2.5–97.5%)
m', s', md', ensemble 10% and 90% quantiles (relative), log climatological mean, CHIRPS coefficient of variation
200 trees, at least 40 samples per leaf, half the predictors per split, pooled
CDF, quantiles, tercile probabilities
Tercile RPSS and amount CRPSS of the Phase 3 distribution models against the published pipeline.