Confirmation · May initialization · JJAS and its months
The screening isolates the amount correction. The confirmation evaluates what the site publishes: each method’s corrected
members go through the full probability pipeline (tercile counts, smoothing and the climatology blend), with every fit and the
blend weight inside the cross-validation folds. Two selection-aware procedures then choose a method for each
left-out year using only the other training years — SEL-P by the lowest inner blended RPS (probabilities),
SEL-A by the lowest inner CRPS (amounts) — so their scores include the effect of choosing among the 14
declared methods. ✓ = better than Mean-Variance Scaling with Holm-adjusted p < 0.05 (methods: across all methods and targets;
SEL-P/SEL-A: across the targets) and not worse over 2017–2025. This is the benchmark’s own decision, as run (Holm); the method selection applies Benjamini–Hochberg to the same p-values (since 17 October 2026). Scope: All Ethiopia.
The season at a glance
JJAS season through the full pipeline (outer leave-one-year-out 1993–2016; test years exploratory)
| Method | Blended RPSS 1993–2016 | Blended RPSS 2017–2025 | BSS below 1993–2016 | BSS above 1993–2016 | Amount CRPSS 1993–2016 | Amount CRPSS 2017–2025 | Mean blend weight λ |
|---|
| Mean-Variance Scaling (operational) | +0.027 | +0.039 | +0.034 | +0.019 | -0.028 | -0.018 | 0.59 |
| Linear Scaling | +0.025 | +0.039 | +0.033 | +0.018 | -0.059 | -0.082 | 0.60 |
| Variance Scaling (unbounded) | +0.027 | +0.041 | +0.035 | +0.019 | -0.027 | -0.016 | 0.58 |
| Square-root-space Mean-Variance Scaling | +0.027 | +0.041 | +0.035 | +0.019 | -0.025 | -0.014 | 0.58 |
| Power Transformation | +0.027 | +0.040 | +0.034 | +0.019 | -0.027 | -0.016 | 0.58 |
| Delta Change | +0.029 | +0.031 | +0.036 | +0.023 | -0.030 | -0.058 | 0.61 |
| Local Intensity Scaling | +0.025 | +0.040 | +0.033 | +0.018 | -0.059 | -0.081 | 0.61 |
| Empirical Quantile Mapping | +0.019 | +0.030 | +0.026 | +0.013 | -0.040 | -0.032 | 0.62 |
| Gamma Quantile Mapping | +0.026 | +0.041 | +0.035 | +0.018 | -0.025 | -0.012 | 0.58 |
| Detrended Quantile Mapping | +0.026 | +0.032 | +0.032 | +0.019 | -0.056 | -0.078 | 0.66 |
| Quantile Delta Mapping | +0.023 | +0.032 | +0.026 | +0.020 | -0.062 | -0.097 | 0.68 |
| Scaled Distribution Mapping | +0.023 | +0.030 | +0.026 | +0.021 | -0.078 | -0.110 | 0.70 |
| CDF-transform | +0.022 | +0.032 | +0.027 | +0.018 | -0.061 | -0.081 | 0.68 |
| MVS with all-zero guard (variant) | +0.027 | +0.039 | +0.034 | +0.019 | -0.028 | -0.018 | 0.59 |
| SEL-P | +0.025 | +0.031 | +0.031 | +0.019 | -0.029 | -0.058 | — |
| SEL-A | +0.023 | +0.041 | +0.030 | +0.016 | -0.036 | -0.012 | — |
Probabilities by target
Blended tercile RPSS by target, outer leave-one-year-out 1993–2016
| Method | Jun | Jul | Aug | Sep | JJAS |
|---|
| Mean-Variance Scaling (operational) | -0.000 | +0.014 | +0.012 | +0.003 | +0.027 |
| Linear Scaling | -0.001 | +0.013 | +0.012 | +0.003 | +0.025 |
| Variance Scaling (unbounded) | -0.001 | +0.015 | +0.012 | +0.004 | +0.027 |
| Square-root-space Mean-Variance Scaling | -0.002 | +0.014 | +0.012 | +0.003 | +0.027 |
| Power Transformation | -0.002 | +0.014 | +0.012 | +0.003 | +0.027 |
| Delta Change | -0.004 | +0.010 | +0.008 | +0.001 | +0.029 |
| Local Intensity Scaling | -0.001 | +0.013 | +0.012 | +0.003 | +0.025 |
| Empirical Quantile Mapping | -0.007 | +0.007 | +0.006 | -0.000 | +0.019 |
| Gamma Quantile Mapping | -0.002 | +0.014 | +0.012 | +0.003 | +0.026 |
| Detrended Quantile Mapping | -0.003 | +0.009 | +0.007 | +0.001 | +0.026 |
| Quantile Delta Mapping | -0.004 | +0.011 | +0.007 | +0.002 | +0.023 |
| Scaled Distribution Mapping | -0.002 | +0.011 | +0.007 | +0.003 | +0.023 |
| CDF-transform | -0.003 | +0.008 | +0.008 | +0.001 | +0.022 |
| MVS with all-zero guard (variant) | -0.000 | +0.014 | +0.012 | +0.003 | +0.027 |
| SEL-P | -0.008 | +0.015 | +0.012 | +0.003 | +0.025 |
| SEL-A | -0.002 | +0.010 | +0.012 | +0.003 | +0.023 |
Amounts by target and the selection-aware procedures
Amount CRPSS by target, outer leave-one-year-out 1993–2016
| Method | Jun | Jul | Aug | Sep | JJAS |
|---|
| Mean-Variance Scaling (operational) | -0.096 | -0.007 | -0.028 | -0.060 | -0.028 |
| Linear Scaling | -0.121 | -0.029 | -0.064 | -0.090 | -0.059 |
| Variance Scaling (unbounded) | -0.093 | -0.000 ✓ | -0.021 ✓ | -0.058 | -0.027 |
| Square-root-space Mean-Variance Scaling | -0.088 | -0.003 | -0.023 ✓ | -0.057 | -0.025 |
| Power Transformation | -0.091 | -0.001 | -0.022 ✓ | -0.060 | -0.027 |
| Delta Change | -0.105 | +0.001 | -0.043 | -0.067 | -0.030 |
| Local Intensity Scaling | -0.118 | -0.028 | -0.061 | -0.087 | -0.059 |
| Empirical Quantile Mapping | -0.100 | -0.017 | -0.035 | -0.067 | -0.040 |
| Gamma Quantile Mapping | -0.088 | -0.006 | -0.021 ✓ | -0.056 | -0.025 |
| Detrended Quantile Mapping | -0.134 | -0.026 | -0.064 | -0.090 | -0.056 |
| Quantile Delta Mapping | -0.145 | -0.025 | -0.071 | -0.098 | -0.062 |
| Scaled Distribution Mapping | -0.162 | -0.037 | -0.084 | -0.107 | -0.078 |
| CDF-transform | -0.138 | -0.026 | -0.065 | -0.098 | -0.061 |
| MVS with all-zero guard (variant) | -0.096 | -0.007 | -0.028 | -0.060 | -0.028 |
| SEL-P | -0.102 | -0.001 | -0.023 | -0.084 | -0.029 |
| SEL-A | -0.088 ✓ | -0.007 | -0.022 ✓ | -0.056 ✓ | -0.036 |
Selection-aware procedures against the operational method (Δ: RPS for SEL-P, CRPS for SEL-A; negative favours the procedure)
| Procedure | Target | Methods chosen (1993–2016, years) | Chosen for 2017–2025 | Δ score vs MVS | Holm p (targets) | Δ 2017–2025 | Confirmed |
|---|
| SEL-P | Jun | Power Transformation 13, Linear Scaling 5, Delta Change 4, Square-root-space Mean-Variance Scaling 2 | Power Transformation | +0.0032 | 1.0000 | -0.0019 | no |
| SEL-P | Jul | Variance Scaling (unbounded) 21, Square-root-space Mean-Variance Scaling 3 | Variance Scaling (unbounded) | -0.0002 | 1.0000 | -0.0010 | no |
| SEL-P | Aug | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.0003 | 1.0000 | -0.0013 | no |
| SEL-P | Sep | Linear Scaling 14, Gamma Quantile Mapping 8, Local Intensity Scaling 1, Scaled Distribution Mapping 1 | Scaled Distribution Mapping | +0.0000 | 1.0000 | +0.0008 | no |
| SEL-P | JJAS | Delta Change 23, Variance Scaling (unbounded) 1 | Delta Change | +0.0008 | 1.0000 | +0.0040 | no |
| SEL-A | Jun | Gamma Quantile Mapping 24 | Gamma Quantile Mapping | -0.1309 | 0.0293 | -0.1626 | yes |
| SEL-A | Jul | Delta Change 14, Variance Scaling (unbounded) 10 | Delta Change | +0.0018 | 1.0000 | +0.1943 | no |
| SEL-A | Aug | Variance Scaling (unbounded) 16, Gamma Quantile Mapping 6, Power Transformation 2 | Variance Scaling (unbounded) | -0.1075 | 0.0000 | -0.0849 | yes |
| SEL-A | Sep | Gamma Quantile Mapping 24 | Gamma Quantile Mapping | -0.0719 | 0.0071 | -0.0947 | yes |
| SEL-A | JJAS | Gamma Quantile Mapping 21, Delta Change 2, Square-root-space Mean-Variance Scaling 1 | Gamma Quantile Mapping | +0.3864 | 1.0000 | -0.2451 | no |