Confirmation · September initialization · ONDJ and its months
The screening isolates the amount correction. The confirmation evaluates what the site publishes: each method’s corrected
members go through the full probability pipeline (tercile counts, smoothing and the climatology blend), with every fit and the
blend weight inside the cross-validation folds. Two selection-aware procedures then choose a method for each
left-out year using only the other training years — SEL-P by the lowest inner blended RPS (probabilities),
SEL-A by the lowest inner CRPS (amounts) — so their scores include the effect of choosing among the 14
declared methods. ✓ = better than Mean-Variance Scaling with Holm-adjusted p < 0.05 (methods: across all methods and targets;
SEL-P/SEL-A: across the targets) and not worse over 2017–2024. This is the benchmark’s own decision, as run (Holm); the method selection applies Benjamini–Hochberg to the same p-values (since 17 October 2026). Scope: All Ethiopia.
The season at a glance
ONDJ season through the full pipeline (outer leave-one-year-out 1993–2016; test years exploratory)
| Method | Blended RPSS 1993–2016 | Blended RPSS 2017–2024 | BSS below 1993–2016 | BSS above 1993–2016 | Amount CRPSS 1993–2016 | Amount CRPSS 2017–2024 | Mean blend weight λ |
|---|
| Mean-Variance Scaling (operational) | +0.039 | +0.075 | +0.028 | +0.051 | +0.070 | +0.178 | 0.20 |
| Linear Scaling | +0.037 | +0.071 | +0.018 | +0.056 | +0.070 | +0.131 | 0.28 |
| Variance Scaling (unbounded) | +0.039 | +0.074 | +0.029 | +0.050 | +0.063 | +0.182 | 0.21 |
| Square-root-space Mean-Variance Scaling | +0.040 | +0.080 | +0.030 | +0.050 | +0.080 | +0.189 | 0.18 |
| Power Transformation | +0.038 | +0.075 | +0.028 | +0.049 | +0.071 | +0.188 | 0.19 |
| Delta Change | +0.026 | +0.029 | +0.016 | +0.035 | +0.093 | +0.114 | 0.23 |
| Local Intensity Scaling | +0.037 | +0.071 | +0.018 | +0.056 | +0.070 | +0.131 | 0.29 |
| Empirical Quantile Mapping | +0.028 | +0.062 | +0.016 | +0.040 | +0.053 | +0.168 | 0.22 |
| Gamma Quantile Mapping | +0.041 | +0.082 | +0.031 | +0.050 | +0.082 | +0.192 | 0.18 |
| Detrended Quantile Mapping | +0.020 | +0.045 | +0.007 | +0.032 | +0.072 | +0.115 | 0.32 |
| Quantile Delta Mapping | +0.018 | +0.041 | +0.003 | +0.033 | +0.075 | +0.118 | 0.31 |
| Scaled Distribution Mapping | +0.015 | +0.040 | -0.000 | +0.030 | +0.061 | +0.108 | 0.38 |
| CDF-transform | +0.022 | +0.049 | +0.009 | +0.036 | +0.074 | +0.118 | 0.32 |
| MVS with all-zero guard (variant) | +0.039 | +0.075 | +0.028 | +0.051 | +0.070 | +0.178 | 0.20 |
| SEL-P | +0.040 | +0.082 | +0.030 | +0.050 | +0.081 | +0.192 | — |
| SEL-A | +0.026 | +0.029 | +0.016 | +0.035 | +0.093 | +0.114 | — |
Probabilities by target
Blended tercile RPSS by target, outer leave-one-year-out 1993–2016
| Method | Oct | Nov | Dec | Jan | ONDJ |
|---|
| Mean-Variance Scaling (operational) | +0.013 | +0.040 | +0.003 | +0.001 | +0.039 |
| Linear Scaling | +0.013 | +0.038 | +0.006 | -0.000 | +0.037 |
| Variance Scaling (unbounded) | +0.013 | +0.041 | +0.004 | +0.001 | +0.039 |
| Square-root-space Mean-Variance Scaling | +0.014 | +0.050 | +0.007 | +0.007 | +0.040 |
| Power Transformation | +0.012 | +0.039 | +0.005 | +0.003 | +0.038 |
| Delta Change | +0.004 | +0.035 | -0.000 | -0.006 | +0.026 |
| Local Intensity Scaling | +0.012 | +0.043 | +0.006 | +0.001 | +0.037 |
| Empirical Quantile Mapping | +0.002 | +0.034 | +0.005 | -0.003 | +0.028 |
| Gamma Quantile Mapping | +0.014 | +0.044 | +0.006 | +0.005 | +0.041 |
| Detrended Quantile Mapping | +0.003 | +0.028 | +0.001 | -0.006 | +0.020 |
| Quantile Delta Mapping | +0.004 | +0.021 | +0.006 | -0.005 | +0.018 |
| Scaled Distribution Mapping | +0.004 | +0.028 | +0.006 | -0.002 | +0.015 |
| CDF-transform | +0.005 | +0.023 | -0.001 | -0.003 | +0.022 |
| MVS with all-zero guard (variant) | +0.013 | +0.040 | +0.003 | +0.001 | +0.039 |
| SEL-P | +0.014 | +0.050 | +0.007 | +0.007 | +0.040 |
| SEL-A | +0.004 | +0.050 | +0.002 | +0.007 | +0.026 |
Amounts by target and the selection-aware procedures
Amount CRPSS by target, outer leave-one-year-out 1993–2016
| Method | Oct | Nov | Dec | Jan | ONDJ |
|---|
| Mean-Variance Scaling (operational) | -0.002 | +0.084 | +0.028 | -0.048 | +0.070 |
| Linear Scaling | -0.016 | +0.073 | -0.008 | -0.132 | +0.070 |
| Variance Scaling (unbounded) | -0.004 | +0.082 | +0.031 | -0.007 | +0.063 |
| Square-root-space Mean-Variance Scaling | +0.006 | +0.089 | +0.037 | +0.034 ✓ | +0.080 |
| Power Transformation | +0.001 | +0.082 | +0.020 | -0.025 | +0.071 |
| Delta Change | +0.021 | +0.085 | +0.026 | -0.047 | +0.093 |
| Local Intensity Scaling | -0.016 | +0.079 | +0.010 | -0.104 | +0.070 |
| Empirical Quantile Mapping | -0.012 | +0.075 | +0.033 | +0.010 | +0.053 |
| Gamma Quantile Mapping | +0.007 | +0.088 | +0.030 | +0.015 ✓ | +0.082 |
| Detrended Quantile Mapping | +0.005 | +0.075 | -0.001 | -0.065 | +0.072 |
| Quantile Delta Mapping | +0.010 | +0.076 | +0.014 | -0.027 | +0.075 |
| Scaled Distribution Mapping | -0.002 | +0.068 | -0.021 | -0.216 | +0.061 |
| CDF-transform | +0.002 | +0.073 | -0.019 | -0.108 | +0.074 |
| MVS with all-zero guard (variant) | -0.002 | +0.084 | +0.028 | -0.048 | +0.070 |
| SEL-P | +0.007 | +0.089 | +0.037 | +0.034 | +0.081 |
| SEL-A | +0.021 | +0.089 | +0.034 | +0.034 ✓ | +0.093 |
Selection-aware procedures against the operational method (Δ: RPS for SEL-P, CRPS for SEL-A; negative favours the procedure)
| Procedure | Target | Methods chosen (1993–2016, years) | Chosen for 2017–2024 | Δ score vs MVS | Holm p (targets) | Δ 2017–2024 | Confirmed |
|---|
| SEL-P | Oct | Gamma Quantile Mapping 23, Square-root-space Mean-Variance Scaling 1 | Gamma Quantile Mapping | -0.0007 | 1.0000 | +0.0025 | no |
| SEL-P | Nov | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.0043 | 0.7338 | -0.0037 | no |
| SEL-P | Dec | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.0021 | 1.0000 | +0.0009 | no |
| SEL-P | Jan | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.0028 | 1.0000 | -0.0212 | no |
| SEL-P | ONDJ | Gamma Quantile Mapping 18, Square-root-space Mean-Variance Scaling 6 | Gamma Quantile Mapping | -0.0003 | 1.0000 | -0.0033 | no |
| SEL-A | Oct | Delta Change 24 | Delta Change | -0.6772 | 0.2275 | +0.9372 | no |
| SEL-A | Nov | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.0931 | 0.2275 | -0.1049 | no |
| SEL-A | Dec | Square-root-space Mean-Variance Scaling 23, Empirical Quantile Mapping 1 | Square-root-space Mean-Variance Scaling | -0.0424 | 0.2027 | -0.0334 | no |
| SEL-A | Jan | Square-root-space Mean-Variance Scaling 24 | Square-root-space Mean-Variance Scaling | -0.4787 | 0.0018 | -0.6902 | yes |
| SEL-A | ONDJ | Delta Change 24 | Delta Change | -1.0435 | 0.2275 | +3.2167 | no |