May initialization · JJAS and its months
Research benchmark. The published forecasts still use Mean-Variance Scaling (operational). This benchmark compares 12 other rainfall-amount bias-correction methods with it, member by member, for every grid cell and target: a screening of the amount correction alone, then a confirmation through the full probability pipeline that also accounts for how a method is chosen.
What the evidence supports
- Published tercile probabilities: no tested method demonstrates a statistically significant improvement over Mean-Variance Scaling under this validation protocol (several have slightly higher point estimates); the selection-aware procedure is confirmed for none.
- Rainfall amounts: confirmed against Mean-Variance Scaling (family-wide, full pipeline): Variance Scaling (unbounded) for Jul and Aug; Square-root-space Mean-Variance Scaling for Aug; Power Transformation for Aug; Gamma Quantile Mapping for Aug. The selection-aware procedure is confirmed for Jun, Aug and Sep. These comparisons establish improvement against the reference, not that one candidate is better than another.
- The operational method stays Mean-Variance Scaling until a change is decided by initialization, target and product, and recorded in the docs, on this evidence; confirmed improvements limited to some months may justify a per-target choice.
- If amounts and probabilities come to use different correction paths, the product metadata must record both methods so that every output can be traced.
Suggested method
The platform suggests a bias-correction method for every target and product from the historical evidence (the confirmation against CHIRPS, counting every method tried) and the independent validation seasons scored since (the shadow and validation runs), by a rule fixed before those seasons are scored. A suggestion is advice for a decision: the published forecast always uses Mean-Variance Scaling until a change is decided. 4 of 10 target–product combinations have a candidate.
The rule
- Candidate: A method is a candidate only if the confirmation (CHIRPS, outer leave-one-year-out over the training years, full pipeline) finds it better than the operational method with Benjamini-Hochberg-adjusted p < 0.05 (false discovery rate) across every declared method and target, and not worse over the test years. Variants outside the declared family are not candidates.
- Choice: Among candidates, the method the selection-aware procedure chooses from all training years (training-only information); if that method is not a candidate, the candidate with the best training-period score. Other candidates are listed as alternatives: a confirmation establishes improvement against the operational method, not a ranking among candidates.
- Validation: Independent validation seasons are forecast years scored after the benchmark (bias_validation.py). A candidate is 'recommended' once at least minimum_seasons have been scored for it and the operational method, and its mean score over them is not worse; with fewer seasons it stays a 'candidate'; with enough seasons and a worse mean it is 'not supported by validation'. Minimum: 3 seasons.
| Target | Product | Suggested method | Status | Evidence |
|---|---|---|---|---|
| Jun | Published tercile probabilities | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for Jun on the historical evidence. |
| Jun | Rainfall amounts | Gamma Quantile Mapping | Candidate — validation 1 of 3 seasons | Gamma Quantile Mapping is confirmed better than Mean-Variance Scaling for Jun (CRPS (mm) difference -0.1309 over the training years, Benjamini–Hochberg p 0.0488; -0.1626 over the test years). Independent validation: 1 of 3 season(s), mean difference +0.3062. Also confirmed: Square-root-space Mean-Variance Scaling. |
| Jul | Published tercile probabilities | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for Jul on the historical evidence. |
| Jul | Rainfall amounts | Variance Scaling (unbounded) | Candidate — validation 1 of 3 seasons | Variance Scaling (unbounded) is confirmed better than Mean-Variance Scaling for Jul (CRPS (mm) difference -0.1386 over the training years, Benjamini–Hochberg p 0; -0.1399 over the test years). Independent validation: 1 of 3 season(s), mean difference -0.4168. Also confirmed: Square-root-space Mean-Variance Scaling, Power Transformation. |
| Aug | Published tercile probabilities | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for Aug on the historical evidence. |
| Aug | Rainfall amounts | Variance Scaling (unbounded) | Candidate — validation 1 of 3 seasons | Variance Scaling (unbounded) is confirmed better than Mean-Variance Scaling for Aug (CRPS (mm) difference -0.129 over the training years, Benjamini–Hochberg p 0; -0.08487 over the test years). Independent validation: 1 of 3 season(s), mean difference -0.1859. Also confirmed: Square-root-space Mean-Variance Scaling, Power Transformation, Gamma Quantile Mapping. |
| Sep | Published tercile probabilities | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for Sep on the historical evidence. |
| Sep | Rainfall amounts | Gamma Quantile Mapping | Candidate — validation 1 of 3 seasons | Gamma Quantile Mapping is confirmed better than Mean-Variance Scaling for Sep (CRPS (mm) difference -0.07192 over the training years, Benjamini–Hochberg p 0.0133; -0.0947 over the test years). Independent validation: 1 of 3 season(s), mean difference -0.1197. Also confirmed: Variance Scaling (unbounded), Square-root-space Mean-Variance Scaling. |
| JJAS | Published tercile probabilities | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for JJAS on the historical evidence. |
| JJAS | Rainfall amounts | Mean-Variance Scaling | Keep the operational method | No method is confirmed better than Mean-Variance Scaling for JJAS on the historical evidence. |
Explore the evidence
- Interpretation
What the results mean, statement by statement (each checked against the numbers), and the caveats. - Data and methods
The data, the 13 methods and the reference, and the evaluation designs. - Screening
The amount correction alone: skill by method and target, every region, statistics, diagnostics. - Confirmation
Every method through the full probability pipeline, and the selection-aware procedures. - Ratio limits
The operational MVS limits (0.5–2) against wider limits and none. - Shadow and validation
The shadow forecast of this initialization and every method scored on the seasons after the benchmark. - Figures
Summary figures, per-cell maps, rank histograms and the forecast members. - Downloads
Result files, provenance, code and references.