Dr Bobby Antonio

Abstract:

Abstract The fractions skill score (FSS) is a widely used metric for assessing forecast skill, with applications ranging from precipitation to volcanic ash forecasts. By evaluating the fraction of grid squares exceeding a threshold in a neighborhood, the intuition is that it can avoid the pitfalls of pixelwise comparisons and identify length scales at which a forecast has skill. The FSS is typically interpreted relative to a “useful” criterion, where a forecast is considered skillful if its score exceeds a simple reference score. However, the typical reference score used is problematic, since it is not derived in a way that provides obvious meaning, does not scale with neighborhood size, and may not be exceeded by forecasts that have skill. We, therefore, provide a new method to determine forecast skill from the FSS, by deriving an expression for the FSS achieved by a random forecast, which provides a more robust and meaningful reference score to compare with. Through illustrative examples, we show that this new method considerably changes the length scales at which a forecast would be regarded as skillful and reveals subtleties in how the FSS should be interpreted. Significance Statement Forecast verification metrics are crucial to assess accuracy and identify where forecasts can be improved. In this work, we investigate a popular verification metric, the fractions skill score, and derive a more robust method to decide if a forecast has sufficiently high skill. This new method significantly improves the quality of insights that can be drawn from this score.

Abstract:

Existing weather models are known to have poor skill at forecasting rainfall over East Africa. Improved forecasts could reduce the effects of extreme weather events and provide significant socioeconomic benefits to the region. We present a novel machine learning based method to improve precipitation forecasts in East Africa, using postprocessing based on a conditional generative adversarial network (cGAN). This addresses the challenge of realistically representing tropical rainfall, where convection dominates and is poorly simulated in conventional global forecast models. We postprocess hourly forecasts made by the European Centre for Medium-Range Weather Forecasts Integrated Forecast System at 6-18h lead times, at 0.1° resolution. We combine the cGAN predictions with a novel neighbourhood version of quantile mapping, to integrate the strengths of machine learning and conventional postprocessing. Our results indicate that the cGAN substantially improves the diurnal cycle of rainfall, and improves predictions up to the 99.9th percentile (∼ 10mm/hr). This improvement extends to the 2018 March–May season, which had extremely high rainfall, indicating that the approach has some ability to generalise to more extreme conditions. We explore the potential for the cGAN to produce probabilistic forecasts and find that the spread of this ensemble broadly reflects the predictability of the observations, but is also characterised by a mixture of under- and overdispersion. Overall our results demonstrate how the strengths of machine learning and conventional postprocessing methods can be combined, and illuminate what benefits ma38 chine learning approaches can bring to this region.