Tim Palmer

Abstract:

© 2018 Royal Meteorological Society Studies conducted by the UK Met Office reported significant skill in predicting the winter North Atlantic Oscillation (NAO) index with their seasonal prediction system. At the same time, a very low signal-to-noise ratio was observed, as measured using the “ratio of predictable components” (RPC) metric. We analyse both the skill and signal-to-noise ratio using a new statistical toy model, which assumes NAO predictability is driven by regime dynamics. It is shown that if the system is approximately bimodal in nature, with the model consistently underestimating the level of regime persistence each season, then both the high skill and high RPC value of the Met Office hindcasts can easily be reproduced. Underestimation of regime persistence could be attributable to any number of sources of model error, including imperfect regime structure or errors in the propagation of teleconnections. In particular, a high RPC value for a seasonal mean prediction may be expected even if the model's internal level of noise is realistic.

Abstract:

Atmospheric seasonal predictability in winter over the Euro-Atlantic region is studied with an emphasis on the signal-to-noise paradox of the North Atlantic Oscillation. Seasonal hindcasts of the ECMWF model for the recent period 1981-2009 show, in agreement with other studies, that correlation skill over Greenland and parts of the Arctic is higher than the signal-to-noise ratio implies. This leads to the paradoxical situation where the real world appears more predictable than the models suggest, with the forecast ensembles being overly dispersive (or underconfident). However, it is demonstrated that these conclusions are not supported by the diagnosed relationship between ensemble mean RMSE and ensemble spread which indicates a slight underdispersion (overconfidence). Furthermore, long atmospheric seasonal hindcasts suggest that over the 110-year period from 1900 to 2009 the ensemble system is well calibrated (neither over- nor underdispersive). The observed skill changed drastically in the middle of the 20th Century and paradoxical regions during more recent hindcast periods were strongly underdispersive during mid-Century decades.

Due to non-stationarities of the climate system in the form of decadal variability, relatively short hindcasts are not sufficiently representative for longer-term behaviour. In addition, small hindcast sample size can lead to skill estimates, in particular of correlation measures, that are not robust. It is shown that the relative uncertainty due to small hindcast sample size is often larger for correlation-based than for RMSE-based diagnostics. Correlation-based measures like the RPC are shown to be highly sensitive to the strength of the predictable signal, implying that disentangling of physical deficiencies in the models on the one hand, and the effects of sampling uncertainty on the other hand, is difficult. Given the current lack of a causal physical mechanism to unravel the puzzle, our hypotheses of non-stationarity and sampling uncertainty provide simple yet plausible explanations for the paradox.