One of the main characteristics of seasonal forecasts in the extratropics is that forecast signals are relatively weak while noise levels are high. Seasonal predictions of winter atmospheric circulation over parts of the North Atlantic and Greenland during recent decades show that the correlation skill between observations and the ensemble mean is larger than one would expect from low signal-to-noise ratios. This is sometimes called the ‘signal-to-noise paradox’. It implies that the real world appears more predictable than the forecast model suggests and that the forecast ensembles are underconfident (over-dispersive). A new study analyses the behaviour of the ECMWF atmospheric model over a much longer hindcast period, from 1900 to 2009. It was found that the paradox disappears if the full 110 years are considered, but that there is also substantial decadal-scale variability. It follows that relatively short hindcasts are not representative of longer-term behaviour. In addition, correlation-based measures show larger sensitivities to small hindcast sample sizes than alternatives, such as error-based diagnostics.
Multi-decadal variability of predictability
The correlation skill of hindcasts of geopotential height at 500 hPa during winter (December to February) for the period 1981–2009 across Greenland and parts of the North Atlantic is statistically highly significant and larger than expected from purely model-based estimates. Here the hindcasts were generated with an atmosphere–land only configuration of the ECMWF model, but coupled hindcasts perform similarly.
A quantity called the Ratio of Predictable Components (RPC) can be used to measure the ‘signal-to-noise paradox’. A perfect forecast system where the predictable components in the real world and in the model are similar would result in an RPC value of about one. If the model estimate of predictability is larger than the real-world predictable component, the RPC will be smaller than one. The forecasts are overconfident, or under-dispersive, as there is not enough variability in the ensemble. Seasonal forecast models generally tend to be overconfident, especially in the tropics.
In the situation of the ‘paradox’, the predictable components of the real world are larger than the model estimates of predictability, resulting in RPC values larger than one. As the figure shows, RPC in the period 1981–2009 is larger than one over Greenland and some eastern parts of the North Atlantic.
The availability of seasonal hindcasts over many decades makes it possible to trace the long-term evolution of RPC values. While the RPC of the North Atlantic Oscillation is very close to one if estimated over the full 110‑year hindcast period 1900 to 2009, 30‑year seasonal forecast skill and RPC vary on multi-decadal scales. For example, as shown in the figure, the mid-century period around the 1950s was characterised by a pronounced drop in skill. The corresponding RPC is either negative or close to 1, suggesting that the ‘paradox’ itself undergoes multi-decadal fluctuations.
The figure also shows the temporal evolution of the RPC for the NAO index during the 20th century. Within specific decades, RPC values can vary considerably. In contrast, the relation between the root-mean-square error (RMSE) and ensemble spread is much more stable, with no indication of underconfidence in any specific period. It is hypothesized that sampling uncertainty due to short hindcasts and non-stationarities in the climate system can contribute substantially to the occurrence of the ‘signal-to-noise paradox’.
Further information can be found in an article by Antje Weisheimer et al. in the Quarterly Journal of the Royal Meteorological Society, doi:10.1002/qj.3446.