The year 2020 continued the upward trend of global forecast performance, driven by regular updates of the Integrated Forecasting System (IFS).
The beneficial effect of new model cycles is fully visible only after 12 months. That is why the improvements shown here are due to Cycle 46r1, implemented in June 2019, as well as to Cycle 47r1, implemented in June 2020.
Headline scores show positive effects in medium-range as well as extended-range forecasts. They concern both ensemble forecasts (ENS) with a resolution of 18 km and high-resolution forecasts (HRES) with a resolution of 9 km.
Main results of 46r1 and 47r1
ECMWF’s headline scores are computed as 12-month running averages to filter out the annual cycle and better identify trends in forecast performance.
The first figure shows the significant improvement of upper-air ENS skill due to Cycle 46r1.
Skill of the ENS at day 5 for three upper-air parameters in the northern extratropics, relative to a Gaussian-dressed ERA5 forecast as a reference. The values are running 12-month averages, and verification is performed against own analysis.
The second figure shows that the beneficial effect of this cycle includes surface parameters, specifically a further reduction of the fraction of large ENS errors in 2 m temperature.
Evolution of the fraction of large 2 m temperature errors (continuous ranked probability score > 5 K) in the ENS at day 5 in the extratropics. Verification is performed against SYNOP observations. The 12-month running mean is shown in red, the 3-month running mean in blue.
Substantial improvements are also seen in the precipitation forecast. Compared to forecasts from other global modelling centres, ECMWF has been able to maintain the overall lead in the medium range, both for upper-air and surface parameters.
It is worth noting that the medium-range forecast performance of the IFS did not show any obvious degradation due to reduced aircraft observations from March 2020 onwards as a result of COVID-19.
Parallel pre-operational testing showed that Cycle 47r1 brings substantial improvements in the stratosphere as well as slight improvements in the troposphere. These will be fully visible in the operational scores by June 2021.
Tropical cyclone and ocean scores
The position error for forecasts of tropical cyclones increased compared to the previous year due to atmospheric variability. This is indicated by forecasts based on the ERA5 reanalysis system, which show a very similar increase.
HRES tropical cyclone intensity errors, as measured by the error in central pressure, have reached their lowest value so far. However, the decrease relative to the previous year is also seen in ERA5, which suggests it is due to year-to-year atmospheric variability.
For ocean waves in the extratropics, ECMWF leads other global wave forecasting systems in terms of significant wave height but ranks closer to average for peak period. In the tropics, ECMWF leads in terms of peak period.
The extended range
Verification of ensemble forecasts of 2-metre temperature anomalies in week two in the northern extratropics shows a statistically significant positive trend. There has also been a significant improvement in the headline score which monitors ENS probabilistic skill for weekly mean 2-metre temperature in week three.
Re-forecasts are a useful additional resource for documenting trends in skill. The third figure shows the re-forecast skill of the ENS in predicting 2 m temperature anomalies in week three in the northern extratropics. The most recent increase in skill is mainly due to the more accurate ERA5 reanalysis replacing ERA-Interim to provide initial conditions for the re-forecasts. Due to this change, the re-forecast verification now gives a better indication of real-time forecast skill, as it takes into account improvements in the data assimilation that happened between ERA-Interim and ERA5.
Skill of ENS re-forecasts in predicting weekly mean 2 m temperature anomalies (in terms of terciles) in week three in the northern extratropics. Verification is carried out for the previous 20 years. The results are shown for the own analysis in blue and for SYNOP observations in red. The verification metric is the Ranked Probability Skill Score. The marked increase in 2020 is mainly due to the more accurate ERA5 reanalysis replacing ERA-Interim to provide initial conditions for the re-forecasts.
Because of the lack of a strong El Niño or La Niña signal, a very strong positive phase of the Indian Ocean Dipole (IOD) peaking towards the end of 2019 became the main tropical driver of global long-range forecast skill.
As a result, 2 m temperature anomaly patterns in the northern hemisphere winter (DJF 2019–20) were reasonably well predicted over ocean areas, including the North Atlantic. In mid- and high-latitude regions of the American and Eurasian continents, however, forecast skill was lower.
In spring 2020, both the IOD and temperature anomalies in the tropical Pacific returned to close-to-neutral values, leaving the 2020 northern hemisphere summer without two strong drivers on seasonal timescales. Forecast skill was accordingly low in many areas.
Further information
More information can be found in the recent Newsletter article on Forecast performance 2020.
The complete set of annual verification results is available in the ECMWF Technical Memorandum No. 880 on ‘Evaluation of ECMWF forecasts, including the 2020 upgrade’, downloadable from the list of Technical Memoranda.