Some 30 years ago ECMWF embarked upon a major project to rewrite the computer code of its forecasting system. One of the key players in the project, Mats Hamrud, says the ‘Integrated Forecasting System’ he and his colleagues went on to develop has stood the test of time but needs to keep evolving to meet new challenges.
Mats joined ECMWF in 1984 after completing a master’s in mathematical engineering and a PhD in meteorology at Stockholm University.
Three years into the job, he was given the task of assisting his colleague Philippe Courtier in what Mats describes as a “massive piece of work”: to rewrite the code of the forecasting system so that it would work with a variational data assimilation system!
“The plan to introduce three-dimensional variational data assimilation (3D-Var) and later 4D-Var meant we needed tangent linear and adjoint versions of the model, which was not easily possible with the existing code,” Mats remembers.
“We had to reproduce the science of the existing spectral model in a new model written in new code.”
After ten years of work involving a growing number of scientists, 3D-Var and then 4D-Var were finally introduced into operations.
Mats proposed a new name to reflect the close integration of the model with the new data assimilation system: the Integrated Forecasting System (IFS).
For several years he managed the implementation of IFS upgrades before returning to the scientific front line.
New challenges
Over the years the IFS has considerably grown in complexity.
In 1992 ECMWF introduced ensemble forecasting. It has since repeatedly increased the model resolution and added more Earth system components.
All this has required the use of ever more powerful supercomputers to continue to produce forecasts in a timely manner.
Today the IFS comprises more than 2 million lines of code. How can this complex system be made as computationally efficient and as scalable as possible?
Mats has been involved in several projects that aim to do just that, including the Centre’s first ‘scalability project’ in 2008.
The goal was to understand to what extent 4D-Var and other parts of the IFS were scalable, in other words how much they could be speeded up by adding more computer processors.
Mats found that scalability was a particular challenge for the data assimilation system.
The underlying issues were not technical but more fundamental: the algorithm’s sequential nature prevented it from running faster just by increasing the number of processors.
Other projects Mats has been involved in include the development of a fast Legendre transform to speed up the spectral model, and alternative data assimilation methods, such as ensemble Kalman filters and hybrid methods.
A common thread is that these projects are closely related to what is today a major stream of work at ECMWF: the Scalability Programme.
It is no coincidence that the 18th Workshop on High-Performance Computing in Meteorology at ECMWF from 24 to 28 September 2018 will focus on scalability and I/O of weather and climate applications on HPC systems.
ECMWF’s high-performance computing facility (HPCF) is one of the most powerful of its kind in Europe. The Centre’s next HPCF will be located at its new data centre to be built in Bologna, Italy.
OOPS!
As part of the Scalability Programme, the OOPS (Object-Oriented Prediction System) project has created a unified, easy-to-use framework for running different variational data assimilation formulations with a variety of forecast models.
Mats has worked on OOPS for the last few years as his last major assignment before his retirement at the end of June.
OOPS is a major international effort, involving input from ECMWF, Météo-France and the HIRLAM-ALADIN community.
OOPS will soon replace the control layer of data assimilation code that has supported ECMWF for the past 20 years.
“As my work at ECMWF comes to an end, it is clear that the IFS will have to continue to undergo major change to meet the challenges of tomorrow,” says Mats.
“I am pleased to have been able to make a sustained contribution to the development of the IFS to help get it to the point where we are today.”