Parts of ECMWF’s Integrated Forecasting System (IFS) are becoming open source, and the merits of moving the full IFS to open source in the future will be reviewed in consultation with Member States.
Currently, a distinction in licensing approach is made between the source code of the IFS and other ECMWF codes. Generally, non-IFS software is available under an open-source licence (Apache‑2), while the approach to the IFS has been more restrictive. The IFS source is not publicly available, accessible only by ECMWF and its Member and Co-operating States. An ‘open’ version of the IFS does exist. It is known as OpenIFS (see ECMWF Newsletter No. 170) and is available to institutions for meteorological research. However, this is not fully open source and using it requires a bespoke licence.
In the June 2020 session of ECMWF’s Council, options for making all or parts of the IFS open source were presented, along with a recommendation to allow carefully selected parts of the IFS to be released under an Apache-2 licence. The argument for open source for non-IFS code, that the benefits outweigh the risks, also applies to at least some parts of the IFS. Accordingly, it was agreed that the open-source approach should be extended to selected parts of the current IFS and to other developments envisaged to be part of the future IFS.
Aims
The aim is primarily enhanced collaboration. Removing restrictions on redistribution could make working with ECMWF more attractive to collaborators who wish to work with their partners. In some instances, further value can be provided through those wider networks. Additionally, contributing to open-source codes may be more attractive to academic partners.
A second purpose is efficiency. The present mixture of bespoke licences and open-source is relatively complex and ad-hoc, with significant time spent devising case-by-case solutions. A secondary consideration is that some journals require open access to codes used. So far, workarounds have been found (typically by providing code to editors/referees, for the purposes of the review), but this is time-consuming.
Furthermore, collaboration on scalability of the IFS with external organisations has been difficult. The OpenIFS licence allows only for meteorological research, not computer science research, and the version of the IFS used in the procurement of high-performance computing facilities is licensed only for benchmarking, not research. A third reason for considering open source is therefore to further position ECMWF and its Member States at the centre of international efforts on scalability and emerging high-performance computing architectures, by encouraging work on the IFS by computational science experts in academia and vendors.
Current availability
A GitHub space has been created to host open-source IFS components: https://github.com/ecmwf-ifs (see the image). This is distinct from the main ECMWF GitHub space (https://github.com/ecmwf), to signify a difference in purpose and support. The main space contains supported packages provided for the benefit of the community; the IFS space contains code released primarily to support pre-existing collaborations. While available to anyone, code in the IFS space is not generally supported.
ecRad: The first component to become available was the ecRad radiation package. This was previously available under a bespoke licence but is now freely available under Apache‑2.
ecTrans: The parallel spectral transforms from the IFS have been extracted and adapted in a standalone form useful to collaborators.
FIAT: A subset of IFS auxiliary routines required by multiple other packages has been released as FIAT (Fortran IFS and Arpège Toolkit). It includes routines used in the IFS for tracing, timing, controlling precision, and parallel communication.
CLOUDSC: A standalone version of the cloud scheme from the IFS. It was extracted in the EU-funded ESCAPE project (2015–2018) and was previously available under an ESCAPE licence.
CLOUDSC2: A new, simplified cloud microphysics scheme, including code to test the tangent-linear and adjoint versions of the scheme. Its purpose is to explore the impact of compiler optimisations and rounding behaviour on the symmetry of tangent-linear and adjoint solutions.
ecBundle: The bundle management tool used to build the IFS and other apps. It downloads code including dependencies and allows architecture-specific configuration for builds.
The future
Further components agreed to become open source soon are:
ecLand: This is the standalone land-surface scheme in the IFS, key to ongoing and future external collaborations, such as the addition of CaMa-Flood for hydrology modelling and EU-funded projects such as CoCO2, CONFESS and NextGEMS. Collaborations target developments on vegetation and photosynthesis modelling, urban tiles, snow and soil hydrology, and tight snow–ocean–sea-ice coupling.
ecWAM: This is the standalone wave model in the IFS, which will in future be even more tightly coupled to the IFS atmosphere, adopting the same grid. A crucial forthcoming activity is the GPU adaptation of ecWAM.
In addition to making components of the IFS open source, it was agreed at Council that ECMWF would also encourage further reviews, in consultation with Member States, of the merits of moving the full IFS to open source. It was noted that extending the approach to the whole IFS could have some significant advantages for collaboration and efficiency. Discussions are expected to begin in 2022.