 |
 |
Julia Wagemann and Esperanza Cuartero, Programme managers of ECMWF Summer of Weather Code
Follow ESoWC on Twitter
For the past three years, summers at ECMWF have been full of coding, open-source developments and innovations:
- a chatbot to assist users in finding answers to common support queries;
- a model based on artificial intelligence (AI) to monitor, and forecast possible disruption to, the delivery of data to users;
- a study that showed the potential to compress the volume of data in the Copernicus Atmosphere Monitoring Service by a factor of 48.
These are just three of the innovative outcomes of this year’s ECMWF Summer of Weather Code (ESoWC), when nine developer teams worked closely with ECMWF and Copernicus mentors on a range of open-source projects. The outcomes were presented at our final ESoWC Day 2020, on 16 October 2020 during a virtual event.
Some of the participants at the final ESoWC day virtual event, 16 October 2020.
Close mentor-developer relationships are at the heart of the programme
The idea for ESoWC came in 2017 when multiple teams at ECMWF needed small developments that would facilitate the team’s work, but extra resources were needed to work on these projects. So the principle behind ESoWC is that ECMWF staff propose ideas for developments, and developers apply by proposing a solution to the proposed challenge. After the coding period, and provided that all milestones and deliverables are fulfilled, the developers receive a stipend of £5,000.
Throughout the coding period, the developers are closely guided by ECMWF mentors leading to a mutual learning experience. This close working relationship is at the core of the ESoWC philosophy.
Collaboration with Copernicus and cloud-processing system providers
The first ESoWC took place in 2018 with five projects. Since 2019, ESoWC has been co-funded by the Copernicus services at ECMWF, which allowed the programme to grow to nine projects in 2020.
For the first time this year, the ESoWC programme was supported by two cloud computing systems, the Data and Information Access Service WEkEO (implemented by Copernicus) and the European Weather Cloud (implemented by ECMWF). This allowed ESoWC to offer sufficient storage and processing capacities for the computing-intensive projects. This year’s projects were at the intersection of machine-learning, open-source, cloud computing and open data.
Four teams explored the use of machine learning and artificial intelligence in a range of applications:
- ‘Virtual assistant for users of ECMWF online products and services’ – the team developed a chatbot that assists ECMWF users to find answers to common support queries.
- ‘Applying AI capabilities to address operational challenges in ECMWF products team’ – the team applied AI capabilities to better understand sudden disruptions or failures of ECMWF production chain systems and predict future spikes and surges.
- ‘Exploring machine/deep learning techniques to detect and track tropical cyclones' - the team developed a deep-learning algorithm which recognises and classifies tropical cyclones based on their intensities.
- ‘DeepGEFF’ – the team explored whether fire danger forecasting using deep learning achieves comparable skill to the Global ECMWF Fire Forecasting (GEFF) system
Three teams put a focus on Copernicus open data:
- ‘Classification of air quality’ – the team developed a classification scheme to easily validate and remove outliers from surface air quality observations. This allowed the comparison of station data with air quality forecasts from the Copernicus Atmosphere Monitoring Service.
- ‘Detecting anomalies in Air Quality Stations (DAAQS)’ – the team applied clustering algorithms to provide reliability and representativeness scores for air quality stations.
- ‘UNSEEN-Open’ – the team developed an open, reproducible and transferable workflow based on climate reanalysis and seasonal forecast data to assess and anticipate climate extremes beyond the observed record.
And another two teams worked on challenges related to ECMWF’s model performance, data storage and archiving:
- ‘HPC performance profiling tool’ – the team developed an interface to interactively visualise HPC performance data in order to better track and analyse the performance of ECMWF’s integrated forecasting system (IFS).
- ‘Compressing atmospheric data into its real information’ – based on the global real-time forecast dataset from the Copernicus Atmosphere Monitoring Service, the team tested different configurations in order to estimate data encoding errors and the potential to compress the volume of data
The many benefits of ESoWC
Benefits for ECMWF and Copernicus are manifold. The projects that implement different machine-learning algorithms are a great pool of examples of ML applications, and they strongly support current research activities in this field. Another key benefit is that mentors and participants mutually learn from each other and project participants. Most participants come from disciplines other than meteorology and so bring new perspectives to the challenges.
The best outcome though is to see that many project teams continue to collaborate even after ESoWC has finished. This is a proof of the great external talent ESoWC attracts and the high-quality developments.
Stay tuned – ESoWC 2021 is around the corner
Planning for the next ECMWF Summer of Weather Code has just started. Each year in February, the application phase opens and a new list of ESoWC challenges is published on Github. Developer teams have until April to submit their proposals, and on 1 May 2021, a new ‘Summer of Weather Code’ starts – full of open-source developments and innovations.
For any updates, follow ECMWF Summer of Weather Code on Twitter and watch the project presentations on ESoWC’s YouTube account.