ECMWF received the first delivery of nine frames of computer equipment from IBM for the mid-life upgrade of the High Performance Computing Facility (HPCF).
ECMWF has a long history of high-performance computing (HPC) in a production environment for weather forecasting, going back to the installation of its first supercomputer in 1978. Over time, various architectures have been used.
The delivery from IBM consisted of a test system, four compute frames and four frames of disk storage providing over 3 petabytes of useable disk. The four compute frames have more than 12 thousand POWER7® processor cores and will deliver more computational power than one of the current clusters on ECMWF codes. A further thirteen compute frames are expected.
ECMWF's HPCF upgrade will continue the successful design of two independent clusters that can cross-mount storage. Dual clusters add significantly to the resiliency of the system, allowing flexibility in performing maintenance and upgrades and when combined with separate resilient power and cooling systems provide protection against a wide range of possible failures.
The upgrade is based on nodes made up of 4 IBM POWER7 processors each with eight cores. Eight nodes make up a drawer and 4 drawers make a super-node. A low latency high speed network connects each node to every other node in a super-node and each super-node to every other super-node. The bandwidth of this interconnect is 23 terabytes per second per compute cluster.
Each compute cluster will weigh more than 26 metric tons and will comprise:
- 24 super nodes, made up of 32 nodes;
- 732 "normal memory" application nodes with 64GB of memory;
- 20 "large memory" applications nodes with 256GB of memory;
- 10 "Availability Plus" spare nodes;
- 6 "service" nodes;
- 24,576 POWER7 processor cores;
- 53TB of memory
The completed system will provide almost three times the sustained performance of the current system and have a theoretical peak performance of about 1.5 petaflops. The upgraded HPCF will be made available progressively during 2012.
More information about ECMWF's supercomputer can be found here.