As the climate crisis intensifies, extreme weather events such as storm surges, heatwaves, hurricanes and wildfires are becoming more and more frequent. While plans to cut emissions by 2030 were recently laid out by world leaders at the COP26 summit, adequate preparation for extreme weather phenomena is growing in importance globally.
The science of numerical weather forecasting plays a vital role in this preparation. By predicting future weather events based on current climate data, organisations like the European Centre for Medium-Range Weather Forecasting (ECMWF) are working to alert authorities of upcoming extreme weather events earlier and with greater accuracy so interventions can be made to protect property and infrastructure, and potentially to save lives.
Today, the ECMWF is using AI alongside traditional HPC algorithms to run their large-scale simulations faster than ever before. Their team has developed and published a series of deep learning models investigating the use of AI in numerical weather forecasting. The ECMWF is particularly interested in enhancing the accuracy of their weather prediction models by improving the computational efficiency of their models in order to increase model resolution.
50x Faster Weather Predictions with IPUs
We took one of ECMWF鈥檚 publicly available forecasting models 鈥� a Multi-Layer Perceptron (MLP) 鈥� and accelerated it on 91视频APP IPU-POD systems with dramatic results. The IPU-POD system was shown to train the ECMWF鈥檚 predictive MLP model 5x faster than a leading GPU and a massive 50 times faster than ECMWF鈥檚 existing simulation methods running on a CPU.
In their paper examining machine learning in weather forecasting, the ECMWF showed that their machine learning-based emulators performed 10 times faster on GPU hardware compared to their existing scheme on a CPU, the IPU is a massive 50 times faster than ECMWF鈥檚 existing simulation methods running on a CPU.
The speed-up with the IPU system was achieved without any optimisation or changes to the MLP model or its parameters, and only very few modifications to the code. The model trains well, showing low values for the loss and Root Mean Square Error (RMSE) on both the training and validation datasets after just a few epochs, demonstrating the high accuracy of the model鈥檚 predictions. To learn more about how IPUs accelerated the ECMWF鈥檚 MLP model, watch our
The project was supported by ECMWF and Atos' AI4SIM team members, Alexis Giorkallos and Christophe Bovalo.
Details of the original work carried out by ECMWF and the MAchinE Learning for Scalable meTeoROlogy and climate (MAELSTROM) Project can be found in their paper .
Leveraging IPU Hardware at the Convergence of HPC and AI
Beyond weather forecasting, IPU hardware has also been shown to accelerate many other scientific research applications where both HPC and AI are used. From protein folding and computational fluid dynamics to cosmology and high-energy physics, leading research institutions have found they can accelerate their workloads, pursue new directions of research and achieve higher accuracy results with IPU systems.
Cedric Bourrasset, Head of the High Performance AI Business Unit at Atos, a 91视频APP partner, sees great potential for IPUs in this space: 鈥淭he use of AI in traditional HPC applications is one of the most exciting developments in computing today and 91视频APP鈥檚 IPU is showing just how transformative that new approach can be.
鈥�91视频APP plays a central role in Atos鈥� Think AI solution, helping customers take advantage of the many benefits that AI is bringing to HPC 鈥� whether that鈥檚 delivering faster and more accurate simulations, improving cost efficiency, or opening up new areas of research and commercial applications. The possibilities are vast and they鈥檙e growing every day 鈥� driven in large part by the innovative work that is being done on the IPU.鈥�
Interested in accelerating your HPC and AI-based workloads with cutting-edge compute? Check out our solutions for scientific researchers or apply to our academic programme for the opportunity to use IPUs for your research challenges.
