Skip to main content
Article
Strong Scaling for Numerical Weather Prediction at Petascale with the Atmospheric Model NUMA
The International Journal of High Performance Computing Applications (2018)
  • Andreas Müller, Naval Postgraduate School
  • Michal A. Kopera, Naval Postgraduate School
  • Simone Marras, New Jersey Institute of Technology
  • Lucas C. Wilcox, Naval Postgraduate School
  • Tobin Isaac, Georgia Institute of Technology
Abstract
Numerical weather prediction (NWP) has proven to be computationally challenging due to its inherent multiscale nature. Currently, the highest resolution global NWP models use a horizontal resolution of 9 km. At this resolution, many important processes in the atmosphere are not resolved. Needless to say, this introduces errors. In order to increase the resolution of NWP models, highly scalable atmospheric models are needed. The non-hydrostatic unified model of the atmosphere (NUMA), developed by the authors at the Naval Postgraduate School, was designed to achieve this purpose. NUMA is used by the Naval Research Laboratory, Monterey as the engine inside its next generation weather prediction system NEPTUNE. NUMA solves the fully compressible Navier–Stokes equations by means of high-order Galerkin methods (both spectral element as well as discontinuous Galerkin methods can be used). NUMA is capable of running middle and upper atmosphere simulations since it does not make use of the shallow-atmosphere approximation. This article presents the performance analysis and optimization of the spectral element version of NUMA. The performance at different optimization stages is analyzed using a theoretical performance model as well as measurements via hardware counters. Machine-independent optimization is compared to machine-specific optimization using Blue Gene (BG)/Q vector intrinsics. The best portable version of the main computations was found to be about two times slower than the best non-portable version. By using vector intrinsics, the main computations reach 1.2 PFlops on the entire IBM Blue Gene supercomputer Mira (12% of the theoretical peak performance). The article also presents scalability studies for two idealized test cases that are relevant for NWP applications. The atmospheric model NUMA delivers an excellent strong scaling efficiency of 99% on the entire supercomputer Mira using a mesh with 1.8 billion grid points. This allows running a global forecast of a baroclinic wave test case at a 3-km uniform horizontal resolution and double precision within the time frame required for operational weather prediction.
Disciplines
Publication Date
April 5, 2018
DOI
10.1177/1094342018763966
Publisher Statement
Date provided is the online early release publication date.
Citation Information
Andreas Müller, Michal A. Kopera, Simone Marras, Lucas C. Wilcox, et al.. "Strong Scaling for Numerical Weather Prediction at Petascale with the Atmospheric Model NUMA" The International Journal of High Performance Computing Applications (2018)
Available at: http://works.bepress.com/michal-kopera/5/