top of page
xNUMA
​
Welcome to the xNUMA page.  

​​​

​

What's new with xNUMA
​​

A new version of NUMA has been developed.  This new model (xNUMA) is similar to NUMA except that it can generate an arbitrary number of multi-domains to support multiscale modeling framework (i.e., super-parameterization) regions and scientific machine learning  strategies. xNUMA is GPU capable via OpenACC directives and has shown to achieve excellent strong scaling up to 256 Nvidia A100 GPUs and achieves 80% weak scaling on this hardware.

​​​

​

​xNUMA Documentation
​​
xNUMA documentation can be found here [PDF].​
​​
​
What is xNUMA
​

xNUMA consists of  a leaner light-weight version of NUMA that is written in a purely object-oriented approach.  Both the spatial discretization and the time-integration methods are encapsulated within objects.  This then offers the flexibility to instantiate numerous NUMA simulations having their own grids, spatial discretization, and time-integration methods. This was required to allow for constructing the multi-scale modeling framework (MMF) -- the general family of methods that includes standard super-parameterization.  In this idea, a coarse-grid model (Large-Scale-Process or LSP) calls numerous LES (fine-grid) models (Small-Scale-Process or SSP). The LES models enrich the LSP model with the sub-grid scale processes (replacing many of the components of a classical physical parameterization).  

​​​

​

​What can xNUMA be used for
 

Figure 1 shows an xNUMA configuration where the coarse-grid model spans the entire 2D domain with 4x4 elements. The LES (fine-grid) model comprises 2x2 elements within each of the coarse-grid model elements. In practice, however, we embed NxN elements inside each coarse-grid model element with the coarse-grid model comprised of NxN elements. In this setup, we can use N+1 MPI ranks or accelerators.

​​

The next figure (LSP Grid -> LSP Column -> SSP Grid) shows a second xNUMA configuration using the classical MMF approach where each vertical column of the LSP model (black points) contain SSP models that are used to enrich the LSP solution, column by column.

​​​

xNUMA has been used for moist simulations such as the 2D squall line and 3D supercell test cases.  These results can be found here [arXiv] where we show that MMF yields results between the uniform fine and uniform coarse results while costing less than the uniform fine.

 

xNUMA has been used recently for running high resolution tropical cyclone simulations on GPUs. These results can be found here [link].

​​​​

​

​Numerical Methods in xNUMA
​

xNUMA uses the continuous Galerkin method in discontinuous storage (CGd) for set2NC; however, the discretization allows for contravariant forms of all the derivatives.  For stabilization, xNUMA uses 2nd and 4th order tensor viscosity methods and filters.  xNUMA allows for no-flux, periodic, and implicit Rayleigh damping boundary conditions. The time-integrators use explicit RK methods as well as 3d-IMEX RK methods of orders 2 through 5. 

​​​

​

Who built xNUMA
​

xNUMA was developed in the group of Prof. Frank Giraldo in the Computational Mathematics Laboratory of the Department of Applied Mathematics at the Naval Postgraduate School.  National Research Council postdocs are contributing to the construction of xNUMA. Currently, the main developer is Dr. Soonpil Kang. 

​​​​

​

Who funded xNUMA
​

xNUMA is funded by the Office of Naval Research (ONR), Marine Meteorology Directorate.

​​

​

xNUMA Development
​

There are two precursors to xNUMA: (1) sNUMA and (2) pNUMA.  xNUMA has subsumed the other components.  The "s" stands for "serial", the "p" for parallel, and the "x" for accelerator (but can also be run in parallel or serial).  xNUMA uses a general parallelization strategy allowing for more MPI ranks to solve the coarse-grid model; in addition, we have added moist microphysics to xNUMA.  xNUMA is able to use N MPI ranks to solve the coarse-grid model with the LES models run on as many MPI ranks that have separate GPU cards. xNUMA is now running on GPUs as well.

​​​

​

How to get access to xNUMA

​​

xNUMA is currently housed in a private Github organization here. We expect to release xNUMA as an open-source code in the future.  Since the code is going through rapid development, we cannot support users but are open to giving collaborators early access. Email Frank Giraldo at frank.giraldo@gmail.com with the subject line: Access to xNUMA followed by a reason for needing access to the code.

LSP_2_SSP.png
bottom of page