xNUMA

Welcome to the xNUMA page.

xNUMA Documentation

xNUMA documentation can be found here [PDF].

What is xNUMA

xNUMA is the main component of the mini-NUMA project. mini-NUMA contains a leaner light-weight version of NUMA that is written in a purely object-oriented approach. Both the spatial discretization and the time-integration methods are encapsulated within objects. This then offers the flexibility to instantiate numerous NUMA simulations having their own grids, spatial discretization, and time-integration methods. We needed to do this in order to allow for a multi-scale modeling framework (MMF), (the general family of methods that includes standard super-parameterization). In this idea, a coarse-grid model (Large-Scale-Process or LSP) calls numerous LES (fine-grid) models (Small-Scale-Process or SSP). The LES models enrich the LSP model with the sub-grid scale processes (replacing many of the components of a classical physical parameterization).

What can xNUMA be used for

Figure 1 shows an xNUMA configuration where the coarse-grid model spans the entire 2D domain with 4x4 elements. The LES (fine-grid) model comprises 2x2 elements within each of the coarse-grid model elements. In practice, however, we embed NxN elements inside each coarse-grid model element with the coarse-grid model comprised of NxN elements. In this setup, we can imagine using, e.g., N+1 MPI ranks or accelerators.

The next figure (LSP Grid -> LSP Column -> SSP Grid) shows a second xNUMA configuration using the classical MMF approach where each vertical column of the LSP model (black points) contain SSP models that are used to enrich the LSP solution, column by column.

As a proof-of-concept, we ran a "do no harm" test whereby we simulate the rising thermal bubble problem on the LSP grid enriched by SSP grids along each vertical column and reproduce the expected result.

Numerical Methods in xNUMA

Currently, xNUMA only uses the continuous Galerkin method in discontinuous storage (CGd) for set2NC; however, the discretization allows for contravariant forms of all the derivatives. For stabilization, xNUMA uses 2nd and 4th order tensor viscosity methods and filters. xNUMA allows for no-flux, periodic, and implicit Rayleigh damping boundary conditions. The time-integrators use explicit RK methods as well as 3d-IMEX RK methods of orders 2 through 5.

Who built xNUMA

xNUMA is being developed in the group of Prof. Frank Giraldo in the Computational Mathematics Laboratory of the Department of Applied Mathematics at the Naval Postgraduate School. National Research Council postdocs are contributing to the construction of xNUMA. Currently, the main developer is Dr. Soonpil Kang.

Who funded xNUMA

xNUMA is funded by the Office of Naval Research (ONR), Marine Meteorology Directorate.

xNUMA Development

There are two precursors to xNUMA: (1) sNUMA and (2) pNUMA. A third component is under development; xNUMA will subsume the other components. The "s" stands for "serial", the "p" for parallel, and the "x" for accelerator (but can also be run in parallel or serial). pNUMA is able to run N LES models plus 1 coarse-grid model using a total of N MPI ranks. A more general parallelization allowing for more MPI ranks to solve the coarse-grid model is currently under way; in addition, we have added moist microphysics to xNUMA. In xNUMA, we plan to use N MPI ranks to solve the coarse-grid model whereas the LES models will be run on as many MPI ranks that have separate GPU cards.

How to get access to xNUMA

xNUMA is currently housed in a private Github organization here. We expect to release xNUMA as an open-source code in the future. Since the code is going through rapid development, we cannot support users but are open to giving collaborators early access. Email Frank Giraldo at frank.giraldo@gmail.com with the subject line: Access to xNUMA followed by a reason for needing access to the code.