The Exascale Computing Project’s (ECP’s) Co-design Center for Particle Applications (CoPA) aims to prepare particle applications for exascale computing. CoPA provides proxy applications (apps) and libraries that enable exascale readiness. CoPA’s research focuses on submotifs that address (1) short-range particle–particle interactions, which often dominate molecular dynamics (MD) and smoothed particle hydrodynamics methods; (2) long-range particle–particle interactions, used in electrostatic MD and gravitational N-body methods; (3) particle-in-cell methods; and (4) linear-scaling electronic structure and quantum molecular dynamics (QMD) algorithms. An article written by CoPA researchers reviews particle application developments at the Center. It was published online in July 2021 as a special issue paper in the ECP issue of the International Journal of High Performance Computing Applications.
Proxy apps such as CoPA-developed ExaMiniMD, CabanaMD, CabanaPIC, and ExaSP2 are used to evaluate the viability of incorporating various algorithms, data structures, and architecture-specific optimizations and associated trade-offs. CoPA’s two main libraries—Cabana Particle Toolkit and PROGRESS/BML QMD Libraries—optimize data structure, layout, and movement to deliver performance portability, flexibility, and scalability across architectures with and without GPU acceleration. The Cabana Particle Toolkit applies to short-ranged, long-ranged, and particle-grid interactions, while PROGRESS/BML QMD Libraries provide methods and solvers for quantum mechanical interactions. The libraries are used by ECP and non-ECP application partners. CoPA includes members from the following ECP partner applications: WDMapp/XGC, EXAALT/LAMMPS/LATTE, ExaSky/HACC, and ExaAM/PicassoMPM.
CoPA projects have shown that proxy apps are beneficial for rapid prototyping of different ideas and performance speedup and that co-design teams of domain scientists, computational scientists, and expert programmers in hardware-specific languages and programming models is an effective work mode for advancing particle applications. Specific project impacts include the transitioning of XGC from FORTRAN to C++ using the Cabana Particle Toolkit and Kokkos and the development of a hybrid quantum mechanical/molecular mechanical QM/MM protein simulation capability, which has proven useful in biomedical research including studies of SARS-CoV-2 proteins.
CoPA’s ongoing work on particle application readiness for exascale computing includes using Cabana/Kokkos in XGC for GPU off-loading of the ion particle operation in the ITER tokamak for performance-portable simulations on exascale platforms; improving additive manufacturing simulations with the PicassoMPM code; and developing HACC-based proxy apps towards a potential full N-body cosmological simulation code.
Mniszewski, Susan M., James Belak, Jean-Luc Fattebert, Christian F.A. Negre, Stuart R. Slattery, Adetokunbo A. Adedoyin, Robert F. Bird, Choongseok Chang, Guangye Chen, Stéphane Ethier, Shane Fogerty, Salman Habib, Christoph Junghans, Damien Lebrun-Grandié, Jamaludin Mohd-Yusof, Stan G. Moore6, Daniel Osei-Kuffuor, Steven J. Plimpton, Adrian Pope, Samuel Temple Reeve, Lee Ricketson, Aaron Scheinberg, Amil Y. Sharma, and Michael E. Wall. “Enabling Particle Applications for Exascale Computing Platforms.” 2021. Special Issue Paper, International Journal of High Performance Computing Applications (July).
https://doi.org/10.1177/10943420211022829
Anatomy of a time step is shown for each of the particle application submotifs addressed by the Exascale Computing Project’s Co-design Center for Particle Applications. Communication-intensive steps and compute/memory-intensive steps are shown in yellow and blue, respectively.
