A computation being performed on one part of a large system often needs to access or provide data to another part of the system in order to complete a scientific simulation. The Partitioned Global Address Space (PGAS) programming model provides the appearance of shared memory accessible to all the compute nodes while implementing this shared memory behind the scenes using physical memory distributed across the nodes and primitives such as remote direct memory access (RDMA). The Pagoda project, as part of the US Department of Energy (DOE) Exascale Computing Project (ECP) is developing a productive and performant PGAS programming system to be deployed on exascale supercomputers.
The Pagoda project is developing a programming system to support exascale application development using the PGAS model, with a focus on supporting applications exhibiting irregular data structures and communication. There are two components to Pagoda: (1) a portable, high- performance, global-address-space communication library (GASNet-EX) and (2) a C++ template library (UPC++) that provides convenient abstractions for accessing and using the global address space. Together, these components enable the agile, lightweight communications that occur in applications, libraries, and frameworks running on exascale systems.
Pagoda enables effective scaling by avoiding the overhead of long, branchy serial code paths, leveraging hardware-offload support for RDMA, and supporting efficient fine-grained communication for both single- and multi-threaded environments. The importance of these properties is exacerbated by application trends; many applications in the ECP require the use of adaptive meshes, sparse matrices, dynamic load balancing, or similar techniques. Pagoda’s low-overhead communication mechanisms can maximize the injection rate and network utilization, tolerate latency through overlap, streamline unpredictable communication events, minimize synchronization, and efficiently support small- to medium-sized messages arising in many applications. Pagoda complements other programming models, enabling developers to focus their efforts on optimizing performance-critical communication.
The Pagoda team is focusing on developing new features that will support application and library requirements unique to the ECP and performance improvements that will enable the ECP software stack to exploit the best-available communication mechanisms. These include novel features being developed by vendors, such as remote direct memory access mechanisms offered by network hardware and on-chip communication between distinct address spaces. Recent work includes delivering accelerated communication of GPU-resident data for the systems on DOE’s exascale roadmap.