The many software technologies being delivered to the Exascale Computing Project (ECP) application developers pose challenges, especially if the application uses several software technology products. The software development kit (SDK) efforts identify meaningful aggregations of products within the programming models and runtimes, development tools, math libraries, and data and visualization technical areas. They also leverage composable scientific workflow technologies with the goal of increasing the interoperability, availability, quality, and sustainability of the software technologies being developed in the ECP while improving developer productivity for the software and application development teams. The SDKs are packaged and delivered through the Extreme-Scale Scientific Software Stack (E4S).
The forthcoming exascale systems require a sustainable, high-quality software ecosystem, and the ECP is chartered with delivering such an ecosystem that will continuously be improved by a robust research and development effort, deployed on advanced computing platforms, and broadly adopted by application teams and software developers to accelerate their science. The E4S and SDK efforts support a variety of activities, including establishing community policies aimed at increasing the interoperability between and sustainability of software technologies developed by the ECP and coordinating the delivery of those products through the E4S.
The Programming Models and Runtimes SDK effort identifies meaningful aggregations of products in this technical area. It provides the software infrastructure needed to enable and accelerate the development of exascale applications that perform well and are correct and robust while reducing the cost of initial development, ongoing porting, and maintenance.
The Development Tools SDK is a collection of independent projects specifically targeted to address performance analysis at scale. The team actively works to leverage techniques for common and identified problem patterns and creates new techniques for software quality assurance related to performance analysis tools while also supporting advanced techniques, such as autotuning and compiler integration, for upcoming heterogeneous architectures.
The Math Libraries SDK, or Extreme-Scale Scientific Software Development Kit (xSDK), predates ECP and is the largest ECP SDK effort. See more under Math Libraries.
The Data and Visualization SDK focuses on the delivery of efficient data management and storage libraries; services such as checkpoint/restart, monitoring, code coupling, and compression; and an efficient in situ visualization and analysis pipeline. The goal is to improve the deployment and usage of I/O and analysis capabilities.
The Scientific Workflows SDK is a new effort to create a rigorously tested community-curated collection of composable workflow technologies. The goal of this SDK is to ensure the exascale readiness of many scientific and other relevant ECP simulation workflows and improve their long-term sustainability. Toward this goal, the team is applying a community approach to design the SDK, integrate libraries and components, and define component interfaces.
The Software Ecosystem SDK effort coordinates activities between the various SDK projects, manages the release and testing of the E4S, and ensures that the software technologies within E4S can be built from source via the Spack package manager or used via prebuilt container images. Application developers can build the subset of the software technologies needed for their specific application. This effort fosters collaboration between software technologies and interacts heavily with the Hardware and Integration (HI) focus area to facilitate software product installation at the facilities.
Spack is a package manager for building and installing software on systems ranging from Linux and macOS laptops to exascale supercomputers. It was created to make HPC software complexity manageable, and allow users to easily build and install optimized, reproducible, and reusable HPC software. Spack has a broad appeal as it isn’t tied to a particular language; users can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, and easily swap compilers or target specific microarchitectures.
The Exascale Computing Project (ECP) adopted Spack as the standard tool to build the ECP software stack on all target platforms. (See E4S for more information about the Extreme Scale Software Stack and its use in scientific computing, industry and the cloud.) This decision was motivated by the observation that the space of possible ways to build software is combinatorial in size, and software reuse is hindered by the complexity of integrating a large number of packages and by issues such as binary compatibility. Building modern software to run correctly and with the highest possible performance on a system is a difficult task. It simply is no longer practical for users to manually download the source code for the packages they wish to install — including all the associated software that must be built and installed to get the software to run — and then iteratively build, debug, and manually tune all this software (potentially many packages) to achieve high performance.
Spack bridges this divide through automation and portable package recipes; specialized packages can be built per-site if needed, or lowest-common denominator packages can be built for those cases that do not need highly optimized performance. Packages are relocatable and can be used outside their original build environment. They can also be installed directly, without building, from the Spack binary cache.
These reasons motivated the ECP to adopt this essential technology to provide a packaging solution that can deploy on bare metal, in a container, or be rebuilt for a new machine on demand. Moreover, Spack provides environments that enable a number of software packages to be deployed together either on an HPC system, in the cloud, or in a container.
Spack automated much of the package build and deploy process. This means that collections of packages can be built consistently and compatibly from source and for multiple configurations including different package versions and combinations of versions; different compilers and operating systems; different low-level support libraries for the CUDA, HIP, and SYCL GPU layers; MPI libraries such as Open MPI and MPICH; and low-level mathematical libraries such as BLAS, LAPACK, and solvers.
The Spack solution has three main components.
Spack has broad adoption in the open-source community as an elegant solution toward solving many of the challenges presented by building software with many dependencies. Within the ECP, Spack plays a central role in the software development and delivery processes by supporting turnkey builds of the ECP Software Technology (ST) software stack for the purposes of continuous integration testing, installation, and seamless multiproduct builds. Spack’s cloud CI system runs over 115,000 builds per week—both in Amazon Web Services instances and on the U. The Oregon Frank cluster comprises a variety of the ECP-relevant GPU nodes that can be used for testing.
The Spack development, maintenance, and testing effort has been ongoing throughout the life of the ECP. All of the L4 ECP projects have both an entry in the E4S test suite as well as ’smoke test’ functionality in their Spack packages. These testing modalities are automatically executed by the E4S team when assessing releases. The Spack smoke tests are also executed by the Spack CI pipeline when evaluating pull requests for a package as well as part of preparation for a Spack release.
The E4S Spack Build Cache now includes binaries for ppc64le as well as x86_64 and includes over 53,000 total binaries. The E4S containers now support custom images for ECP applications, such as WDMApp, ExaWind, and Pantheon. A regular cadence of E4S releases will continue, with updated ST products, broader facility adoption, and inclusion in vendor offerings.
Spack continues to gain penetration across the ECP, and is the de facto delivery method for ECP ST products building from source. The ECP provided Spack packaging assistance for ST users and DOE facilities and has developed new capabilities for Spack that enable automated deployments of software at facilities, in containerized environments, and as part of continuous integration.
The E4S project continues, but funding is needed to continue this essential service.