xSDK4ECP

The large number of software technologies being delivered to application developers poses challenges, especially if the application needs to use more than one technology simultaneously, such as using a linear solver from the PETSc/TAO mathematics library in conjunction with a time integrator from the SUNDIALS library. The Extreme-Scale Scientific Software Development Kit (xSDK) project is an effort to create a value-added aggregation of mathematics and scientific libraries and increase the combined usability, standardization, and interoperability of these libraries.

Project Details

As architectures become more complex and applications become reliant on multiple libraries to supply performant capabilities on those architectures to achieve their exascale performance and science goals, the ability to incorporate multiple libraries into one executable is necessary. The xSDK project is an effort to provide the turnkey installation and use of popular scientific packages needed for next-generation scientific applications. The xSDK project is working to (1) enable the seamless combined use of diverse, independently developed numerical libraries as needed by exascale applications; (2) develop interoperability layers among numerical libraries to improve code quality, access, usability, interoperability, and sustainability; and (3) provide an aggregate build and install capability for the numerical libraries that supports hierarchical, modular installation.

The xSDK project focuses on community development and a commitment to combined success via quality improvement policies, better build infrastructure, and the ability to use numerical libraries in combination to solve large-scale multiphysics and multiscale problems. The project represents a different approach to coordinating library development and deployment. Previously the xSDK, scientific software packages were cohesive with one team effort but not across these efforts. The xSDK goes a step further by developing community policies followed by each independent library included in the xSDK. This policy-driven, coordinated approach enables independent development that still results in compatible and composable capabilities. Moreover, the xSDK provides a forum for collaborative numerical library development, helping independent teams accelerate the adoption of best practices, enabling the interoperability of independently developed libraries, and improving developer productivity and library sustainability.

The xSDK project also entails a coordinated effort to investigate and deploy multiprecision functionality in the Exascale Computing Project (ECP) software technology (ST) ecosystem to enable the use of low-precision hardware function units, reduce the pressure on memory and communication interfaces, and achieve improved performance. After conducting a comprehensive analysis of existing theory and multiprecision functionalities, the project focuses on developing multiprecision capabilities in xSDK member libraries, acknowledging the cross-library interoperability as a first-class design consideration. After a thorough investigation of the numerical robustness and performance potential, the project will start integrating multiprecision functionality into ECP application projects.

Principal Investigator(s):

Ulrike Yang, Lawrence Livermore National Laboratory

Collaborators:

Lawrence Livermore National Laboratory; Argonne National Laboratory; Sandia National Laboratories; Lawrence Berkeley National Laboratory; Oak Ridge National Laboratory; University of California, Berkeley; University of Tennessee, Knoxville; University of Oregon; Karlsruhe Institute of Technology; University of Manchester; Charles University at Prague; Tech-X Corporation; NexGen Analytics

Progress to date

  • The xSDK team released version 0.6.0, which added two new xSDK members—heFFTe and SLATE—to the other xSDK libraries—AMRex, ButterflyPACK, deal.II, DTK, Ginkgo, hypre, libEnsemble, MAGMA, MFEM, Omega_h, PETSc, PHIST, PLASMA, preCICE, PUMI, SLEPc, STRUMPACK, SUNDIALS, SuperLU, Tasmanian, and Trilinos—and the two domain components Alquimia and PFLOTRAN.
  • The team continued to develop community policies. Recent changes include a new recommended policy on documentation quality and Spack variant guidelines that replace the installation policies of previous versions.
  • The team developed a suite of example codes that demonstrate interoperabilities between select xSDK libraries, xsdk-examples. The suite includes a build system and documentation in the subfolders of the codes and can be built with Spack.
  • The first version of the GPTune autotuning software for the parameter optimization of high-performance computing codes was released. It was evaluated by tuning several ECP math libraries and applications codes. It outperformed two state-of-the-art tuners, OpenTuner and HpBandster, with a speedup of up to 2.5 when tuning ScaLAPACK QR.
  • The project developed a cross-platform “memory accessor” that separates the main memory precision format from the arithmetic format used in the on-core calculations and handles the conversion between the user-selected formats of on-the-fly in processor registers.
  • Based on the memory accessor, a Compressed Basis GMRES (generalized minimal residual method) was developed that stores the Krylov basis vectors in lower than working precision, significantly accelerating iterations with little impact on convergence.

National Nuclear Security Administration logo U.S. Department of Energy Office of Science logo