Time integrators are at the core of every time-dependent simulation application. Additionally, many applications require the solution of linear algebraic systems of equations, whether through the use of an implicit approach for integrating the time dependence or for solution of steady-state systems. The SUNDIALS-hypre project is enhancing the SUNDIALS library for integrating differential systems in time by using state-of-the-art adaptive and additive time integration technologies and the hypre library for solving large systems of linear equations, both for use on exascale systems.

Project Details

Many exascale applications depend on efficient time integrators and linear solvers yet do not use state-of-the-art algorithms and cannot easily take advantage of algorithmic advances. Through flexible and efficient libraries, applications can more easily take advantage of new algorithms and more efficient implementations that will allow for easier adaptations to exascale architectures. The SUNDIALS-hypre project is enhancing the SUNDIALS and hypre libraries, which collectively deliver time integrators, nonlinear solvers, linear solvers, and preconditioners for use in scientific applications running on exascale systems.

SUNDIALS provides adaptive multistep and multistage time integrators designed to evolve systems of ordinary differential equations and differential algebraic equations. This suite also includes both Newton and fixed-point nonlinear solvers and scaled Krylov methods with hooks for user-supplied data structures and solvers. The SUNDIALS team is adding capabilities to support the required programming models for AMD and Intel GPU architectures. Additionally, the team is adding interfaces to new linear solvers that work with these programming models and that provide the solvers necessary for use in Exascale Computing Project (ECP) applications that rely on SUNDIALS for their systems integration. Additionally, the SUNDIALS team is adding a performance assessment layer and integrating with the Caliper package for greater performance understanding, as well as integrating with GitLab to ensure regular testing with high performance systems. Furthermore, the SUNDIALS team is continuing work with ECP applications in the integration of SUNDIALS into their applications, as well as assessing and improving performance within ECP applications on pre-exascale and exascale systems.

Hypre is a software library of high-performance preconditioners and solvers for the solution of large, sparse linear systems of equations on massively parallel computers. The library includes parallel multigrid solvers for structured and unstructured grid problems and features conceptual interfaces, which include a structured interface, a semi-structured interface, and a traditional linear-algebra-based interface. The hypre team continues to port the library to various GPU architectures, assesses the performance of these ports, examines performance bottlenecks, and develops new algorithms or algorithm variants that are better suited for pre-exascale and exascale architectures. The team also collaborates with various application developers to integrate hypre into their codes and optimize hypre’s performance.

Principal Investigator(s):

Carol Woodward, Lawrence Livermore National Laboratory


Lawrence Livermore National Laboratory; Southern Methodist University

Progress to date

  • The SUNDIALS team released new versions of the SUNDIALS suite that include support for CUDA for NVIDIA GPUs, HIP for AMD GPUs, and SYCL for Intel GPUs.  In addition, new support for use with Kokkos structures is included as well as interfaces to more GPU-enabled solvers in Kokkos Kernels and oneMKL Dense, MAGMA Dense, and Ginkgo.  The team also supported ECP applications through enhancements to the build system, review and revisions to the documentation, and through evaluation and optimization of algorithm usage.
  • The hypre team released new versions of the hypre library that are capable to run all CUDA-enabled hypre components on AMD GPUs via HIP and structured solvers, Krylov solvers and AMG on Intel GPUs using SYCL. The team also added a new factorized sparse approximate inverse (FSAI) preconditioner and smoother and add more capabilities to the MGR solver. The library now also includes the ability to switch between CPUs and GPUs at run time. The team also enabled the use of Umpire in hypre for additional performance gains.
  • More information on hypre can be found here.
  • More information on SUNDIALS can be found here.

National Nuclear Security Administration logo U.S. Department of Energy Office of Science logo