The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The November webinar is titled Reducing Technical Debt with Reproducible Containers, and will be presented by Tanu Malik (DePaul University). The webinar will take place on Wednesday, November 4, 2020 at 1:00 pm ET.
Abstract:
Computational experiments can be challenging to reproduce; researchers have to choose between pursuing a fast-paced research agenda and developing well-organized, sufficiently documented, and easily reproducible software. Like incurring fiscal debt, there are often tactical reasons to take on technical debt in scientific software—such as deferring documentation, organization, refactoring, and unit tests when pursuing a new idea or meeting a conference deadline. However, more often than not, researchers do not repay this technical debt, leading to irreproducible experiments.
The webinar will describe different levels of technical debt and quantify the cost of not repaying the technical debt. The presenter will introduce isolation in containers as a powerful mechanism for reducing portability debt and describe limitations of current container tools. The presenter will introduce a vision of a reproducible container that aims to automate repayment of different types of technical debt, and will describe the current state of this vision with three tools that use isolation, encapsulation, and monitoring to include necessary and sufficient content in the container—both in terms of software and data, and describe the contents of the container. Finally, the presenter will show results of using reproducible containers on domain science and HPC use cases, and provide guidance.
UPC++: An Asynchronous RMA/RPC Library for Distributed C++ Applications
The UPC++ Team will offer a tutorial at SC20 introducing the basic concepts and advanced optimization techniques of UPC++, a C++ library supporting Partitioned Global Address Space (PGAS) programming. The tutorial will introduce the UPC++ memory and execution models and examine basic algorithm implementations. Participants will gain hands-on experience incorporating UPC++ features into several application examples. The presenters will also examine two irregular applications (metagenomic assembler and multifrontal sparse solver) and describe how they leverage UPC++ features to optimize communication performance.
Presenters:
- Katherine A. Yelick (Lawrence Berkeley National Laboratory, University of California, Berkeley)
- Amir Kamil (Lawrence Berkeley National Laboratory, University of Michigan)
- Dan Bonachea (Lawrence Berkeley National Laboratory)
- Paul H. Hargrove (Lawrence Berkeley National Laboratory)
More information about the tutorial (including registration) is available at the SC20 website.
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The December webinar is titled Software Design for Longevity with Performance Portability, and will be presented by Anshu Dubey (Argonne National Laboratory and University of Chicago). The webinar will take place on Wednesday, December 9, 2020 at 1:00 pm ET.
Abstract:
In the era of simultaneously increasing heterogeneity in hardware and application software, the topics of performance portability and longevity may seem at cross purposes. Key to achieving either objective individually is software design. Achieving both simultaneously is a much harder challenge, yet, in today’s scientific computing landscape neither objective can be ignored. Questions that science is posing to computation are more complex, which imply greater investment in building science capabilities in the software, and therefore longevity is important. Those questions need more capable hardware, which can only be obtained only through evermore heterogeneous platforms. This webinar will present a few basic principles of scientific software design that have been instrumental in mitigating some of the challenges that applications developers are facing. These principles represent a combination of experience from the presenter’s own project and from the Exascale Computing Project Performance Portability Panel Series that took place during summer of 2020.
Scientific visualization is a key component of simulation workflows. Visualization tools transform mesh data into pictures that help simulation users digest and understand complex science results. Visualization is often done post-hoc using files written to disk, however at extreme scales that model is strained by gaps between I/O and compute speeds. In memory processing, a type of in situ processing, can help avoid these I/O bottlenecks and allow users to analyze more data.
We are developing Ascent to support production in memory visualization for HPC simulations. It leverages both distributed-memory (MPI) and many-core (CUDA or OpenMP) parallelism to provide ray-tracing, contouring, etc using the same HPC resources as a host simulation application. Ascent is one of the infrastructure thrusts of the ECP ALPINE Software Technology Project and will be used to deploy new automated data reduction algorithms developed by ALPINE. The ALPINE is team actively working to help integrate Ascent into ECP application codes and support visualization needs.
This tutorial will cover how to present data to Ascent, transform data, render pictures, and export results. Attendees can follow along and execute tutorial examples using cloud hosted Jupyter Notebooks or a public Ascent install.
Please join us to learn about Ascent, 10am – 11am PT, 1pm – 2pm ET!
Presenters:
- Cyrus Harrison (Lawrence Livermore National Laboratory)
- Matthew Larsen (Lawrence Livermore National Laboratory)
- Hank Childs (University of Oregon)
Useful Links:
The tutorial has been recorded and it is available at the ECP Youtube Channel. The slides are available here.
In response to the COVID-19 pandemic and need for many to transition to unplanned remote work, the IDEAS-ECP Productivity project launched the panel series Strategies for Working Remotely, which explores important topics in this area.
Abstract:
- Earlier this year many workers abruptly transitioned from a primarily on-site to a primarily remote work experience due to a global pandemic. As we bring 2020 to a close, what have we learned so far, and what do we have yet to learn about working remotely, and working effectively in hybrid configurations? In this fireside chat, we look at key highlights from each of the Strategies for Working Remotely panel discussions in the series and dig deeper. What has worked, why, and where can we improve? What do we have yet to learn, or unlearn? “Ask me anything” questions can be submitted by the audience in advance to [email protected].
Panelists:
- Lori Diachin, Lawrence Livermore National Laboratory
- Tom Evans, Oak Ridge National Laboratory
- Elaine Raybourn, Sandia National Laboratories
Moderator:
- Ashley Barker, Oak Ridge National Laboratory
E4S Hackathon
We encourage ECP ST and related team members to attend the E4S Hackathon on January 12, 2021. The hackathon will focus on the porting of numerical tools, libraries, and applications to three different GPU architectures — NVIDIA, Intel, AMD — and making them available through Spack. The hackathon will also help developers update their product repositories to better map files for updating the content in the E4S DocPortal. The Spack and E4S teams will be available to answer questions and help ECP software and application developers. The hackathon will be a virtual Zoom based event. The Spack and E4S teams will enable access to a testbed at the University of Oregon featuring the three GPU architectures with a common filesystem for porting and testing.
Important: Participants are required to enter a temporary password when registering (through “Tickets”). Please do not use any important password and immediately change the password when logging in for the first time on the testbed at the University of Oregon.
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The January webinar is titled Extreme-scale Scientific Software Stack (E4S), and will be presented by Sameer Shende (University of Oregon) and David Honegger Rogers (Los Alamos National Laboratory). The webinar will take place on Wednesday, January 13, 2021 at 1:00 pm ET.
Abstract:
With the increasing complexity and diversity of the software stack and system architecture of high performance computing (HPC) systems, the traditional HPC community is facing a huge productivity challenge in software building, integration and deployment. Recently, this challenge has been addressed by new software build management tools such as Spack that enable seamless software building and integration. Container based solutions provide a versatile way to package software and are increasingly being deployed on HPC systems. The DOE Exascale Computing Project (ECP) Software Technology focus area is developing an HPC software ecosystem that will enable the efficient and performant execution of exascale applications. Through the Extreme-scale Scientific Software Stack (E4S), it is developing a curated, Spack-based, comprehensive and coherent software stack that will enable application developers to productively write highly parallel applications that can portably target diverse exascale architectures. E4S provides both source builds through the Spack platform and a set of containers that feature a broad collection of HPC software packages. E4S exists to accelerate the development, deployment, and use of HPC software, lowering the barriers for HPC and AI/ML users. It provides container images, build manifests, and turn-key, from-source builds of popular HPC software packages developed as Software Development Kits (SDKs). This effort includes a broad range of areas including programming models and runtimes (MPICH, Kokkos, RAJA, OpenMPI), development tools (TAU, PAPI), math libraries (PETSc, Trilinos), data and visualization tools (Adios, HDF5, Paraview), and compilers (LLVM), all available through the Spack package manager. The webinar will describe the community engagements and interactions that led to the many artifacts produced by E4S, and will introduce the E4S containers that are being deployed at the HPC systems at DOE national laboratories.
The presenters will discuss the recent efforts and techniques to improve software integration and deployment for HPC platforms, and describe recent collaborative work on reproducible workflows between E4S and the Pantheon project. Pantheon provides a set of working examples of end-to-end workflows using ECP apps, infrastructure and postprocessing, focused on common vis/analysis operations and workflows of interest to application scientists and show a video of the workflow.
ALCF Developer Sessions Webinar Series
This webinar will cover the NVIDIA HPC SDK, a comprehensive, integrated suite of compilers, libraries, and tools for ThetaGPU and other NVIDIA HPC platforms. The SDK is equipped with new features that continue to open GPU computing to a wider audience of developers and users, including automatic acceleration and tensor core programmability in standard languages, and novel libraries for compute and communication. The webinar will be presented by Tim Costa, HPC Software Product Manager at NVIDIA.
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The February webinar is titled Good Practices for Research Software Documentation, and will be presented by Stephan Druskat (Friedrich Schiller University Jena) and Sorrel Harriet (Leeds Trinity University). The webinar will take place on Wednesday, February 10, 2021 at 1:00 pm ET.
Abstract:
This webinar aims to introduce the importance of software documentation and the different approaches that may be taken at various stages, and on various levels, in the software development life cycle. Through the sharing of examples and stimulative questions, the speakers aim to encourage the audience to reflect on the relationship between documentation and process, and to make informed choices about when and how to document their software.
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The March webinar is titled An Overview of the RAJA Portability Suite, and will be presented by Arturo Vargas (Lawrence Livermore National Laboratory). The webinar will take place on Wednesday, March 10, 2021 at 1:00 pm ET.
Abstract:
The RAJA Portability Suite is a collection of open-source software libraries that enable developers to write single-source applications that are portable across a wide range of HPC architectures. The Suite contains tools for portable loop execution (RAJA) and memory management (Umpire and CHAI). The development of the Suite is motivated by the needs of production multiphysics codes, which must run efficiently on laptops, commodity clusters, and massively parallel advanced technology systems at any point in time as well as across multiple platform generations. The scale and complexity of these applications require that they be able to employ system-appropriate native programming models, such as OpenMP, CUDA, and HIP, without significant source code modification. The abstractions that the RAJA Portability Suite provides enable such portable single-source application development. The Suite is used in a diverse range of production codes at Lawrence Livermore National Laboratory (LLNL). It is also funded as a Software Technology Project in DOE’s Exascale Computing Project, where the Suite supports a number of key applications. The webinar will provide an overview of the Suite and its capabilities and discuss status and plans to support applications on exascale platforms. The webinar will present code examples that illustrate basic usage and compare to programming with native programming models, and performance results for several applications that rely on the Suite for platform portability.
