The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The June webinar is titled Using the PSIP Toolkit to Achieve Your Goals – A Case Study at The HDF Group, and will be presented by Elena Pourmal (The HDF Group), Reed Milewicz (Sandia National Laboratories) and Elsa Gonsiorowski (Lawrence Livermore National Laboratory). The webinar will take place on Wednesday, June 9, 2021 at 1:00 pm ET.
Abstract:
Productivity and Sustainability Improvement Planning (PSIP) is a lightweight, iterative workflow that allows software development teams to identify development bottlenecks and track progress toward goals to overcome them. In this talk, we present an overview of the PSIP methodology and toolkit, and describe how the HDF5 Group used PSIP to make improvements in three key areas of their software development process.
Getting Started with E4S for Industry and Agencies Workshop
The Extreme-scale Scientific Software Stack (E4S) project aims to tame both the complexity and portability problems by creating an ecosystem of numerical libraries, runtime systems, and tools that lowers the barrier for entry for the HPC and AI/ML developer communities. E4S is a community effort to provide open source software packages for developing, deploying, and running scientific applications on HPC platforms. It aims to deliver a modular, interoperable, and deployable software stack based on the Spack package manager. The “Getting Started with E4S for Industry and Agencies” workshop was held on June 14-15, 2021. The intended audience was technical people from companies and US government agencies that are considering using E4S in their environment.
Workshop Agenda (Slides)
Day 1: Monday, June 14
8:00 AM – 9:30 AM – Overview E4S, Mike Heroux, Director of ECP Software Technology (Video)
- Who should attend:
- Technical Leaders and developers who may or may not be familiar with E4S;
- Prospective and first-time developers using E4S
- Topics addressed: This overview will discuss the philosophy behind E4S, the E4S components, and why industry should understand and consider adopting some of these components.
10:00 AM – 12:00 PM – Getting Started with E4S for Industry and Agency Tutorial, Sameer Shende (ECP Technical Lead for E4S) and team (Video)
- Who should attend: Prospective and first-time developers of E4S.
- Topics addressed: Hands-on instructions on the E4S build environment. Participants will learn how to incorporate E4S products into codes. Participants may use their own code or one provided.
Day 2: Tuesday, June 15
8:00 AM- 9:00 AM: E4S Q&A (Video)
Opportunity to discuss with Sameer Shende and Mike Heroux any high-level questions on what products to consider for specific industry/agency applications
- Who should attend:
- Technical Leaders and developers who may or may not be familiar with E4S and have questions about applicability to industry problems
- All Tutorial participants
- Topics addressed: High-level questions on what products to consider for specific industry/agency applications, unanswered questions from Day 1
9:00 AM – 11:45 AM: Continuation of hands-on session focusing on AI & ML, GPUs, Sameer Shende, E4S (ECP Technical Lead for E4S) and team (Video)
- Who should attend: Day 1 tutorial participants.
- Topics addressed: Continued hands-on instructions on the E4S environment
Topics addressed in tutorial
- Pantheon demo – David Rogers, LANL
- Performance evaluation tools: TAU
- E4S AI & ML packages
- E4S GPU support
- GPU runtime support: OneAPI, ROCm, and CUDA
- Advanced Spack: E4S Spack build cache
In response to the COVID-19 pandemic and transition to remote work, ECP and the IDEAS Productivity project launched the panel series Strategies for Working Remotely, which explores important topics in this area.
Abstract:
- Many virtual student internship programs across the national labs complex, industry, and academia are underway with students working remotely while geographically dispersed. What does a “virtual internship” lack, and what changes to mentoring or onboarding must be made? How can students get the most out of their internships—what opportunities should they look for, and how can they prepare for challenges? In the ninth installment of the panel discussion series, seasoned scientists offer career advice for students and early career scientists on weathering a pandemic, time management, and how to adjust to long-term changes while working remotely.
Panelists:
- Dorian Arnold, Emory University
- Rebecca Hartman-Baker, Lawrence Berkeley National Laboratory
- Christine Harvey, The MITRE Corporation
- Jay Lofstead, Sandia National Laboratories,
Moderators:
- Ashley Barker, Oak Ridge National Laboratory
- Elaine Raybourn, Sandia National Laboratories
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The July webinar is titled Mining Development Data to Understand and Improve Software Engineering Processes in HPC Projects, and will be presented by Boyana Norris (University of Oregon). The webinar will take place on Wednesday, July 7, 2021 at 1:00 pm ET.
Abstract:
The webinar will explore the role of software-related data mining tools in supporting productive development of high-performance scientific software. The webinar will discuss a variety of existing and emerging tools for analyzing code, git, emails, issues, test results, and dependencies, with the long-term goal of improving the understanding of development processes and enhancing developer productivity. The webinar will include specific analysis examples by applying a subset of those tools to ECP projects.
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The August webinar is titled Software Engineering Challenges and Best Practices for Multi-Institutional Scientific Software Development, and will be presented by Keith Beattie (Lawrence Berkeley National Laboratory). The webinar will take place on Wednesday, August 4, 2021 at 1:00 pm ET.
Abstract:
Scientific software is increasingly becoming the backbone of obtaining and validating scientific results. This is no longer just the case for traditionally computationally intensive areas but is now true across a wide variety of scientific disciplines. This circumstance elevates how scientific software is developed, independent of the field, to a new level of importance. Further, the multi-institutional nature of many science projects presents unique challenges to how scientific software can be effectively developed and maintained over the long term. In this webinar we present the challenges faced in leading the development of scientific software across a distributed, multi-institutional team of contributors, and we describe a set of best-practices we have found to be effective in producing impactful and trustworthy scientific software.
1st Variorum Lecture Series August 2021
The Variorum team will provide its first Variorum Lecture Series, where attendees will learn everything necessary to start using Variorum on various platforms to write portable power management code. The team will provide support through GitHub and Variorum mailing list during and after the lecture series. This Variorum Lecture Series will consist of two modules, each of 1.5 hours each. We will hold two sessions to accommodate different time zones as well as attendee schedules.
-
Module 1: August 6, 8:30am-10:00am PT / 11:30am-1:00pm ET, targeting US/European attendees, and August 20, 4:00pm-5:30pm PT / 7:00pm-8:30pm ET, targeting US/Asian attendees.
Module 2: August 13, 8:30am-10:00am PT / 11:30am-1:00pm ET, targeting US/European attendees, and August 27, 4:00pm-5:30pm PT / 7:00pm-8:30pm ET, targeting US/Asian attendees.
What is Variorum?
Variorum is a production-grade, open-source, vendor-neutral software infrastructure for exposing low-level control and monitoring of a system’s underlying hardware features. It can easily be ported to different hardware devices, as well as different generations within a particular device. This allows users to manage power, performance and thermal information seamlessly across hardware from different vendors. More specifically, Variorum’s flexible design supports a set of features that may exist on one generation of hardware, but not on another. Variorum can also be included as part of the system software stack for power management: such as runtime systems, resource managers, and other profiling tools. At present, Variorum supports 5 platforms (IBM, Intel, AMD, ARM and NVIDIA) and a total of ten microarchitectures across these platforms.
Contents of the Lectures
Module 1: Introduction to Variorum
- Challenges in Power Management and The HPC Power Stack
- Understanding Power Management Knobs on Intel, IBM, NVIDIA, ARM, and AMD platforms
- Variorum Library
- Build, dependencies, and setup
- Monitoring user applications non-intrusively
- Vendor-neutral Variorum API across diverse architectures
- Using Variorum for finer-grained monitoring, power capping, and management
Module 2: Integrating Variorum with System Software and Tools
- The HPC Power Stack revisited: need for power management at various levels
- GEOPM: job-level power management
- Kokkos and Caliper: application and workflow power management
- SLURM (Research Extensions): system-level power management
- Upcoming Features in Variorum
- The HPC Power Stack Roadmap
How to Attend
- The lecture series is available to everyone, and participants are welcome to attend any/all sessions.
- No-cost registration is necessary, meeting link and password will be sent to registrants. See “Tickets” above.
- Presenters will show in-depth demos during the lecture series. Presenters can provide support during and after the lecture series with setup and usage on supported architectures.
Presenters
- Stephanie Brink, Tapasya Patki, Aniruddha Marathe and Barry Rountree (Lawrence Livermore National Laboratory)
Module 1
Module 2
1st Variorum Lecture Series August 2021
The Variorum team will provide its first Variorum Lecture Series, where attendees will learn everything necessary to start using Variorum on various platforms to write portable power management code. The team will provide support through GitHub and Variorum mailing list during and after the lecture series. This Variorum Lecture Series will consist of two modules, each of 1.5 hours each. We will hold two sessions to accommodate different time zones as well as attendee schedules.
-
Module 1: August 6, 8:30am-10:00am PT / 11:30am-1:00pm ET, targeting US/European attendees, and August 20, 4:00pm-5:30pm PT / 7:00pm-8:30pm ET, targeting US/Asian attendees.
Module 2: August 13, 8:30am-10:00am PT / 11:30am-1:00pm ET, targeting US/European attendees, and August 27, 4:00pm-5:30pm PT / 7:00pm-8:30pm ET, targeting US/Asian attendees.
What is Variorum?
Variorum is a production-grade, open-source, vendor-neutral software infrastructure for exposing low-level control and monitoring of a system’s underlying hardware features. It can easily be ported to different hardware devices, as well as different generations within a particular device. This allows users to manage power, performance and thermal information seamlessly across hardware from different vendors. More specifically, Variorum’s flexible design supports a set of features that may exist on one generation of hardware, but not on another. Variorum can also be included as part of the system software stack for power management: such as runtime systems, resource managers, and other profiling tools. At present, Variorum supports 5 platforms (IBM, Intel, AMD, ARM and NVIDIA) and a total of ten microarchitectures across these platforms.
Contents of the Lectures
Module 1: Introduction to Variorum
- Challenges in Power Management and The HPC Power Stack
- Understanding Power Management Knobs on Intel, IBM, NVIDIA, ARM, and AMD platforms
- Variorum Library
- Build, dependencies, and setup
- Monitoring user applications non-intrusively
- Vendor-neutral Variorum API across diverse architectures
- Using Variorum for finer-grained monitoring, power capping, and management
Module 2: Integrating Variorum with System Software and Tools
- The HPC Power Stack revisited: need for power management at various levels
- GEOPM: job-level power management
- Kokkos and Caliper: application and workflow power management
- SLURM (Research Extensions): system-level power management
- Upcoming Features in Variorum
- The HPC Power Stack Roadmap
How to Attend
- The lecture series is available to everyone, and participants are welcome to attend any/all sessions.
- No-cost registration is necessary, meeting link and password will be sent to registrants. See “Tickets” above.
- Presenters will show in-depth demos during the lecture series. Presenters can provide support during and after the lecture series with setup and usage on supported architectures.
Presenters
- Stephanie Brink, Tapasya Patki, Aniruddha Marathe and Barry Rountree (Lawrence Livermore National Laboratory)
Module 1
Module 2
The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.
As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The September webinar is titled What I Learned from 20 Years of Leading Open Source Projects, and will be presented by Wolfgang Bangerth (Colorado State University). The webinar will take place on Wednesday, September 15, 2021 at 2:00 pm ET. This webinar will start one hour later than the usual time.
Abstract:
Scientific software has grown from loose collections of individual routines working on relatively simple data structures to very large packages of 100,000s to millions of lines of code, with dozens of contributors, and hundreds or thousands of users. In the process, the approaches to software development have also drastically changed: both the software packages as well as their development are professionally managed, with version control, extensive test suites, and automatic regression checks for every patch. Maybe more interestingly, the approaches to managing the *community* of software developers and users have also dramatically changed.
Having led two large, open source software projects (the finite element package deal.II, and the Advanced Simulator for Problems in Earth ConvecTion ASPECT) for more than 20 years, the presenter will share lessons learned about both the technical management of scientific software projects, as well as the social side of these projects.
In response to the COVID-19 pandemic and transition to remote work, ECP and the IDEAS Productivity project launched the panel series Strategies for Working Remotely, which explores important topics in this area.
Abstract:
- Many organizations abruptly transitioned from a primarily on-site to a primarily remote work experience last spring. However, organizations still have training needs that were once largely accomplished through in-person events such as workshops, hackathons, and tutorials. This panel shared what they learned during the past year in their efforts to bring more virtualization to what historically has worked for in-person training events. What worked well? What did not work? This panel shared their insights about lessons learned over the past year and how those experiences will inform plans moving forward when organizations can safely offer in-person training again.
Panelists:
- Kelly Barnes, The Carpentries
- Helen He, NERSC, Lawrence Berkeley National Laboratory
- Julia Levites, Nvidia Corporation
- Thomas Papatheodore, OLCF, Oak Ridge National Laboratory
Moderators:
- Ashley Barker, Oak Ridge National Laboratory
- Osni Marques, Lawrence Berkeley National Laboratory
New Features in the HDF5 1.13.0 Release
This webinar will cover the the major new features of the HDF5 1.13.0 release. It will cover pluggable virtual file drivers (VFDs) and changes to the virtual object layer (VOL), and show how to build and use the async, pass-through, and cache VOL connectors.
More information about the webinar, including registration, can be found here.
