Simplifying Scientific Python Package Installation and Usage

The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC, and the DOE Exascale Computing Project (ECP), organizes the webinar series on Best Practices for HPC Software Developers.

As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The September webinar is titled Simplifying Scientific Python Package Installation and Usage; and will be presented by Amiya Maji (Purdue University). The webinar will take place on Wednesday, September 13, 2023, at 1:00 pm ET.

Abstract:

With the growing popularity of Python, installation and management of Python packages in HPC environments is emerging as a critical problem for researchers; the problem is exacerbated by the need to provide consistency across traditional batch workloads and interactive notebooks. This webinar will discuss how to simplify scientific Python package installation by streamlining environment management, dependency tracking, and runtime customizations through easy-to-use tools. The webinar will discuss challenges for installing Python packages in HPC environments and present the best practices suggested by various HPC centers. Many of these best practices have been incorporated into a tool, conda-env-mod, developed by the speaker and his collaborators. HPC centers can further customize the tool and its module templates to incorporate additional software dependencies and provide descriptive help messages. The deployment of the tool has significantly reduced errors and enabled sharing of Python package installations among users. The webinar will give an overview of installing Python packages with conda-env-mod.