Best Practices for Using Proxy Applications as Benchmarks

The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.

As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The next webinar in the series was titled Best Practices for Using Proxy Applications as Benchmarks, and was presented by David Richards (Lawrence Livermore National Laboratory) and Joe Glenski (Hewlett Packard Enterprise). The webinar will took place on Wednesday, April 15, 2020 at 1:00 pm ET.


Proxy applications have many uses in software development and hardware/software co-design. Because most proxies are easy to build, run, and understand, they are especially appealing for use in benchmark suites and studies. This webinar examined the role of proxy apps as benchmarks and explained why run rules and a figure of merit are essential for a proxy application to function as an effective benchmark. The presenters showed how to evaluate the fidelity of benchmarks as a model for actual workloads and provided tips on creating problem specifications and other run rules. The presenters discussed what DOE facilities are looking for when they assemble benchmark suites for use in procurements. Finally, the presenters explained how system vendors use their benchmark suites and what practices they view as most (and least) effective.