Researchers working with the nuclear fusion device simulation application WDMApp for the Exascale Computing Project have developed EFFIS, a workflow framework currently used in complex core-edge code coupling problems to automate visualizations of scientific, diagnostic, and performance results using a collaborative web-based dashboard. EFFIS consists of a library, command line utilities, and a collection of runtime daemons. The framework eases the difficulty of implementing workflows involving both strong and weak coupling of workflow components, delivering easier integration of efficient data movement strategies to achieve optimal workflow coupling. The work was published in May 2021 as a special issue paper in the ECP issue of the International Journal of High Performance Computing Applications.
Exascale simulations rarely involve running a single executable program, without the need for any additional processing to extract meaningful scientific insight. Workflows more often run simulations and then use the data in subsequent analysis calculations and visualizations. In addition, multiple applications may be run together in a coupled fashion, each solving a different piece of the problem. EFFIS optimizes the tasks application scientists typically execute before, during, and after runs with a common interface that aligns with best-practice strategies. EFFIS’s streaming approach to workflows allows for asynchronous analysis or visualization of larger datasets. The dashboard facilitates collaboration among application scientists.
The researchers have built EFFIS to be broadly applicable to sciences benefitting from exascale computing. Their future work will develop flexible, efficient command-and-control methods for high-performance computing workflows. For example, performance monitoring tools could notice when the edge application experiences long delays waiting for data from the core and prompt the workflow manager to redistribute resources or reconfigure the job. Another example is runtime anomaly detection, dynamically launching a new analysis process that monitors the region of the anomaly over time and computes statistics. Continuing work will also research how EFFIS could be used in preprocessing to validate the compatibility of components in multiple-application workflows, as well as enhance EFFIS’s postprocessing actions including automatically delivering the data to different storage devices or computing sites and refactoring/reducing the data for accelerated analysis.
Eric Suchyta, Scott Klasky, Norbert Podhorszki, Matthew Wolf, Abolaji Adesoji, C. S. Chang, Jong Choi, Philip E. Davis, Julien Dominski, Stéphane Ethier, Ian Foster, Kai Germaschewski, Berk Geveci, Chris Harris, Kevin A Huck, Qing Liu, Jeremy Logan, Kshitij Mehta, Gabriele Merlo, Shirley V. Moore, Todd Munson, Manish Parashar, David Pugmire, Mark S. Shephard, Cameron W Smith , Pradeep Subedi, Lipeng Wan, Ruonan Wang, and Shuangxi Zhang. “The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling Whole Device Modeling in Fusion Science.” 2021. Special Issue Paper, International Journal of High Performance Computing Applications (May).
https://doi.org/10.1177/10943420211019119
Schematic summarizing EFFIS usage in WDMApp workflows. EFFIS services are shown in pink, EFFIS technology in orange, and enabling technology in blue.
