Principal Investigator: Michael Lang, Los Alamos National Laboratory
The LANL ATDM Data and Visualization project focuses on developing scalable systems software for the generation and analysis of the data produced by ECP applications. Scientific simulations running on Exascale platforms will continue to have access to enormous solid-state storage tiers that provide opportunities for rapid data acquisition. Similarly, massive campaign storage systems built from affordable media offer the opportunity for maintaining large data sets over longer campaigns to support longer duration simulations. Finally, advanced monitoring frameworks built using time-series databases and analysis storage systems are deploying within HPC data centers. Each of these systems requires significant systems software development to adequately leverage to improve the time to insight for scientists using extreme scale scientific simulations.
The LANL ATDM Data and Visualization project is essential for ECP because it is developing the systems software required to deploy advanced data collection and analysis capabilities into HPC data centers so that scientists can effectively use the data generated by extreme scale simulations. MarFS is the only campaign storage system within the DOE complex and is also the only storage system built using scale out principles with affordable SMR hard drives. The LANL monitoring stack is leveraging available open source software to build dashboards for monitoring both the data center and scientific application performance. Finally, the application level software technologies, Cinema and HXHIM, are being developed in coordination with LANL’s ECP application NGC to ensure that data collected during the simulation execution is of appropriate frequency, resolution, and viewport for later analysis and visualization by scientists. Cinema is an innovative way of capturing, storing and exploring extreme scale scientific data. Cinema is essential for ECP because it embodies approaches to maximize insight from extreme-scale simulation results while minimizing data footprint.