Software Technology

Data and Visualization

DataLib: Data Libraries and Services Enabling Exascale Science

Principal Investigator: Rob Ross, Argonne National Laboratory

This project is centered on extending, enhancing, and tuning the ROMIO, PnetCDF, Darshan I/O libraries and the Mercury suite of I/O services for exascale platforms. It approaches the challenge of providing effective storage and I/O solutions via three complementary thrusts. First, it will support the widely used ROMIO and Parallel netCDF libraries as standards-based methods for interfacing to storage from HPC codes and extending these to better leverage new technologies (e.g., burst buffers). Second, it will  enhance the Darshan I/O characterization tool to provide better intelligence on how application codes are using I/O systems in order to assist in debugging and performance optimization at scale. Third, it will harden and employ the Mercury RPC library and Mochi suite of tools to enable applications to employ specialized data services for their specific needs and to accelerate the development of other ST technologies.