Software Technology

Data and Visualization

EZ: Fast, Effective, Parallel Error-bounded Exascale Lossy Compression for Scientific Data

Principal Investigator: Franck Cappello, Argonne National Laboratory

The EZ project is endeavoring to provide a production-quality lossy compressor for scientific datasets, respecting user-set error bounds (Absolute, Relative, PSNR), which will have two major impacts: 1) allow for datasets to be transferred and stored faster, reducing the time to completion of the simulations, 2) allow for smaller foot print of datasets in network, storage and memory, increasing the size of problems that can be run.

The EZ project extends and improves the SZ lossy compressor for structured and unstructured scientific datasets respecting user-set error controls. SZ is applicable to simulation datasets as well as datasets form experiments. EZ improves the effectiveness of SZ’s three core compression algorithms: prediction, quantization, and coding. The EZ project focuses on compression performance improvement, memory cost minimization, parallelization, additional error controls, and integration of SZ into parallel I/O environments (PnetCDF, ADIOS, HDF5).

SZ is released under a BSD license.