The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:DRBSD-7: The 7th International Workshop on Data Analysis and Reduction for Big Scientific Data
Authors: Yuanjian Liu (University of Chicago); Sheng Di (Argonne National Laboratory (ANL)); Kai Zhao (University of California, Riverside); Kyle Chard (University of Chicago); Wentao Ding and Sian Jin (Washington State University); and Cheng Wang, Ian Foster, and Frank Cappello (Argonne National Laboratory (ANL))
Abstract: Multiple lossy compression frameworks have been proposed to address the vast volumes of data being produced by scientific simulations. Setting different precisions to different ranges of data based on researchers' interests appears to be a promising approach to further improve the compression ratios of many scientific datasets. However, previous researches have not clearly demonstrated how to apply different precisions to different ranges of data and not many real-world datasets are evaluated to show the effectiveness of this idea. In this work, we investigate a specific compression method that can set multiple error bounds based on the SZ framework. We carefully assess its effectiveness using real-world datasets which have concrete demands on multiple precisions. The experimental results show that the multi-error-bounded lossy compression can achieve a 15% improvement in compression ratio, with negligible overhead in compression time.