No Travel? No Problem.

Remote Participation
Error-Controlled, Progressive, and Adaptable Retrieval of Scientific Data with Multilevel Decomposition
Event Type
Big Data
Data Analytics
Data Management
File Systems and I/O
Memory Systems
Registration Categories
TimeThursday, 18 November 20211:30pm - 2pm CST
DescriptionExtreme-scale simulations and high-resolution instruments are generating an increasing amount of data, which poses significant challenges to both data storage and retrieval. The challenges in satisfying various analysis needs while minimizing I/O overhead should never be left unmanaged. In this paper, we propose a data refactoring/compressing/retrieval framework capable of: fine-grained data refactoring with regard to precision; incremental retrieving and recomposing data toward requested error bounds; and adaptively retrieving data in multi-precision and multi-resolution with respect to analysis. Our framework reduces the amount of data retrieved when multiple incremental precisions are requested. Experiments show that the amount of data retrieved under the same progressively requested distortion using our method is 64% less than that using state-of-the-art approaches. Parallel experiments with up to 1024 cores and ~600GB data show that our approach yields 1.36x and 2.52x performance over existing approaches in writing to and reading from persistent storage systems, respectively.
Back To Top Button