SC21 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Utilizing Persistent Memory in Parallel I/O Libraries


Authors: Luke Logan (Illinois Institute of Technology); Jay Lofstead, Scott Levy, and Patrick Widener (Sandia National Laboratories); and Xian-He Sun and Anthony Kougkas (Illinois Institute of Technology)

Abstract: Scientific applications use parallel I/O (PIO) libraries to manage the complexity of I/O to distributed storage efficiently. PIO libraries, however, have not adequately adapted to the emergence of persistent memory (PMEM), which provides comparable performance to DRAM. They interact with PMEM using either POSIX or MPI-IO, which forces unnecessary network communications and data copies to occur when PMEM is available. In addition, PIO libraries introduce complicated APIs and configuration spaces that cause significant burden on programmers to store basic data structures. In this work, we explore the use of memory mapping as an alternative to POSIX and MPI-IO for interacting with PMEM through pMAP: a lightweight I/O library with a simplistic key-value store interface. We demonstrate that our approach can yield up to 2x improvement in performance over other popular PIO libraries by eliminating unnecessary data copies and network communications.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF


Back to Poster Archive Listing