SC21 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Support in OpenMP for Multi-GPU Parallelism


Authors: Raul Torres (Barcelona Supercomputing Center (BSC)), Vivek Kale and Abid Malik (Brookhaven National Laboratory), Tom Scogland (Lawrence Livermore National Laboratory), Roger Ferrer (Barcelona Supercomputing Center (BSC)), and Barbara Chapman (Stony Brook University)

Abstract: Nodes of emerging supercomputers have multiple GPUs, i.e., a multi-GPU, on them. Applications are often parallelized across the GPUs of a multi-GPU using MPI, but a more performant and portable solution for parallelizing across the GPUs is needed. OpenMP, which is used to parallelize computation within a multi-core or a GPU, could facilitate parallelization of computation across the GPUs in a performant and portable way through, e.g., low memory requirements compared to MPI and directive-based parallelization. In this work, we present a solution that provides support in OpenMP for parallelizing an application across GPUs of a multi-GPU through language extensions and compiler optimizations developed in LLVM's OpenMP implementation. Preliminary experimentation of our solution using the Stream benchmark on a cluster’s node having four GPUs suggests that our approach can be a performant, portable and easy-to-use solution for application programmers to harness the computational power of the GPUs of a node.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF


Back to Poster Archive Listing