SC21 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Code Generation and Optimization for Deep-Learning Computations on GPUs via Multi-Dimensional Homomorphisms

Authors: Richard Schulze, Ari Rasch, and Sergei Gorlatch (University of Muenster)

Abstract: We present our work-in-progress code generation and optimization approach for DL computations based on the algebraic formalism of multi-dimensional homomorphisms (MDH). We show that popular DL computations can be expressed in the MDH formalism, thereby exploiting the already existing MDH GPU code generation and optimization approach which so far has not been focused on DL. Furthermore, we show that the MDH formalism is more expressive than the state-of-the-art DL abstractions (e.g., as provided by TensorFlow): for example, MDH can express multiple DL computations (e.g., multiple element-wise computations) as a single MDH expression, enabling MDH optimizations (like tiling and parallelization) across the computations. Our experiments confirm that our MDH-based approach achieves better performance than the state-of-the-art, including Apache TVM and Facebook’s TC.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF

Back to Poster Archive Listing