The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:LLVM-HPC2021: The Seventh Workshop on the LLVM Compiler Infrastructure in HPC
Authors: Dounia Khaldi, Yuanke Luo, Bing Yu, and Alexey Sotkin (Intel Corporation); Bruno Morais (Northeastern University, Intel Corporation); and Milind Girkar (Intel Corporation)
Abstract: In this paper, we introduce a DPC++ matrix ex-tension to unify different tensor hardware: Intel® Advanced Matrix Extensions (Intel® AMX) to CPUs, NVIDIA® TPUs, IBM® POWER® MMA, etc. These tensor hardware units are usually accessed by low-level intrinsics or assembly to perform matrix operations. It is hard for scientists to program these domain-specific devices without the kind of high-level abstractions and efficient implementations we introduce here. We also extend the existing LLVM matrix intrinsics to represent this DPC++ extension and yield efficient Intel AMX code generation. Based on our case study of implementing this interface on Intel AMX hardware, we discuss some of the limitations of existing LLVM Intermediate Representation (IR) and how they can be overcome to exploit tensor hardware.