Accelerating Applications using Edge Tensor Processing Units

Authors: Kuan-Chieh Hsu and Hung-Wei Tseng (University of California, Riverside)

Abstract: Neural network (NN) accelerators have been integrated into a wide range of computer systems. NN accelerators provide native hardware support for operations on multidimensional tensor data. Therefore, NN accelerators are theoretically tensor processors that can improve system performance for any problem using tensors as inputs and outputs.

This paper introduces General-Purpose Computing on Tensor Processing Units (GPTPU), an open-source, open-architecture framework that allows the developer and research communities to discover opportunities that NN accelerators enable for applications. GPTPU includes a powerful programming interface with efficient runtime system-level support; similar to that of CUDA and OpenCL in GPGPU computing; to bridge the gap between application demands and mismatched hardware/software interfaces.

We built GPTPU machine using Edge Tensor Processing Units (Edge TPUs). By leveraging the underlying Edge TPUs to perform main compute kernels, our results reveal that GPTPU achieves a 2.06× speedup over high-end CPUs and reduces energy consumption by 90%.




Back to Technical Papers Archive Listing