The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:7th Workshop on Machine Learning in High Performance Environment
Authors: Vincent Dumont (Lawrence Berkeley National Laboratory (LBNL))
Abstract: We present a new software, HYPPO, that enables the automatic tuning of hyperparameters of various deep learning models. Unlike other hyperparameter optimization methods, HYPPO uses adaptive surrogate models and directly accounts for uncertainty in model predictions to find accurate and reliable models that make robust predictions. Using asynchronous nested parallelism, we are able to significantly alleviate the computational burden of training complex architectures and quantifying the uncertainty. HYPPO is implemented in Python and can be used with both TensorFlow and PyTorch libraries. We demonstrate various software features on time-series prediction and image classification problems as well as a scientific application in computed tomography image reconstruction. Finally, we show that we can reduce by an order of magnitude the number of evaluations necessary to find the most optimal region in the hyperparameter space and reduce by two orders of magnitude the throughput for such HPO process to complete.
Back to 7th Workshop on Machine Learning in High Performance Environment Archive Listing