The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:7th Workshop on Machine Learning in High Performance Environment
Authors: Gaurav Verma (Stony Brook University); Murali Emani (Argonne National Laboratory (ANL)); Chunhua Liao, Pei-Hung Lin, and Tristan Lucas Vanderbruggen (Lawrence Livermore National Laboratory); Xipeng Shen (North Carolina State University); and Barbara Chapman (Stony Brook University)
Abstract: Artificial Intelligence (AI) is being adopted in different domains at an unprecedented scale. A significant interest in the scientific community also involves leveraging machine learning (ML) to run high-performance computing applications at scale effectively. Given multiple efforts in this arena, there are often duplicated efforts when existing rich data sets and ML models could be leveraged instead. The primary challenge is a lack of an ecosystem to reuse and reproduce the models and datasets. In this work, we propose HPCFAIR, a modular, extensible framework to enable AI models to be Findable, Accessible, Interoperable, and Reproducible (FAIR). It enables users with a structured approach to search, load, save and reuse the models in their codes. We present the design, implementation of our framework and highlight how it can be seamlessly integrated into ML-driven applications for high-performance computing applications and scientific machine learning workloads.
Back to 7th Workshop on Machine Learning in High Performance Environment Archive Listing