Magic Castle - Enabling Scalable HPC Training through Scalable Supporting Infrastructures
Event Type
Workshop
Education and Training and Outreach
Emerging Technologies
HPC Training and Education
Workforce
W
TimeMonday, 15 November 202110:30am - 11am CST
Location230
DescriptionMagic Castle aims to recreate the supercomputer user experience in public or private clouds. To define the virtual machines, volumes, and networks that are required in a cloud-provider agnostic way, it uses the open-source software Terraform and HashiCorp Language (HCL). These resources are then configured using the configuration management and deployment tool Puppet to replicate a virtual HPC infrastructure with a full scientific software stack, and including a feature-rich JupyterHub environment. The final resource is accessible both through a web browser and via SSH, making it trivially OS agnostic for the trainees.
Through the use of Magic Castle, we demonstrate that it is possible to dynamically provision virtual HPC system(s) in public or private cloud infrastructure easily, quickly and cheaply. We also show that such infrastructures can support accelerators and fast interconnects, meaning that they can still be considered "true" HPC resources.
Through the use of Magic Castle, we demonstrate that it is possible to dynamically provision virtual HPC system(s) in public or private cloud infrastructure easily, quickly and cheaply. We also show that such infrastructures can support accelerators and fast interconnects, meaning that they can still be considered "true" HPC resources.