The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:CANOPIE-HPC: Containers and New Orchestration Paradigms for Isolated Environments in HPC
Authors: Gregory J. Zynda (NVIDIA Corporation) and Shweta Gopaulakrishnan and John Fonner (University of Texas)
Abstract: Software containers are an important common currency for portability and reproducibility in the modern world of computing. While they are easy to share through public registries, usage documentation is often lacking, effectively leaving users with black boxes. RollingGantryCrane (RGC) is an open-source tool that takes generic software containers and automatically exposes the internal software through LMOD environment modules. Users provide the container URLs they wish to use, and RGC pulls the containers, collects descriptive metadata from public repositories, scans for non-standard executables on each container’s search path, and generates LMOD modulefiles with help text and shell functions that transparently expose applications directly to the command line interface. RGC has been used in production since early 2019 on five production systems at The Texas Advanced Computing Center (TACC), allowing users to create bespoke modules and serving over 3000 unique tools from the BioContainers project.