RollingGantryCrane: Automation for Unpacking Containers into HPC Environments
Event Type
Workshop
Applications
Emerging Technologies
Reproducibility and Transparency
Software Engineering
System Software and Runtime Systems
Workflows
W
TimeSunday, 14 November 20212:30pm - 3pm CST
Location231-232
DescriptionSoftware containers are an important common currency for portability and reproducibility in the modern world of computing. While they are easy to share through public registries, usage documentation is often lacking, effectively leaving users with black boxes. RollingGantryCrane (RGC) is an open-source tool that takes generic software containers and automatically exposes the internal software through LMOD environment modules. Users provide the container URLs they wish to use, and RGC pulls the containers, collects descriptive metadata from public repositories, scans for non-standard executables on each container’s search path, and generates LMOD modulefiles with help text and shell functions that transparently expose applications directly to the command line interface. RGC has been used in production since early 2019 on five production systems at The Texas Advanced Computing Center (TACC), allowing users to create bespoke modules and serving over 3000 unique tools from the BioContainers project.