The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop:XLOOP 2021: The 3rd Annual Workshop on Extreme-Scale Experiment-in-the-Loop Computing
Authors: Soren Bear (University of California, Santa Barbara); Dula Parkinson (Lawrence Berkeley National Laboratory (LBNL), Advanced Light Source); Lisa Gerhardt (Lawrence Berkeley National Laboratory (LBNL), National Energy Research Scientific Computing Center (NERSC)); Mel Sibony (Lawrence Berkeley National Laboratory (LBNL), Advanced Light Source); and Bjoern Enders (Lawrence Berkeley National Laboratory (LBNL), National Energy Research Scientific Computing Center (NERSC))
Abstract: Today, beamlines at DOE light sources produce vast amounts of data that can easily outgrow local compute and storage capacity. Beamline operators need to find mechanisms that facilitate easy access for experimenters to their data (with appropriate access control) but that host it externally, to quickly free up constrained local storage capacity. Here, we show how HPC and network facilities can be coupled to a light source facility to build an efficient, low-maintenance data distribution platform.