Strengthening Reproducibility for SC21 and Beyond

Authors: Carlos Maltzahn (University of California, Santa Cruz), Ivo Jimenez (University of California, Santa Cruz; Vectorized, Inc), Anjo Vahldiek-Oberwagner (Intel Labs), Le Mai Weakley (Indiana University), Stephen Lien Harrell (Texas Advanced Computing Center (TACC), University of Texas), Scott Michael (Indiana University), Tanu Malik (DePaul University)

Abstract: Come and join a Reproducibility BoF with the leadership of the SC21 Reproducibility Initiative! Hear about their experience, the experiences of authors and reviewers, and share what worked for you, what didn’t, and ways to improve the reproducibility initiatives of future SCs. This interactive session will include short presentations about outcomes, challenges and innovations of this year’s Initiative and will provide ample opportunity for you to chime in with questions and comments.

Long Description: The SC21 Reproducibility Initiative builds on experience gathered at SC20 by including members of the SC20 Transparency and Reproducibility Initiative, brings in outside artifact evaluation expertise from top-tier systems conferences such as SOSP’19, USENIX Security’20, and OSDI’20, and pushes forward on artifact evaluation by encouraging functional artifacts that reproduce claims, organizing access to shared infrastructure for authors and reviewers, and help streamline the SC22 Reproducibility Challenge. This year’s Initiative also achieved a clear separation of responsibilities between technical paper program committees and the Artifact Description / Artifact Evaluation committee and introduced the Best Reproducibility Advancement Award (pending approval by the SC Steering Committee at time of this submission).

URL: https://sc21.supercomputing.org/submit/reproducibility-initiative/


Back to Birds of a Feather Archive Listing