Student Cluster Competition

Benchmarks

 

LINPACK Benchmark
http://top500.org/project/linpack/

The Linpack Benchmark is a measure of a computer’s floating-point rate of execution. It is determined by running a computer program that solves a dense system of linear equations. It is used by the TOP 500 as a tool to rank peak performance. The benchmark allows the user to scale the size of the problem and to optimize the software in order to achieve the best performance for a given machine. This performance does not reflect the overall performance of a given system, as no single number ever can. It does, however, reflect the performance of a dedicated system for solving a dense system of linear equations. Since the problem is very regular, the performance achieved is quite high, and the performance numbers give a good correction of peak performance.

 

HPCG Benchmark
http://hpcg-benchmark.org/

The High Performance Conjugate Gradients (HPCG) Benchmark project is an effort to create a new metric for ranking HPC systems. HPCG is intended as a complement to the High Performance LINPACK (HPL) benchmark, currently used to rank the TOP500 computing systems. The computational and data access patterns of HPL are still representative of some important scalable applications, but not all. HPCG is designed to exercise computational and data access patterns that more closely match a different and broad set of important applications, and to give incentive to computer system designers to invest in capabilities that will have impact on the collective performance of these applications.

 

IO500 Benchmark
http://io500.org

The IO500 benchmark is a benchmark suite for High-Performance IO. It harnesses existing and trusted open-source benchmarks such as IOR and MDTest and bundles execution rules and multiple workloads with the purpose to evaluate and analyze the storage devices for various IO patterns. The IO500 benchmark is designed to provide performance boundaries of the storage for HPC applications regarding data and metadata operations under what are commonly observed to be both easy and difficult IO patterns from multiple concurrent clients. Moreover, there is a phase that scans for previously-created files that match certain conditions using a (possibly file system-specific) parallel find utility to evaluate the speed of namespace traversal and file attribute retrieval. The final score that is used to rank submissions in the list is a combined score across all the executed benchmarks.

 

Applications

 

Cardioid

Cardioid is a cardiac simulation suite for simulating clinical cardiac phenomena. It is capable of simulating both electrophysiological and mechanical organ-level simulations, and has processing tools for computing cardiac meshes, reconstructions of torso ECGs, and generating realistic cardiac fiber orientations. The Cardioid electrophysiology solver was a Gordon Bell finalist and has strong-scaled to all of the Vulcan supercomputer for a clinically relevant problem. The code is parallelized using MPI, and has separate optimized loops for taking advantage of OpenMP, SIMD instruction sets for CPU architectures, and CUDA for GPU architectures.

 

Quantum ESPRESSO

https://www.quantum-espresso.org/project/manifesto

Quantum ESPRESSO is a software package for first-principles electronic-structure calculations and materials modeling based on density-functional theory, plane wave basis sets, and pseudopotentials.

 

Mystery Application
At the start of the competition, teams will be given an application and datasets for a mystery application. Students will be expected to build, optimize and run this mystery application all at the competition.

 

Reproducibility Challenge
Once again, students in the cluster competition will be asked to replicate the results of a publication from the previous year’s SC conference. For this challenge, students will take on the role of reviewing an SC20 paper to see if its results are reproducible. The SC21 Reproducibility Committee has selected the paper “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by Ankit Srivastava, Sriram P. Chockalingam, and Srinivas Aluru to be the Student Cluster Competition (SCC) benchmark for the Reproducibility
Challenge this year.

Last year, thanks to the adoption of automatically generating an Artifact Descriptor (AD) during submission time, all accepted papers from SC20 featured an AD in their appendix. A team of reviewers selected the paper from these past papers based on the AD, author interviews, and its
suitability for the SCC. The authors and the Reproducibility Committee have been working to create a reproducible benchmark that builds on the paper’s results. During the SCC, the student teams will be asked to run the benchmark, attempting to reproduce the findings from the original paper under different settings with different data sets.

Safety Briefing

Saturday, November 13, 2021

This will be a virtual briefing and will be recorded and posted. Students must participate in the live briefing or view the recording before the competition begins.

 

Competition

Monday–Wednesday, November 15–17, 2021

The competition will run continuously Monday–Wednesday, November 15–17, 2021. Both benchmarks and applications will be released at the SCC Kickoff on Monday, November 15, 2021. Results can be submitted throughout the competition.

Violation of any rule may result in a team’s disqualification from the competition or point penalization at the discretion of the SCC committee. Any unethical conduct not otherwise covered in these rules will also be penalized at the discretion of the SCC Committee.

The following violations will result in immediate disqualification:

• Having anyone other than the 6 registered team members working on the team’s cloud resources during competition hours.
• Any communication between your cloud resource and a network other than the approved cloud networks.

 

General Competition Rules

Safety First
All SCC operations are always subject to safety as first consideration. If a task cannot be done safely, then it is unacceptable. When in doubt, ask a SCC committee member or your SCC team liaison.

No Assistance From Non-team Members
Once the competition starts, student teams will not be allowed to receive assistance from anyone, including their advisor.

Stay Under Budget
The Microsoft Azure Cloud budget allowed for the S21 SCC will be given to teams at the start of the competition. Point penalties will be assessed if teams go over this budget.

No External Computational Assistance
All benchmarks and applications workloads must be run on Microsoft Azure Cloud and Oracle Cloud using the budget allotted and the hardware provided. Submitting results or using computational resources during the competition other than the provided Microsoft Azure Cloud and Oracle Cloud resources is not permitted and will result in disqualification.

Teams Must Conduct Themselves Professionally
Teams must conduct themselves professionally and adhere to the SC21 Code of Conduct. Students must compete fairly and ethically.

 

System Software

Only run on the specified cloud resources. Benchmarks and applications must be run on the specified cloud as follows:

Microsoft Azure Cloud

• Benchmarking
• Quantum ESPRESSO
• Mystery Application

Oracle Cloud

• Cardioid
• Reproducibility Application

Teams may choose any operating system and software stack that will run the applications and display visualizations to conference attendees.

Teams may study and tune the open-source benchmarks and applications for their platforms. Any changes to application source code must be shared with the SCC committee.

Teams shall not modify or obstruct the data collection of metrics for their cloud systems. Trying to purposefully modify or obstruct the metrics database can result in a full disqualification.

Metrics collection software has been pre-installed and pre-configured on student systems. They should not be disabled or removed. List includes: InfluxDB, WAA Agent.