How GPT-3 is Spearheading the Fourth Industrial Revolution
Session Leader
Additional Session Leaders
Event Type
Birds of a Feather
Accelerator-based Architectures
Cloud and Distributed Computing
Machine Learning and Artificial Intelligence
TP
XO / EX
TimeTuesday, 16 November 202112:15pm - 1:15pm CST
Location229
DescriptionScaling a large language training model such as GPT-3 across 64 NVIDIA A100 GPUs and beyond can be complex and difficult to tune for high performance. This BoF will bring industry experts and users together to discuss the current trends, challenges and approaches to effectively scale these trainings across large clusters of GPUs in the cloud. The goal is to foster discussions amongst the HPC and AI communities to share best practices, and for the platforms that host these workloads to understand how best to support those running these applications.
Archive view
Session Leader
Additional Session Leaders
