BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20211207T055400Z
LOCATION:Second Floor Atrium
DTSTART;TZID=America/Chicago:20211118T083000
DTEND;TZID=America/Chicago:20211118T170000
UID:submissions.supercomputing.org_SC21_sess243_spostu116@linklings.com
SUMMARY:Increasing Mini-Batch Size While Preserving Accuracy for Distribut
 ed Deep Learning
DESCRIPTION:ACM Student Research Competition: Graduate Poster, ACM Student
  Research Competition: Undergraduate Poster, Posters\n\nIncreasing Mini-Ba
 tch Size While Preserving Accuracy for Distributed Deep Learning\n\nMcCord
 \n\nData-parallel distributed training is an effective way to reduce train
 ing times for complex deep learning models and large-scale datasets. In di
 stributed training, each worker requires enough work to justify the commun
 ication overhead and maintain a balanced computation-to-communication rati
 o. This issue can be addressed by using large mini-batch sizes, which refe
 r to the size of the chunk of training data that is processed through the 
 network during a single training iteration. However, beyond a certain size
  limit, the benefits of large mini-batch sizes come at the cost of accurac
 y. This research focuses on understanding the effects of large mini-batch 
 sizes on accuracy and the underlying training dynamics. The techniques of 
 gradual warmup and linear scaling of the learning rate were analyzed. Ther
 e is evidence that the optimal number of warmup epochs is linearly associa
 ted with mini-batch size, and linear scaling does not appear to impact min
 i-batch sizes as large as 30K.\n\nTag: In-Person Only\n\nRegistration Cate
 gory: Tech Program Reg Pass, Exhibit Hall Only
END:VEVENT
END:VCALENDAR