DLRG Mini-Course: Data Parallel Deep Learning using Multiple GPUs 1

Title: Data Parallel Deep Learning using Multiple GPUs 1
Date: Oct 23, 2023
Time: 1-3:30pm CST
Location: Online

About This Workshop:

This workshop teaches you techniques for training deep neural networks on multi-GPU technology to shorten the training time required for data-intensive applications. Working with deep learning tools, frameworks, and workflows to perform neural network training, you’ll learn concepts for implementing Horovod multi-GPUs to reduce the complexity of writing efficient distributed software and to maintain accuracy when training a model across many GPUs.

Learning Objectives:

• At the conclusion of the workshop, you’ll have an understanding of:
• Stochastic gradient descent (SGD), a crucial tool in parallelized training
• Batch size and its effect on training time and accuracy
• Transforming a single-GPU implementation to a Horovod multi-GPU implementation
• Techniques for maintaining high accuracy when training across multiple GPUs

When: Friday, July 29th, 12 Noon – 5 PM CT

Mode of Instruction: Virtual (using zoom)

Prerequisites: Experience with gradient descent model training

Technologies: TensorFlow, Keras, Horovod

Certificate: Upon successful completion of the assessment, participants will receive an NVIDIA DLI certificate to recognize their subject matter competency and support professional career growth.

Assessment Type: Code-based

Registration: https://forms.gle/o8t4NAnEcfBE1GJ6A