February 2
@
8:00 am
–
February 27
@
5:00 pm
Parallelism in Deep Learning
In this course, “Parallelism in Deep Learning,” you will learn about the need for parallelism in deep learning and how to use different methods of parallelism in deep learning. You will also learn about leveraging data parallelism and model parallelism workflows for your AI models on HPC infrastructures.
In this course, you will engage in hands-on activities, homework, and instructor consulting to make learning parallelism in deep learning enjoyable and rewarding. You will also be able to tackle real-world model training problems on HPC clusters. By the end of this course, you’ll have the skills and confidence to train your AI models at scale using multiple GPUs and nodes.
Course At a Glance
Instructional Period: February 2 – February 27, 2026
Time to Complete Badge: 60 days
Last Day to Earn Badge: March 31, 2026
Expertise Level: Intermediate/Advanced
Scientific Machine Learning
Parallelism in Deep Learning
- Basic Python programming
- Basic understanding of deep learning
- Basic understanding of graphical concepts
- Basic PyTorch programming
Audience: The course is intended for a broad audience within the spectrum of the software and technology industry, including software engineers, data scientists, data engineers, data analysts, research scientists, and software developers. The course is designed to provide a basic understanding of high-performance computing for deep learning and how to use these models for a broad range of audiences.
By the end of the course, you should be able to:
Explain the need for parallelism in deep learning and its impact on scalability
Evaluate optimization dynamics, including advanced techniques optimization methods.
Construct workflows leveraging data, model, and hybrid parallelism in distributed environments.
Refine resource utilization strategies by maximizing GPU/CPU efficiency during distributed training
- 2 Quizzes to learn basic knowledge of parallelism in Deep Learning
- 1 exercise in which you will setup a compute environment for parallelism in deep learning based on hands-on activities.
- 1 Coding assignment to train a model using data parallelism, and model parallelism.
Module 1: Introduction to Parallelism in Deep Learning
Module 2: Implementing Parallelism (Data, Model, and Hybrid)
Module 3: Optimization Dynamics for Large-Scale Training
Module 4: Case Studies of Parallelism in Deep Learning
The course starts on February 2, 2026. All coursework must be completed by March 31, 2026, in order to earn the micro-credential badge. You will continue to have access to the course materials until January 1, 2027. The approximate time to complete this course is 16 hours.
This course has an instructional period from February 2 to February 27, 2026. During this instructional period, course materials will be released weekly and live synchronous sessions will be held. You may complete the course materials at your own pace. Live Zoom meetings will be conducted for interactive coding sessions. A suitable time for these live sessions will be determined through a group poll. The recordings of those sessions will be available soon after each meeting.
You will receive the Parallelism in Deep Learning micro-credential badge upon successful completion of the course assessments.
Course materials are provided within the course. No additional purchase is required.
Registration
Register for the course as a 1-credit independent study course with a maximum of 3 such TrAC courses per semester.
Students must register for the independent study by emailing benearl@iastate.edu and cc-ing baditya@iastate.edu. Also, fill this Google form for our records: https://forms.gle/d7PUHaqjko6sPHVg7
Note: You cannot cancel your registration after April 1 for any course.
Industry Professionals/ISU Staff/Post Docs
$
500
.00
ISU Professionals/Staff and Government Employees: $300
Aditya Balu, Data Scientist
Aditya Balu is a data scientist in the Translational AI Center (TrAC) at Iowa State University. With 14 years of experience working in the field of AI, he brings extensive expertise in distributed deep learning to this course. His rich experience includes developing and implementing AL solutions for industry-level applications, making him uniquely qualified to guide students through this course. Aditya is passionate about bridging the gap between academic research and practical industry applications, ensuring students gain both theoretical knowledge and hands-on skills needed for real-world AI challenges.
Nicole Hayungs
nhayungs@iastate.edu
