5449: Introduction to High-Performance Deep Learning

Instructors: Prof. Dhabaleswar K. (DK) Panda and Dr. Hari Subramoni
Autumn 2021

Course Number: 5449

Class Number: 36352 (Grad) and 36351 (Undergrad)

Credits: 3

Course Time: WF 11:10 am - 12:30 pm

Classroom: Pomerene Hall 250

Course Description:

Recent advancements in Artificial Intelligence (AI) have been fueled by the resurgence of Deep Neural Networks (DNNs) and various Deep Learning (DL) frameworks like Tensorflow, PyTorch, MXNet, Caffe, and Chainer. DNNs have found widespread applications in classical areas like Image Recognition, Speech Processing, Textual Analysis, as well as areas like Cancer Detection, Medical Imaging, Physics, Materials Science, and even Autonomous Vehicle systems. However, scaling distributed training with scale-up and scale-out approaches are still challenging. This is leading to the emergence of a new field called "High-Performance Deep Learning".

The objectives of this course are to understand the principles and the practice of this emerging trend, the open set of challenges, how modern HPC technologies can be used to accelerate DL trainings, etc.

Topics to be Covered


Selected papers from the literature including papers focusing on past and on-going research activities in the group.

Laboratory Exercises:

The course will involve laboratory expercises for students to experiment with Deep Learning Frameworks. These exercises will be carried out on OSC (Ohio Supercomputing Center) clusters using GPUs. This will provide hands-on knowledge to the students in the area of high-performance deep learning.

Last Updated: May 5, 2021