This course covers the the methods available to write high performance parallel codes in Julia, including threading, GPU acceleration, and distributed programming with MPI and Dagger.jl.
By the end of the course students will be able to use Julia to write write portable and performant code that scales from their laptop CPUs to GPU-enabled supercomputer clusters like NERSC Perlmutter. These skills will be illustrated with a set of applications including scientific codes like Gray-Scott, and training large scale AI applications like LLMs.
Lectures: Mondays, Wednesdays, and Fridays 11 AM - 12 AM in room 36-144.
Prerequisites: 18.03, 18.06, or equivalents, and some programming experience. You should have taken the first half-semester numerical "hub" 18.S190/16.S090. Familiarity with Julia is assumed.
Instructors: A. Edelman
Teaching Assistants: Raye Kimmerer ([email protected]), Eveylne Ringoot, Rabab Alomairy
Office Hours:
- Raye on Tuesdays and Thursdays 12:45PM - 1:45PM on Zoom room
rayegun.
Lecture Recordings: Unfortunately we are not teaching in a recorded classroom this semester.
Links: Worth bookmarking:
-
6 Homeworks: 90%
-
Class Participation: 10%
| Homework | Assigned | Due | Topic | Notes | Solution |
|---|---|---|---|---|---|
| HW0 | April 2 | April 7 @ 11:59PM | Logistics | ||
| HW1 | April 7 | April 14 @ 11:59PM | Threading | Updated 9/4 | |
| HW2 / Perlmutter Instructions | April 20 | April 28 @ 11:59PM | Threading |
| # | Day | Date | Lecturer | Topic | Slides / Notes | Notebooks |
|---|---|---|---|---|---|---|
| Week 1 - Overview of Parallel Computing | ||||||
| 1 | M | 3/31 | Edelman | Parallel Computing may not be what you think, top500, matmul vs matadd | Intro | Language Horse Race |
| 2 | W | 4/2 | Edelman | If you live with one programming language, you dont know what you are missing | Slides | |
| 3 | F | 4/4 | Kimmerer | Allocations and other serial performance tips | PerformantSerial Julia Pluto Notebook | |
| Week 2 - Parallelism Concepts,Julia Performance, and Multithreading | ||||||
| 4 | M | 4/7 | Edelman | If you see an algorithm that doesn't look like it should be parallel it's probably Parallel Prefix | Slides | reduce,prefix pluto |
| 5 | W | 4/9 | Kimmerer, Ringoot | Multithreading, hyperthreading, and pipelining | Slides part 1 Slides part 2 | threading pluto notebook, JuliaParallel Notebook |
| 6 | F | 4/11 | Edelman | Prefix Continued | ||
| Week 3 - Attaching to a supercomputer, Multitasking & Intro GPU | ||||||
| 7 | M | 4/14 | Kimmerer, Edelman | Perlmutter, GPU pictures | ||
| 8 | W | 4/16 | Alomairy | GPU background | colab link gpu slides | |
| 9 | F | 4/18 | Alomairy, Ringoot | GPU software | Slides | |
| 10 | W | 4/23 | Alomairy | GPU Acceleration | Slides | |
| Week 4 - GPU Computing | ||||||
| Week 5 - GPU Computing | ||||||
| Week 6 - Distributed Computing (MPI + Dagger.jl) | ||||||
| Week 7 - Parallelism Concepts and Julia Performance |