Introduction to Parallel High Performance Programming - A Practical Course for Engineers

Content

Since the early 2000s, as Moore’s Law began to slow down, computer architects have shifted their focus to increasing performance by adding more cores to a single computer. This transition marked a fundamental shift toward parallel computing. Today, it’s clear that the future of computing lies in this parallel world—one that impacts nearly every field, particularly engineering. In engineering, one of the most significant applications of parallel computing is the use of numerical methods to solve complex partial differential equations that describe the behavior of mechanical systems, such as structural mechanics or fluid dynamics. 

To develop software that powers the designs of tomorrow, engineers must not only understand these numerical methods but also master how to implement them efficiently on modern parallel computer architectures. Whether running on a local workstation or on a supercomputer like LRZ, the ability to leverage parallel computing effectively is crucial for tackling real-world engineering challenges.

The course “Introduction to Parallel High Performance Programming - A Practical Course for Engineers” introduces the core concepts of parallel programming using C++ and demonstrates how these can be applied to solve real-world engineering problems on parallel computer architectures including both shared and distributed memory architectures. Through hands-on projects, students will develop a finite difference-based heat equation solver and explore various parallelization techniques, gaining practical experience in parallel computing. Throughout this project, students will not only learn to implement parallel programming techniques but also dive into classical numerical methods such as the conjugate gradient method and finite difference methods, adapting them to take full advantage of modern parallel computing architectures.

The course begins with a brief introduction to computer architecture and a review of essential C++ basics. From there, we will develop a finite difference solver for the heat equation over the course of the semester, progressively applying and combining the following parallel programming tools:
• C++ Threads
• SIMD Vectorization
• OpenMP (Open Multi-Processing)
• MPI (Message Passing Interface)
In a series of lectures, students will be introduced to these tools, along with common pitfalls and best practices for parallel programming. Each lecture will be followed by hands-on practical sessions where students can experiment with the newly learned concepts, starting with simpler examples like matrix-matrix multiplication.