Supercomputing Course: OpenMP

OpenMP is the industry standard shared memory programming model. First developed in 1997, the OpenMP standard is updated by an architecture review board (ARB).

There are several really great advantages to OpenMP. First, it allows you to take an incremental approach to parallelization. You can parallelize the the parts that will have the most impact before parallelizing the rest, optimizing your effort-to-improvement ratio. Second, the constructs you use to parallelize the code are simple to read, and most are one-liners, so your code doesn't grow that much. And finally, you can use the same code as parallel or serial code -- just compile it with or without OpenMP enabled, depending on what you want.

The OpenMP API is a combination of directives, runtime library routines, and environment variables. These components of the API fall into three categories: expression of parallelism, data sharing among threads, and synchronization.

The OpenMP parallelism model is a fork-and-join model. That is, when your program starts running, there is a single thread of control. When it reaches a parallel section, it forks into multiple threads, which rejoin into the single thread upon reaching the end of the parallel section.

The nuclear physicist in our previous installment could write a hybrid MPI/OpenMP program to solve a bigger problem. She would reproduce the basis functions once on each node, and then use OpenMP to parallelize the computations across the node. As we will see in the next installment of this course, there are many OpenMP directives that can be used to parallelize many common features of algorithms.