Last updates: Tue May 8 19:16:06 2001 Fri Nov 12 15:26:10 2004 Thu Nov 13 18:30:20 2008 Mon Mar 1 16:28:36 2010
OpenMP is a relatively new (1997) development
in parallel computing. It is a language-independent
specification of multithreading, and implementations are
available from several vendors, including
OpenMP is implemented as comments or directives
in Fortran, C, and C++ code, so that its presence is
invisible to compilers lacking OpenMP support.
Thus, you can develop code that will run everywhere, and
when OpenMP is available, will run even faster.
The OpenMP Consortium maintains a very useful
Web site at
http://www.openmp.org/,
with links to vendors and resources.
There is an excellent overview of the advantages of
OpenMP over POSIX threads ( pthreads )
and PVM/MPI in the paper
OpenMP: A Proposed Industry Standard API for Shared
Memory Processors,
also available in
HTML
and
PDF
.
This is a must-read if you are getting started in
parallel programming. It contains two simple examples
programmed with OpenMP, ptheads,
and MPI.
The paper also gives a very convenient tabular comparison of
OpenMP directives with Silicon Graphics
parallelization directives.
OpenMP can be used on uniprocessor and
multiprocessor systems with shared memory. It can also be
used in programs that run on homogeneous or heterogeneous
distributed memory environments, which are typically
supported by systems like Linda, MPI
, and PVM, although the OpenMP
part of the code will only provide parallelization
on those processors providing shared memory.
In distributed memory environments, the programmer must
manually partition data between processors, and make special
library calls to move the data back and forth. While that
kind of code can also be used in shared memory systems,
OpenMP is much simpler to program.
Thus, you can start parallelization of an application using
OpenMP, and then later add MPI or
PVM calls: the two forms of parallelization
can peacefully coexist in your program.
An extensive bibliography on multithreading, including
OpenMP, is available at
http://www.math.utah.edu/pub/tex/bib/index-table-m.html#multithreading.
MPI and PVM are covered in a
separate bibliography:
http://www.math.utah.edu/pub/tex/bib/index-table-p.html#pvm
OpenMP benchmark: computation of pi
This simple benchmark for the computation of pi is taken
from the paper above. Its read statement has
been modified to read from stdin instead of the
non-redirectable /dev/tty, and an extra final
print statement has been added to show an
accurate value of pi.
Follow this link for the
source code
a
shell script
to run the benchmark, a UNIX
Makefile,
and a small
awk program
to extract the timing results for inclusion in tables like
the ones below.
OpenMP directives during compilation:
Once you have compiled with OpenMP support, the
executable may still not run multithreaded, unless you
preset an environment variable that defines the number of
threads to use. On most of the above systems, this variable
is called OMP_NUM_THREADS. This has no effect on the
IBM systems; I'm still trying to find out what is expected
there.
When the Compaq/DEC benchmark below was run, there was one
other single-CPU-bound process on the machine, so we should
expect to have only 3 available CPUs. As the number of
threads increases beyond the number of available CPUs, we
should expect a performance drop, unless those threads have
idle time, such as from I/O activity. For this simple
benchmark, the loop is completely CPU bound. Evidently,
3 threads make almost perfect use of the machine, at a cost
of only two simple OpenMP directives added to
the original scalar program.
The previous two systems were essentially idle when the benchmark was run, and, as expected, the optimal speedup is obtained when the thead count matches the number of CPUs.
The next one is a large shared system on which the load average was about 40 (that is, about 2/3 busy) when the benchmark was run. With a large number of CPUs, the work per thread is reduced, and eventually, communication and scheduling overhead dominates computation. Consequently, the number of iterations was tripled for this benchmark. Since large tables of numbers are less interesting, the speedup is shown graphically as well. At 100% efficiency, the speedup would be a 45-degree line in the plot. With a machine of this size, it is almost impossible to ever find it idle, though it would be interesting to see how well the benchmark would scale without competition from other users for the CPUs.