Differences and relations between high-performance/throughput computing and parallel computing?

Question

I wonder if high-performance computing and parallel computing always mean the same? If not,

• what are the differences and relations between them?
• what are some examples of high-performance computing that are not parallel computing?
• what are some examples of parallel computing that are not high-performance computing?

Are high-throughput computing and high-performance computing the same concept?

• Yuval Filmus: Apparently HPC always means parallel computing: insidehpc.com/hpc-basic-training/what-is-hpc.
• Jukka Suomela: You can do parallel computing on the multicore CPU of your laptop or mobile phone. People usually wouldn't call it HPC.
• Tim: So HPC is a type of parallel computing. What about high-throughput computing?
• Raphael: Tim, what are you trying to accomplish by these questions? These are "only" notions. People are using them in different ways. There is (probably) no insight to be gained just by creating a glossary.
• Tim: I am trying to understand what people are saying/writing. @Raphael

Answer 1

As noted by @JukkaSuomela, you can do parallel computing on low-end resources such as your laptop and even on your mobile phone (if they are equipped with a multicore processor). However, HPC (High Performance Computing) is, roughly stated, parallel computing on high-end resources, such as small to medium sized clusters (ten to hundreds of nodes) up to supercomputers (thousands of nodes) costing millions of dollars. Therefore, the difference is mainly in the hardware used.

Finally, HTC (High Throughput Computing) refers to executing the maximum number of tasks (or jobs if you prefer) per time unit. A classical example of HTC computation is the so called parameter sweep, in which you must run the same executable, but varying in each execution a set of parameters (thus the sweep spans the whole parameter space).

• Tim: Thanks! Does High Throughput Computing necessarily or often involve parallel computing?
• Massimo Cafaro: Yes, in order to achieve high throughput, the idea is to run several executable at once on different processors/cores.

Answer 2

I would say parallelism is included in HPC. HPC uses subsidiary tools other than paralelism, the most important being vectorization, including matrix operations with diagonalisation and more general SVD speed boost. So HPC wherewithal are often engaged for big problems only well fitted for matrix(Linpack), well conditioned : simulation, PDE. So it ranges from weather predictions, brain simulation, PDE resolution in near real time for High Frequency Trading, and so on. Further applications on www.top500.org

Answer 3

the field of supercomputing has undergone a significant paradigm shift in the last decade or so such that it used to require highly specialized hardware and designs. a supercomputing company that embodied this approach is Cray supercomputers.

however a new concept/trend has emerged that extremely high performance can be achieved with "off-the-shelf" components such as those used in consumer electronics and heavy internetworking using standard networking components. supercomputer designs have always utilized a large amount of parallelism but this trend has been amplified with the proliferation of the off-the-shelf type approach. also parallelism is much more being supported with sophisticated software architectures such as MPI, MapReduce, etc

so the systems are still highly parallel but the particular means of achieving that parallelism has shifted in the hardware and software. the parallelism is more visible at the application layer and not abstracted away.

also recently power consumption has become much more of an issue and supercomputers are starting to focus on energy-efficient designs and low-energy CPUs such as those used in mobile computing. [1] is a nice ref on key issues in current trends/barriers on high performance computing scalability.

[1] Next-Generation Supercomputers Kogge

• Dave Clarke: I don't see how this answers the question.