About multithreading, concurrency, and parallelism

Question

Recently I had confusion with understanding concepts: multithreading, concurrency, and parallelism. In order to reduce confusion, I've tried to organize my understanding about these and drawn my conclusion. My question is,

Is there a misunderstanding or something wrong from conclusion below?

References I took can be found here.

1. Concurrency and parallelism are different level of category.

It is not either concurrent or parallelism. It is either concurrent or not, and either parallel or not.

For example,

• Not concurrent (Sequential) / Not parallel
• Not concurrent (Sequential) / Parallel
• Concurrent / Not parallel
• Concurrent / Parallel

2. Parallelism is not subset of concurrency.

3. How does threading or multithreading relates to concurrency and parallelism?

Definition of thread clarifies this. Thread is "unit of execution flow". This "execution flow" can be managed independently by a scheduler, which is typically a part of the operating system.

• Having a thread means having one unit of execution flow.
• Having multiple threads (Multithreading) means having multiple units of execution flow.

And,

• Having multiple units of execution flow is having multiple things making progress, which is definition of concurrency.

And,

• Multiple units of execution flow is done by time slicing in single core hardware environment.
• Multiple units of execution flow is done in parallel in multi core hardware environment.

4. Is multithreading concurrent or parallel?

• Multithreading, or having multiple units of execution flow, is concurrent.
• Multithreading itself is just having multiple units of execution flow. This has nothing to do with parallelism.
• How operating system deals with multiple units of execution flow relates to parallelism.
• Parallelism is achieved by operating system and hardware environment.
• "Code can be concurrent, but not parallel." (Parallelism implies concurrency but not the other way round right?, stackexchange)

Detailed description will be truly appreciated.

Answer 1

Parallelism Refers to any system in which a single application can make use of more computing hardware than a single CPU can provide. There are a number of different types of parallel computing architecture, but when people say "parallelism" they often are talking about one in particular...

...A Symmetric MultiProcessing (SMP) system is a computer with one memory system, and two or more traditional CPUs that have equal access to it. Most modern workstations, most mobile devices, and many server systems* are SMP.

Multithreading is a model of concurrent computing.** A computer scientist might tell you that two threads run concurrently when the order in which the operations they perform are interleaved is not strictly determined by the program itself. A software developer is more likely to say that two threads run concurrently with each other when both threads have been started and neither of them has finished.

One way to achieve parallelism in an application running on an SMP system is to use multiple concurrent threads.

---

* Some servers are NUMA, which is a close cousin to SMP. In a NUMA system, the CPUs all access the same memory system, just like in SMP, except that each CPU "owns" part of the physical memory space, and it can access its own memory locations more quickly than it can access memory locations that are owned by other CPUs.

** There are other models of concurrent computing. Some, such as Actors, are used in production software. Others are mostly of academic interest.