Sequential Consistency

The atomics are the base of the C++ memory model. Per default, sequential consistency is applied.

The strong C++ memory model

In 2004 Java 5.0 gets its current memory model, and in 2011 C++.  Before that, Java had an erroneous memory model, and C++ had no memory model. Who thinks that this is the endpoint of a long process is wrong. The foundations of multithreading programming are 40 to 50 years old. So Leslie Lamport defined 1979 the concept of sequential consistency.

Sequential consistency provides two guarantees.

1. The instructions of a program are executed in source code order.
2. There is a global order of all operations on all threads.

Before I look deeper into these two guarantees, I explicitly emphasize them. The statements only hold for atomics but influence non-atomics.

The simple graphic displays two threads. Each thread is storing its variable x or y, loads the other variables y and x, and stores them in the variable res1 or res2.

The operations take place on atomics, and so they are atomic. By default, sequential consistency applies. But the question is. In which order can the statements take place?

The first guarantee of sequential consistency is that the instruction will be executed in the order of the source code. That is easy. No store operation can overtake a load operation.

The second guarantee of sequential consistency is that all threads' instructions must follow a global order. That means, in that concrete case, that thread 2 sees the operations of thread 1 in the same order in which thread 1 executes them. This is the key observation. Thread 2 sees all operations of thread 1 in the source code order of thread 1. The same holds from the perspective of thread 1. So you can think about characteristic 2 as a global counter, which all threads must obey. The global counter is the global order.

We're not done with our riddle right now. What is still missing is to look at the different interleaving executions of the two threads. So the following six interleavings of the two threads are possible.

That was easy. Or?

From the strong to the weak memory model

I want to refer to the picture of the contract between the programmer and the system.

The programmer uses atomics in this particular example. So he obeys his part of the contract by using them correctly. The system guarantees him a well-defined program behavior without data races. In addition to that, the system can execute the four operations in each combination. If the programmer uses the relaxed semantic, the pillars of the contract dramatically change. On the one hand, it is much more difficult for the programmer to apply the contract correctly. On the other hand, the system has a lot more optimization possibilities. With the relaxed semantic - also called weak memory model - many more combinations of the four operations are possible. The counter-intuitive behavior is that thread 1 can see the operations of thread 2 in a different order. So there is no picture of a global counter. From the perspective of thread 1 it is possible that the operation res= y.load() overtakes x.store().

There are a few more models between the sequential consistency and the relaxed-semantic. The most important one is the acquire-release semantic. I think you already guess it. With acquire-release semantics, the programmer must obey weaker rules than sequential consistency. But the system has more optimization possibilities. The acquire-release semantic is the key to a deeper understanding of multithreading programming because the threads will be synchronized at specific synchronization points in the code. Without these synchronization points, there is no well-defined behavior of threads, tasks or condition variables possible. More about that in the following post.  

What's next?

The next post will provide a deeper look into atomics. But we stick with the strong C++ memory model. (Proofreader Alexey Elymanov)