<\/span><\/b><\/p>\n
\n<\/p>\n
<\/p>\n
![\"Undo\"](\"https:\/\/www.modernescpp.com\/wp-content\/uploads\/2021\/02\/Undo.png\")
<\/p>\n
C and C++ developers requiring high performance often employ various <\/span>concurrent computing<\/span><\/a> techniques in which several threads of execution operate simultaneously.<\/span><\/p>\n Recent Modernes<\/b><\/span> C++<\/b> blog posts explain how to avoid concurrency bugs in multithreaded C++ systems by using <\/span>Latches<\/span><\/a>, <\/span>Semaphores<\/span><\/a>, and <\/span>Barriers and Atomic Smart Pointers<\/span><\/a>.<\/span><\/p>\n These practices are important because if one does not pay careful attention to thread locking and memory access, there’s a risk of introducing new failure modes (e.g.<\/span> nondeterministic behaviour, race conditions, etc.), which can make the application very difficult to debug. <\/span><\/p>\n In this article, we will take a closer look at concurrency bugs and explain how Time Travel Debugging<\/b> (a.k.a. reverse debugging) can be used to resolve them more quickly.<\/span><\/p>\n Concurrency bugs are nondeterministic defects that arise when the execution of a thread disrupts the behaviour of other threads running at the same time. <\/span><\/p>\n Nondeterministic factors such as thread-switching or external data can affect the order or timing of thread execution, resulting in unpredictable application behaviour such as miscalculations, crashes, or hangs. Below are a few concurrency defect<\/span> examples.<\/span><\/p>\n If the execution of Thread 1 below is interrupted by Thread 2 immediately after passing the <\/span>if<\/span><\/b> <\/span>test, then Thread 1 will crash with a memory access violation.<\/span><\/p>\n Thread 1:<\/span><\/p>\n if (foo->bar)<\/span><\/p>\n {<\/span><\/p>\n do_something<\/span>(foo->bar);<\/span><\/span><\/p>\n }<\/span><\/p>\n <\/span><\/p>\n Thread 2:<\/span><\/p>\n foo->bar = nullptr<\/span><\/span>;<\/span><\/span><\/p>\n It is possible for Thread 1 to wait indefinitely for Thread 2 to unlock L2, while Thread 2 is waiting for Thread 1 to release L2.<\/span><\/p>\n Thread 1:<\/span><\/p>\n std::<\/span><\/span>lock_guard<\/span><\/span><std::mutex> lock(L1); std::lock_guard<\/span><std::mutex> lock(L2);<\/span><\/p>\n <\/span><\/p>\n Thread 2:<\/span><\/p>\n std::<\/span><\/span>lock_guard<\/span><\/span><std::mutex> lock(L2); std::lock_guard<\/span><std::mutex> lock(L1);<\/span><\/p>\n A race condition is a type of software defect that occurs when separate threads interact in an unforeseen way and disrupt the expected timing and order of operations. <\/span><\/p>\n An example is where two threads try to change shared data at the same time, leading to unpredictable system behaviour. That is, multiple threads are in a race and different threads might win the race depending on non-deterministic events.<\/span><\/p>\n If you have a defective multithreaded program, your starting point is likely to be the program entering a confused state or just crashing. From this starting point, you might find yourself following the steps below to find the cause of a concurrency defect and correct it:<\/span><\/p>\n These steps can take days or weeks. What#s<\/span> worse is that, since concurrency defects can be so hard to reproduce, causes often go undiscovered and uncorrected. <\/span><\/p>\n Brian Kernighan famously wrote:<\/span><\/p>\n “Debugging involves backward reasoning, like solving murder mysteries. Something impossible occurred, and the only solid information is that it really did occur. So<\/span> we must think backwards from the result to discover the reasons.”<\/span><\/i><\/p>\n In other words, we really want the debugger to be able to tell us what happened in the past. You need to know what your program actually did<\/span>, as opposed to what you expected it was going to do. This is why<\/span> debugging typically involves reproducing the bug many times, slowly teasing out more and more information until you pin it down.<\/span><\/p>\n Time Travel Debugging takes away all that guesswork and trial and error; the debugger can tell you directly what just happened.<\/span><\/p>\n Time Travel Debugging is the ability of a debugger to stop after a failure in a program has been observed and to go back into the history of the execution to uncover the reason for the failure.<\/span><\/p>\n For hard-to-reproduce concurrency bugs like race conditions, Time Travel Debugging allows you to start from the point of failure and step backwards to find the cause. This is a very different approach from the typical process of running and rerunning a program, again and again, adding logging logic as you go until you find the cause. <\/span><\/p>\n Let’s<\/span><\/span> look at the impact Time Travel Debugging has on debugging a race condition.<\/span><\/p>\n You start by loading and running a program <\/span>(<\/span>race<\/span>) <\/span>in a <\/span>debugger that supports time travel<\/span><\/a> (such as <\/span>UDB<\/span><\/a>). In our example below, you can see the program crashes:<\/span><\/p>\n .\/<\/span><\/span>udb<\/span><\/span> race<\/span><\/p>\n (udb<\/span>) run<\/span><\/span><\/p>\n …<\/span><\/p>\n s_threadfn<\/span><\/span>: it=110000<\/span><\/p>\n s_threadfn<\/span><\/span>: it=70000<\/span><\/p>\n race: race.cpp:34: void s_threadfn<\/span><\/span>(<\/span>): Assertion `s_value<\/span> == old_value<\/span> + a’ failed.<\/span><\/p>\n <\/span><\/p>\n [New Thread 3441.3452]<\/span><\/p>\n <\/span><\/p>\n Program received signal SIGABRT, Aborted.<\/span><\/p>\n [Switching to Thread 3441.3452]<\/span><\/p>\n <\/span><\/p>\n In our example, the error is all about the<\/span><\/p>\n Assertion `s_value<\/span> == old_value<\/span> + a’ failed<\/span><\/span><\/p>\n We want to know why one thread expects one value but gets another. We then use <\/span>reverse-finish <\/span>until we arrive at a line in our program:<\/span><\/p>\n (udb<\/span>) rf<\/span><\/p>\n 79 <\/span><\/span> raise<\/span> (SIGABRT);<\/span><\/p>\n (udb<\/span>) rf<\/span><\/p>\n 92 <\/span><\/span> abort<\/span> ();<\/span><\/p>\n (udb<\/span>) rf<\/span><\/p>\n 101 <\/span><\/span> _<\/span>_assert_fail_base<\/span> (_(“%s%s%s<\/span>:%u: %s%sAssertion<\/span> `%s’ failed.\\n%n<\/span>“),<\/span><\/p>\n (udb<\/span>) rf<\/span><\/p>\n 34 <\/span><\/span> assert( s<\/span><\/span>_value<\/span> == old_value<\/span> + a);<\/span><\/p>\n <\/span><\/p>\n <\/span><\/p>\n This brings us to where our program What are concurrency bugs?<\/span><\/h2>\n
Atomicity violation<\/span><\/h3>\n
Deadlock<\/span><\/h3>\n
Race condition<\/span><\/h3>\n
The hard way to find and fix concurrency defects<\/span><\/span><\/h2>\n
\n
The easier way: Time Travel Debugging<\/span><\/h2>\n