Lecture 20: Synchronization Patterns, Deadlock

» Lecture video (Brown ID required)
» Lecture code
» Post-Lecture Quiz (due 11:59pm Monday, April 20)

C++ mutex patterns

Before we talk about condition variables, let's look at a C++ pattern that both makes working with mutexes easier and forms the basis for the standard library condition variable API.

It is very common that we write some synchronized function where we need to lock the mutex first, do some work, and then unlock the mutex before the function returns. Doing this repeatedly can be tedious. Also, if the function can return at multiple points, it is possible to forget a unlock statement before a return, resulting errors. C++ has a pattern to help us deal with these problems and simplify programming: a scoped lock.

We use scoped locks to simplify programming of the bounded buffer in bbuffer-scoped.cc. The write() method now looks like the following:

ssize_t bbuffer::write(const char* buf, size_t sz) {
    std::unique_lock guard(this->mutex_);
    assert(!this->write_closed_);
    size_t pos = 0;
    while (pos < sz && this->blen_ < bcapacity) {
        size_t bindex = (this->bpos_ + this->blen_) % bcapacity;
        this->bbuf_[bindex] = buf[pos];
        ++this->blen_;
        ++pos;
    }
    ...
}

Note that in the first line of the function body we declared a std::unique_lock object, which is a scoped lock that locks the mutex for the scope of the function. Upon initialization of the std::unique_lock object, the mutex is automatically locked, and when this object goes out of scope, the mutex is automatically unlocked. These special scoped lock objects lock and unlock mutexes in their constructors and destructors (a special C++ method on classes that gets invoked before an object of that class is destroyed) to achieve this effect.

This design pattern is also called Resource Acquisition is Initialization (or RAII), and is a common pattern in software engineering in general. The use of RAII simplify coding and also avoids certain programming errors.

Condition Variables

So far we implemented everything in the spec of the bounded buffer except blocking. It turns out mutex alone is not enough to implement this feature -- we need another synchronization object. In standard C++ this object is called a condition variable.

A condition variable supports the following operations:

• wait(std::unique_lock& lock): In one atomic step, it unlocks the lock, blocks until another thread calls notify_all(). It also relocks the lock before returning (waking up).
• notify_all(): Wakes up all threads blocked by calling wait().

Condition variables are designed to avoid the sleep-wakeup race conditions we briefly visited when we discussed signal handlers. The atomicity of the unlocking and the blocking guarantees that a thread that blocks can't miss a notify_all() message.

Logically, the writer to the bounded buffer should block when the buffer becomes full, and should unblock when the buffer becomes nonfull again. Let's create a condition variable, called nonfull_, in the bounded buffer, just under the mutex. Note that we conveniently named the condition variable after the condition under which the function should unblock. It will make code easier to read later on. The write() method implements blocking is in bbuffer-cond.cc. It looks like the following:

ssize_t bbuffer::write(const char* buf, size_t sz) {
    std::unique_lock guard(this->mutex_);
    assert(!this->write_closed_);
    while (this->blen_ == bcapacity) {  // #1
        this->nonfull_.wait(guard);
    }
    size_t pos = 0;
    while (pos < sz && this->blen_ < bcapacity) {
        size_t bindex = (this->bpos_ + this->blen_) % bcapacity;
        this->bbuf_[bindex] = buf[pos];
        ++this->blen_;
        ++pos;
    }
    ...

The new code at #1 implements blocking until the condition is met. This is a pattern when using condition variables: the condition variable's wait() function is almost always called in a while loop, and the loop tests the condition in which the function must block.

On the other hand, notify_all() should be called whenever some changes we made might turn the unblocking condition true. In our scenario, this means we must call notify_all() in the read() method, which takes characters out of the buffer and can potentially unblock the writer, as shown in the inserted code #2 below:

ssize_t bbuffer::read(char* buf, size_t sz) {
    std::unique_lock guard(this->mutex_);
    ...
    while (pos < sz && this->blen_ > 0) {
        buf[pos] = this->bbuf_[this->bpos_];
        this->bpos_ = (this->bpos_ + 1) % bcapacity;
        --this->blen_;
        ++pos;
    }
    if (pos > 0) {                   // #2
        this->nonfull_.notify_all();
    }

Note that we only put notify_all() in a if, but we put the wait() in the earlier code snippet inside a while loop. Why is it necessary to have wait() in a while loop?

wait() is almost always used in a loop because of what we call spurious wakeups. Since notify_all() wakes up all threads blocking on a certain wait() call, by the time when a particular blocking thread locks the mutex and gets to run, it's possible that some other blocking thread has already unblocked, made some progress, and changed the unblocking condition back to false. For this reason, a "woken-up" must revalidate the unblocking condition before proceeding further, and if the unblocking condition is not met it must go back to blocking. The while loop achieves exactly this.

Deadlock

Locking is difficult not only because we need to make sure that we include all accesses to shared state in our critical sections, but also because it's possible to get it wrong in ways that cause our programs to get stuck indefinitely.

To explain this, let's consider a new example program. passtheball.cc contains the basic, unsychronized code for a game in which players pass a ball between them. Each player runs in a separate thread, and the example program has three players (P₀ to P₂). At the start, P₀ has a ball, indicated by its has_ball member variable. The players then pass the ball by invoking pass(), which changes the ball state (has_ball) of the target player as well as their own ball state to move the ball.

Clearly, has_ball is shared state that gets written and read here. Consequently, it's not surprising that passtheball.cc (which lacks synchronization) both quickly ends up going awry when run (it either loses the ball or duplicates it because the threads can get descheduled between changing the two ball states) and generates a bunch of race condition warnings when compiled with thread sanitizers enabled (make TSAN=1).

passtheball-atomic.cc, passtheball-mutex.cc, and passtheball-cond.cc implement synchronized versions of the ball game using, respectively, atomics, mutexes, and a condition variable. These implementations differ in performance, but they work correctly for the setting with a single ball.

Now consider a modified version of this game, where players generate a random number to decide whether to pass the ball to their left or to their right (each with a 50% likelihood), and where we additionally introduce a second ball to the game.

When we run this code (passtheball-deadlock), it quickly gets stuck in a setting similar to the one shown below. (The specific players involved may differ if you run the code, but the overall situation will be the same.) P₀ has a ball and is trying to pass it to P₂, who also has a ball and is concurrently trying to pass that ball to P₀.

Why does the code get stuck? Consider what happens in player_threadfunc() and pass():

1. First, in player_threadfunc(), the passing player locks its own ball state mutex (the unique_lock over me->m.
2. Second, in pass(), the passing player attempts to lock the ball state mutex of the target player (target->m.lock()).

No thread can ever make progress again, and the program becomes deadlocked.

Summary

Today, we looked at more complex synchronization settings. In particular, we learned about scoped locks in C++, and then about condition variables, and deadlock.

Condition variables are another type of synchronization object that make it possible to implement blocking of threads until a condition is satisfied (e.g., there is space in a bounded buffer again). This improves efficiency of the bounded buffer program, as threads no longer spin; for some other programs that require threads to wait, condition variables are actually required for correctness. To wait, a thread calls wait() on the condition variable while holding a mutex lock. If the condition does not hold, the mutex is released and the thread is blocked. Any waiting threads for a condition variable are unblocked by a call to notify_all (or notify_one) from another thread. Threads should call the notify functions only when the condition has changed. Once notified, a waiting thread seeks to reacquire the mutex it held when calling wait(), and once it succeeds, returns to the caller, which needs to recheck the condition to avoid problems with spurious wakeups.

Finally, we saw an example of deadlock, which occurs when a thread blocks on acquiring a lock that is already held and never given up. Deadlock can occur with two threads that try to mutually take locks on resources that they have already locked. With C++ standard library mutexes, deadlock can also happen in other situations, such as when a thread is trying to take a lock it already holds.