CS1950Y Lecture 16: Motivating Model Checking
Monday, March 12 2018

Announcements

• Reminder that final project partners are due on Thursday
• We’ll be releasing the list of final project topics soon
• What makes a good final project? There are two parts: the model itself, and the outcome. It’s not enough to just have a neat model, there also has to be something you’re trying to get out of it.

TLA+

We’ll be doing a bit of Alloy today, but we’re shifting gears to a new tool called TLA+. TLA+ is a model checker, used by companies like Amazon to verify their distributed systems algorithm.

Today, we’ll be talking about something simpler than what we’ve been used to: a traffic light (it’ll get somewhere interesting, we promise).

What do we need to model? In Alloy, what would the sigs be?

• We need the direction. This could be an explicit sig, or two lights where we implicitly know which is for which direction
• We need the colors - we need to say that this light is green or that light is red.
• Finally, we’re modeling a stateful system, where there’s change over time. That means we need some kind of State sig. Either we could make our Light sig work as a state or add a separate sig for state.

What are some things we want to show with our model? In terms of the final project, this is the outcome part

• Each direction always eventually turns green
• At every time, either the north-south light or the east-west light is red

These two properties have somewhat different characters. The second describes a bad state - if we’re ever in a state where this is false, there’s something wrong with our system, so we’ll call it “safety.” The second doesn’t describe a particular state, but rather an eventual outcome we want, so we’ll call it “liveness.”

Make sure to look at the Alloy model code for this section

In our model, we’re going to have it be a particular light’s turn at each time. This is so lights can’t each cycle through by themselves and get us into bad states.

How many possible states do we have? There are 3 possible states for each light, and we have two lights, and 2 possible states for whose turn it is. This gives us 3 * 3 * 2 = 18 states, of the form (Red, Yellow, 1). We don’t actually want this particular state to be reachable, because if it’s light 1’s turn, it’ll turn green while the other light is still yellow.

So, we have a model now. Let’s check some things with it! First, we’ll check safety:

assert alwaysOneRed {
    -- We're saying always, so we need to quantify over something
    all t: Time | {
        -- There's always at least one red light
        some l : Light | {
            l.myColor[t] = Red
        }
    }
}

check alwaysOneRed for 3 Color, 2 Light, 10 Time

Alloy doesn’t find any counterexamples. As we expect, the unsat core contains pretty much everything - if we didn’t have any of these properties, we could end up in a bad state. Why are we looking at a core here? Cores are a really good way to tell if your assertion passes because the property is true or because of a bug in the model. If there’s an extra contradiction in your model, or just in the assertion, then the assertion will pass when it shouldn’t (for example, if it’s impossible to instantiate Times). This is called a vacuous pass, when the assertion passes because the model is unsatisfiable - there can’t be counterexamples if there are no examples to examine. We’re dwelling on this now because final projects are coming up. With bigger specs, this is really important to keep in mind. Alloy is good at using all the freedom you give it to find examples, but if you don’t give it any freedom, it can’t help.

Now, let’s check liveness:

assert alwaysTurnGreen {
    -- For both lights at all times
    all l: Light | all t: Time | {
        -- We have a light that's red at some time. What comes next?
        l.myColor[t] = Red implies {
            -- There must be some later time (using the transitive closure of next)
            -- where the light is green
            some t': t.^next | l.myColor[t'] = Green
        }
    }
}

check alwaysTurnGreen for 3 Color, 2 Light, 10 Time

This is so nice! Isn’t Alloy amazing?

In principle, it is in fact true that this model should satisfy the liveness property because of how we’re modeling turn. But the assertion fails! Why?

We run out of time! We only have 10 Time, so there’s always some last Time. At that time, there’s going to be a red light that never has a chance to turn green - since alwaysOneRed passes, we know there’s always a red light. We need infinite time to really check this, since examples where every light always eventually turns green are infinite.

We know that there’s only 18 states, with a bunch of allowed transitions between them - a graph. Given this, could you find an infinite sequence where some light never turns green? There’s a finite number of states, so if we’re looking for an infinite path, we’re looking for a cycle. We’re looking for a reachable cycle where a light never turns green. The cycle itself is finite in length - it goes through at most 18 states. We might be able to express this in Alloy, sort of similarly to the livelock problem in Alloy 3. But think about a more complicated system - we can’t handle a million states in Alloy. And besides, it would be a pain to specify the cycle detection logic each time.

We need to use the right tool for the right job! Alloy is really good when you have a few states but they’re really rich and interesting. It’s really good at checking safety properties and handling lots of relations among elements of the model. The tool we’ll be looking at starting on Wednesday handles simple states with really long traces. That’s why we’re looking at both paradigms - they each have their own set of tasks they work well for, and it’s important to know which way to approach a problem.