CS1950Y Lecture 9: Analysis Pitfalls
2/12/2018

Analysis Pitfalls

		
			sig Node {edges: set Node}

			pred something {
				some n: Node | some n.edges
			}
What happens if I go in and add a set there? ie. some n: set Node | some n.edges
Answer: 
		
			pred higherOrder {
				some subEdges: edges | subEdges = edges
				#edges > 1
			}
			run higherOrder for 4
Is higherOrder going to be satisfiable?
Answer: If alloy is sneaking in higher order quantification, then yes. Otherwise, if it is first order quantifiaction, then no.

When the thing you are quantifying over has more than one column, then the default is higher order quantification. If it only has one column, the default is first order quantification.

Why does this matter?
Answer: We saw this last week! Alloy just doesn't run. It gives us the skolemization error.

Instead of trying to quantify over edges, you can do something like: all n1, n2: Node | n1->n2 in edges implies...

Natural Numbers

		
			sig N { succ: one N }
			one sig Z extends N {}

			pred largestN {
				some n: N | all n2: N | n in n2.^succ
			}
			run largestN for 5 Node, 5 N
If I run this predicate, will we get an instance or unsatisfiable?
Answer: There are only going to be 5 natural numbers, so there will be one that is the biggest. But also, we haven't said there can't be any cycles. So it will be satisfiable.

We've limited the successors so that everyone has a distinct successor, so what if we add a fact to say that there are no cycles? 
		
			fact {
				all n: N | n not in n.^succ
			}
We don't get any instances when we run with this new fact! This is because alloy can't model the infinite number of natural numbers.

Takeaway: If you are writing facts that force many many things to exist, beware! In your text, this is called the unbounded universal quantifier problem.

Symmetry in Alloy

		
			sig Node {edges: set Node}
			run {no edges & iden} for exactly 2 Node
What does no edges intersection iden force?
Answer: No self loops!
How many examples do we expect to see?
Answer: We only got three, but we thought we would get four. The key here is the error message: "due to symmetry breaking and other optimizations, some equivalent solutions may have been omitted". So alloy is ruling out isomorphic instances in order to keep the example count as small as possible.
What if we give these two Nodes names?
Answer: By naming the nodes, we got the other symmetric edge! Because we named the nodes, alloy doesn't consider the two nodes the same, so we get the next instance.


Sidebar: Under the hood, alloy is adding more constraints on top of the constraints we've given it in order to limit the space it has to search. This helps alloy find unsatisfiability. 

		
			pred someOut {
				some c: Node | some c.edges
			}

			pred someOut2 {
				some Node.edges
			}
			run someOut
			run someOut2
The difference between running someOut and someOut2, is that alloy doesn't label which node satisfies our predicate in someOut2. If you are having debugging problems with your constraints, you should start out with writing plain quantifiers, because alloy will label in your instances what it picked to satisfy the "some".

Skolemization

		
			pred allSome {
				all c: Node | some c2: Node | c->c2 in edges
			}
When we look at instances, we see that each city has an "escape route", or a road leaving it (an outgoing edge). It would be nice though if alloy labeled this for us. The reason alloy didn't label it for us is because there is a setting called "skolem depth" which limits how many "alls" alloy will go in and label for us. We can change that setting to view more labels, which is incredibly helpful when debugging! 
		
			pred completeGraph {
				all c, c2: Node | c->c2 in edges
			}
			run {allSome and completeGraph} for exactly 5
Will we get any instances if we run this? If yes, how many?
Answer: One idea is that we will have the same underlying graph, but alloy will highlight different c/c2. When we look at an instance, we get 5 allSome_c2 labels and 25 edges.
We got loads of instances. Every city is going to produce a new instance with the complete graph underneath it. With skolemization turned on, we get many new instances.

To turn on skolemization, go to options -> skolem depth and then you can change the skolem depth limit (but only in normal alloy! Not in the alloy we use for labs!) 

		
			one sig Skolem {
				witness: set Node -> one Node
			}
			pred allSome2 {
				all c: Node | c->(Skolem.witness)[c] in edges
			}
Now, rather than existentially quantifying, we are telling alloy to look up the c.
Is alloy going to be doing it's default skolemization anywhere?
Answer: No, because we don't have the some quantifier anywhere.
This is essentially what skolemization does. It converts existential quantifiers into functions to get rid of them.