Satisfiability Warsaw lectures V.W. Marek Department of Computer - - PowerPoint PPT Presentation

Satisfiability

Satisfiability

Warsaw lectures

V.W. Marek

Department of Computer Science University of Kentucky October 2013

1 / 1

Satisfiability

What it is about?

◮ Satisfiability is a popular logic-based formalism to

represent finite-domain Constraint Satisfaction Problems (see Apt’s book on Constraint Satisfaction)

◮ It is one of several such formalisms (one is called

ASP and will be mentioned during the last lecture)

◮ It has a long history, grounded in the work of George

Boole and Augustus De Morgan in 19th century

◮ Was developed by philosophical logicians and

mathematicians of 20th century (quite number of them in Warsaw, esp. Alfred Tarski)

◮ Found to be useful by electrical and computer

engineers right before WW II (Claude Shannon)

◮ (In fact Computer Engineering is still a driving force

for development of SAT)

2 / 1

Satisfiability

What it is about, cont’d

◮ Actually. some of the milestones in SAT development

were realized in ECE departments (for instance at Michigan, and at Princeton)

◮ Surprisingly some of these developments involved

interesting algebraic considerations (for instance analysis of the notion of Boolean Constraint Propagation)

◮ There are reasons for use of SAT in Computer

Engineering

◮ Studies of circuits, esp. combinational circuits (but

also to some extent sequential circuits) directly relate to SAT

◮ One of the “killer apps” for SAT is so-called

Equivalence Checking where one tests if the implementation of a circuit is functionally equivalent to the specification

◮ Tools of this sort (e.g. ABC out of Berkeley) use SAT

solvers as “back-end solvers”

3 / 1

Satisfiability

Plan

◮ Basics of SAT ◮ Example ◮ DPLL algorithm and its improvements ◮ Backdoors to SAT (“easy-cases”) ◮ The future

4 / 1

Satisfiability

Formulas of Logic

◮ Defined inductively; via BNF

ϕ ::= ⊥|⊤|p|¬ϕ|ϕ ∧ ϕ|ϕ ∨ ϕ|ϕ → ϕ|ϕ + ϕ

◮ (Here p ranges over a fixed set of atomic

propositions At)

◮ In reality, formulas are labeled binary trees with

leaves labeled by ⊥, ⊤ and atoms

5 / 1

Satisfiability

Formulas, cont’d

◮ A theory is a set of formulas ◮ There are infinitely many formulas over a finite set of

atoms At

◮ (But there is an equivalence relation ≡ describing the

meaning of formulas, with finitely, actually 22n classes for |At| = n)

◮ This relation is defined via semantics of formulas ◮ Abstractly, formulas represent Boolean functions

6 / 1

Satisfiability

Semantics

◮ Semantics is provided by valuations a.k.a. (truth)

assignments, that is functions from At to Bool, the type consisting of 0 and 1

◮ Given a valuation v, the value v(ϕ) is defined

inductively (v(⊥) = 0, v(ϕ ∧ ψ) = min(v(ϕ), v(ψ)), etc.)

◮ (This is Tarski evaluation function) ◮ We think about formulas as constraints on valuations

(because we seek valuations satisfying formulas)

7 / 1

Satisfiability

Semantics, cont’d

◮ The set of literals, Lit, consists of atoms and negated

atoms

◮ A set of literals is complete if it contains exactly one

literal of each pair {p, ¬p}

◮ There is a one-to-one correspondence between

complete sets of literals and valuations (assignments)

◮ We write v |

= ϕ when v(ϕ) = 1

◮ We write v |

= T if v | = ϕ for all v ∈ T

◮ Obviously v |

= ϕ or v | = ¬ϕ (but not both)

◮ Formulas ϕ and ψ are semantically equivalent if for

all valuations v, v | = ϕ if and only if v | = ψ

◮ Semantic equivalence extends to theories

8 / 1

Satisfiability

Conjunctive Normal Forms

◮ A clause is a disjunction of literals ◮ A CNF is a theory consisting of clauses ◮ For every theory T there is CNF theory T ′ so that T

and T ′ are semantically equivalent

◮ Such theory T ′ is not unique but often some CNFs

are more natural than others

◮ Thus the CNF is not, unlike various other

representations such as tables, BDDs, or Boolean polynomials, canonical

◮ (Often we write ¯

p instead ¬p and define ¯ p to be p)

9 / 1

Satisfiability

Properties of clauses

◮ A tautology is a formula that takes value 1 under

every valuation (really, tautologies are not constraints)

◮ A clause is a tautology if and only if it contains a pair

• f complementary (i.e. dual) literals

◮ It is sometimes convenient to think about clauses as

sets of literals (which is confusing, because an assignment - which is a conjunction of literals - can also be interpreted as a set of literals)

◮ A non-tautological clause C is falsified by exactly one

assignment of its variables

◮ When a clause consists of just one literal, it is called

unit clause

10 / 1

Satisfiability

Partial assignments

◮ A partial assignment is a partial function from At to

Bool

◮ A set of literals is consistent if it does not contain a

pair of complementary literals

◮ There is a one-to-one correspondence between

partial assignments and consistent sets of literals

◮ Let us add a third logic value u ◮ Then there is a one-to-one correspondence between

partial assignments and functions from the set of atoms At to {0, 1, u}

◮ (How would you defined it?)

11 / 1

Satisfiability

Partial assignments, cont’d

◮ There is a natural ordering on the family of partial

assignments, ≤k

◮ When we talk about partial functions from At to Bool,

we define it as w ≤k w′ if for all p ∈ Dom(w), w′(p) = w(p)

◮ Ordering ≤k is due to Stephen Kleene, and

describes increased knowledge

◮ We define the ordering ≤k on truth values as

reflexive closure of relation u ≤k 0 and u ≤k 1

◮ (But 0 and 1 are not compatible in ≤k, so this is not

what is normally done with truth values in, for instance, 3-valued Post logic)

◮ We extend the two-valued semantics of propositional

logic to three-valued by setting w(p) = u when p is not in the domain of w

◮ (This is what we did above)

12 / 1

Satisfiability

Partial assignments, cont’d

◮ Then we define three-valued truth tables for the

functors of logic

◮ For instance: 0 ∧ u = 0, but 1 ∧ u = u etc. ◮ What we get is 3-valued truth-function of Kleene (as

• pposed to 2-valued truth function of Tarski we saw

above) which we denote w3(ϕ)

◮ For instance, if ϕ := p ∨ ¬q, w(p) = 0, w(q) = u then

w3(ϕ) = . . . (you guess!)

◮ (And if you program in C or like then maybe it looks

familiar?)

13 / 1

Satisfiability

Partial assignments, cont’d

◮ Kleene theorem says that for all partial truth

assignments w, w′ and all formulas ϕ, if w ≤k w′, then w3(ϕ) ≤k (w′)3(ϕ)

◮ This is a fundamental result for all satisfiability

algorithms where one searches for satisfying assignment by building ever-bigger partial assignments

◮ (Of course for clausal ϕ, Kleene Theorem is obvious)

14 / 1

Satisfiability

Partial assignments, cont’d

◮ Kleene Theorem says is that if you are building a

complete assignment, have a current partial assignment and that approximation assigns to a formula ϕ a Boolean value, then you are stuck with that value until the “bitter end” (unless you backtrack)

◮ In fact the current basic algorithm is doing exactly

this: building a sequence of partial assignments that, ultimately, satisfies the constraints

◮ (Of course backtracks are necessary) ◮ Also, assignments are partial assignments (which

happen to be complete) and if w is complete, then w(ϕ) = w3(ϕ)

◮ There are 2n assignments of a set of atoms At such

that |At| = n, but 3n of partial assignments on that set

15 / 1

Satisfiability

Satisfiability Problems (SAT)

(1) (General form, decision version) Given a propositional theory T decide if there is an assignment v so that v | = T (2) (General form, search version) Given a propositional theory T find an assignment v so that v | = T, if there is one (3) (Clausal form, decision version) Given a clausal theory T decide if there is an assignment v so that v | = T (4) (Clausal form, search version) Given a clausal theory T find an assignment v so that v | = T, if there is one

16 / 1

Satisfiability

Satisfiability Problem, cont’d

◮ (Cook - Levin) Problems (1) and (3) are

NP-complete, and in (3), limitation to clauses of length ≤ 3 does not change the status

◮ Problems (2) and (4) are FNP-complete ◮ Problems (2) and (4) can be solved by a linear (in the

number of variables) applications of algorithms for (1) or for (3)

◮ (Cook-Levin Theorem tells us that immense number

• f problems polynomially reduce to SAT, namely all

problems in the (decision) class NP and (search) class FNP

◮ Knowledge Representation is the area of AI that

deals with representing problems so we can solve them using suitable software

◮ Thus we should pay attention on how we reduce, and

• ur example below will be an exercise in Knowledge

Representation

17 / 1

Satisfiability

Solving and designing sudoku’s

◮ (This example is “generic” - SAT is a general purpose

tool, but we assume everybody saw Sudoku)

◮ I guess a billion people daily solve Sudoku puzzles (I

for one solve a Sudoku, and my Department publishes daily (actually two) Sudoku-like puzzles)

◮ So what is Sudoku, and how it relates to SAT? ◮ Sudoku (or Number-place) is a kind of a game where

a 9 × 9 grid is partially filled with numbers from (1..9)

◮ The goal of this (solitaire) game is to fill the grid so

that each row, each column, and each 3 × 3 section contains each number

18 / 1

Satisfiability

Sudoku’s story

◮ According to Wikipedia, Sudoku was invented by a

retired architect from Connersville, IN, Howard Garns, in 1979 and published regularly in Dell Puzzles

◮ (But surely Leonhard Euler must have known

sudokus because sudoku is just an Euler square with an additional constraints)

◮ It was popularized by Japanese (because it is hard to

produce meaningful crossword puzzles in Japanese)

◮ A correct Sudoku puzzles possesses exactly one

solution (which is sometimes used in the solving process)

19 / 1

Satisfiability

Sudokus, cont’d

◮ But what does it have to do with SAT? ◮ Let us start with a set of propositional variables that

has exactly 729 elements (why 729?)

◮ We will list all these variables as a three-dimensional

array, but instead of a[i][j][k] we will write ai,j,k with i, j, k ranging over (1..9)

◮ The intention is that that the atom ai,j,k is true if the

cell in the ith row, jth column contains the number

◮ So, Sudoku problem is then something like this: the

designer tells us that a certain number of these 729 variables are true

◮ And she tells us: there is a solution (fill of the grid), in

fact exactly one

◮ And she tells us: find it!

20 / 1

Satisfiability

Sudokus, cont’d

◮ If we think about our problem, then we see that the

data given to us by the designer is a partial assignment of our 729 variables

◮ The issue is to find an extension of the given partial

assignment (data) to a complete assignment so that all variables are assigned values, and the grid is completely filled

◮ Now we have two choices. We can either use paper

and pencil to solve it, or maybe, just maybe, out of laziness, we use a computer to solve it

21 / 1

Satisfiability

Sudokus, cont’d

◮ We could write a program in C that will search the

grid for solutions

◮ Or we could write once and for all a single collection

• f constraints T so that out of a satisfying

assignment for T ∪ D (where D is the representation

• f data provided by the designer) we can read-off the

solution

◮ (Of course there are numerous Sudoku solvers on

WWW, so the alternative above is not complete)

22 / 1

Satisfiability

Sudokus, cont’d

◮ So we now have these 729 variables and what do we

do with them?

◮ We write couple of clauses

For instance, we want to say that every cell contains at most one number: ¬ai,j,k1 ∨ ¬ai,j,k2 whenever k1 < k2

◮ Actually, there is

9

2

• , i.e. 36 of these. As there are 81

cells, we have 2916 of such clauses

◮ But it is not all ...

23 / 1

Satisfiability

Sudokus, cont’d

◮ We need to make sure that each row has each

number in it: a1,1,1 ∨ a1,2,1 ∨ . . . ∨ a1,9,1 (And this says ...)

◮ 9 rows, 9 numbers, so 81 clauses (each of length 9),

altogether 81 such clauses

◮ And similar 81 clauses for columns...

24 / 1

Satisfiability

Sudokus, cont’d

◮ And now about sections:

a1,1,1 ∨ a1,2,1 ∨ a1,3,1 . . . ∨ a3,3,1 (This one was about the symbol 1 in the first section)

◮ Altogether 81 clauses of this sort ◮ Thus 2916 + 243 clauses, i.e. 3159 clauses

(assuming there were no errors ...)

◮ The following is pretty obvious: there is a one-to-one

correspondence between sudoku solutions and the satisfying assignments for the collection of clauses defined above

25 / 1

Satisfiability

Sudokus, cont’d

◮ But this was about grids, not about solutions of the

individual sudoku problems

◮ We have the following fact, not especially deep (a

kind of Bayes Theorem, only simpler): Let T, S be two propositional theories in the language LAt. Then models of T ∪ S are precisely the models of T that satisfy S

◮ So, adding clues (such as a1,7,2) selects out of all

valid sudoku grids those which satisfy the clues

26 / 1

Satisfiability

Sudokus, cont’d

◮ But what is the pragmatics of encoding? Surely

solvers can not understand expressions such as a1,7,2

◮ Indeed, there is a standard, called DIMACS format

for describing SAT (clausal) theories

◮ Actually, it is quite simple; the literals are represented

as signed integers, and the number 0 is used as a clause separator

◮ Since the only functors are negation (applied to

variables) and disjunction, we drop disjunction, and a clause is modeled as a list of signed integers separated by spaces and ending with 0

◮ We also need to tell the solver how many

propositional variables are there, and the number of lines (although surely we can derive this)

27 / 1

Satisfiability

Coding sudoku, cont’d

◮ Let us represent variable ai,j,k by the integer

81 ∗ (i − 1) + 9 ∗ (j − 1) + k (why k and not k − 1?)

◮ So, for instance a1,2,1 is 10 ◮ And the clause ¬a1,2,1 ∨ ¬a1,2,7 becomes:

−10 − 16

◮ Altogether we get 3159 clauses and then the header

line: p cnf 729 3159

◮ (the comment lines use c as a first two characters)

28 / 1

Satisfiability

Coding sudoku, cont’d

◮ In the process the meaning of this collection of line

as a sudoku description entirely disappears

◮ So the translation of whatever (if anything) solver

returns us is needed to recover the meaning

◮ Let us observe that the knowledge representation is

a major step in the use of solvers

◮ Knowledge Representation is not automated in SAT -

• nly solving is push button

29 / 1

Satisfiability

Solving

◮ So let us assume we downloaded the minisat or any

• ther reasonable SAT solver

◮ We also assume that we wrote a script that produced

the sudoku constraints

◮ We also grabbed the sudoku problem from the

newspaper

◮ That problem had 32 clues ◮ For instance 4 in the (1,2) cell; this generated the

(unit) clause 140

◮ We computed all 32 lines describing the clues ◮ Then we added these at the end and changed the

header line to p cnf 729 3191

◮ Saved the file as sudoku.cnf

30 / 1

Satisfiability

Solving, cont’d

◮ We now run:

» minisat sudoku.cnf > res

◮ We get some gibberish telling us which variables are

true

◮ We decode the content of the grid that is the solution

31 / 1

Satisfiability

Formats

◮ Every Boolean function is represented by a CNF T

so that f(v) = 1 if and only if v | = T

◮ Thus every Boolean function is represented by a

DIMACS file

◮ There are competing formats; we can represent

functions by DNFs and then the natural way to think is in terms of equations f(v) = 0

◮ There is a representation standard for

And-Inverter-Graphs (you sure know who invented it)

◮ An important format is for representing BDDs (much

liked by EE)

32 / 1

Satisfiability

So now we know that solving can be done

◮ But how is it done? ◮ What kind of algorithms are used? ◮ The fundamental algorithm for solving is so-called

DPLL algorithm named after Davis, Putnam, Logemann and Loveland

◮ This algorithm is based on two basic ideas:

(1) Boolean constraint propagation (2) Backtracking search in the space of partial assignments

◮ (Significant improvements in the idea (2), based on

the analysis of the idea (1) resulted in the power of modern solvers)

33 / 1

Satisfiability

Boolean constraint propagation (BCP)

◮ The idea is simple: Say I have a clause C

l1 ∨ l2 ∨ . . . ∨ ln−1 ∨ ln

◮ And I already established that the literals ¯

l1, . . . ,¯ ln−1 are true (whatever it means ...)

◮ Therefore to make the clause C true I have to

commit to the truth of ln

◮ (The order of literals in a clause is immaterial) ◮ In terms of partial assignments this means that if my

current partial assignment w contains ¯ l1, . . . ,¯ ln−1 then I need to commit to ln

◮ But can I always do this? ◮ What if my w contains ¯

ln?

34 / 1

Satisfiability

BCP, cont’d

◮ We can formalize BCP as an operation BCP(·, ·)

where the first argument is a partial assignment, and the second argument is a clausal theory

◮ We need to be a bit careful, because the result does

not have to be a partial assignment (i.e. a consistent set of literals); it may become inconsistent

◮ Example: we have w = {¬p, q, r}, and T contains a

clause: p ∨ ¬q ∨ ¬r

◮ We compute ¬r and we no longer have a partial

assignment, but a set of literals that is inconsistent

◮ This means that the choices we did on our way to the

assignment w resulted in the inconsistency and hence a backtrack will be needed

35 / 1

Satisfiability

BCP, cont’d

◮ But if we are careful, and extend the definition of

BCP so that it accepts as the first argument a set of literals, then we can check that BCP is monotone, so we can use Knaster-Tarski theorem and claim existence of a least fixpoint

◮ We are interested in the case when the first

argument is a partial assignment, and we would like to have the output that is a partial assignment, but if it is not, then we also get useful information

◮ But, what is BCP, and does it have a logic meaning,

• r just the algebraic one?

◮ It turns out it has logical and algebraic and

graph-theoretical meanings

36 / 1

Satisfiability

Resolution

◮ Resolution is the following rule of proof:

C1 ∨ l C2 ∨¯ l C1 ∪ C2

◮ So what happens is that if one of clauses contains l

and another ¯ l then we eliminate l,¯ l from both and take the disjunction of the remaining literals

◮ Example: Given C1 := p ∨ q ∨ ¬r, C2 := p ∨ r ∨ s, we

conclude the clause C1 ⊗ C2 := p ∨ q ∨ s

◮ Resolution rule is sound, if an assignment v satisfies

both C1 and C2, then it also satisfies C1 ⊗ C2

37 / 1

Satisfiability

Subsumption (weakening)

◮ The subsumption rule is

C D where C ⊆ D

◮ Subsumption Rule is also sound ◮ Quine Theorem: A non-tautological clause C is a

consequence of a clausal theory T if and only if some resolution consequence D of T subsumes C

◮ Thus the proof system based on resolution and

subsumption is complete for clausal logic

◮ (Actually Resolution alone is not complete for clausal

logic)

◮ A slightly stronger form of completeness, limiting

proofs to at most one application of subsumption is also valid by the second Quine Theorem

38 / 1

Satisfiability

Resolution and satisfiability

◮ Resolution is a (partial) binary function in the set of

all clauses

◮ So, given a clausal theory T, we define Res(T) as

the closure of T under resolution

◮ Quine Theorem: a CNF theory T is satisfiable if and

• nly if empty clause does not belong to Res(T)

◮ Unfortunately, there is O(3n) clauses (why?) ◮ So, testing satisfiability via closure under resolution

is not a feasible option

39 / 1

Satisfiability

Resolution and BCP

◮ BCP(v, T) is closely related to resolution, though ◮ Specifically, unit resolution is a restriction of

resolution where one of the inputs must be a unit

◮ Then it turns out that the literals computed with

BCP(T) are precisely literals that possess a unit resolution proof out of T

◮ Thus we now have a proof-theoretic representation

• f BCP

◮ Of course, a proof is a labeled directed acyclic graph ◮ When we think about BCP in these terms, we get

graphs labeled both with clauses and literals

◮ Specifically, literals labeling immediate predecessors

form inputs to a clause labeling the node

◮ (So a node is kind of a machine, on some inputs it

produces outputs)

40 / 1

Satisfiability

Reduct of a CNF by a partial assignment

◮ Reduct of a clausal theory T by a partial assignment

w is defined as follows:

◮ If there is a literal l ∈ w such that l ∈ C - eliminate C

(i.e. C becomes null

◮ If there is a literal l ∈ w such that ¯

l ∈ C - eliminate ¯ l from C

◮ This is done until fixpoint (i.e. no more eliminations

possible)

◮ The result is called reduct of T by w ◮ (We will denote it by T/w)

41 / 1

Satisfiability

Reduct and satisfiability

◮ Example: v = {p, ¯

q}, T = {p ∨ r, ¬p ∨ s ∨ t, q ∨ u}

◮ The first clause is eliminated, the second shortened,

the third one is also shortened, producing a new literal (which could be further used if we had more clauses)

◮ (Reduct of a theory by a partial assignment may be

contradictory, can you craft an example?)

◮ A clausal theory T is satisfiable if and only if BCP(T)

is a partial assignment, and the reduct of T by that partial assignment is satisfiable

◮ In fact when the r.h.s. is true every satisfying

assignment for the reduct expands to a satisfying assignment for T

42 / 1

Satisfiability

BCP and satisfiability, cont’d

◮ The last fact of the previous viewgraph tells us that

we can use BCP as a preprocessing and also inprocessing in every node of the search tree

◮ (We do not know yet how this search tree looks like,

but we know that we will have such tree)

◮ It turns out that BCP contains much more

information, due to the fact that the we have the proof-theoretic interpretation, and that there is

• rdering of decisions on each branch of the search

tree

43 / 1

Satisfiability

Basic DPLL algorithm

◮ Given an input CNF theory T we compute BCP(T) ◮ If BCP(T) is contradictory we leave with failure ◮ O/w v := BCP(T) ◮ If v3(T) = 1 we return any complete assignment of

At extending v

◮ O/w we select a literal l whose underlying variable is

not in the domain in v, set variable first to 1

◮ We set w := BCP(v ∪ {l}, T) ◮ If w is non-contradictory, we set v := w and call our

algorithm recursively (thus new decision will be made, and the variable first will be reset to 1)

◮ If w is contradictory and first equals to 1 we set first

to 0 and call the algorithm recursively (with the same v, not w)

◮ If w is contradictory and first equals to 0 then we

backtrack to the last decision where the first was 1, change first to 0 and continue

44 / 1

Satisfiability

Data structures

◮ We need appropriate data structures to support this

algorithm

◮ The basic one is the history that tells us:

◮ What decisions were taken and at what level ◮ What were the values of the first bit ◮ What were the results of the BCP after the decisions ◮ Additional information that will support conflict

analysis, for instance levels of the literals computed by BCP

45 / 1

Satisfiability

Autarkies

◮ A partial valuation v touches a clause C if one of the

variables on which v is defined occurs (positively or negatively) C

◮ For instance p touches ¬p ∨ q ∨ r ◮ A partial valuation v is an autarky for a clausal theory

T if for all clauses C of T, if v touches C then v satisfies C (i.e. v3(C) = 1)

◮ A pure literal for T is a literal l so that ¯

l does not

• ccur in T

◮ Of course, if l is pure in T then {l} is an autarky for T ◮ Autarkies are very desirable, because if v is an

autarky for T then T is satisfiable if and only if the reduct T/v is satisfiable

◮ But unfortunately existence of nontrivial autarkies is

an NP-complete problem (but see below, when we discuss Krom clauses)

46 / 1

Satisfiability

’Easy cases’ of SAT

◮ There are numerous classes of SAT problems where

solving (i.e. searching for solution) are easy (i.e. in the class P)

◮ In fact there are various hierarchies of SAT problems

with the increasing degree of difficulty

◮ We will discuss here only 7 classes

◮ Positive formulas ◮ Negative formulas ◮ Krom formulas ◮ Horn formulas ◮ Dual-Horn formulas ◮ Renameable-Horn formulas ◮ Systems of linear equations over Z2 (affine formulas) 47 / 1

Satisfiability

Positive formulas

◮ A positive formula is one where on any path from the

root to a leaf there is an even number of nodes labeled by negation symbol

◮ (Equivalently: if we “push negation inward”, the result

will have no negations)

◮ It is easy to test a positive CNF for satisfiability; it is

satisfiable if and only if when none of the clauses is ⊥

◮ For non-CNF and positive it is a bit harder, but also

easy

◮ Moreover if such formula is satisfiable then it is

satisfied by constant function 1At

48 / 1

Satisfiability

Negative formulas

◮ A negative formula is one where on any path from

the root to a leaf there is an odd number of nodes labeled by negation symbol

◮ (Equivalently: if we “push negation inward”, the result

will have negations over all atoms in the leaves)

◮ It is easy to test a negative CNF for satisfiability; it is

satisfiable if and only if when none of the clauses is ⊥

◮ For non-CNF and negative it is a bit harder, but also

easy

◮ Moreover if such formula is satisfiable then it is

satisfied by constant function 0At

◮ (There is also a proof using permutations, can you

guess it?)

49 / 1

Satisfiability

Krom CNFs

◮ Krom clause is one of size ≤ 2 ◮ Krom CNF is one consisting of Krom clauses ◮ Collection of all Krom clauses is closed under

resolution

◮ When BCP is run over a set of Krom clauses and we

do not get inconsistency, the result is a partial assignment and a set of clauses of length exactly 2

◮ When l ∨ m is a Krom clause can assign to it two

edges; one from ¯ m to l, another from ¯ l to m

◮ Then, a set K of clauses of length 2 determines a

directed graph GK

50 / 1

Satisfiability

Krom formulas, cont’d

◮ (What is a strongly connected component in a

digraph?)

◮ A Krom CNF is satisfiable if and only if no strongly

connected component in GK contains a pair of dual literals

◮ This can be used to devise a polynomial time

algorithm for finding a satisfying assignment for a given Krom CNF K

◮ We topologically sort (what is it?) strongly connected

components of GK and then in that order, going backwards, we assign the values to literals

◮ Because no component has a pair of dual literals set

• f duals of a connected component is also a

connected component

51 / 1

Satisfiability

Krom formulas, cont’d

◮ So, we repeatedly take the first (remember the

reverse order) unassigned strongly connected component, set on it value 1 and set the value 0 on its dual, repeat till all variables are assigned values

◮ The resulting assignment satisfies K ◮ The reason is that if both literals l, m in a clause l ∨ m

are evaluated as 0 then ¯ l (which is evaluated as 1) must be assigned 1 after ¯ m (because ¯ l precedes m in GK) and likewise ¯ m must be assigned 1 after ¯ l

52 / 1

Satisfiability

Krom formulas, example

◮ Let us look at this K:

◮ ¬p ∨ q ◮ ¬q ∨ r ◮ ¬r ∨ p ◮ p ∨ s

◮ The graph GK has 4 strongly connected components ◮ The two of these are: {p, q, r}, another is {¬s} ◮ (The other two are immaterial, why?) ◮ We get an assignment s = 0, p = q = r = 1

53 / 1

Satisfiability

Another SAT algorithm for Krom theories

◮ Krom CNFs have the following desirable property:

given any partial valuation v, if w = BCP(v, T) is consistent then it is an autarky for T

◮ So here is an algorithm (we assume BCP(∅, T) = ∅):

◮ Select any l, compute w = BCP({l}, T), set visited

to 1

◮ If w consistent: ◮ Reduce T by W, and if the reduct empty, return any

complete extension of w

◮ O/w call the algorithm recursively ◮ O/w set visited to 0, call the algorithm recursively but

with ¯ l instead of l. If the computed BCP is inconsistent, leave with failure

◮ Otherwise reduce, choose new literal, continue 54 / 1

Satisfiability

Krom formulas, cont’d

◮ Operation maj is a ternary Boolean function that

returns the value of the majority of arguments

◮ (Thus maj(1, 0, 0) is ...) ◮ Various programming languages have built-in bitwise

• perations on Boolean strings of same length, say

&&, or ||

◮ We can imagine a ternary operation maj on Boolean

strings of some fixed length n

◮ (When the ordering of propositional variables in At is

fixed, such string is a code for an assignment)

55 / 1

Satisfiability

Krom formulas, cont’d

◮ Let V be a set of assignments of the set of atoms At

and let us think about V as a collection of Boolean strings

◮ Then there is Krom CNF K such that V is a set of all

satisfying assignments for K if and only if V is closed under bitwise maj

◮ This can be used to devise yet another polynomial

time algorithm for finding a satisfying assignment for a given Krom CNF K

56 / 1

Satisfiability

Krom clauses, conclusion

◮ Krom clauses are quite useful in processing of

clausal theories (we will see it below)

◮ One important property is thet if |C| = n, D is Krom

and C, D rsolvable, then |C ⊗ D| = n so resolving with Krom clauses does not increase sizes of the resolvent

◮ (There is plenty of literature on the use of Krom

clauses)

57 / 1

Satisfiability

Horn clauses

◮ Horn clause is one that has at most one positive

literal (i.e. one or none)

◮ A clause with one positive literal is called (depending

• n the community) definite Horn clause, or program

clause

◮ A clause with no positive literal is sometimes called a

constraint (quite misleading...)

◮ The reason why program clause is so called is that

the clause p ∨ ¯ q1 ∨ . . . ∨ ¯ qn is really an implication: (q1 ∧ . . . ∧ qn) ⇒ p that is a rule: once q1, . . . , qn are computed (derived), compute p

◮ A Horn CNF splits naturally into two parts: P –

consisting of program rules, and C consisting of constraints

58 / 1

Satisfiability

Horn clauses, cont’d

◮ A definite Horn theory (i.e. a program) is satisfiable

(obviously, why?)

◮ Given a set of atoms A and a definite clause

C := (q1 ∧ . . . ∧ qn) ⇒ p, we say that A matches C if q1, . . . , qn all belong to A

◮ (p is called the head of C, denoted head(C)) ◮ Then we define an operator TP in the family of

subsets of At by: TP(A) = {head(C) : C ∈ P and M matches C}

◮ The operator TP is monotone, so it possesses a

fixpoint, called MP

59 / 1

Satisfiability

Horn clauses, cont’d

◮ We can think about assignments as sets of atoms ◮ The reason is that an assignment is the

characteristic function of a subset M ⊆ At

◮ If H is a definite-Horn (i.e. consists of definite Horn

clauses) then there is a ⊆-least set that satisfies H

◮ This set coincides with the least fixed point of the

• perator TH

◮ When H is a Horn theory but not definite-Horn, then

H uniquely splits into the disjoint union H1 ∪ H2 where H1 is definite-Horn and H2 consists of constraints

◮ If H is Horn, then H is consistent if and only if the

least model of H1 satisfies H2

60 / 1

Satisfiability

Horn clauses, cont’d

◮ This last fact implies an algorithm for testing Horn

theories for satisfiability and computing a satisfying assignment (if satisfiable)

◮ Given input Horn theory H, we first split H into

H1 ∪ H2 (where H1 consists of definite-Horn clauses, H2 consists of constraints)

◮ Next, we compute the least fixpoint of the operator

TH1

◮ Finally, we test if the least model, MH1 satisfies H2 ◮ (But are these two above steps algorithmic? and if

so, how complex?)

61 / 1

Satisfiability

Computing the least model of H1, cont’d

◮ It should be obvious that we can do this in quadratic

time

◮ The reason is that we can start with the empty set of

atoms, and each time the body of a clause is matched, add the head

◮ We will run more than m times (where m is the

number of clauses)

◮ This gives the quadratic upper bound on the

complexity

62 / 1

Satisfiability

Computing the least model of H1, cont’d

◮ But we can do better with Dowling - Gallier algorithm ◮ In this algorithm we associate with each

definite-Horn clause C a counter holding the number

• f atoms in the body of C which are not yet computed

◮ When that counter drops to 0, we add the head of C

to the list MP

◮ With appropriate data structure this results in linear

number of steps (each atom can be computed only

• nce)

63 / 1

Satisfiability

Testing satisfaction of constraints

◮ Once MH1 is computed, we go through the list of

constraints only once

◮ For each constraint ¬q1 ∨ . . . ∨ ¬qm in H2 we test if

MH1 ∩ {q1, . . . , qm} = ∅

◮ If so we leave with failure ◮ If all constraints satisfied - we return MH1 ◮ With appropriate data structures this can be done in

linear time

◮ (Dowling - Gallier algorithm allows for computation of

BCP(T) for clausal theory T in linear time, why?)

64 / 1

Satisfiability

Horn clauses, cont’d

◮ Assuming At finite, any family of sets closed under

intersections (equivalently: family of assignments closed under bitwise-conjunction) is the family of models for suitably chosen Horn clausal theory H

◮ (Actually, an old but important piece by Makowsky

explains why Horn theories matter in CS)

◮ Any monotone operator in finite set of atoms At is of

the form TH for suitably chosen definite-Horn theory H

◮ Any Horn theory that contains no positive units is

satisfiable (why?)

65 / 1

Satisfiability

So, maybe this could be of use?

◮ Let T be a CNF, we can split T = H ∪ R where H is

Horn part of T

◮ Then we could quickly compute MH. These guys

have to be in evaluated as 1 for every assignment satisfying H , thus T

◮ But the gain is illusory. The reason is that elements

• f MH are computed by as BCP(H) and since BCP is

monotone, also out of T

◮ Thus DPLL will compute MH in the first step

66 / 1

Satisfiability

Horn Theories, example

◮ H := {p ∨ ¬q ∨ ¬r, r ∨ ¬s, r, q, ¬q ∨ ¬p} ◮ H1 := {p ∨ ¬q ∨ ¬r, r ∨ ¬s, r, q} ◮ H2 := {¬q ∨ ¬r} ◮ MH1 = {p, q, r} ◮ MH1 does not satisfy H2 because {p, q, r}∩ {q, r} = ∅ ◮ If instead of ¬q ∨ ¬r we have a constraint ¬q ∨ ¬s,

MH1 satisfies H2, thus H

◮ (Why do we give such trivial examples?)

67 / 1

Satisfiability

Permutations of atoms

◮ Given a permutation π of the set At we can extend

the actions of π to formulas

◮ Since the formulas are trees with atoms and

constants in the leaves, we just put π(x) for x in the corresponding leaf

◮ Then, we can extend the action of permutations to

assignments, namely π(v)(x) = v(π(x))

◮ Then, kind of obviously, v |

= ϕ iff π(v) | = π(ϕ)

◮ But there is more to permutations ◮ Since clauses consist of literals, we want to deal with

permutations of literals

68 / 1

Satisfiability

Permutations of literals

◮ Obviously, we need to be careful, because if we are

not then we may not preserve logical properties

◮ For instance, say we have a formula ϕ := p ∧ q, we

map literal p to r, and literal q to ¬r

◮ Then ϕ is satisfiable, but π(ϕ) is not satisfiable ◮ This motivates the following definition: a permutation

π of Lit is consistent if for all literals l, π(¯ l) = π(l)

◮ Thus we can change variable, and we can change

sign, but in a consistent way

◮ Clearly, a consistent permutation of literals is

determined by two objects:

◮ A permutation of variables, and ◮ A binary sequence (keep the sign or not) 69 / 1

Satisfiability

Permutations of literals, cont’d

◮ Thus there is precisely 2n · n! of consistent

permutations of literals when |At| = n

◮ As before we can extend the action of consistent

permutations of literals to formulas, and to assignments (partial assignments, too)

◮ The permutation lemma above holds for consistent

permutations of literals

◮ A shift permutation of literals is one which does not

change variables, but only the signs (that is moves l to l or to ¯ l)

70 / 1

Satisfiability

Permutations of literals, cont’d

◮ Shifts form a group, of size 2n ◮ Shifts commute with permutations of atoms ◮ Square of a shift is always the identity ◮ Image of a Horn clause under permutation of atoms

is Horn; satisfiability is preserved

◮ But a consistent permutation of literals does not have

to preserve Horn theories (although it preserves satisfiability)

◮ Image of a Krom clause under consistent

permutation of literals is Krom, satisfiability is preserved

71 / 1

Satisfiability

Dual-Horn theories

◮ A dual-Horn clause is one that has at most one

negative literal

◮ A dual-Horn theory is a CNF consisting of dual-Horn

clauses

◮ The shift sending l → ¯

l sends Horn theories to dual-Horn and conversely, too

◮ Thus every result we had on Horn clauses has a

corresponding result for dual-Horn theories

72 / 1

Satisfiability

Dual-Horn theories, cont’d

◮ Also the algorithms that we have for Horn theories

can be “refurbished” to be used for dual-Horn

◮ And, of course, we can shift dual-Horn theory to a

Horn one, do something, and then shift back

◮ (State theorems, get proofs) ◮ Sets of all assignments for dual-Horn CNFs can be

characterized by a property analogous to that characterizing Horn formulas; can you characterize it?

73 / 1

Satisfiability

Renameable-Horn theories

◮ A renameable-Horn theory is one for which a shift

permution transforming it into a Horn theory exist

◮ Certainly it is a very desirable property (why?) ◮ The issue is if we could find such shift, if one exists ◮ And of course at a non-significant cost ◮ Miraculously, this can be done

74 / 1

Satisfiability

Example

◮ Here is a simple example of what happens when we

try

◮ Let us look at a clause C := p ∨ q ∨ ¬r¬s ∨ ¬u ◮ We introduce new atom shift(x) for each atom x ◮ What are the constraints on shift? ◮ We need to shift at least one of p or q (if there were

positive literals, we would have more similar constraints)

◮ We can not shift r and s at the same time, likewise r

and u at the same time and s and u at the same time

◮ If we shift r then we must shift both p and q (and the

same for s and for u

◮ I believe that I specified 6 constraints on our shift (but

check)

◮ Let us denote this set of clauses by SC

75 / 1

Satisfiability

Renameable-Horn theories, cont’d

◮ The clauses in SC are all Krom ◮ Given a CNF T, let us form ST = C∈T SC ◮ Clearly, ST is Krom, and |ST| = O(|T|2) ◮ Hence, in principle, we can solve ST ◮ (Why “in principle”?)

76 / 1

Satisfiability

Renameable-Horn theories, cont’d

◮ Lewis Theorem: There is a one-to-one

correspondence between satisfying assignments for ST and shift permutations that transform T into a Horn

◮ Thus we can form ST, use the algorithm for solving

Krom theories

◮ If we succeed, we have a shift π turning T into a T ′

that is Horn

◮ Now we solve T ′ ◮ If we succeed, we get a solution w for T ′; the

assignment π(w) is a solution for T theory T (how?)

77 / 1

Satisfiability

Algebraic methods

◮ The structure Z2 := Bool, +, ∧, 0, 1 is a field ◮ (Here + is XOR, not ∨, ∧ is in our context denoted ·) ◮ Z2 is a very nice field because over that field every

function is a polynomial (actually, the only field with this property)

◮ That is, for every function f : Booln → Bool there is

g ∈ Bool[x1, . . . , xn] so that f ≡ g

◮ Also, Boolean polynomials provide a canonical

representation of Boolean functions (once the order

• f variables is fixed), like tables, and ROBDDs)

◮ (Actually, polynomials are very closely related to

ROBDDs)

78 / 1

Satisfiability

Algebraic methods, cont’d

◮ This has consequences; algebraic methods such as

Gröbner bases, etc. apply (not that I am a specialist)

◮ For a time, esp. in the 90ies, there was a great hope

that so-called Nullstellensatz of Hilbert would provide a processing methods for satisfiability

◮ Here is the idea: Let ϕ[x1, . . . , xn] be a formula. Then

there is a polynomial f[x1, . . . , xn] so that for any assignment a1, . . . , an f(a1, . . . , an) = 0 if and only if a1, . . . , an | = ϕ

◮ But now, satisfiability of a set of formulas (maybe

clauses) is transformed to finding common zeros of a set of polynomials

◮ Here is when Nullstellensatz comes in, for it gives a

criterion on a (power of) polynomial belonging to the ideal generated by a set of polynomials

79 / 1

Satisfiability

Affine formulas

◮ A linear equation over At = {x1, . . . , xn} is one of the

form ε1x1 + ε2x2 + . . . + εnxn + εn+1 = 0

◮ An affine formula is a conjunction of linear equations ◮ Since Z2 is a field, therefore we have various

techniques for solving affine formulas (i.e. finding satisfying assignments)

80 / 1

Satisfiability

Affine formulas

◮ Gaussian elimination, Gauss-Jordan variation ◮ (See Apt’s book) ◮ (Because the characteristics of Z2 is 2 the

determinants do not apply)

◮ It is possible to extract out of a CNF implied affine

formula

◮ This can (and in fact was) used for preprocessing

and inprocessing (see below)

81 / 1

Satisfiability

Affine formulas, cont’d

◮ The idea is the same which failed us at Horn case

but succeeds here;

◮ Let us extract out of a CNF T a set of linear

equations A

◮ If A determines some specific values - we can assign

these values for all putative satisfying assignments

◮ For other values, we get additional, quickly

computable constraints

◮ Example: if we found that A entails p1 = 0 we can

skip the branch where p1 = 1; if we found that p2 = p5 + p7 + 1 then each time we assign values to p5 and p7 we know how to assign the value to p2

82 / 1

Satisfiability

Affine formulas, cont’d

◮ There is a characterization of the collections of

assignments that are satisfying assignments for a set

• f linear equation

◮ Let A be a set of assignments. Then there is an

affine formula ϕ such A is the set of satisfying assignments for ϕ if and only if A is closed under bitwise sum of three

◮ Generally, affine formulas are closely related to linear

error-correcting codes - which is an important (and very applicable) area of Combinatorics

83 / 1

Satisfiability

Why these characterizations?

◮ In various characterizations we had “inverse” results

(all of these were of the form A is a set of assignments satisfying a theory T of some syntactic form)

◮ These characterizations connect logic of SAT with

Universal Algebra, to be precise with an important

• bject called Post Lattice studied by people in

computational Universal Algebra

◮ (They talk about clones, i.e. classes of functions

closed under substitutions and polymorphisms - and those are not related to polymorphic data structures, etc.)

◮ (The mathematics of Post Lattice is breathtakingly

beautiful)

◮ (See the book by Dietlinde Lau, Function Algebras

• n Finite Sets)

84 / 1

Satisfiability

Are there more “easy classes”?

◮ Of course there are ◮ Trivial example is a class of formulas consisting of

Horn clauses and a bounded number (say 7) of other clauses

◮ But there are others, one is called SLUR, single

literal unit resolution class of Franco and van Gelder, where contradictory sets of clauses generate contradiction via BCP

◮ There is a beatiful result of Kuˇ

cera et.al. showing that every Boolean function possesses such representation (but checking that a representation is

• f this kind is a co-NP-complete problem)

85 / 1

Satisfiability

Opportunity for ’Divide-and-Conquer’?

◮ Let us explore the case when T can be represented

as T1 ∪ T2, both T1 and T2 belonging to “easy cases”

◮ Unfortunately this approach fails spectacularly ◮ Here is an example: There is a polynomial-time

transformation of clause sets T into clause sets T ′ so that:

◮ T ′ = T1 ∪ T2 ◮ T1 consists of positive clauses ◮ T2 consists of Krom clauses ◮ There is a one-to-one polynomial time

correspondence between the satisfying assignments

• f T and satisfying assignments for T ′

86 / 1

Satisfiability

‘Divide and conquer’, cont’d

◮ Let us look at an example. Say I have a clause

C := p ∨ q ∨ ¯ s ∨ ¯ t ∨ ¯ u

◮ I introduce three new atoms s′, t′, and u′ ◮ I introduce six new Krom clauses: s ∨ s′, ¯

slors′ (you guess the remaining 4) and get T ′

◮ (s′ plays the role of ¬s) ◮ My original clause becomes C′ := p ∨ q ∨ s′ ∨ t′ ∨ u′ ◮ Obviously, there is a one-to-one, polynomial

correspondence between satisfying assignments for T := {C} and T ′

◮ (Here T1 consists of C′, T2 consists of newly

introduced Krom clauses)

87 / 1

Satisfiability

‘Divide and conquer’, cont’d

◮ The construction outlined above is general, we do

this for all clauses in T, take union, get T ′, obviously polynomial-time in T

◮ T ′ = T1 ∪ T2, T1 is positive, T2 is Krom, and there is

• ne-to-one polynomial correspondence between

satisfying assignments for T and T ′

◮ So, if we can solve the case positive/Krom in

polynomial time, we can solve the general case in polynomial time

88 / 1

Satisfiability

‘Divide and conquer’, cont’d

◮ With seven classes I considered above there is 21

(why?) cases to consider

◮ There are two trivial cases (all negative clauses are

Horn, all positive clauses are dual-Horn)

◮ The remaining 19 cases are all NP-complete ◮ (We are not finite-group theorists and do not

consider so many cases)

◮ The only mildly interesting case is the case of linear

equations and Krom clauses

◮ In that case we first reduce SAT to 3-SAT (what is

3-SAT?) and then transform a 3-clause to one linear equation and two Krom clauses

89 / 1

Satisfiability

Closer look at BCP

◮ Let us fix F and write BCP(v) instead of BCP(v, F) ◮ As we progress on a branch of the search-tree we

make consecutive assumptions

◮ The branch itself determines the order of these

assumptions

◮ Let us use notation l1@1, l2@2, . . . ◮ Example: Our clause set contains among others

p1 ∨ p17 ∨ ¯ p9, ¯ p9 ∨ p7 and at level 1 we make decision p9 (i.e. p9@1) and at level 2 we make decision ¯ p8 (i.e. ¯ p8@2)

◮ Literal p7 is computed (using BCP) also at level 1, so

we write p7@1

◮ Formally, if l1@1, . . . , ln@n is the sequence of

decisions taken on the branch B, then we write m@k if m ∈ BCP({l1, . . . , lk} \ BCP({l1, . . . , lk−1}

◮ Thus on a given branch, every computed literal gets

its level

90 / 1

Satisfiability

What about a contradiction?

◮ Say we are on a branch B, we made the sequence of

decisions l1@1, . . . ln@n, and BCP({l1, . . . , ln}) is contradictory

◮ The choice of decisions {l1, . . . , ln} is called no-good ◮ When {l1, . . . , ln} is no-good then

F | = ¯ l1 ∨ . . .¯ ln

◮ The reason is that a unit-resolution proof of ⊥ can be

constructed (recall the connection between BCP and unit resolution discussed above)

◮ Let us also observe that BCP(v) = BCP(BCP(v)) ◮ Thus there may be other no-goods on the branch,

kind of secondary

91 / 1

Satisfiability

Contradiction analysis

◮ Let us look at a kind of drastic case ◮ In our F we may have ¯

p1 ∨ p34 ∨ p57 and also p34 ∨ ¯ p57

◮ We make decisions p1@1, p12@2, ¯

p17@3 and then at level 4 we make the decision ¯ p34

◮ We get a contradictory BCP but if we look closely, we

see that really only decisions p1@1 and ¯ p34@4 are responsible for contradiction

◮ This is an important discovery, because instead a

longer no-good ¯ p1 ∨ ¯ p12 ∨ p17 ∨ p34 we get a short one: ¯ p1 ∨ p34

◮ We say that we learned ¯

p1 ∨ p34

92 / 1

Satisfiability

Contradiction analysis, cont’d

◮ During the current search the original no-good

¯ p1 ∨ ¯ p12 ∨ p17 ∨ p34 can not be reused to close branches in other parts of the search tree

◮ But the new one, ¯

p1 ∨ p34 can, because if we ever encounter p1 and ¯ p34 on some other branch then we now know we need to backtrack because the reasoning that generated ⊥ out of F, p1, and ¯ p34 can be repeated

◮ So, the issue is how can we learn new no-goods ◮ (This was discovery, in the GRASP project, that was

a major improvement to DPLL)

93 / 1

Satisfiability

Contradiction analysis, cont’d

◮ The literature reports various techniques ◮ The one used in GRASP and then fine-tuned in

zChaff and now used in most solvers is based on a graph-theoretic method

◮ Whenever BCP({l1, . . . , ln}) is contradictory (i.e.

contains a pair of contradictory literals) we can build a directed graph G describing the contradiction proof

94 / 1

Satisfiability

Contradiction analysis, cont’d

◮ The decisions are sources in that graph; the sink is ⊥ ◮ Contradiction may be obtained out of {l1, . . . , ln} in

more than one way; we select one

◮ (Can you cook an example where there is more than

• ne contradiction after some decision?)

◮ Nodes are labeled by clauses used to derive a literal

that is the output of such node

◮ (Incoming) edges are labeled by literals used to

reduce the clause to a single literal

95 / 1

Satisfiability

Contradiction analysis, cont’d

◮ In our example we had a node labeled with the

clause ¯ p1 ∨ p34 ∨ p57

◮ Into that node came in edges labeled with p1, and ¯

p34

◮ The first one was the decision, the second came

from some other derivation

◮ The third one was the output ◮ Actually, in our example we had, eventually, the node

producing the output p57 and another with ¯ p57 (which resulted in ⊥)

96 / 1

Satisfiability

Contradiction analysis, cont’d

◮ Graph-theorists know that there is a node x at level n

with the following property: every path from the decision point of level n to the sink must pass through x

◮ That node is called (first) unique implication point

1-UIP

◮ (There are other implication points, also used) ◮ Then we look at that UIP and do the following: all

literals to which are proved (remember our graph is a proof) after x are on one side of our cut (this side is called by Malik and coauthors of zChaff conflict side

◮ The remaining literals (including u) are on the other

side of the cut (called reason side)

97 / 1

Satisfiability

Contradiction analysis, cont’d

◮ The maximal elements of the reason side have the

property that they entail the contradiction, because

• nce we have them, we can use our graph G to get

formally the contradiction

◮ Let R be conjunction of these maximal elements ◮ We see that F |

= ¬R

◮ Therefore F |

=

l∈R¯

l

◮ This clause l∈R¯

l is what we learn

98 / 1

Satisfiability

Contradiction analysis, cont’d

◮ But really, it should be obvious that the following

holds: Let RBCP(L) where L is the set of decisions made when we got a contradictory BCP be conjunction of inclusion-minimal set of literals such that BCP(R) is contradictory. Then F | = ¬R, that is F | =

l∈R¯

l

◮ Thus we have a method to learn new clauses ◮ The method to learn from 1-UIP is effective (with the

appropriate data structures); the general method mentioned on this slide is not

◮ Let us observe, though, that such inclusion-minimal

set must contain at least one element of level n (why?)

◮ So, 1-UIP method tells us which element of the last

level to choose, and moreover it chooses a specific

• ne

99 / 1

Satisfiability

Contradiction analysis, cont’d

◮ But there is no reason to learn just one clause in this

fashion

◮ As mentioned above each proof of contradiction (and

there can be several) allows to learn a new clause

◮ But we do not have to stop a 1-UIP (i.e. at the last

level)

◮ We can do more then one UIP

, and learn more clauses

◮ But this is ‘embarrasse de richesses”. The reason is

that one, presumably, learns at each backtrack

◮ As there are, in principle O(3m) potential backtracks

• ne needs to manage the learned clauses

◮ zChaff proposed a heuristics for managing learned

clauses

◮ This heuristics, called VSIDS drops learned clauses

(why only learned?) when they are no longer likely to contribute

100/ 1

Satisfiability

Backjumping

◮ DPLL algorithm backtracks to the last decision point

where the variable first is set to 1, that is we make a decision on a literal l, but explored only the branch with l, but not with ¯ l

◮ What can be learned if, at some decision point we

explored both a literal l and also ¯ l and found contradiction in both places?

◮ So, l is the nth decision point on a branch B. ◮ That is l = ln, and there were no contradiction found

in {l1, . . . , ln−1}, i.e. BCP({l1, . . . , ln−1) is not contradictory

◮ However BCP({l1, . . . , ln−1, ln})

BCP({l1, . . . , ln−1,¯ ln}) are both contradictory

101/ 1

Satisfiability

Backjumping, cont’d

◮ We need to be a bit careful because we analyze both

w1 := BCP({l1, . . . , ln−1, ln}) and w2 := BCP({l1, . . . , ln−1,¯ ln})

◮ These two sets agree on assignment of levels up to

n − 1

◮ Here is what may happen:

◮ We may find that for a literal m so that m has a level

n in w1, and ¯ m has level n in w2 and for some R1 where all literals are of level smaller than n − 1 and some R2 where also all literals are of level smaller than n − 1 we have BCP({m} ∪ R1) is contradictory and also BCP({ ¯ m} ∪ R2) is contradictory

102/ 1

Satisfiability

Backjumping, cont’d

◮ Then, F |

= ¯ m ∨

r∈R1 ¬r ◮ Also, F |

= m ∨

r∈R2 ¬r ◮ But then, the clauses on the right hand sides are

resolvable

◮ Thus we learn a new clause:

• r∈R1∪R2

¯ r

◮ Let k be maximum of levels of literals occurring in

R1 ∪ R2. Then decisions l1, . . . , lk contradict F!

◮ This means that the entire part of the search tree

below {l1, . . . , lk} can be abandoned; if we did not explore some branches below it, we should not do this because it is guaranteed that there will be no solution to F in that part of the search tree

◮ In other words we backtrack not to the ¯

ln−1 but to ¯ lk−1

◮ This k − 1 may be much smaller than n − 1 and

hence “jump”

103/ 1

Satisfiability

Which variable and which polarity

◮ A major non-logical problem is how to select variable

as next literal on a branch

◮ It is not only this; once a variable is selected we need

to select polarity of that variable to be explored first

◮ There are various heuristics that can be used to

select the variable and its polarity. Clearly the idea is to assume a literal that offers maximum gain, that is decreases the size of the remaining search space below it

◮ Various heuristics were proposed and surely one can

try other ones

◮ Learning new clauses changes statistical properties

• f the clause set, and therefore after learning a

clause some the literal selected for expansion may change

104/ 1

Satisfiability

Restarts

◮ If we do not find a solution for a “long time” (whatever

it means) we may want to restart the process

◮ The point is that while we start anew, we do not start

with the same clause set

◮ The new clause set contains, besides the original

clauses all or some learned clauses

◮ Once we change the clause set (even though it has

exactly the same satisfying assignments (why?)) the input to heuristic functions governing the process of search change, and so we do not construct the same search tree

◮ Many modern SAT solvers use restarts

105/ 1

Satisfiability

Watched literals

◮ It should be clear that BCP works by means of

shortening of clauses and in this process finding new literals entailed by previous choices

◮ A clause C generates new literal in BCP if it is

shortened to the length 1

◮ (All literals but one are eliminated by means of unit

resolution)

◮ But if a clause contains at least two unassigned

literals we do not care if it is shortened

106/ 1

Satisfiability

Watched literals, cont’d

◮ The technique of “watched literals” is based on the

above observation; we assign two literals as watched

• if none of these is assigned we do not need to

process the clause for shortening

◮ But which literals to watch? The original proposal

was to watch the first and the last one; an innovation

• f zChaff was that randomly selected two literals are

watched

◮ Of course, if one (or both) of these literals are

assigned value(s) then we have to visit the clause and attempt to see if there are still two unassigned literals

◮ If there are two such literals - we change what we

watch

◮ Else, if there is just one, we add it to BCP, ◮ If none - backtrack is forced ◮ (From what gain comes?)

107/ 1

Satisfiability

Pure literals elimination

◮ Eliminating clauses containing a pure literal l does

preserve satisfiability

◮ Eliminating one pure literal may result in another

literal becoming pure (It is easy to create an example, try it!)

◮ Thus pure literals can be eliminated until no more

found

◮ Work decreasing by half or more each time

108/ 1

Satisfiability

Variable elimination (VER)

◮ Let us assume that we have a clause set F, and we

select a variable x

◮ Then F splits into union of three clause sets: Fx, set

• f clauses containing a positive occurrence of x, F¯

x,

set of clauses containing a negative occurrence of x and the rest, F x

0 set of clauses that do not contain x

at all

◮ If both Fx, F¯ x are non-empty, then every clause from

Fx resolves with every clause from F¯

• x. Let Gx be the

result of execution of these resolution instances (we also include F x

0 into Gx) and eliminating both Fx and

F¯

x

109/ 1

Satisfiability

Variable elimination, cont’d

◮ (Davis and Putnam) F and Gx are equisatisfiable, in

fact a stronger property holds

◮ Every assignment satisfying F satisfies Gx ◮ Every assignment satisfying Gx expands to an

assignment satisfying F

◮ But Gx does not have occurrences of x so the set of

variables of Gx has at least one variable less

◮ Clearly VER generalizes pure literal elimination

(why?)

110/ 1

Satisfiability

Variable elimination, cont’d

◮ (What if x occurs in only one of Fx, F¯ x?) ◮ The effect of executing variable elimination on a

variable x reduces the amount of work by half or more (why possibly more?)

◮ Let us observe that we can select another variable,

say y and can repeat the process

◮ Eventually we can eliminate all variables ◮ The effect is that we get an empty set of clauses or a

nonempty set of clauses containing empty clause

◮ In this first case we can “bootstrap” to get a satisfying

assignment for F

◮ In the second case F is unsatisfiable

111/ 1

Satisfiability

Variable elimination, cont’d

◮ The process of “bootstrapping” from a satisfying

assignment for Gx to a satisfying assignment for F is entirely constructive and polynomial

◮ So why not to take it to the limit and forget about

DPLL and do variable elimination resolution (VER) down to the clause set without variables, and then back?

◮ There is a reason; in principle, if |F| = m then

|Gx| = O(m2)/4

◮ So for some time we may experience exponential

growth in the size of the clause set!

112/ 1

Satisfiability

But the miracle may happen

◮ What if we select some variable x and the size of Gx

does not increase?

◮ Then the miracle happens; the size of the clause set

remains the same (or smaller) but the amount of work goes down by half

◮ This technique was pioneered in a solver called

NiVER, and is currently included as one of preprocessing method of modern SAT solvers

◮ This, of course, subsumes finding of a pure literal

(i.e. of a very simple autarky)

◮ So we test variables for non-increasing variable

elimination and if it does not increase the size, we apply it (and remember that when we find the satisfying assignment for Gx we need to expand that assignment to include x or ¯ x)

◮ (VER as a preprocessing tool is used in particular in

Lingeling out of Linz)

113/ 1

Satisfiability

Yet another simplification

◮ (This time eliminating clauses) ◮ Recall that a binary clause l ∨ m really represents

two implications: ¯ l ⇒ m and ¯ m ⇒ l

◮ Let us fix a clause C we do the following until nothing

more can be done (i.e. fixpoint is reached)

◮ Whenever l ∈ C, D := l ∨ m belongs to F, add m to

C (i.e. C := C ∪ {m})

◮ The resulting clause is called HLA(C) ◮ (Hidden literal addition)

◮ (Heule, Järvisalo and Biere): F is equivalent to

(F \ {C}) ∪ {HLA(C)}

◮ The reason is that whenever l ∈ HLA(C) \ C then

there must be a chain of implications from some literal m of C to l using implications listed above

114/ 1

Satisfiability

Hidden tautology elimination

◮ So, the following makes sense: if HLA(C) is a

tautology then we can eliminate C altogether

◮ The reason is that in reality C transforms into a

tautology, thus not a constraint

◮ Example: Say F = {p ∨ q ∨ r, q ∨ ¬s, r ∨ s, r ∨ ¬t ◮ Then we get:

HLA(p ∨ q ∨ r) = p ∨ q ∨ r ∨ s ∨ ¬s ∨ ¬t

◮ Thus p ∨ q ∨ s can be eliminated

115/ 1

Satisfiability

Looking ahead

◮ If BCP({l}) and BCP(¯

l) both contain a literal m then m is a consequence of F (why?)

◮ Finding such literal m reduces work by half ◮ The process of trying both l and ¯

l is called lookahead

◮ Lookahead allows us also to measure which of l, ¯

l is better to choose for expansion

◮ One does not have to limit lookahead to one level

116/ 1

Satisfiability

Backdoors

◮ A set of variables A is called backdoor for a clause

set F if any assignment of A reduces F to an easy case

◮ Example (trivial, out of necessity):

F := {p ∨ q ∨ s, ¯ p ∨ ∨t ∨ ¯ s, ¯ q ∨ ¯ t}

◮ The set {p} is a backdoor; setting p reduces F to

2-SAT

◮ Each notion of “easy case” generates its own notion

• f backdoor

◮ Unfortunately, finding a backdoor is hard (if it were

not, SAT would be easy)

◮ Nevertheless this idea was tried

117/ 1

Satisfiability

All SAT solvers are equal?

◮ Not really; some are better on some classes of

problems, some on others

◮ In the meanwhile computation became “dirt cheap” ◮ A portfolio of algorithms is an arrangement where

several implementations of different algorithms for solving the same task are run in parallel

◮ Satzilla is a portfolio of SAT solvers out of University

• f British Columbia

118/ 1

Satisfiability

Portfolio, cont’d

◮ The value of portfolio is not only in bringing together

a number of algorithms (any reasonable scripting language like Python or Perl can do this for us) but also in estimating which algorithm (program) is better for the job

◮ Moreover, one can machine learn from the the

already solved cases!

◮ At least to some extent the idea works; Satzilla won

a number of SAT competitions in various categories

◮ But one wonders if the process is parasitic

119/ 1

Satisfiability

What about the cloud?

◮ We have 1024 processors, so we consider all partial

assignments to 10 randomly selected variables, send that assignment and the clause set F to one of its processors

◮ When a processor ends its work it either sends back

to the master satisfying assignment or the signal “I am free” and the master asks a processor with the biggest part of the search space for half of its work which is sent to the free processor

◮ Nice, but fails, due to communication costs (which

form the weaker part of cloud computing and the reason why we have “map-reduce” in the first place)

120/ 1

Satisfiability

Cloud, cont’d

◮ But if we had a reasonable estimate of the work at Pv

(P a processor, v a partial assignment) the scheme described above would make sense

◮ And indeed, a system called Cube-and-conquer tries

to do that

◮ We will see in the future how good it is, surely there

will be competing schemes

◮ Finding such scheme will be a major research push

for SAT

121/ 1

Satisfiability

How can we be sure?

◮ When a solver finds a solution, then it is easy and

inexpensive to test that it is really a solution

◮ But if it does not find a solution and claims that the

entire space was searched, what can we do?

◮ This is an important issue, for instance in

Equivalence Checking discussed above

◮ (Computer Engineering research shows cases where

strange cases happen, like a circuit correctly computing product of two 32 bit numbers, except in just one case)

122/ 1

Satisfiability

Converting claims of unsatisfiability

◮ The idea is natural: convert the claim of

unsatisfiability into a proof of ⊥

◮ But these are of cosmic proportions - remember 3n

partial assignments

◮ More issues are at hand, and I will speak about this

separately, when discussing Experimental Mathematics

◮ One issue is to keep the data (recall that DPLL was

design so not to keep too much data)

◮ Anyway, “proofs out of UNSAT” is a major research

effort right now

123/ 1

Satisfiability

Resources for SAT

◮ I “dropped names” of various SAT solvers ◮ But the stakes in SAT research are serious, esp.

since SAT has applications in Computer Engineering

◮ Since Shannon’s M.Sc. thesis (MIT, 1937) it is known

that, really, combinational (and also sequential) circuits are about Boolean functions

◮ Since propositional logic provides one of faithful

representations of circuits (there are others, for instance BDDs), logic was and is widely used in the design, implementation, optimization, and verification

• f digital circuits

◮ SAT solvers are (esp. today) back-solvers for these

applications

124/ 1

Satisfiability

Resources for SAT, cont’d

◮ There are annual conferences for SAT (organized by

Computer Scientists, but also by Computer Engineers)

◮ Conferences include competitions for various classes

• f problems

◮ Industrial ◮ Crafted (for instance very hard combinatorial

problems)

◮ Random

◮ There is JSAT, Journal on Satisfiability, Boolean

Modeling and Computation

◮ There is (of course) Handbook of Satisfiability, A.

Biere, M. Heule, H. van Maaren, T. Walsh (eds.), IOS Press, 2009, ISBN-13 978-1-58603-929-5.

◮ It is a bit dated, but most of discoveries mentioned

here are there

125/ 1

Satisfiability

But we promised more...

◮ SAT is not the only declarative formalism for

declaring and solving constraints:

◮ There are various alternatives ◮ There is venerable language called PROLOG (now

40 years old)

◮ Based on the concept of a rule in quantifier-free

fragment of predicate logic (we discussed above propositional logic, only)

◮ Because of extra-logical concepts such as

negation-as-failure there is no generally-accepted semantics for this system

◮ After original cult following and merciless hyping

PROLOG (IMHO) is, essentially dead

126/ 1

Satisfiability

ECLiPSe

◮ A practical and quite developed system extending

Logic Programming, Prolog-style by libraries allowing to handle combinatorial optimization that is tasks such as planning, scheduling, tabling etc

◮ It looks impressive - when it works ◮ (Of course it my be my own incompetence) ◮ In particular ECLiPSe has a library for handling

finite-domain problems

◮ (Last I looked at it, it did not work, so I gave up)

127/ 1

Satisfiability

DATALOG

◮ DATALOG is a form of Logic Programming without

function symbols (but with variables)

◮ Originated in Database community ◮ In principle expresses aggregate-free, negation-free

fragment of SQL (what is it?)

◮ DATALOG implements Closed-world assumption -

• nly those objects that are mentioned in the input

program are considered, and this implies that one deals with finite domains

◮ But a form of recursion is included ◮ DATALOG is implemented in DB2 RDBMS out of IBM

and standardized in SQL standards

128/ 1

Satisfiability

Answer Set Programming

◮ Generalizes DATALOG by admitting negation in the

bodies of rules

◮ Here is how the rules look like:

p ← q1, . . . qm not r1, . . . , not rn

◮ (p, q, rs may contain variables but not function

symbols)

◮ An ASP program is a finite set of such rules ◮ (Taking queue from Database practice, often

programs split into data and program proper, called extensional database and intentional database, resp.)

129/ 1

Satisfiability

Answer Set Programming, cont’d

◮ There is a semantics for such programs (notice that

exotic constructs such as cut or negation-as-failure disappeared)

◮ But negation is not the same negation as used in our

SAT considerations but a default negation

◮ Semantics, called stable semantics or

Gelfond-Lifschitz semantics (not this Gelfond, and not this Lifschitz) is based on the notion of reduct that we have seen in our SAT consideration, but tailored to the present situation

130/ 1

Satisfiability

Answer Set Programming, cont’d

◮ The expressive power of ASP is precisely the same

as that of SAT

◮ Programming in ASP is easier than in SAT ◮ (Just try to write 3-coloring in ASP - you will see) ◮ Generally, ASP pays more attention to Knowledge

Representation

◮ Various reasonable extensions of ASP

, allowing for weight constraints, cardinality constraints and aggregates are available

131/ 1

Satisfiability

Answer Set Programming, cont’d

◮ Various implementations (smodels, dlv, clasp, and

• thers)

◮ Techniques pioneered by SAT community used in

implementations

◮ Performance compatible with SAT solvers (which

• ften serve as back-solvers)

◮ Quite vibrant research community today ◮ But the fact that dealing with non-standard semantics

(understanding requires some effort) is an impediment

132/ 1

Satisfiability

Did we cover everything?

◮ Certainly not; here are issues we did not cover:

◮ Random SAT and phase transition (physicists,

physicists, and then statisticians!)

◮ Incomplete solvers for SAT (ditto) ◮ SAT as a mechanism for handling fragments of

decidable first-order theories (what is it?). This is called Satisfiability-modulo-theories or SMT. Industrial strength SMT solvers (such as Z3 out of Microsoft) are in use

133/ 1

Satisfiability

Conclusions

◮ Logic turned out to be a computational declarative

formalism

◮ What is more important, it is not a useless technique,

but with many applications, all over the place

◮ The reason is that finite-domain CSPs are all over

the place

◮ (And of course decidable fragments of logical

theories are useful, too)

134/ 1

Satisfiability

Message to take home

◮ (Late Gian-Carlo Rota (who was he?) taught that

every presentation must have such message)

◮ The message is that declarative programming (that

seemed moribund with PROLOG) is here to stay

◮ And that there is much to be done; both in

researching the area, and in the implementations. In complete solvers and in incomplete ones - take your pick

135/ 1