Constraint t Networks ( slides courtesy o ( y of Natalia - - PowerPoint PPT Presentation

Constraint

( slides courtesy o ( y based on slides court

t Networks

• f Natalia Flerova,

tesy of Rina Dechter)

Read

Russell and Norvig Ch t 6 (3rd d ) Ch Chapter 6 (3rd ed.); Cha For next week: Ch t 7 (f T d ) Chapter 7 (for Tuesday) Chapter 8 (for Thursday

ding

t 5 (2nd d ) apter 5 (2nd ed.) ) y)

Good source Good source inform

Rina Dechter, Constraint Processing, g, Morgan Kaufmann

e of advanced e of advanced mation

3

Out

 Constraint Satisfaction Pro

Definition and simple mod



Definition, and simple mod

 Representing constraints

C i l i h (

 Consistency algorithms (ar  General search strategy



Look-ahead meth

tline

• blems (CSPs):

deling examples deling examples

h d i i ) rc-, path- and i-consistency) hods

4

Constra

Example

Variables - countries (A Values - colors (e.g., r Constraints:

A≠ B,

A B A

red green red black green red green black bl k

B D

black green black red

C

aint Satisfaction

: map coloring

A,B,C,etc.) red, green, black)

A≠ D, D≠ E , etc.

E D F G

Constraint Netw



A constraint network is: R=(X,D,C

 X variables

X= {X

 D domains  C constraints

X {X

D= {D

 C constraints  R expresses allowed tuples over scopes

C= {C



A solution is an assignment to all vari all relations). T k i t ? ll l ti



Tasks: consistency?, one or all solutio

work; Definition

C)

X1 X } X1,... ,X n}

D1,..., Dn}, Di= {v1,...vk}

s

1,...Ct},,,Ci=

Si ,Ri

ables that satisfies all constraints (join of ti ti i ti

• ns, counting, optimization

6

Example: The N-queens

The network has four variables, all w (a) The labeled chess board. (b) The c

Sp

problem

with domains Di = {1, 2, 3, 4}. constraints between variables.

ring 2009 7

Example The 4 p

Place 4 Queens on a chess board of 4x4 such that no two queens reside in the same row, column or diagonal.

Q Q Q Q Q Q Q Q Q Q

D {1 2 3 4}

• Domains:

Di= {1,2,3,4}.

• Constraints: There are = 6 constr

4 2

( )

2

R12= { 1,3

• 1,4
• 2,4
• 3,1
• 4,1
• 4,2

}

R13= { 1,2

• 1,4
• 2,1
• 2,3
• 3,2
• 3,4
• 4,1
• 4

R = { 1 2

• 1 3
• 2 1
• 2 3
• 2 4
• 3 1
• 3 2
• 3

R14= { 1,2

• 1,3
• 2,1
• 2,3
• 2,4
• 3,1
• 3,2
• 3

R23= { 1,3

• 1,4
• 2,4
• 3,1
• 4,1
• 4,2

}

R24= { 1,2

• 1,4
• 2,1
• 2,3
• 3,2
• 3,4
• 4,1
• 4

R { 1 3

• 1 4
• 2 4
• 3 1
• 4 1
• 4 2

} R34= { 1,3

• 1,4
• 2,4
• 3,1
• 4,1
• 4,2

}

4-queen problem q p

Standard CSP formulation of the problem:

• Variables: each row is a variable.

1 2 3 4

Q Q

X1 X 2

1 2 3 4

Q Q Q

X 4

X 3

2

raints involved:

4,3 }

• 3 4
• 4 2
• 4 3

}

• Constraint Graph :

X1

X 3

• 3,4
• 4,2
• 4,3

}

• 4,3

}

X 2 X 4

A solution and a p A solution and a p tup

Not all consistent instantiations are part o (a)A consistent instantiation that is not pa (b) Th l t f th (b) The placement of the queens correspon solution (2, 4, 1,3). c) The placement of the queens correspon (3, 1, 4, 2).

Sp

partial consistent partial consistent ple

• f a solution:

art of a solution. di t th nding to the nding to the solution

ring 2009 9

Example: configu

Sp

uration and design

ring 2009 10

Configuratio



Want to build: recreation area, apartm dump

 Recreation area near lake  Steep slopes avoided except for recreation a

P il id d f d l t

 Poor soil avoided for developments  Highway far from apartments, houses and r  Dump not visible from apartments, houses  Lots 3 and 4 have poor soil  Lots 3 and 4 have poor soil  Lots 3, 4, 7, 8 are on steep slopes  Lots 2, 3, 4 are near lake  Lots 1, 2 are near highway

g y

Sp

• n and design

ments, houses, cemetery,

area recreation and lake

ring 2009 11

Out

 CSP: Definition, and simp

Representing constraints

 Representing constraints  Consistency algorithms (ar  General search strategy



Look-ahead meth

tline

ple modeling examples rc-, path- and i-consistency) hods

12

Constraint’s re

 Relation: allowed tuples

p

 Algebraic expression:

X

 Propositional formula:

X

Sp

epresentations

X Y Z 1 3 2

X Y 2 10 X≠ Y

2 1 3

XY 2≤ 10 , X≠ Y

b

• ab

¬c

ring 2009 13

Operations w

 Intersection

U i

 Union  Difference  Selection  Projection

j

 Join

Composition

 Composition

with relations

14

Figure 1.8: Example of set operation applied to

Sp

ns intersection, union, and difference

• relations.

ring 2009 15

selection, projection, and jo

Sp

• in operations on relations.

ring 2009 16

Constrain

Primal, Dual an



A (primal) constraint graph: a node per variable t t i d i bl



arcs connect constrained variables.



A dual constraint graph: a node per constraint’s sc arc connect nodes sharing variables =hypergraph 1 2 4 1,2,3,4,5 5 3 6 9 7 11 5 12 8 10 5,7,11 Sp 13 (a)

nt Graphs:

nd Hypergraphs

cope, an 12,13 3,6,9,12 3 12 13 9 8,9,10,11 10,13 10 11 ring 2009 17 (b)

Out

 CSP: Definition, and simp

Representing constraints

 Representing constraints  Consistency algorithms (ar  General search strategy



Look-ahead meth

tline

ple modeling examples rc-, path- and i-consistency) hods

18

Consistenc

 Constraint propagation

constraints constraints

 Can get such an explic

ill fi d th l ti will find the solution w

 Approximation of infer

 Arc, path and i-consiste

 Methods that transform

into a tighter and tighte

cy methods

n – inferring new it network that the search ith t d d d without dead-ends. rence:

ency

m the original network

19

er representations

Arc-cons Arc-cons

• infer constraints based on p

1  X Y Z T  3 X 3 2, 1, 1  X, Y, Z, T  3 X  Y Y = Z T  Z



3 2, 1, T  Z X T , , T Insures that every legal value in the dom a legal match In the domain of any othe

sistency sistency

pairs of variables

3 2, 1, Y

 =

3 2, 1,



, , Z



20

main of a single variable has er selected variable

Arc-con

1  X Y Z T  3 X 1 1  X, Y, Z, T  3 X  Y Y = Z T  Z



T  Z X  T 2 T

i t nsistency

Y



3

= 

3 Z



21

Arc-con i t nsistency

22

Revise for arc

D D ∩

• R

Di Di∩π i R

c-consistency

23

R D Rij D j

AC AC



Complexity (Mackworth and Freuder, 1986):



e = number of arcs, n variables, k values



(ek^2, each loop, nk number of loops), best



Arc-consistency is:

ek 2

C-1 C-1

O enk 3

s

24

t-case = ek,

AC AC

O ek 3



Complexity:



Best case O(ek) since each arc may



Best case O(ek), since each arc may



e = number of arcs, n variables, k valu

C-3 C-3

25

be processed in O(2k) be processed in O(2k) ues

Distributed ar Distributed ar (Constraint p



Implement AC-1 distributedly.



Node x j sends the message to node _j g x_i



Node x_i updates its domain:



Messages can be sent asynchronously or scheduled in a topological order

rc-consistency rc-consistency propagation)

Di Di∩ π i Rij D j

hi

j π i

Rij D j

Di Di∩ π i Rij D j = D D ∩ h j Di Di∩ hi

j

26

Distributed Ar

A i b

 Arc-consistency can be

algorithm:

rc-Consistency

f l d di ib d formulated as a distributed

A B C D F G 27

a Constraint network a Constraint network

Relational Ar

The message that R2 sends to R1 is R1 updates its relation and domains and send messages to neighbors

rc-consistency

A 1 2 3 A B 1 2 1 3

R1 R2

R3

A 3 A C 1 2 3 2 2 1 2 3 3 1 3 2 A B C 3 2 B C F

R5

D F

ds

B C F 1 2 3 3 2 1 A B D 1 2 3 1 3 2

R4

D F G 28 1 3 2 2 1 3 2 3 1 3 1 2 D F G 1 2 3 2 1 3

R6

3 2 1 2 1 3

A 1

R1

DRAC on

2 3 A B 1 2

R2

1 2 1 3 2 1 2 3 2

A

3 1 3 2

R4

AB

B

4

A

A B D 1 2 3 1 3 2 2 1 3

4 ABD

4

D

2 1 3 2 3 1 3 1 2 3 2 1 D 1

R6

1 2 A

1the dual join-graph

R

1 A C 1 2 3 2

R3

A

3

A

1

R5

AC

5

B C

B C F 1 2 3 3 2 1

BCF DFG

5 6

F

29 F G 2 3

DFG

2 3 1 3

Sud Constraint

• Constraint

P ti

• Propagation
• I nference

Each row, column and major blo “Well posed” if it has unique solu

doku – t Satisfaction f

• Variables: empty slots
• Domains =

{ 1,2,3,4,5,6,7,8,9}

• Constraints:



27 all-different 2 3 4 6

2

30

• ck must be alldifferent

ution: 27 constraints

Path-consistency

31

Path-consistency

32

Revi Revi

 Complexity: O(k^3)  Best-case: O(t)  Worst-case O(tk)



e = number of arcs, n variables, k valu e number of arcs, n variables, k valu constraint relation

ise-3 ise-3

33

ues, t = tightness: number of tuples in the ues, t tightness: number of tuples in the

PC

 Complexity:



e = number of arcs, n variables, k values constraint relation

O n5k5

constraint relation

C-1

34

s, t = tightness: number of tuples in the

PC- PC-

O n3k5

 Complexity:  Optimal PC-4:

O n3k3



(each pair deleted may add: 2n-1 triplets, numb O(n^3 k^2), processing is O(k^3))

b f i bl k l t

O n k



e = number of arcs, n variables, k values, t constraint relation

• 2
• 2

35

ber of pairs: O(n^2 k^2)  size of Q is

t ti ht b f t l i th t = tightness: number of tuples in the

Example: before Example: before consis



PC-1 requires 2 processings of each ar



Can we do path-consistency distribute

e and after path- e and after path- stency

36

rc while PC-2 may not edly?

I-consistency

37

Out

 CSP: Definition, and simp

Representing constraints

 Representing constraints  Consistency algorithms (ar  General search strategy



Look-ahead meth

tline

ple modeling examples rc-, path- and i-consistency) hods

38

Backtrackin

 No matter how much we rea

are still left with choices → possible choices.

 Backtracking search – assi

ki h i turn, making sure each assig values assigned thus far. E t i bl ith d

 Encounter variable with no d

previous assignment – dead- the value of previous variabl

ng algorithm

ason about the problem – we need to search the space of ign values to each variable in d l i i i h gned value is consistent with d i l i t t ith domain value consistent with

• end → backtrack – change

les.

39

Backtracking

The sear

 A tree of all partial solu

A i l l i ( 1

 A partial solution: (a1,

relevant constraints

 The size of the underly

• n:

 Variable ordering  Level of consistency po

rch space

utions j) i f i ll …,aj) satisfying all ying search space depends

• ssessed by the problem

S h d h Search space and th ff f d i he effect of ordering

Z evenly divides x,y,l 20 legal states 1 dead-end leaf 48 legal states 18 dead-end leaves

Dependency on c Dependency on c



After arc-consistency z=5 and l=5 are removed



After path-consistency



R’_zx



R’_zy



R’_zl



R’_xy



R’_xl



R’_yl

consistency level consistency level

Backtrack-free network

Backtr Backtr racking racking



Complexity of extending a partial solution:



Complexity of consistent O(e log t), t bounds tuples, e constraints



Complexity of selectvalue O(e k log t) log t)

Improving b

 Before search: (reducing

Arc consistenc path consist

 Arc-consistency, path-consist  Variable ordering (fixed)

 During search:

g

 Look-ahead schemes:

 value ordering,  variable ordering (if not fixe  variable ordering (if not fixe

 Look-back schemes:

 Backjump

Constraint recording

 Constraint recording  Dependency-directed backtr

backtracking

g the search space)

tenc tency

ed) ed) racking

Out

 CSP: Definition, and simp

Representing constraints

 Representing constraints  Consistency algorithms (ar  General search strategy



Look-ahead meth

tline

ple modeling examples rc-, path- and i-consistency) hods

51

Improving b

 Before search: (reducin

A i t th

 Arc-consistency, path-c  Variable ordering (fixe

During search:

 During search:

Look-ahead schemes:

 Look-ahead schemes:



value order maximizes future assig



variable or i t ti t instantiate variable

backtracking

ng the search space)

i t consistency ed) ring - first assign value that s the number of options for p gnments rdering (if not fixed)- first t hi hl t i d most highly constrained

Look-ahead: v

 Intuition:



Choose value least likely to yield a y y



Approach: apply propagation at eac



Forward-checking



(check each unassigned variable sep (check each unassigned variable sep



Maintaining arc-consistency (MAC)



(apply full arc-consistency)



Full look-ahead



Full look ahead



One pass of arc-consistency (AC-1)



Partial look-ahead

directional-arc-consistency



directional-arc-consistency

value orderings

dead-end ch node in the search tree parately) parately) )

Generalized look-ahead

Forward-

 The most limited form

propagation during sea propagation during sea

 Propagates the effect o

selection to each future selection to each future

 If domain of one of the

becomes empty – value next value.

• checking
• f constraint

arch arch f a tentative value e variable separately e variable separately e future variables e is not selected → try

Forward-checking example

Forward-

Complexity of selectValue-forward-che p y

• checking

O ek 2

cking at each node:

O ek

g

Arc-consisten

(Gashni

 Applies full arc-consistenc

variables following each v variables following each v current variable.

 Complexity:

p y

 If optimal arc-consisten

 SelectValue-Arc-Consiste

essentially AC-1 with som

 Maintaining-Arc-Consiste

arc-consistency after each

cy look-ahead

ig, 1977)

cy on all un-instantiated value assignment to the value assignment to the

O ek 3

ncy is used:

O ek

ency: The repeat...until loop – me variables instantiated. ency (MAC): Performs full h domain value is rejected.

Arc-consisten (maintaining arc (maintaining arc- cy look-ahead: consistency MAC)

• consistency MAC)

Full and parti

 Full looking ahead:

M k f ll th

 Make one full pass thro

(remove repeat-until fro consistency) consistency)

 Partial look-ahead:

Applies (similar to) dir

 Applies (similar-to) dir

future variables. - futur compared with those va

O e

 Complexity: also  More efficient than MA

O e

ial look-ahead

h f t i bl

• ugh future variables
• m selectValue-arc-

rectional arc consistency to rectional arc-consistency to re variables are only ariables following them.

ek 3

AC

ek

Example of partial look-ahead

Out

 CSP: Definition, and simp

Representing constraints

 Representing constraints  Consistency algorithms (a  General search strategy



Look-ahead meth



Branching-ahead

tline

ple modeling examples cr-, path- and i-consistency) hods d

62

Branching-ahead: Dy

Rank order the promise in non



Rank functions Rank functions

 MC (min conflict) – choose

• f values from domains of f

 MD (min domain) – choose  MD (min domain) – choose

domain size in future variab

 SC (expected solution count

• f solutions by multiplying t
• so ut o s by

u t p y g t after incompatible values ar

ynamic Value Ordering

n-rejected values

s value that removes the smallest number future variables s value that creates the largest minimum s value that creates the largest minimum bles ts) – computes upper bound on number the domains sizes of each future variable t e do a s s es o eac utu e va ab e re removed.

Branchin Dynamic Variable

 Following constraint propa

constrained variable constrained variable

 Intuition: early discovery  Highly effective: the single

g y e ec ve: t e s g e cut down search space

 Most popular with FC  Dynamic search rearrangem

(Purdon,1983)

ng-ahead: e Ordering (DVO)

agation, choose the most

• f dead-ends

e most important heuristic to e

• st

po ta t eu st c to ment (Bitner and Reingold, 1975)