Slides Set 8: Search for Constraint Satisfaction Rina Dechter ( - - PowerPoint PPT Presentation

▶
slides set 8
SMART_READER_LITE
LIVE PREVIEW

Slides Set 8: Search for Constraint Satisfaction Rina Dechter ( - - PowerPoint PPT Presentation

Jan 05, 2024 562 likes •1.38k views

Algorithms for Reasoning with graphical models Slides Set 8: Search for Constraint Satisfaction Rina Dechter ( Dechter2 chapters 5-6, Dechter1 chapter 6 ) slides7 828X 2019 Sudoku Approximation: Constraint Propagation Variables: empty

slide-1
SLIDE 1

Algorithms for Reasoning with graphical models

Slides Set 8:

Rina Dechter

slides7 828X 2019

Search for Constraint Satisfaction

(Dechter2 chapters 5-6, Dechter1 chapter 6)

slide-2
SLIDE 2

Sudoku – Approximation: Constraint Propagation

Each row, column and major block must be alldifferent “Well posed” if it has unique solution: 27 constraints

2 3 4 6

2

  • Variables: empty slots
  • Domains =

{1,2,3,4,5,6,7,8,9}

  • Constraints:
  • 27 all-different
  • Constraint
  • Propagation
  • Inference

slides7 828X 2019

slide-3
SLIDE 3

Outline: Search in CSPs

 Improving search by bounded-inference

(constraint propagation) in looking ahead

 Improving search by looking-back  The alternative AND/OR search space

slides7 828X 2019

slide-4
SLIDE 4

Outline: Search in CSPs

 Improving search by bounded-inference

(constraint propagation) in looking ahead

 Improving search by looking-back  The alternative AND/OR search space

slides7 828X 2019

slide-5
SLIDE 5

What if the CN is Not Backtrack- free?

 Backtrack-free in general is too costly,

so what to do?

 Search?  What is the search space?  How to search it? Breadth-first? Depth-

first?

slides7 828X 2019

slide-6
SLIDE 6

The Search Space for a CN

slides7 828X 2019

slide-7
SLIDE 7

The Effect of Variable Ordering

2,3,5

2,5,6 2,3,4

2,3,4

Z X Y L

slides7 828X 2019

slide-8
SLIDE 8

The Effect of Consistency Level

After arc-consistency z=5 and l=5 are removed

After path-consistency

R’_zx

R’_zy

R’_zl

R’_xy

R’_xl

R’_yl

2,3,5 2,5,6 2,3,4

2,3,4

Z X Y L

slides7 828X 2019

slide-9
SLIDE 9

The Effect of Variable Ordering

z divides x, y and t

slides7 828X 2019

slide-10
SLIDE 10

Sudoku – Search in Sudoku. Variable ordering? Constraint propagation?

Each row, column and major block must be alldifferent “Well posed” if it has unique solution: 27 constraints

2 3 4 6

2

  • Variables: empty slots
  • Domains =

{1,2,3,4,5,6,7,8,9}

  • Constraints:
  • 27 all-different
  • Constraint
  • Propagation
  • Inference

slides7 828X 2019

slide-11
SLIDE 11

Sudoku

Each row, column and major block must be alldifferent “Well posed” if it has unique solution

slides7 828X 2019

Alternative formulations: Variables? Domains? Constraints?

slide-12
SLIDE 12

Backtracking Search for a Solution

slides7 828X 2019

Second ordering = (1,7,4,5,6,3,2)

slide-13
SLIDE 13

Backtracking Search for a Solution

slides7 828X 2019

slide-14
SLIDE 14

Backtracking Search for All Solutions

slides7 828X 2019

slide-15
SLIDE 15

Backtracking search for *all* solutions

For all tasks Time: O(𝒍𝒐) Space: linear n= number of variables K = max domain size

slides7 828X 2019

slide-16
SLIDE 16

slides7 828X 2019

Traversing Breadth-First (BFS)?

Not-equal

BFS memory is O(𝒍𝒐) while no Time gain  use DFS

slide-17
SLIDE 17

Improving Backtracking

 Before search: (reducing the search space)

 Arc-consistency, path-consistency  Variable ordering (fixed)

 During search:

 Look-ahead schemes:

 value ordering,  variable ordering (if not fixed)

 Look-back schemes:

 Backjump  Constraint recording or learning  Dependency-directed backtacking

slides7 828X 2019

slide-18
SLIDE 18

Look-Ahead: Value Orderings

 Intuition:

Choose value least likely to yield a dead-end

Approach: apply constraint propagation at each node in the search tree

Forward-checking

(check each unassigned variable separately

Maintaining arc-consistency (MAC)

(apply full arc-consistency)

Full look-ahead

One pass of arc-consistency (AC-1)

Partial look-ahead

directional-arc-consistency

slides7 828X 2019

slide-19
SLIDE 19

Forward-Checking for Value Ordering

slides7 828X 2019

slide-20
SLIDE 20

Forward-Checking for Value Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O

FW overhead:

slides7 828X 2019

slide-21
SLIDE 21

Forward-Checking, Variable Ordering

) (

2

ek O

slides7 828X 2019

FW overhead:

) ( ) (

3 2

ek O ek O

slide-22
SLIDE 22

Forward-Checking, Variable Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O

After X1 = red choose X3 and not X2

slides7 828X 2019

FW overhead:

slide-23
SLIDE 23

Forward-Checking, Variable Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O

After X1 = red choose X3 and not X2

slides7 828X 2019

FW overhead:

slide-24
SLIDE 24

Forward-Checking, Variable Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O

After X1 = red choose X3 and not X2

slides7 828X 2019

FW overhead:

slide-25
SLIDE 25

Arc-consistency for Value Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O FW overhead: MAC overhead:

slides7 828X 2019

slide-26
SLIDE 26

Arc-Consistency for Value Ordering

) (

2

ek O

) ( ) (

3 2

ek O ek O FW overhead: MAC overhead: Arc-consistency prunes x1=red Prunes the whole tree

Not searched By MAC slides7 828X 2019

slide-27
SLIDE 27

Branching-Ahead for SAT: DLL

example: (~AVB)(~CVA)(AVBVD)(C)

Only enclosed area will be explored with unit-propagation Backtracking look-ahead with Unit propagation= Generalized arc-consistency (Davis, Logeman and Laveland, 1962)

slides7 828X 2019

slide-28
SLIDE 28

Constraint Programming

 Constraint solving embedded in programming

languages

 Allows flexible modeling with algorithms  Logic programs + forward checking  Eclipse, ILog, OPL,minizinc  Using only look-ahead schemes (is that

true?)

 Numberjeck (in Python)

slides7 828X 2019

slide-29
SLIDE 29

Outline: Search in CSPs

 Improving search by bounded-inference in

branching ahead

 Improving search by looking-back  The alternative AND/OR search space

slides7 828X 2019

slide-30
SLIDE 30

Look-Back: Backjumping / Learning

 Backjumping:

 In deadends, go back to the most recent

culprit.

 Learning:

 constraint-recording, no-good learning,

Deep-learning, shallow learning

 good-recording  Clause learning

slides7 828X 2019

slide-31
SLIDE 31

Look-Back: Backjumping

(X1=r,x2=b,x3=b,x4=b,x5=g,x6=r,x7={r,b})

(r,b,b,b,g,r) conflict set of x7

(r,-,b,b,g,-) c.s. of x7

(r,-,b,-,-,-,-) minimal conflict-set

Leaf deadend: (r,b,b,b,g,r)

Every conflict-set is a no-good

slides7 828X 2019

slide-32
SLIDE 32

Jumps At Leaf Dead-Ends (Gascnnig-style 1977)

slides7 828X 2019

slide-33
SLIDE 33

Jumps at Leaf Dead-Ends (Gascnnig 1977)

slides7 828X 2019

slide-34
SLIDE 34

Graph-Based Backjumping Scenarios Internal Deadend at X4

 Scenario 1, deadend at x4:  Scenario 2: deadend at x5:  Scenario 3: deadend at x7:  Scenario 4: deadend at x6:

slides7 828X 2019

slide-35
SLIDE 35

Graph-Based Backjumping

 Uses only graph information to find culprit  Jumps both at leaf and at internal dead-ends  Whenever a deadend occurs at x, it jumps to the most

recent variable y connected to x in the graph. If y is an internal deadend it jumps back further to the most recent variable connected to x or y.

 The analysis of conflict is approximated by the graph.  Graph-based algorithm provide graph-theoretic bounds.

slides7 828X 2019

slide-36
SLIDE 36

Properties of Graph-Based Backjumping

slides7 828X 2019

slide-37
SLIDE 37

Graph-based Backjumping on DFS ordering

slides7 828X 2019

slide-38
SLIDE 38

Backjumping Styles

 Jump at leaf only (Gaschnig 1977)

 Context-based

 Graph-based (Dechter, 1990)

 Jumps at leaf and internal dead-ends, graph

information

 Conflict-directed (Prosser 1993)

 Context-based, jumps at leaf and internal dead-ends

slides7 828X 2019

slide-39
SLIDE 39

DFS of graph and induced graphs

Spanning-tree of a graph; DFS spanning trees, Pseudo-tree Pseudo-tree is a spanning tree that does not allow arcs across branches.

slides7 828X 2019

slide-40
SLIDE 40

Complexity of Backjumping Uses Pseudo-Tree Analysis

Simple: always jump back to parent in pseudo tree Complexity for csp: exp(tree-depth) Complexity for csp: exp(w*log n)

slides7 828X 2019

slide-41
SLIDE 41

Complexity of Backjumping

  • Simple: always jump back to parent in pseudo tree
  • Complexity for csp: exp(w*log n), exp(m), m= depth
  • From exp(n) to exp(w*logn) while linear space
  • (proof details: exercise)

Graph-based and conflict-based backjumpint

slides7 828X 2019

slide-42
SLIDE 42

Look-back: NoGood Learning

(x1=2,x2=2,x3=1,x4=2) is a dead-end

Conflicts to record:

(x1=2,x2=2,x3=1,x4=2) 4-ary

(x3=1,x4=2) binary

(x4=2) unary

Learning means recording conflict sets used as constraints to prune future search space.

slides7 828X 2019

slide-43
SLIDE 43

Learning, Constraint Recording

 Learning means recording conflict sets  An opportunity to learn is when deadend is

discovered.

 Goal of learning is to not discover the same

deadends.

 Try to identify small conflict sets  Learning prunes the search space. slides7 828X 2019

slide-44
SLIDE 44

No-good Learning Example

slides7 828X 2019

slide-45
SLIDE 45

Learning Issues

 Learning styles

 Graph-based or context-based  i-bounded, scope-bounded  Relevance-based

 Non-systematic randomized learning  Implies time and space overhead  Applicable to SAT: CDCL (Conflict-Directed

Clause Learning)

slides7 828X 2019

slide-46
SLIDE 46

Deep Learning

 Deep learning: recording all and only minimal

conflict sets

 Example:  Although most accurate, or “deepest”,

  • verhead can be prohibitive: the number of

conflict sets in the worst-case:

r

r r 2 2 /         

slides7 828X 2019

https://medium.com/a-computer-of-ones-own/rina-dechter-deep-learning-pioneer- e7e9ccc96c6e

slide-47
SLIDE 47

Bounded and Relevance-Based Learning

Bounding the arity of constraints recorded.

When bound is i: i-ordered graph-based,i-order jumpback or i-order deep learning.

Overhead complexity of i-bounded learning is time and space exponential in i.

slides7 828X 2019

slide-48
SLIDE 48

Graph-Based Learning Scenarios Internal Deadend at X4, conflicts?

 Scenario 1, deadend at x4:  Scenario 2: deadend at x5:  Scenario 3: deadend at x7:  Scenario 4: deadend at x6:

slides7 828X 2019

slide-49
SLIDE 49

Complexity of Backtrack-Learning for CSP

For graph-based learning the number of dead ends is bounded by Number of constraint tests per dead-end are Space complexity is Time complexity is

) ( ) (

1 ) ( * 2 ) ( * 

d w d w

k n O nk O  The complexity of learning along d is time and

space exponential in w*(d):

) (

) *(d w

nk O

) (e O

slides7 828X 2019

slide-50
SLIDE 50

Proof of Complexity NG learning

slides7 828X 2019

slide-51
SLIDE 51

Relationships between various backtracking algrithms

slides7 828X 2019

slide-52
SLIDE 52

Moving to New Queries

 Consistency and one solution.  Counting  Enumerating

slides7 828X 2019

slide-53
SLIDE 53

Bucket-elimination for counting

slides7 828X 2019

slide-54
SLIDE 54

#CSP - Tree DFS Traversal

1 0 1 1

C D F E B A

A E C B F D

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

0 1 1 1

0 1 1

0 1 1 1 1 2

0 1 1

0 1 1 1 0 1 1 3

Value of node = number of solutions below it

5

1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 1 1 1

0 1 0 1 1 1 1 1 1 0 1 0 1 0 1 1 1 2 1 2 1 1 2 2 2 2 1 3 3 6 6 9 1 1 1 1 2 3 14

slides7 828X 2019

slide-55
SLIDE 55

Outline

 Improving search by bounded-inference in

branching ahead

 Improving search by looking-back  The alternative AND/OR search space

slides7 828X 2019

slide-56
SLIDE 56

OR Search Space

A D B C E F

1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

E C F D B A

1

Ordering: A B E C D F

slides7 828X 2019

slide-57
SLIDE 57

AND/OR Search Space

A

OR AND

1 B

OR

B

AND

1 1 E

OR

C E C E C E C

OR

D F D F D F D F D F D F D F D F

AND

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

AND

1 1 1 1 1 1 1 1

A D B C E F A D B C E F

Primal graph DFS tree

A D B C E F A D B C E F slides7 828X 2019

slide-58
SLIDE 58

AND/OR vs. OR

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1 A

OR AND

1 B

OR

B

AND

1 1 E

OR

C E C E C E C

OR

D F D F D F D F D F D F D F D F

AND

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

AND

1 1 1 1 1 1 1 1

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1

AND/OR OR

A D B C E F A D B C E F

AND/OR size: exp(4), OR size exp(6)

slides7 828X 2019

slide-59
SLIDE 59

AND/OR vs. OR

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1 A

OR AND

1 B

OR

B

AND

1 1 E

OR

C E C E C E C

OR

D F D F D F D F D F D F D F D F

AND

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

AND

1 1 1 1 1 1 1 1

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1

E

1 1 1 1

C

1 1 1 1 1 1 1 1

F

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

B

1 1

A

1

AND/OR OR

A D B C E F A D B C E F No-goods (A=1,B=1) (B=0,C=0)

slides7 828X 2019

slide-60
SLIDE 60

AND/OR vs. OR

F

AND/OR

A D B C E F A D B C E F E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1 A

OR AND

1 B

OR

B

AND

1 E

OR

C E C E C

OR

D F D F D F D F

AND

1 1 1 1 1 1 1 1

AND

1 1 1 1 1 1

E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1

E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1

OR

(A=1,B=1) (B=0,C=0)

slides7 828X 2019

slide-61
SLIDE 61

AND/OR vs. OR

F

AND/OR

A D B C E F A D B C E F E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1 A

OR AND

1 B

OR

B

AND

1 E

OR

C E C E C

OR

D F D F D F D F

AND

1 1 1 1 1 1 1 1

AND

1 1 1 1 1 1

E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1

E

1 1 1

C

1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

D

1 1 1 1 1 1 1 1

B

1

A

1

OR

(A=1,B=1) (B=0,C=0)

Space: linear Time: O(exp(h)) O(w* log n) Linear space, Time: O(exp(n))

slides7 828X 2019

slide-62
SLIDE 62

AND/OR vs. OR Spaces

width depth OR space AND/OR space Time (sec.) Nodes Time (sec.) AND nodes OR nodes 5 10 3.15 2,097,150 0.03 10,494 5,247 4 9 3.13 2,097,150 0.01 5,102 2,551 5 10 3.12 2,097,150 0.03 8,926 4,463 4 10 3.12 2,097,150 0.02 7,806 3,903 5 13 3.11 2,097,150 0.10 36,510 18,255 Random graphs with 20 nodes, 20 edges and 2 values per node

slides7 828X 2019

slide-63
SLIDE 63

#CSP – AND/OR Search Tree

A E C B F D

A D B E C F

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

A

OR AND

B

OR AND OR

E

OR

F F

AND

1 1

AND

1 C D D 1 1 1 1 E F F 1 1 1 C D D 1 1 1 1 B E F F 1 1 1 C D D 1 1 1 1 E F F 1 1 1 C D D 1 1 1

slides7 828X 2019

slide-64
SLIDE 64

#CSP – AND/OR Tree DFS

A E C B F D

A D B E C F

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

A

OR AND

B

OR AND OR

E

OR

F

AND

1

AND

1 C D D 1 1 1

1 1 1 1 2 1 1 2 1 1 3 1 3 9 9

1 E F F 1 1 1 C D 1 1 1 B E F 1 1 C D D 1 1 1 1 E F 1 1 C D 1 1

1 1 1 1 1 2 1 2 2 1 2 2 3 6 1 1 1 1 1 1 1 2 1 1 1 2 3 1 2 1 1 1 2 1 2 3 2 5 5 14

OR node: Marginalization operator (summation) AND node: Combination operator (product)

slides7 828X 2019

slide-65
SLIDE 65

Pseudo-Trees

(Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91)

(a) Graph

4 6 1 3 2 7 5

(b) DFS tree depth=3 (c) pseudo- tree depth=2 (d) Chain depth=6

4 6 1 3 2 7 5 2 7 1 4 3 5 6 4 6 1 3 2 7 5

h <= w* log n

slides7 828X 2019

slide-66
SLIDE 66

a 1 2 b c 3 a c 4 OR AND OR AND OR AND AND OR c b 4 b 3 a b 4 c b 4 c 7 b c 5 a c 6 c b 6 b 5 a b 6 c b 6 c b 2 a c 3 b c 4 c a 4 a 3 a b 4 c 4 c a 7 a c 5 b c 6 c a 6 a 5 a b 6 c 6 c a c 2 a b 3 b c 4 a 4 b a 3 a c 4 b 4 b a 7 a b 5 b c 6 a 6 b a 5 a c 6 b 6 b a a 1 3 a 4 b c OR AND OR AND OR AND 2 b c b 4 c 2 c c 4 b 2 b 5 a 7 b c 6 b c b 7 c 6 c c 7 b 6 b b 3 b 4 a c 2 a c a 4 c 2 c c 4 a 2 a c 3 c 4 a b 2 a b a 4 b 2 b b 4 a 2 a 5 b 7 a c 6 a c a 7 c 6 c c 7 a 6 a 5 c 7 a b 6 a b a 7 b 6 b b 7 a 6 a

4 1 6 5 7 2 3 4 6 7 2 1 5 3 4 1 6 5 7 2 3

(a) (b) (c)

slides7 828X 2019

slide-67
SLIDE 67

AND/OR search tree for graphical models

The AND/OR search tree of R relative to a tree, T, has:

Alternating levels of: OR nodes (variables) and AND nodes (values)

Successor function:

The successors of OR nodes X are all its consistent values along its path

The successors of AND <X,v> are all X child variables in T

A solution is a consistent subtree

Task: compute the value of the root node

A B E

OR AND OR AND OR AND

C

OR AND

D 1 F 1 1 D 1 F 1 1 1 E C D 1 F 1 1 D 1 F 1 1 1 B E C D 1 F 1 1 D 1 F 1 1 1 E C D 1 F 1 1 D 1 F 1 1

A D B C E F A D B C E F

slides7 828X 2019

slide-68
SLIDE 68

slides7 828X 2019

The end

slide-69
SLIDE 69

From Search Trees to Search Graphs

 Any two nodes that root identical subtrees

(subgraphs) can be merged

slides7 828X 2019

slide-70
SLIDE 70

From Search Trees to Search Graphs

 Any two nodes that root identical subtrees

(subgraphs) can be merged

slides7 828X 2019

slide-71
SLIDE 71

From Search A/O Trees to Search A/O Graphs

 Any two nodes that root identical

subtrees/subgraphs can be merged

 Minimal AND/OR search graph:

closure under merge of the AND/OR search tree

 Inconsistent sub-trees can be pruned too.  Some portions can be collapsed or reduced.

slides7 828X 2019

slide-72
SLIDE 72

AND/OR Tree

A D B C E F A D B C E F G H J K G H J K

A

OR AND

1 B

OR

B

AND

1 1 E

OR

C E C E C E C

OR

D F D F D F D F D F D F D F D F

AND AND

1 1 1 1 1 1 1 1

OR OR AND AND

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

slides7 828X 2019

slide-73
SLIDE 73

An AND/OR Graph: Caching Goods

A D B C E F A D B C E F G H J K G H J K

A

OR AND

1 B

OR

B

AND

1 1 E

OR

C E C E C E C

OR

D F D F D F D F D F D F D F D F

AND AND

1 1 1 1 1 1 1 1

OR OR AND AND

G H H

0101 1

1 G H H

0101 1

J K K

0101 1

1 J K K

0101 1

slides7 828X 2019

slide-74
SLIDE 74

Context-based Caching

Caching is possible when context is the same

context = current variable + parents connected to subtree below

A D B C E F A D B C E F G H J K G H J K

context(B) = {A, B} context(c) = {A,B,C} context(D) = {D} context(F) = {F}

slides7 828X 2019

What is the context size? Induced-width

slide-75
SLIDE 75

Complexity of AND/OR Graph

 Theorem: Traversing the AND/OR search

graph is time and space exponential in the induced width/tree-width.

 If applied to the OR graph complexity is time

and space exponential in the path-width.

slides7 828X 2019

slide-76
SLIDE 76

#CSP – AND/OR Tree DFS

A E C B F D

A D B E C F

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

A

OR AND

B

OR AND OR

E

OR

F

AND

1

AND

1 C D D 1 1 1

1 1 1 1 2 1 1 2 1 1 3 1 3 9 9

1 E F F 1 1 1 C D 1 1 1 B E F 1 1 C D D 1 1 1 1 E F 1 1 C D 1 1

1 1 1 1 1 2 1 2 2 1 2 2 3 6 1 1 1 1 1 1 1 2 1 1 1 2 3 1 2 1 1 1 2 1 2 3 2 5 5 14

slides7 828X 2019

slide-77
SLIDE 77

#CSP – AND/OR Search Graph (Caching Goods)

A E C B F D

A D B E C F

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

A

OR AND

B

OR AND OR

E

OR

F F

AND

1

AND

1 C D D 1 1 1 E C D D 1 1 B E F F 1 C 1 E C

slides7 828X 2019

slide-78
SLIDE 78

#CSP – AND/OR Search Graph (Caching Goods)

A E C B F D

A D B E C F

A B C RABC 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A B E RABE 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 A E F RAEF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 B C D RBCD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

A

OR AND

B

OR AND OR

E

OR

F F

AND

1

AND

1 C D D 1 1 1 E C D D 1 1 B E F F 1 C 1 E C

Time and Space O(exp(w*))

slides7 828X 2019

slide-79
SLIDE 79

All Four Search Spaces

Full OR search tree 126 nodes Full AND/OR search tree 54 AND nodes Context minimal OR search graph 28 nodes Context minimal AND/OR search graph 18 AND nodes

1 1 1 1 1 1 1 1 1 1 1 1 01010101010101010101010101010101 010101 0101010101010101 0101010101 1 1 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1 0 1 1 1 1

C D F E B A

1 A

OR AND

B

OR AND OR

E

OR

F F

AND

0 1 0 1

AND

1 C D D 0 1 0 1 1 1 E F F 0 1 0 1 1 C D D 0 1 0 1 1 1 B E F F 0 1 0 1 1 C D D 0 1 0 1 1 1 E F F 0 1 0 1 1 C D D 0 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

C D F E B A

1 A

OR AND

B

OR AND OR

E

OR

F F

AND

0 1

AND

1 C D D 0 1 1 1 E C D D 1 1 B E F F 1 C 1 E C

A E C B F D

slides7 828X 2019

slide-80
SLIDE 80

AND/OR vs. OR DFS Algorithms

 AND/OR tree

 Space: O(n)  Time:

O(n km) O(n kw* log n)

(Freuder85; Bayardo95; Darwiche01)

 AND/OR graph

 Space: O(n kw*)  Time:

O(n kw*)

k = domain size m = tree depth n = # of variables w*= induced width pw*= path width

 OR tree

  • Space: O(n)
  • Time:

O(kn)

 OR graph

  • Space: O(n kpw*)
  • Time:

O(n kpw*)

slides7 828X 2019

slide-81
SLIDE 81

slides7 828X 2019

Summary: Time-Space for Constraint Processing

Constraint-satisfaction, one solution

Naive backtracking

 Space: O(n),  Time: O(exp(n))

Backjumping

 Space: O(n),  Time: O(exp(log n

w*))

Learning no-goods

 Space: O(exp(w*))  Time: O(exp(w*))

Variable-elimination

 Space: O(exp(w*))  Time: O(exp(w*))

Counting, enumeration

Backtracking, backjumping

 Space: O(n),  Time: O(exp(n ))

Learning no-goods

 space: O(exp(w*)) 

Time: O(exp(n))

Search with goods and no- goods learning

 Space: O(exp(pw*))  Time: O(exp(pw*)),

both, O(exp(w*logn))

Variable-elimination

 Space: O(exp(w*))  Time: O(exp(w*))

BFS is time and space O(exp(pw*))