Empirical comparison of the mini- bucket and MPLP algorithms for - - PowerPoint PPT Presentation

▶
empirical comparison of the mini
SMART_READER_LITE
LIVE PREVIEW

Empirical comparison of the mini- bucket and MPLP algorithms for - - PowerPoint PPT Presentation

May 29, 2023 36 likes •199 views

Empirical comparison of the mini- bucket and MPLP algorithms for MPE problem Solving MPE A finite COP is a triple R X , D , F where : Global Cost Funct ion X { X ,..., X } - variables 1 n

slide-1
SLIDE 1

Empirical comparison of the mini- bucket and MPLP algorithms for MPE problem

slide-2
SLIDE 2

Solving MPE

  • Exact by Bucket elimination
  • Bounding scheme: MBE, MPLP, Soft arc-consistency.
  • MBE: capitalize on large cluster processed exactly. Similar to i-

consistency for large i (parameter z), directional, non-iterative.

  • SAC, MPLP: equivalent to arc-consistency. Can be extended to

cluster graphs, but may not be efficient. Based on cost shifting subject t equivalence-preserving transformations (EPT)

scopes S f f F D D D X X X F D X R COP finite

m m n n

      } ,..., {S S ies probabilit

  • }

,..., { domains

  • }

,..., { variables

  • }

,..., { : where , , a triple is A

1 1 1 1

 

) ( max * ) ( ion Cost Funct Global

1

x F F X f X F

x m i i

  

slide-3
SLIDE 3

4

Mini-Bucket Approximation

Split a bucket into mini-buckets => bound complexity

) ( ) ( ) O(e : decrease complexity l Exponentia

n r n r

e O e O

 

bucket (X) = { h1, …, hr, hr+1, …, hn } { h1, …, hr } { hr+1, …, hn }

Bucket(X’)= Bucket(X’’)=

X X’ X’’ [Dechter, Rish 2003]

 

n i i X X

h h

1

max

 

  

 

n r i i X r i i X X

h h g

1 1

) (max ) (max

X X

h g 

slide-4
SLIDE 4

Mini-Bucket Approximation

  • Properties:

– Guaranteed to output upper bound on the exact solution (for maximization) – Control parameter iBound (number of variables in the mini-bucket) allows to flexibly trade space for accuracy

  • If iBound>=treewidth the algorithm turns into the exact bucket

elimination scheme

– The solution can be recovered by the upward pass – If the values assigned to ‘duplicate’ variables agree, the solution found is exact

slide-5
SLIDE 5

Mini-Bucket Approximation

  • Possible ways to improve accuracy:

– Increase i-Bound – Try different heuristics to split the functions into mini- buckets:

  • Scope-based: rely only on functions arguments [Dechter, Rish

1997]

  • Content-based (the product of mini-bucket functions should be as

close as possible to the original bucket function, according to some distance metric (l1,l2…) *Rollon Dechter 2010]

– Force the values of duplicates to be the same -> force max- marginals of the mini-buckets to be the same (‘moment matching’)

slide-6
SLIDE 6

Max Product Linear Programming algorithm (MPLP)

  • Variation of max-product

[Globerson, Jakkola] F( x x x x x

  • Always guaranteed to converge

(but not necessarily at the global optimum)

  • monotonically improves with more iterations
  • If the costs are not tied, the solution found is exact
  • If all variables are binary, MPLP converges to

exact solution

slide-7
SLIDE 7

Empirical evaluation

  • Genetic linkage instances from UAI’2008 competition

– ~300-1000 variables – 15-20 treewidth – Domain size 4-5

  • Mini-buckets:

– iBound=5,10 – Heuristics: scope-based, content-based(l1,l2,linf, KL,HPM,MAS) – Without and with moment-matching before eliminating bucket variable

  • MPLP:

– On original factors, cluster trees constructed by MBE with iBounds=1, 5, 10 – Iterations: 5, 10, 15, 30, 60, 120, 240, 500, 1000, 1500

slide-8
SLIDE 8
  • 105.5
  • 105
  • 104.5
  • 104
  • 103.5
  • 103
  • 102.5
  • 102

5 10 15 30 60 120 240 500 1000 1500 log(MPE) number of iterations

pedigree1: log(MPE) as a function of number of iterations of MPLP, [-104.956 ]

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10 exact solution

slide-9
SLIDE 9

100 200 300 400 500 600 700 800 900 5 10 15 30 60 120 240 500 1000 1500 time, sec number of iterations

pedigree1: time, sec, as a function of number of iterations of MPLP

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-10
SLIDE 10
  • 144
  • 143
  • 142
  • 141
  • 140
  • 139
  • 138
  • 137
  • 136
  • 135

5 10 15 30 60 120 240 500 1000 1500 runtime, sec number of iterations

Pedigree23, log(MPE) as a function of number of iterations of MPLP,[-143.662 ]

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10 exact solution

slide-11
SLIDE 11

100 200 300 400 500 600 700 800 900 1000 5 10 15 30 60 120 240 500 1000 1500 time, sec number of iterations

pedigree23: time, sec, as a function of number of iterations of MPLP

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-12
SLIDE 12
  • 327
  • 322
  • 317
  • 312
  • 307
  • 302
  • 297
  • 292
  • 287

5 10 15 30 60 120 240 500 1000 1500 runtime, sec number of iterations

Pedigree37, log(MPE) as a function of number of iterations of MPLP,[-333.603 ]

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-13
SLIDE 13
  • 200

300 800 1300 1800 2300 5 10 15 30 60 120 240 500 1000 1500 time, sec number of iterations

pedigree37: time, sec, as a function of number of iterations of MPLP

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-14
SLIDE 14
  • 166
  • 165
  • 164
  • 163
  • 162
  • 161
  • 160
  • 159
  • 158
  • 157
  • 156

5 10 15 30 60 120 240 runtime, sec number of iterations

Pedigree13, log(MPE) as a function of number of iterations of MPLP,[-168.952 ]

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-15
SLIDE 15

50 100 150 200 250 5 10 15 30 60 120 240 time, sec number of iterations

pedigree13: time, sec, as a function of number of iterations of MPLP

MPLP_original factors MPLP_iBound=1 MPLP_iBound=5 MPLP_iBound=10 mbeScopeNoMatching_i5 mbeScopeMatching_i5 mbeScopeNoMatching_i10 mbeScopeMatching_i10 mbeBestNoMatching_i5 mbeBestMatching_i5 mbeBestNoMatching_i10 mbeBestMatching_i10

slide-16
SLIDE 16

0.009 0.014 0.019 0.024 0.029 0.034 0.039 0.044 0.049 scope l1 l2 linf KL HPM MAS distance metrics

Average relative bound improvement due to moment matching

iBound=5 iBound=10