Anytime Best First search: Empirical evaluation Natalia Flerova Radu Marinescu Rina Dechter University of California IBM Research Irvine Ireland
Anytime Repairing AOBF (wR-AOBF) (based on ARA* [Likhachev et al. 2003] ) Main idea: Run search iteratively, decreasing w (like wAOBF) BUT re-use the results of previous iterations ! Consider some starting weight w, put start node in OPEN until w=1 or out of time Search for solution in AOBF manner, expanding nodes on OPEN with best f(n), but only if f(n) is better than the current best cost keep track of nodes that are already on CLOSED , but whose g(n) has changed ( INCONS list) output the solution found by Weighted A* Decrease w by fixed positive value δ Move all nodes from INCONS to OPEN re-compute f(n) for all nodes in OPEN with new w
Experiments ● 3 algorithms: – wAOBF Weight schedule: – wR-AOBF – BRAOBB [Otten, Dechter'11] ● Experimental settings: – I-bounds attempted{2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22} – Time limit: ● Main dataset: 1 hour ● Pascal2 dataset: 10 hours ● We also consider short term results: 20 sec, 1 min, 10 min – Memory limit: ● Small memory: 4 Gb ● Large memory: 80 Gb – –
Experiments Benchmarks: – Main set (17 hard problems): ● Pedigrees ● Binary grids ● WCSP – Pascal2 set: ● Alchemy (1 instance) ● CSP (61 instances) ● DBN (116 instances) ● Grids (21 instances) ● Imagealignment (10 instances) ● Objectdetection (37 instances) ● Promedas (86 instances) ● Proteinfolding (10 instances) ● Proteinprotein (11 instances) ● Segmentation (100 instances) ● ●
The main conclusions Impossible to claim absolute dominance of any of 3 algorithms. ● The performance greatly depends on: – benchmark – heuristic strength – memory limit
The main conclusions Memory limit: – Best First schemes (wAOBF and wR-AOBF) greatly benefit from additional memory. – With 80 Gb memory limit they: ● can find solutions to many instances, for which no solutions were found for 4 Gb ● Can find optimal solutions for problems, for which only loose approximations are found for 4 Gb – BRAOBB is less hurt by lack of memory, having similar performance for both 4 Gb and 80 Gb
The main conclusions Benchmarks: ● Main data set: – wAOBF and wR-AOBF perform better compared to BRAOBB on instances with a lot of determinism ( pedigrees and grids ) – BRAOBB performs better on instances with little determinism ( WSCP )
The main conclusions Benchmarks: ● Pascal2 data set: (only results for 4 Gb) CSP: – BF schemes manage to find solutions on considerably fewer instances ● than BRAOBB (e.g. i=2: wAOBF 13 vs BRAOBB 61 instances) DBN: – No solutions by wAOBF and wR-AOBF . BRAOBB finds solutions for 108 ● (i=2) to 60 instances (i=22) Grids: – No solutions by wAOBF and wR-AOBF . BRAOBB finds solutions for 21 ● (i=2) to 13 instances (i=22) Image alignment: – BRAOBB finds solutions for more instances than wAOBF and wR-AOBF . ● –
Impact of heuristics Given a particular instance: (e.g.pedigree31, C*=-130.461) I=8: MBE-ROOT = -123.324 BRAOBB: 1st sol: [0] -164.472 Last sol: [1075] -140.293 wAOBF-sqrt: 1st sol: [421] -176.6877 Last sol: [5400] -143.4375 I=10: MBE-ROOT = -124.32 BRAOBB: 1st sol: [0] -151.657 Last sol: [655] -138.952 wAOBF-sqrt: 1st sol: [0] -172.7116 Last sol: [6035] -136.6440
'Our' grids vs pascal2 grids 75-20-5: MBE-ROOT = -8.24529 C*=-12.7195 BAYES, k=2, ar=3, n=400, f=400, e=1120 w*=27 , h=99 A lot of determinism wAOBF-sqrt: 1st sol: [0] -23.2776 Last sol: [6479] -12.7195 Grid20x20.f10: MBE-ROOT = -1506.39 C*=-1309.72 MARKOV, k=2, ar=2, n=400, f=1200, e=800 w*=44 , h=68 No determinism wAOBF-sqrt: OOM
Why it takes so much time to find the first solution? wAOBF-sqrt: 1st sol: [88] -3669.6776 Nodes: 61248 Last sol: [1200] -2999.9340 Nodes: 377165 No C*, OOM wAOBF-sqrt: 1st sol: [226] -2006.2421 Nodes: 228571 Last sol: [661] -1547.1252 Nodes: 996572 C*=-1547.1252
Recommend
More recommend
