Visualizations to Summarize Search Behavior Ian S. Howell !,# , - - PowerPoint PPT Presentation

Constraint Systems Laboratory

9/8/20 CP 2020 1

Visualizations to Summarize Search Behavior

Ian S. Howell !,#, Berthe Y. Choueiry $, and Hongfeng Yu %

Constraint Systems Laboratory $ Visualizations Laboratory % Department of Computer Science & Engineering University of Nebraska-Lincoln

Constraint Systems Laboratory

Introduction

• Backtrack search when solving a

constraint satisfaction problem (CSP) suffers from thrashing

• Enforcing a consistency property reduces

thrashing at the cost of further processing

• The tradeoff of running a higher-level

consistency (HLC) is poorly understood.

• We propose to summarize search

9/8/20 CP 2020 2

Constraint Systems Laboratory

Background

• A constraint satisfaction problem (CSP)

consists of a set of variables, the variables’ domains, and constraints.

• A solution is a domain to variable

assignment that satisfies all constraints.

• Backtrack search is only sound and

complete method of solving a CSP.

9/8/20 CP 2020 3

Constraint Systems Laboratory

Background

• Thrashing is the main malady of search

– Repeatedly performing same logic

• Applying a consistency algorithm after

variable instantiation reduces thrashing

• Past visualizations focus on debugging

and inspecting search

– Detect thrashing through isomorphic subtrees – Debug individual issues by investigating the state of variables, propagators, etc.

9/8/20 CP 2020 4

Constraint Systems Laboratory

Background

• Woodward et al. propose to measure:

– thrashing with Backtracks per Depth (BpD) – HLC cost with consistency calls per depth (CpD)

9/8/20 CP 2020 5

10 20 30 40 50 60 5,000 10,000 15,000 20,000 25,000 30,000 35,000

1 26 51 76 101 126 151 176 201 226 251 276 301 326 351 376

# Calls to POAC # Backtracks

Depth

APOAC

#BTs Filter No Filter Wipeout

CPU Time: 23.77 sec. #NV: 59,181 #BT: 53,212 #Calls POAC: 11,142

5,000 10,000 15,000 20,000 25,000 30,000 35,000

1 26 51 76 101 126 151 176 201 226 251 276 301 326 351 376

# Backtracks

Depth

GAC

#BTs

CPU Time: 140.81 sec. #NV: 3,978,074 #BT: 3,348,330

Constraint Systems Laboratory

Contributions

• 1. Criteria for computing distance between

two time samples based on the BpD

• 2. A clustering technique for summarizing

search into a history of qualitatively distinct regimes

• 3. A new visualization that examines the

behavior of variable ordering heuristics

9/8/20 CP 2020 6

Constraint Systems Laboratory

Outline

1. Introduction 2. Background 3. Contributions 4. Outline 5. Distance between timestamps 6. Summarizing search history 7. Example: Analyzing structure 8. Visualizing variable ordering 9. Example: Variable ordering

• 10. Conclusions and discussion

9/8/20 CP 2020 7

Constraint Systems Laboratory

Distance between timestamps

9/8/20 CP 2020 8

• Design goal: capture the “shape” change

#BT at time ! #BT at time !′ Depth

Constraint Systems Laboratory

Distance between timestamps

• Adapt Kulbak-Liebler Divergence
• BpD distribution:

– Use additive smoothing to account for zero- probability intervals

9/8/20 CP 2020 9

Constraint Systems Laboratory

Distance between timestamps

9/8/20 CP 2020 10

Constraint Systems Laboratory

Summarizing search history

• Create a clustering tree

– Use agglomerative, hierarchical clustering – Only merge temporally adjacent clusters – Each cluster’s representative is the middle of the interval of included timestamps

9/8/20 CP 2020 11

t0 t1 t2 t11

Divergence time

t3 t4 t5 t6 t7 t8 t9 t10

Constraint Systems Laboratory

Summarizing search history

• Create a summarization tree

– Cut the tree at a user-defined ! or – Cut the tree to include a " regimes

9/8/20 CP 2020 12

t0 t1 t2 t11

Divergence time

t3 t4 t5 t6 t7 t8 t9 t10

r1 r2 r3 r4 r5

δ

Constraint Systems Laboratory

Example: Analyzing structure

9/8/20 CP 2020 13

Constraint Systems Laboratory

Example: Analyzing structure

9/8/20 CP 2020 14

0.1-summarization of GAC on MUG100-25-3

Constraint Systems Laboratory

Example: Analyzing structure

9/8/20 CP 2020 15

Primal graph of MUG100-25-3

Constraint Systems Laboratory

Example: Analyzing structure

9/8/20 CP 2020 16

Primal graph of MUG100-25-3

Many large cycles

Constraint Systems Laboratory

Example: Analyzing structure

9/8/20 CP 2020 17

BpD of GAC and DPPC+ on MUG100-25-3

Constraint Systems Laboratory

Visualizing variable ordering

• Variable Instantiations per Depth (VIpD)

– "($, &, ') is the number of instantiations of variable $ at depth & and time '

• Order the variables of the VIpD according

to each variable’s weighted depth:

9/8/20 CP 2020 18

Constraint Systems Laboratory

Example: Variable ordering

9/8/20 CP 2020 19

VIpD of GAC and POAC with dom/wdeg on MUG100-1-3

Constraint Systems Laboratory

Example: Variable ordering

9/8/20 CP 2020 20

VIpD of POAC with dom/deg on MUG100-1-3

Constraint Systems Laboratory

Example: Variable ordering

9/8/20 CP 2020 21

0.1-summarization of POAC with dom/wdeg on MUG100-1-3

Constraint Systems Laboratory

Example: Variable ordering

9/8/20 CP 2020 22

0.1-summarization of POAC with dom/deg on MUG100-1-3

Constraint Systems Laboratory

Conclusions and discussion

• Summarizations can be used to help

explain the (changing) behavior of search

• Researchers and developers can use

these tools iteratively to better study the impact of a strategy on a given problem

• Summarizations can catch small and large

behavior that a human could not

9/8/20 CP 2020 23

Constraint Systems Laboratory

Conclusions and discussion

• Initial results of BpD with binary branching

give similar results

• We currently provide a ‘post-mortem’
• analysis. Future work includes an ‘in-vivo’

analysis that enables human experts

9/8/20 CP 2020 24

Constraint Systems Laboratory

Acknowledgements

• This research is supported by NSF Grant
• No. RI-1619344 and NSF CAREER Award
• No. III-1652846.
• The experiments were completed utilizing

the Holland Computing Center of the University of Nebraska, which receives support from the Nebraska Research Initiative.

9/8/20 CP 2020 25