A talk with 3 titles By Patrick Prosser Research how not to do it - - PowerPoint PPT Presentation

▶
a talk with 3 titles by patrick prosser research how not
SMART_READER_LITE
LIVE PREVIEW

A talk with 3 titles By Patrick Prosser Research how not to do it - - PowerPoint PPT Presentation

Sep 12, 2022 39 likes •1.13k views

A talk with 3 titles By Patrick Prosser Research how not to do it LDS revisited (aka Chinese whispers) Yet Another Flawed Talk by Patrick Prosser Send reinforcements. Were going to advance. Send three and fourpence. Were going to

slide-1
SLIDE 1
slide-2
SLIDE 2

A talk with 3 titles By Patrick Prosser

slide-3
SLIDE 3

Research … how not to do it

LDS revisited (aka Chinese whispers) Yet Another Flawed Talk by Patrick Prosser

slide-4
SLIDE 4
slide-5
SLIDE 5

Send reinforcements. We’re going to advance.

slide-6
SLIDE 6
slide-7
SLIDE 7

Send three and fourpence. We’re going to a dance!

slide-8
SLIDE 8

A refresher

  • Chronological Backtracking (BT)
  • what’s that then?
  • when/why do we need it?

Quick Intro Limited Discrepancy Search (lds)

  • what’s that then

Then the story … how not to do it

slide-9
SLIDE 9

An example problem (to show chronological backtracking (BT)) Colour each of the 5 nodes, such that if they are adjacent, they take different colours 1 2 3 4 5

slide-10
SLIDE 10

A Tree Trace of BT (assume domain ordered {R,B,G}) 1 2 3 4 5 v1 v2 v3 v4 v5

slide-11
SLIDE 11

A Tree Trace of BT (assume domain ordered {R,B,G}) 1 2 3 4 5 v1 v2 v3 v4 v5

slide-12
SLIDE 12

A Tree Trace of BT (assume domain ordered {R,B,G}) 1 2 3 4 5 v1 v2 v3 v4 v5

slide-13
SLIDE 13

A Tree Trace of BT (assume domain ordered {R,B,G}) 1 2 3 4 5 v1 v2 v3 v4 v5

slide-14
SLIDE 14

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-15
SLIDE 15

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-16
SLIDE 16

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-17
SLIDE 17

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-18
SLIDE 18

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-19
SLIDE 19

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-20
SLIDE 20

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-21
SLIDE 21

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-22
SLIDE 22

A Tree Trace of BT (assume domain ordered {R,B,G}) v1 v2 v3 v4 v5 1 2 3 4 5

slide-23
SLIDE 23

Could do better Improvements:

  • when colouring a vertex with colour X
  • remove X from the palette of adjacent vertices
  • when selecting a vertex to colour
  • choose the vertex with the smallest palette
  • tie break on adjacency with uncoloured vertices

An inferencing step A heuristic (Brelaz)

Conjecture: our heuristic is more reliable as we get deeper in search

slide-24
SLIDE 24

What’s a heuristic?

slide-25
SLIDE 25

Limited Discrepancy Search (LDS)

slide-26
SLIDE 26
slide-27
SLIDE 27

Motivation for lds

slide-28
SLIDE 28

Motivation for LDS

slide-29
SLIDE 29

LDS

  • show the search process
  • assume binary branching
  • assume we have 4 variables only
  • assume variables have 2 values each

Limited Discrepancy Search

slide-30
SLIDE 30

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take no discrepancies (go with the heuristic, go left!)

slide-31
SLIDE 31

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take no discrepancies

slide-32
SLIDE 32

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take no discrepancies

slide-33
SLIDE 33

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take no discrepancies

slide-34
SLIDE 34

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-35
SLIDE 35

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-36
SLIDE 36

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-37
SLIDE 37

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-38
SLIDE 38

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-39
SLIDE 39

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-40
SLIDE 40

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-41
SLIDE 41

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-42
SLIDE 42

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-43
SLIDE 43

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-44
SLIDE 44

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-45
SLIDE 45

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 1 discrepancy

slide-46
SLIDE 46

Now take 2 discrepancies

slide-47
SLIDE 47

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 2 discrepancies

slide-48
SLIDE 48

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 2 discrepancies

slide-49
SLIDE 49

Limited Discrepancy Search (LDS) Ginsberg & Harvey

Take 2 discrepancies

slide-50
SLIDE 50

First proposal For discrepancies 0 to n

slide-51
SLIDE 51

First proposal For discrepancies 0 to n k is remaining discrepancies

slide-52
SLIDE 52

First proposal For discrepancies 0 to n k is remaining discrepancies Go with heuristic

slide-53
SLIDE 53

First proposal For discrepancies 0 to n k is remaining discrepancies Go with heuristic Go against then go with

slide-54
SLIDE 54

The lds search process: how it goes (a cartoon)

slide-55
SLIDE 55

The lds search process: how it goes NOTE: lds revisits search states with k discrepancies When searching with > k discrepancies

slide-56
SLIDE 56

My pseudo code

slide-57
SLIDE 57

lds revisits nodes: Korf’s improvement (AAAI 96)

slide-58
SLIDE 58

Korf’s improvement

slide-59
SLIDE 59

Korf’s 1st mistake! Woops! Do you see it? He’s taking his discrepancies late/deep!

slide-60
SLIDE 60

Wrong way round.

Is that important?

Harvey & Ginsberg Korf

slide-61
SLIDE 61

Korf’s 1st mistake! Wrong way round Richard

slide-62
SLIDE 62

Richard, was that a bug?

slide-63
SLIDE 63

Yes, but so?

slide-64
SLIDE 64

Korf’s 2nd bug

slide-65
SLIDE 65

Woops! Richard, you know there is another bug?

slide-66
SLIDE 66

My pseudo code

slide-67
SLIDE 67

Has anyone noticed Korf’s bug? Have people been using Korf’s LDS? Have people been using Harvey & Ginsberg’s LDS? Has anyone remembered the motivation for LDS?

slide-68
SLIDE 68
slide-69
SLIDE 69
slide-70
SLIDE 70
slide-71
SLIDE 71

Chris, late or early?

slide-72
SLIDE 72

Wafa, late or early?

slide-73
SLIDE 73

Wafa’s response

slide-74
SLIDE 74

My pseudo code I think this has not been reported

slide-75
SLIDE 75

Does it make a difference if we take discrepancies late or early? An empirical study Tests Harvey & Ginsberg’s motivation for LDS

slide-76
SLIDE 76

Car Sequencing Problem

Assessed exercise 2

slide-77
SLIDE 77
slide-78
SLIDE 78
slide-79
SLIDE 79
slide-80
SLIDE 80
slide-81
SLIDE 81
slide-82
SLIDE 82
slide-83
SLIDE 83
slide-84
SLIDE 84
slide-85
SLIDE 85
slide-86
SLIDE 86
slide-87
SLIDE 87
slide-88
SLIDE 88

My empirical study on car sequencing problems Using various search algorithms, heuristics. Question:

  • does the order (late/early) that we take discrepancies in lds matter?
  • is the order sensitive to the heuristics used?
slide-89
SLIDE 89

Performing the experiments (what’s involved)

  • code up lds in JChoco
  • for non-binary domains
  • paramaterised late/early discrepancies
  • using Korf’s improvement
  • code up model of car sequencing problem
  • using Pascal Van Hentenryk’s model
  • code up my BT (as a gold standard)
  • code up a certificate checker
  • is a solution a solution?
  • code up 4 different heuristics
  • 2 published heuristics for car sequencing
  • the 2 anti-heuristics
  • Perform experiments on benchmark problems
  • limits on CPU time (minutes sometimes hours per instance)
  • test that all solutions are solutions (paranoia?)
  • problems typically have 200 cars (non-trivial)
  • NOTE TO SELF
  • also did Golomb rulers
  • started on HC
  • did this to show results were general and not car seqn specific
slide-90
SLIDE 90
slide-91
SLIDE 91
slide-92
SLIDE 92
slide-93
SLIDE 93

and now the results …

slide-94
SLIDE 94
slide-95
SLIDE 95

Well, did you see a pattern? If there is no pattern what does this say about H&G’s hypothesis? And, if no pattern, why is lds any good?

slide-96
SLIDE 96

See anything?

slide-97
SLIDE 97

Got my act together for ECAI08 reject

slide-98
SLIDE 98

How about another problem domain? Hamiltonian Circuit ecai08 rejects

slide-99
SLIDE 99
slide-100
SLIDE 100

What’s involved? ecai08 rejects

slide-101
SLIDE 101
slide-102
SLIDE 102
slide-103
SLIDE 103

HCP: What’s involved?

  • constraint model, single successor
  • subtour elimination constraint
  • heuristic
  • constraint model maintains information
  • resultant model is then Algorithm 595 (a surprise)
  • locate benchmark problems
  • perform experiments
  • late v early discrepancies
  • heuristics static v dynamic

ecai08 rejects

slide-104
SLIDE 104

ecai08 rejects

slide-105
SLIDE 105

Still to do

  • lds - extremely early and extremely late
  • jobshop scheduling using Cheng & Smith’s heuristic
  • repeat H & G’s experiments going late/early
  • number partitioning with CKK
  • repeat Korf’s experiments going late/early
  • fair bit of implementation and analysis
  • 2 months work at least
  • should get 2 more tables 
  • might get new ****09 rejection 
slide-106
SLIDE 106

What lessons can we learn?

  • we can just follow on without question (read the most recent paper)
  • forget the basic/initial hypothesis
  • forget to really look at our results
  • take too much for granted
  • it is not uninteresting to repeat someone elses’s experiments.
  • do not be frightened or disinterested in –ve results or different results (above)
  • don’t do just one set of experiments when you can do many (different domains)
  • published papers may have multiple errors (beware)
  • be paranoid (Am I right? Is he right? How do I know I’m right?)
  • know that we are human
slide-107
SLIDE 107

L a t e s t r e j e c t Available from the kiosk ecai08 rejects

slide-108
SLIDE 108

Start without me

Questions?

slide-109
SLIDE 109