CS4102 Algorithms Fall 2018 Warm up Compare + with + () When - - PowerPoint PPT Presentation

β–Ά
cs4102 algorithms
SMART_READER_LITE
LIVE PREVIEW

CS4102 Algorithms Fall 2018 Warm up Compare + with + () When - - PowerPoint PPT Presentation

Oct 15, 2022 281 likes β€’743 views

CS4102 Algorithms Fall 2018 Warm up Compare + with + () When = () When = () 1 = O() () () () () + +

slide-1
SLIDE 1

Warm up Compare 𝑔 π‘œ + 𝑛 with 𝑔 π‘œ + 𝑔(𝑛) When 𝑔 π‘œ = 𝑃(π‘œ) When 𝑔 π‘œ = Ξ©(π‘œ)

CS4102 Algorithms

Fall 2018

1

slide-2
SLIDE 2

2

π‘œ 𝑛 π‘œ + 𝑛

𝑔(π‘œ) 𝑔(𝑛) 𝑔(π‘œ) 𝑔(𝑛)

𝑔 π‘œ = O(π‘œ)

𝑔 π‘œ + 𝑛 ≀ 𝑔 π‘œ + 𝑔(𝑛)

slide-3
SLIDE 3

3

π‘œ 𝑛 π‘œ + 𝑛 𝑔(π‘œ) 𝑔(π‘œ) 𝑔(𝑛)

𝑔 π‘œ + 𝑛 β‰₯ 𝑔 π‘œ + 𝑔(𝑛)

𝑔 π‘œ = Ξ©(π‘œ)

𝑔(𝑛)

slide-4
SLIDE 4

4

π‘œ 𝑛 π‘œ + 𝑛 𝑔(π‘œ) 𝑔(𝑛) 𝑔(π‘œ) 𝑔(𝑛)

𝑔 π‘œ + 𝑛 = 𝑔 π‘œ + 𝑔(𝑛)

𝑔 π‘œ = Θ(π‘œ)

slide-5
SLIDE 5

Today’s Keywords

  • Divide and Conquer
  • Sorting
  • Quicksort
  • Median
  • Order statistic
  • Quickselect
  • Median of Medians

5

slide-6
SLIDE 6

CLRS Readings

  • Chapter 7

6

slide-7
SLIDE 7

Homeworks

  • Hw2 due 11pm Friday!

– Programming (use Python or Java!) – Divide and conquer – Closest pair of points

  • Hw3 released soon

– Divide and conquer – Written (use LaTeX!)

7

More on HW2

  • You must read from garden.txt file

automatically (it’s a fixed filename)

  • That file has a list of pairs of floats (not

ints)

  • You must only output one floating point

number (minimum distance)

  • Uploaded files:
  • One python file, or
  • One or more java files (uploaded

individually)

  • Don’t use packages!
  • Don’t use subdirectories!
  • DO NOT upload a zip file!
  • Try it yourself:
  • Put the files you are going to upload

in a directory (with a garden.txt file)

  • python closestpair_mst3k.py
  • javac *.java

java ClosestPair

  • Use the one for your language and

you should get a result

slide-8
SLIDE 8

Quicksort

  • Idea: pick a pivot element, recursively sort two sublists around

that element

  • Divide: select an element π‘ž, Partition(π‘ž)
  • Conquer: recursively sort left and right sublists
  • Combine: Nothing!

8

slide-9
SLIDE 9

Partition (Divide step)

  • Given: a list, a pivot π‘ž

9

8 5 7 3 12 10 1 2 4 9 6 11

Goal: All elements < π‘ž on left, all > π‘ž on right

Start: unordered list

5 7 3 1 2 4 6 8 12 10 9 11

slide-10
SLIDE 10

Partition Summary

  • 1. Put π‘ž at beginning of list
  • 2. Put a pointer (Begin) just after π‘ž, and a pointer (End) at the

end of the list

  • 3. While Begin < End:
  • 1. If Begin value < π‘ž, move Begin right
  • 2. Else swap Begin value with End value, move End Left
  • 4. If pointers meet at element < π‘ž: Swap π‘ž with pointer position
  • 5. Else If pointers meet at element > π‘ž: Swap π‘ž with value to

the left

10

slide-11
SLIDE 11

Quicksort Run Time

  • Then we divide in half each time

11

2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + π‘œ

  • If the pivot is always the median:

π‘ˆ π‘œ = 𝑃(π‘œ log π‘œ)

slide-12
SLIDE 12

Quicksort Run Time

  • Then we shorten by 1 each time

12

1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12

π‘ˆ π‘œ = π‘ˆ π‘œ βˆ’ 1 + π‘œ

  • If the partition is always unbalanced:

π‘ˆ π‘œ = 𝑃(π‘œ2)

slide-13
SLIDE 13

Good Pivot

  • What makes a good Pivot?

– Roughly even split between left and right – Ideally: median

  • Can we find median in linear time?

– Yes! – Quickselect

13

slide-14
SLIDE 14

Quickselect

  • Finds 𝑗th order statistic

– 𝑗th smallest element in the list – 1st order statistic: minimum – π‘œth order statistic: maximum –

π‘œ 2

th order statistic: median

14

slide-15
SLIDE 15

Quickselect

  • Finds 𝑗th order statistic
  • Idea: pick a pivot element, partition, then recurse on sublist

containing index 𝑗

  • Divide: select an element π‘ž, Partition(π‘ž)
  • Conquer: if 𝑗 = index of π‘ž, done!

– if 𝑗 < index of π‘ž recurse left. Else recurse right

  • Combine: Nothing!

15

slide-16
SLIDE 16

Partition (Divide step)

  • Given: a list, a pivot value π‘ž

16

8 5 7 3 12 10 1 2 4 9 6 11

Goal: All elements < π‘ž on left, all > π‘ž on right

Start: unordered list

5 7 3 1 2 4 6 8 12 10 9 11

slide-17
SLIDE 17

Conquer

17

2 5 7 3 6 4 1 8 10 9 11 12

All elements < π‘ž All elements > π‘ž Exactly where it belongs!

Recurse on sublist that contains index 𝑗 (add index of the pivot to 𝑗 if recursing right)

slide-18
SLIDE 18

Quickselect Run Time

  • Then we divide in half each time

18

2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12

𝑇 π‘œ = 𝑇 π‘œ 2 + π‘œ

  • If the pivot is always the median:

𝑇 π‘œ = 𝑃(π‘œ)

slide-19
SLIDE 19

Quickselect Run Time

  • Then we shorten by 1 each time

19

1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12

𝑇 π‘œ = 𝑇 π‘œ βˆ’ 1 + π‘œ

  • If the partition is always unbalanced:

𝑇 π‘œ = 𝑃(π‘œ2)

slide-20
SLIDE 20

Good Pivot

  • What makes a good Pivot?

– Roughly even split between left and right – Ideally: median

  • Here’s what’s next:

– An algorithm for finding a β€œrough” split – This algorithm uses Quickselect as a subroutine

20

DΓ©jΓ  vu?

slide-21
SLIDE 21

Good Pivot

  • What makes a good Pivot?

– Both sides of Pivot >30%

21

Or

>30% >30% Select Pivot from this range

slide-22
SLIDE 22

Median of Medians

  • Fast way to select a β€œgood” pivot
  • Guarantees pivot is greater than 30% of elements and less than

30% of the elements

  • Idea: break list into chunks, find the median of each chunk, use

the median of those medians

22

slide-23
SLIDE 23

Median of Medians

23

  • 1. Break list into chunks of size 5
  • 2. Find the median of each chunk
  • 3. Return median of medians (using Quickselect)
slide-24
SLIDE 24

Why is this good?

24

< < < < < < < < < < < < < < < < < < < < < < <

Each chunk sorted, chunks ordered by their medians

MedianofMedians is Greater than all

  • f these

π‘œ 5 5

slide-25
SLIDE 25

Why is this good?

25

MedianofMedians is larger than all

  • f these

Larger than 3 things in each (but one) list to the left

< 3

1 2 β‹… π‘œ 5 βˆ’ 2 β‰ˆ 3π‘œ 10 βˆ’ 6 elements

Similarly:

> 3

1 2 β‹… π‘œ 5 βˆ’ 2 β‰ˆ 3π‘œ 10 βˆ’ 6 elements

π‘œ 5

< < < < < < < < < < < < < < < < < < < < < < <

slide-26
SLIDE 26

Quickselect

  • Divide: select an element π‘ž using Median of Medians,

Partition(π‘ž)

  • Conquer: if 𝑗 = index of π‘ž, done, if 𝑗 < index of π‘ž recurse left.

Else recurse right

  • Combine: Nothing!

26

𝑁 π‘œ + Θ(π‘œ)

≀ 𝑇

7 10 π‘œ

𝑇 π‘œ ≀ 𝑇

7 10 π‘œ + 𝑁 π‘œ + Θ(π‘œ)

slide-27
SLIDE 27

Median of Medians, Run Time

27

  • 1. Break list into chunks of 5
  • 2. Find the median of each chunk
  • 3. Return median of medians (using Quickselect)

Θ(π‘œ) Θ(π‘œ) 𝑇 π‘œ 5 𝑁 π‘œ = 𝑇 π‘œ 5 + Θ(π‘œ)

slide-28
SLIDE 28

Quickselect

28

𝑁 π‘œ = 𝑇 π‘œ 5 + Θ(π‘œ) 𝑇 π‘œ ≀ 𝑇 7π‘œ 10 + 𝑁 π‘œ + Θ(π‘œ) = 𝑇 7π‘œ 10 + 𝑇 π‘œ 5 + Θ(π‘œ) 𝑇 π‘œ = O(π‘œ) = 𝑇 7π‘œ 10 + 𝑇 2π‘œ 10 + Θ(π‘œ) ≀ 𝑇 9π‘œ 10 + Θ(π‘œ)

Master theorem Case 3! Because 𝑇 π‘œ = Ξ©(π‘œ)

slide-29
SLIDE 29

Phew! Back to Quicksort

  • Then we divide in half each time

29

2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + Θ(π‘œ)

  • Using Quickselect, with a median-of-medians

partition:

π‘ˆ π‘œ = Θ(π‘œ log π‘œ)

slide-30
SLIDE 30

Is it worth it?

  • Using Quickselect to pick median guarantees Θ(π‘œ log π‘œ) run

time

  • Approach has very large constants

– If you really want Θ(π‘œ log π‘œ), better off using MergeSort

  • Better approach: Random pivot

– Very small constant (very fast algorithm) – Expected to run in Θ(π‘œ log π‘œ) time

  • Why? Unbalanced partitions are very unlikely

30

slide-31
SLIDE 31

Quicksort Run Time

31

π‘ˆ π‘œ = π‘ˆ π‘œ 10 + π‘ˆ 9π‘œ 10 + π‘œ

  • If the pivot is always π‘œ

10

th order statistic:

slide-32
SLIDE 32

π‘œ

π‘ˆ π‘œ = π‘ˆ π‘œ 10 + π‘ˆ 9π‘œ 10 + π‘œ

π‘œ 10 9π‘œ 10 π‘œ 100 9π‘œ 100 9π‘œ 100 81π‘œ 100

… … … …

1 1 1 1

π‘œ π‘œ/10 9π‘œ/10 π‘œ/100 9π‘œ/100 9π‘œ/100 81π‘œ/100 1 1 1 1

π‘œ π‘œ π‘œ + + + + + + +

log 10

9

π‘œ

slide-33
SLIDE 33

Quicksort Run Time

33

π‘ˆ π‘œ = π‘ˆ π‘œ 10 + π‘ˆ 9π‘œ 10 + π‘œ

  • If the pivot is always π‘œ

10

th order statistic:

π‘ˆ π‘œ = Θ(π‘œ log π‘œ)

slide-34
SLIDE 34

Quicksort Run Time

  • Then we shorten by 𝑒 each time

34

1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12

π‘ˆ π‘œ = π‘ˆ π‘œ βˆ’ 𝑒 + π‘œ

  • If the pivot is always 𝑒th order statistic:

π‘ˆ π‘œ = 𝑃(π‘œ2) What’s the probability of this occurring?

slide-35
SLIDE 35

Probability of π‘œ2 run time

We must consistently select pivot from within the first 𝑒 terms

35

Probability first pivot is among 𝑒 smallest:

𝑒 π‘œ

Probability second pivot is among 𝑒 smallest:

𝑒 π‘œβˆ’π‘’

Probability all pivots are among 𝑒 smallest: 𝑒 π‘œ β‹… 𝑒 π‘œ βˆ’ 𝑒 β‹… 𝑒 π‘œ βˆ’ 2𝑒 β‹… … β‹… 𝑒 2𝑒 β‹… 1 = 1 π‘œ 𝑒 !

slide-36
SLIDE 36

Formal Argument for π‘œ log π‘œ Average

  • Remember, run time counts comparisons!
  • Quicksort only compares against a pivot

– Element 𝑗 only compared to element π‘˜ if one of them was the pivot

36

slide-37
SLIDE 37

Formal Argument for π‘œ log π‘œ Average

  • What is the probability of comparing two

given elements?

37

1 2 3 4 5 6 7 8 9 10 11 12

  • (Probability of comparing 3 and 4) = 1

– Why? Otherwise I wouldn’t know which came first – ANY sorting algorithm must compare adjacent elements

slide-38
SLIDE 38

Formal Argument for π‘œ log π‘œ Average

  • What is the probability of comparing two

given elements?

38

1 2 3 4 5 6 7 8 9 10 11 12

  • (Probability of comparing 1 and 12) = 2

12

– Why?

  • I only compare 1 with 12 if either was chosen as the

first pivot

  • Otherwise they would be divided into opposite sublists
slide-39
SLIDE 39

Formal Argument for π‘œ log π‘œ Average

  • Probability of comparing 𝑗 with π‘˜ (π‘˜ > 𝑗):

– dependent on the number of elements between 𝑗 and π‘˜ –

1 π‘˜βˆ’π‘—+1

  • Expected number of comparisons:

–

1 π‘˜βˆ’π‘—+1 𝑗<π‘˜

39

slide-40
SLIDE 40

Expected number of Comparisons

1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

40

Consider when 𝑗 = 1 Sum so far:

2 2

1 2 3 4 5 6 7 8 9 10 11 12

Compared if 1 or 2 are chosen as pivot (these will always be compared)

slide-41
SLIDE 41

Expected number of Comparisons

1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

41

Consider when 𝑗 = 1 Sum so far:

2 2 + 2 3

1 2 3 4 5 6 7 8 9 10 11 12

Compared if 1 or 3 are chosen as pivot (but never if 2 is ever chosen)

slide-42
SLIDE 42

Expected number of Comparisons

1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

42

Consider when 𝑗 = 1 Sum so far:

2 2 + 2 3 + 2 4

1 2 3 4 5 6 7 8 9 10 11 12

Compared if 1 or 4 are chosen as pivot (but never if 2 or 3 are chosen)

slide-43
SLIDE 43

Expected number of Comparisons

1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

43

Consider when 𝑗 = 1 Overall sum:

2 2 + 2 3 + 2 4 + 2 5 + β‹― + 2 π‘œ

1 2 3 4 5 6 7 8 9 10 11 12

Compared if 1 or 12 are chosen as pivot (but never if 2 -> 11 are chosen)

slide-44
SLIDE 44

Expected number of Comparisons

1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

44

When 𝑗 = 1: 2

1 2 + 1 3 + 1 4 + β‹― + 1 π‘œ

π‘œ terms overall 1 π‘˜ βˆ’ 𝑗 + 1

𝑗<π‘˜

≀ 2π‘œ 1 2 + 1 3 + β‹― + 1 π‘œ

Θ(log π‘œ) Quicksort overall: expected Θ π‘œ log π‘œ