CS 401 Master Theorem / Closest Points Xiaorui Sun 1 Master - - PowerPoint PPT Presentation

CS 401

Master Theorem / Closest Points

Xiaorui Sun

1

Master Theorem

Master Theorem

Suppose ! " = $ !

% & + (") for all " > +. Then,

• If $ < +) then ! " = Θ ")
• If $ = +) then ! " = Θ ")log "
• If $ > +) then ! " = Θ "23456

Works even if it is %

& instead of % & .

We also need $ ≥ 1, + > 1 , : ≥ 0 and ! " = <(1) for " ≤ +.

c: absolute constant

Quiz

Consider the following recurrence. Which case of the master theorem?

• A. ! " = Θ "%&'( ) = *(",../.)
• B. ! " = Θ "log "
• C. ! " = Θ "
• D. Master theorem not applicable

! " = 4 Θ 1 if " = 1 3!( "/2 ) + Θ(") if " > 1

Suppose ! " = = !

> ? + @"A for

all " > B. ! " = Θ "A 4 1 if = < BA log? " if = = BA "%&'D E if = > BA

Quiz

Consider the following recurrence. Which case of the master theorem?

• A. ! " = Θ "
• B. ! " = Θ "log "
• C. ! " = Θ "(
• D. Master theorem not applicable

! " = if " ≤ 1 ! " 5 + ! " − 3 " 10 + 11 5 " if " > 1 Akra–Bazzi theorem Wiki!

How to use master theorem?

For a divide and conquer algorithm

• a: number of subproblems
• b: ratio of problem size / subproblem size
• c⋅nk: running time of divide and combine step

We have recurrence " # = % "

& ' + )#* for all # > ,.

Example: Mergesort have two subproblems of half size of the original problem, and the cost of divide and combine step is O(n), so T(n) = 2 T(n / 2) + c n, which implies T(n) = Θ(n log n)

Quiz

In a divide and conquer algorithm, every problem of size n is divided into 20 subproblems of size n / 10, the cost of divide and combine steps is !", what is the running time of the algorithm?

• A. Θ(!")
• B. Θ(!" log !)
• C. Θ(!-./01 23)
• D. None of above

Suppose 4 ! = 6 4

7 8 + :!; for all ! > =.

4 ! = Θ !; 1 if 6 < =; log8 ! if 6 = =; 6 =;

• ./B 7

if 6 > =;

Proving Master Theorem

! " = $!("/') + *"+

$ " Problem size "/' "/', ' 1

. = log2"

# probs

$2 $ 1 $. cost *"+ $ ⋅ * "/' + $, ⋅ * "/', + $+ ⋅ * "/'+ +

! " = 5

678 97:;<= >

$6* " '6

+

Master Theorem

Suppose ! " = $ !

% & + (") for all " > +. Then,

• If $ < +) then ! " = Θ ")
• If $ = +) then ! " = Θ ")log "
• If $ > +) then ! " = Θ "23456

# of problems increases slower than the decreases of cost. First term dominates. # of problems increases faster than the decreases of cost Last term dominates.

! " = 7

89: ;92345 %

$8( " +8

)

A Useful Identity

Theorem: 1 + # + #$ + ⋯ + #& = ()*+,-

(,-

Proof: Let . = 1 + # + #$ + ⋯ + #& Then, #. = # + #$ + ⋯ + #&/- So, #. − . = #&/- − 1 i.e., . # − 1 = #&/- − 1 Therefore, . = ()*+,-

(,-

Corollary: 1 + # + #$ + ⋯ + #& = 1 2( 1 if # < 1 6 + 1 if # = 1 2( #&/- if # > 1

2( means the hidden constant depends on #

Solve: ! " = $!

% & + (")

Corollary: 1 + + + +, + ⋯ + +. = / Θ1 1 if + < 1 Θ 5 if + = 1 Θ1 +.67 if + > 1 Going back, we have ! " = 9

:;< .;=>?@ %

$:( " A:

)

= (") 9

:;< .;=>?@ %

$ A)

:

Hence, we have ! " = Θ ") 1 if $ < A) log& " if $ = A) $ A)

=>?@ %

if $ > A)

Solve: ! " = $!

% & + (")

! " = Θ ") 1 if $ < /) log& " if $ = /) $ /)

3456 %

if $ > /) For $ < /), we simply have ! " = Θ ") . For $ = /), we have ! " = Θ ") log& " = Θ(") log "). For $ > /), we have ! " = Θ ")

: &; 3456 %

= Θ("3456 :).

$3456 % = (/3456 :)3456 % = (/3456 %)3456 : = "3456 : /) 3456 % = /3456 % ) = ")

Finding the Closest Pair of Points

Recap: Closest Pair of Points (1-dim)

Given ! points, find the closest pair.

Brute force: Check all "

# pairwise distances

1-dim case: 11, 2, 4, 19, 4.8, 7, 8.2, 16, 11.5, 13, 1 find the closest pair Fact: Closest pair is adjacent in ordered list So, first sort, then scan adjacent pairs. Time $(! log !) to sort, if needed, Plus $(!) to scan adjacent pairs

1 2 4 4.8 7 8.2 11 11.5 13 16 19

Recap: Closest Pair of Points (2-dim)

Given ! points in the plane, find a pair with smallest Euclidean distance between them. Idea: make use of 1-dim algorithm (but not in a simple way)

No single direction along which one can sort points to guarantee success!

Divide & Conquer

Divide: draw vertical line ! with ≈ "/2 points on each side. Conquer: find closest pair on each side, recursively. Combine to find closest pair overall Return best solutions

12 21 8 L

How ?

Key Observation

Suppose ! is the minimum distance of all pairs in left/right of ". ! = min 12,21 = 12. Key Observation: suffices to consider points within ! of line ". Almost the one-D problem again: Sort points in 2!-strip by their + coordinate.

12 21 L d=12

7 1 2 3 4 5 6

Almost 1D Problem

Partition each side of ! into "

# × " # squares

Claim: No two points lie in the same "

# × " # box.

Proof: Such points would be within

" # #

+

" # #

= '

( # ≈ 0.7' < '

Let ./ have the 012 smallest 3-coordinate among points in the 2'-width-strip. Claim: If 0 − 6 > 11, then the distance between ./ and .

9 is > '.

Proof: only 11 boxes within ' of 3(./).

d

29 30 31 28 26 25

d

½d ½d

49

i j

27 29

> '

Closest Pair (2 dimension)

Closest-Pair(!", !$, ⋯ , !&) { if(& ≤ $) return |!" − !$| Compute separation line * such that half the points are on one side and half on the other side. +" = Closest-Pair(left half) +$ = Closest-Pair(right half) + = min(+", +$) Delete all points further than d from separation line L Sort remaining points p[1]…p[m] by y-coordinate. for , = ", $, ⋯ , . for k = ", $, ⋯ , "" if , + 0 ≤ . d = min(d, distance(p[i], p[i+k])); return d. }

Closest Pair Analysis

Let !(#) be the number of pairwise distance calculations in the Closest-Pair Algorithm

Closest Pair (2 dimension)

Closest-Pair(!", !$, ⋯ , !&) { if(& ≤ $) return distance(!", !$) Compute separation line ) such that half the points are on one side and half on the other side. *" = Closest-Pair(left half) *$ = Closest-Pair(right half) * = min(*", *$) Delete all points further than d from separation line L Sort remaining points p[1]…p[m] by y-coordinate. for + = ", $, ⋯ , - for k = ", $, ⋯ , "" if + + / ≤ - d = min(d, distance(p[i], p[i+k])); return d. }

111 ≤ 112 distance calculations

Closest Pair Analysis

Let !(#) be the number of pairwise distance calculations in the Closest-Pair Algorithm ! # ≤ & 1 if # ≤ 2 2! # 2 + 11 #

• . w. ⇒ ! # = O(#log #)

BUT, that’s only the number of distance calculations What if we counted running time?

Closest Pair (2 dimension)

Closest-Pair(!", !$, ⋯ , !&) { if(& ≤ $) return distance(!", !$) Compute separation line ) such that half the points are on one side and half on the other side. *" = Closest-Pair(left half) *$ = Closest-Pair(right half) * = min(*", *$) Delete all points further than d from separation line L Sort remaining points p[1]…p[m] by y-coordinate. for + = ", $, ⋯ , - for k = ", $, ⋯ , "" if + + / ≤ - d = min(d, distance(p[i], p[i+k])); return d. }

0(2 log 2) 0(2) 0(1) 0(2 log 2) 0(2)

Closest Pair Analysis

Let !(#) be the number of pairwise distance calculations in the Closest-Pair Algorithm ! # ≤ & 1 if # ≤ 2 2! # 2 + 11 #

• . w. ⇒ ! # = O(#log #)

BUT, that’s only the number of distance calculations What if we counted running time? 4 # ≤ & 1 if # ≤ 2 24 # 2 + 5(# log #)

• . w. ⇒ 4 # = O(#log6 #)

Can we do better?

Closest Pair (2 dimension) Improved

Closest-Pair(!", !$, ⋯ , !&) { if(& ≤ $) return distance(!", !$) Compute separation line ) such that half the points are on one side and half on the other side. (*", +") = Closest-Pair(left half) (*$, +$) = Closest-Pair(right half) * = min(*", *$) +,-./01 = merge(+", +$) (merge sort it by y-coordinate) Let 2 be points (ordered as +,-./01) that is * from line L. for 3 = ", $, ⋯ , 5 for k = ", $, ⋯ , "" if 3 + 7 ≤ 5 d = min(d, distance(S[i], S[i+k])); return d and +,-./01. }

8 9 ≤ : 1 if 9 = 1 28 9 2 + ? 9

• . w.

⇒ 8 9 = ?(9 log 9)

Input sorted by y-coordinate Assume: input sorted by x-coordinate (O(n log n) overhead initially)

?(1) ?(9) ?(1) ?(9) ?(9)

Summary

Closest pair in 2-dimension: Given ! points in the plane, find a pair with smallest Euclidean distance between them. Brute Force: Check all pairs of points in Θ(!$) time. Divde and Conquer:

• Divide: draw vertical line & with ≈ !/2 points on each side.
• Conquer: find closest pair on each side, recursively.
• Combine to find closest pair overall

Exercise: Remove the assumption of “no two points have same ) coordinate”?