CPSC 320: Intermediate Algorithm Design and Analysis July 18, 2014 - - PowerPoint PPT Presentation

β–Ά
cpsc 320 intermediate algorithm
SMART_READER_LITE
LIVE PREVIEW

CPSC 320: Intermediate Algorithm Design and Analysis July 18, 2014 - - PowerPoint PPT Presentation

Aug 09, 2023 370 likes β€’641 views

CPSC 320: Intermediate Algorithm Design and Analysis July 18, 2014 1 Course Outline Introduction and basic concepts Asymptotic notation Greedy algorithms Graph theory Amortized analysis Recursion

slide-1
SLIDE 1

1

CPSC 320: Intermediate Algorithm Design and Analysis

July 18, 2014

slide-2
SLIDE 2

2

Course Outline

  • Introduction and basic concepts
  • Asymptotic notation
  • Greedy algorithms
  • Graph theory
  • Amortized analysis
  • Recursion
  • Divide-and-conquer algorithms
  • Randomized algorithms
  • Dynamic programming algorithms
  • NP-completeness
slide-3
SLIDE 3

3

Recurrence Relations

slide-4
SLIDE 4

4

Recursive Functions

  • Complexity analysis of iterative functions has been covered
  • How can we analyse the complexity of a recursive function?
  • Example:

Algorithm MergeSort(𝐡, 𝑑, 𝑓) If (𝑑 < 𝑓) Then 𝑛 ← ⌊ 𝑑 + 𝑓 2βŒ‹ MergeSort(𝐡, 𝑑, 𝑛) MergeSort(𝐡, 𝑛 + 1, 𝑓) Merge(𝐡, 𝑑, 𝑛, 𝑓)

slide-5
SLIDE 5

5

Recurrence Relations

  • The complexity of MergeSort can be computed as:

π‘ˆ π‘œ = Θ(1) π‘œ = 1 π‘ˆ π‘œ 2 + π‘ˆ π‘œ 2 + Θ(π‘œ) π‘œ β‰₯ 2

  • How can we figure out what the complexity is for the function above?
slide-6
SLIDE 6

6

Guess and Test

  • First approach: guess one expression and test if it works
  • Test usually involves strong induction
  • Example: prove that π‘ˆ π‘œ ≀ π‘‘π‘œ log π‘œ
  • Induction step:

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + π‘’π‘œ ≀ 2𝑑 π‘œ 2 log π‘œ 2 + π‘’π‘œ ≀ 2𝑑 π‘œ 2 log π‘œ 2 + π‘’π‘œ = π‘‘π‘œ log π‘œ βˆ’ 1 + π‘’π‘œ = π‘‘π‘œ log π‘œ βˆ’ 𝑑 βˆ’ 𝑒 π‘œ ≀ π‘‘π‘œ log π‘œ if 𝑑 β‰₯ 𝑒

slide-7
SLIDE 7

7

Guess and Test

  • Base case:
  • π‘œ = 1:

π‘ˆ 1 ≀ 𝑑 β‹… 1 log 1 1 ≀ 0 doesn’t work!

  • π‘œ = 2:

π‘ˆ 2 ≀ 𝑑 β‹… 2 log 2 2π‘ˆ 1 + 2𝑒 ≀ 2𝑑 1 + 𝑒 ≀ 𝑑 ∴ 𝑑 β‰₯ 𝑒 + 1 2 is not enough, since 3

2 = 1

slide-8
SLIDE 8

8

Guess and Test

  • Base case (cont.):
  • π‘œ = 3:

π‘ˆ 3 ≀ 𝑑 β‹… 3 log 3 2π‘ˆ 1 + 3𝑒 ≀ (3 log 3)𝑑 2 + 3𝑒 3 log 3 ≀ 𝑑 ∴ 𝑑 β‰₯ 𝑒 + 1

  • So, for 𝑑 = 𝑒 + 1 and π‘œ β‰₯ π‘œ0 = 2, π‘ˆ π‘œ ∈ 𝑃 π‘œ log π‘œ
slide-9
SLIDE 9

9

Guess and Test

π‘ˆ π‘œ = 1 π‘œ = 1 2π‘ˆ π‘œ 2 + π‘’π‘œ π‘œ > 1

  • Prove that π‘ˆ π‘œ ≀ π‘‘π‘œ
  • Induction step:

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + π‘’π‘œ ≀ 2𝑑 π‘œ 2 + π‘’π‘œ ≀ 2𝑑 π‘œ 2 + π‘’π‘œ = π‘‘π‘œ + π‘’π‘œ = (𝑑 + 𝑒)π‘œ

  • Prove didn’t reach expected result, disproved
  • Careful, even though c+d is a constant, we needed the constant to be c only.
slide-10
SLIDE 10

10

Guess and Test

π‘ˆ π‘œ = 1 π‘œ = 1 2π‘ˆ π‘œ 2 + 𝑒 π‘œ > 1

  • Prove that π‘ˆ π‘œ ≀ π‘‘π‘œ
  • Induction step:

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + 𝑒 ≀ 2𝑑 π‘œ 2 + 𝑒 ≀ 2𝑑 π‘œ 2 + 𝑒 = π‘‘π‘œ + 𝑒

  • Prove didn’t reach expected result, but we got close
  • Let’s try a stronger claim: π‘ˆ π‘œ ≀ π‘‘π‘œ βˆ’ 𝑒 ≀ π‘‘π‘œ
slide-11
SLIDE 11

11

Guess and Test

π‘ˆ π‘œ = 1 π‘œ = 1 2π‘ˆ π‘œ 2 + 𝑒 π‘œ > 1

  • Prove that π‘ˆ π‘œ ≀ π‘‘π‘œ βˆ’ 𝑒
  • Induction step:

π‘ˆ π‘œ = 2π‘ˆ π‘œ 2 + 𝑒 ≀ 2 𝑑 π‘œ 2 βˆ’ 𝑒 + 𝑒 ≀ 2𝑑 π‘œ 2 βˆ’ 2𝑒 + 𝑒 = π‘‘π‘œ βˆ’ 𝑒

  • Base case, π‘œ = 1

π‘ˆ 1 ≀ 𝑑 β‹… 1 βˆ’ 𝑒 1 ≀ 𝑑 βˆ’ 𝑒

  • Proved that 𝑑 ≀ π‘‘π‘œ βˆ’ 𝑒 for 𝑑 = 𝑒 + 1 and π‘œ β‰₯ π‘œ0 = 1
slide-12
SLIDE 12

12

Recursion Trees

slide-13
SLIDE 13

13

Recursion Trees

  • Sometimes guessing isn’t that easy, or that exact
  • Draw a tree representing all calls to the recursion
  • Add the β€œcosts” for each row of the tree
  • Use these sums and the height of the tree to bound the running time
  • If result is not clear, use substitution to prove a better guess
slide-14
SLIDE 14

14

Examples

  • π‘ˆ π‘œ = 3π‘ˆ

π‘œ 4 + Θ(π‘œ2)

π‘œ β‰₯ 4 Θ(1) π‘œ < 4 π‘ˆ π‘œ π‘ˆ π‘œ 4 π‘ˆ π‘œ 4 π‘ˆ π‘œ 4 π‘ˆ π‘œ 16 π‘ˆ π‘œ 16 π‘ˆ π‘œ 16 π‘ˆ π‘œ 16 … … … π‘’π‘œ2 3π‘’π‘œ2 16 9π‘’π‘œ2 256

slide-15
SLIDE 15

15

Examples

  • So we have:

π‘ˆ π‘œ ≀ π‘’π‘œ2 + 3π‘’π‘œ2 16 + 9π‘’π‘œ2 256 + β‹― + 3 16

log4 π‘œ

π‘’π‘œ2 ≀ 𝑒 3 16 + 3 16

2

+ 3 16

3

+ β‹― π‘œ2 ≀ π‘‘π‘œ2 π‘ˆ π‘œ ∈ 𝑃(π‘œ2)

slide-16
SLIDE 16

16

Examples

  • π‘ˆ π‘œ = 3π‘ˆ

π‘œ 2 + Θ(π‘œ)

π‘œ β‰₯ 2 Θ(1) π‘œ = 1

  • First tree level: π‘’π‘œ
  • Second tree level:

3 2 π‘’π‘œ

  • Third tree level: 3

2 2

π‘’π‘œ

  • Number of levels: log2 π‘œ
slide-17
SLIDE 17

17

Examples

  • So we have:

π‘ˆ π‘œ ≀ π‘’π‘œ + 3π‘’π‘œ 2 + 9π‘’π‘œ 4 + β‹― + 3 2

log2 π‘œ

π‘’π‘œ ≀ π‘’π‘œ

𝑗=0 lg π‘œ 3

2

𝑗

= π‘’π‘œ 3 2

1+lg π‘œ

βˆ’ 1 3 2 βˆ’ 1 ≀ 2π‘’π‘œ 3 2 3 2

lg π‘œ

= 3π‘’π‘œ 3lg π‘œ 2lg π‘œ = 3π‘’π‘œ 2lg 3 lg π‘œ π‘œ = 3π‘’π‘œlg 3 π‘ˆ π‘œ ∈ 𝑃(π‘œlg 3)

slide-18
SLIDE 18

18

Examples

  • π‘ˆ π‘œ = π‘ˆ

π‘œ 3 + π‘ˆ 2π‘œ 3

+ Θ(π‘œ) π‘œ β‰₯ 3 Θ(1) π‘œ < 3

  • First tree level: π‘’π‘œ
  • Second tree level: π‘’π‘œ
  • Third tree level: π‘’π‘œ
  • Number of levels: log3 π‘œ on one side, log3

2 π‘œ on the other side (tree is skewed)

slide-19
SLIDE 19

19

Examples

  • So we have:

π‘ˆ π‘œ ≀ π‘’π‘œ β‹… log3

2

π‘œ π‘ˆ π‘œ ∈ 𝑃(π‘œ log π‘œ) π‘ˆ π‘œ β‰₯ π‘’π‘œ β‹… log3 π‘œ π‘ˆ π‘œ ∈ Ξ©(π‘œ log π‘œ) π‘ˆ π‘œ ∈ Θ π‘œ log π‘œ

slide-20
SLIDE 20

20

Examples

  • π‘ˆ π‘œ = π‘ˆ π‘œ βˆ’ 1 + Θ 1

π‘œ β‰₯ 1 Θ 1 π‘œ < 1

  • First tree level: 𝑒
  • Second tree level: 𝑒
  • Third tree level: 𝑒
  • Number of levels: π‘œ
  • Total: π‘ˆ π‘œ ≀ π‘’π‘œ, so π‘ˆ π‘œ ∈ 𝑃(π‘œ)
slide-21
SLIDE 21

21

Examples

  • So we have:

π‘ˆ π‘œ ≀ π‘’π‘œ β‹… log3

2

π‘œ π‘ˆ π‘œ ∈ 𝑃(π‘œ log π‘œ) π‘ˆ π‘œ β‰₯ π‘’π‘œ β‹… log3 π‘œ π‘ˆ π‘œ ∈ Ξ©(π‘œ log π‘œ) π‘ˆ π‘œ ∈ Θ π‘œ log π‘œ

slide-22
SLIDE 22

22

Master Theorem

slide-23
SLIDE 23

23

Master Theorem

  • Let 𝑏 β‰₯ 1, 𝑐 > 1 be real constants, 𝑔 π‘œ : β„• β†’ ℝ+, and π‘ˆ(π‘œ) defined by:

π‘ˆ π‘œ = π‘π‘ˆ π‘œ 𝑐 + 𝑔(π‘œ) π‘œ β‰₯ π‘œ0 Θ(1) π‘œ < π‘œ0

  • where π‘œ

𝑐 could also be π‘œ 𝑐 or π‘œ 𝑐 . Then:

  • 1. If 𝑔 π‘œ ∈ 𝑃 π‘œlog𝑐 π‘βˆ’πœ

for some 𝜁 > 0, then π‘ˆ π‘œ ∈ Θ π‘œlog𝑐 𝑏 .

  • 2. If 𝑔 π‘œ ∈ Θ π‘œlog𝑐 𝑏 log𝑙 π‘œ

for some 𝑙 β‰₯ 0, then π‘ˆ π‘œ ∈ Θ π‘œlog𝑐 𝑏 log𝑙+1 π‘œ .

  • 3. If 𝑔 π‘œ ∈ Ξ© π‘œlog𝑐 𝑏+𝜁

for some 𝜁 > 0, and 𝑏𝑔

π‘œ 𝑐 < πœ€π‘”(π‘œ) for some 0 < πœ€ < 1 and all

π‘œ large enough, then π‘ˆ π‘œ ∈ Θ 𝑔(π‘œ) .

slide-24
SLIDE 24

24

Master Theorem Proof

  • We can prove using a recursion tree.
  • Number of levels: log𝑐 π‘œ
  • First level: 𝑔(π‘œ)
  • Second level: 𝑏𝑔

π‘œ 𝑐

  • Third level: 𝑏2𝑔

π‘œ 𝑐2

  • Close to end: 𝑏log𝑐 π‘œβˆ’1𝑔

π‘œ 𝑐log𝑐 π‘œβˆ’1

  • Last level: 𝑏log𝑐 π‘œπ‘” 1
slide-25
SLIDE 25

25

Master Theorem Proof

π‘ˆ π‘œ = 𝑏log𝑐 π‘œ +

π‘˜=0 log𝑐 π‘œβˆ’1

π‘π‘˜π‘” π‘œ π‘π‘˜ = 𝑐log𝑐 𝑏 log𝑐 π‘œ +

π‘˜=0 log𝑐 π‘œβˆ’1

π‘π‘˜π‘” π‘œ π‘π‘˜ = π‘œlog𝑐 𝑏 +

π‘˜=0 log𝑐 π‘œβˆ’1

π‘π‘˜π‘” π‘œ π‘π‘˜

  • From this result we can prove all three cases.