CS4102 Algorithms Fall 2020 Warm up Simplify: 1 + 2 + 3 + + ( 1) - - PowerPoint PPT Presentation

β–Ά
cs4102 algorithms
SMART_READER_LITE
LIVE PREVIEW

CS4102 Algorithms Fall 2020 Warm up Simplify: 1 + 2 + 3 + + ( 1) - - PowerPoint PPT Presentation

Dec 07, 2023 769 likes β€’982 views

CS4102 Algorithms Fall 2020 Warm up Simplify: 1 + 2 + 3 + + ( 1) + = 1 + 1 1 + 2 + 3 + + ( 1) + = 2 + 1 2 Matrix Multiplication 1 2 3 2 4 6 4 5 6 8 10 12 7 8 9

slide-1
SLIDE 1

1

Warm up Simplify: 1 + 2 + 3 + β‹― + (π‘œ βˆ’ 1) + π‘œ =

CS4102 Algorithms

Fall 2020

slide-2
SLIDE 2

2

1 + 2 + 3 + β‹― + (π‘œ βˆ’ 1) + π‘œ =

π‘œ + 1 π‘œ

π‘œ π‘œ + 1 2

slide-3
SLIDE 3

Matrix Multiplication

3

1 2 3 4 5 6 7 8 9 Γ— 2 4 6 8 10 12 14 16 18 = 60 72 84 132 162 192 204 252 300 = 2 + 16 + 42 4 + 20 + 48 6 + 24 + 54 β‹… β‹… β‹… β‹… β‹… β‹… Run time? 𝑃(π‘œ3) π‘œ π‘œ

slide-4
SLIDE 4

Matrix Multiplication D&C

4

Multiply π‘œ Γ— π‘œ matrices (𝐡 and 𝐢)

𝐡 = 𝑏1 𝑏2 𝑏3 𝑏4 𝑏5 𝑏6 𝑏7 𝑏8 𝑏9 𝑏10 𝑏11 𝑏12 𝑏13 𝑏14 𝑏15 𝑏16 𝐢 = 𝑐1 𝑐2 𝑐3 𝑐4 𝑐5 𝑐6 𝑐7 𝑐8 𝑐9 𝑐10 𝑐11 𝑐12 𝑐13 𝑐14 𝑐15 𝑐16

Divide:

slide-5
SLIDE 5

Matrix Multiplication D&C

5

Multiply π‘œ Γ— π‘œ matrices (𝐡 and 𝐢)

𝐡 = 𝑏1 𝑏2 𝑏3 𝑏4 𝑏5 𝑏6 𝑏7 𝑏8 𝑏9 𝑏10 𝑏11 𝑏12 𝑏13 𝑏14 𝑏15 𝑏16

𝐡1,1 𝐡1,2 𝐡2,1 𝐡2,2

𝐡𝐢 = 𝐡1,1𝐢1,1 + 𝐡1,2𝐢2,1 𝐡1,1𝐢1,2 + 𝐡1,2𝐢2,2 𝐡2,1𝐢1,1 + 𝐡2,2𝐢2,1 𝐡2,1𝐢1,2 + 𝐡2,2𝐢2,2 𝐢 = 𝑐1 𝑐2 𝑐3 𝑐4 𝑐5 𝑐6 𝑐7 𝑐8 𝑐9 𝑐10 𝑐11 𝑐12 𝑐13 𝑐14 𝑐15 𝑐16

𝐢1,1 𝐢1,2 𝐢2,1 𝐢2,2 Run time? π‘ˆ π‘œ = 8π‘ˆ π‘œ 2 + 4 π‘œ 2

2

Cost of additions

Combine:

slide-6
SLIDE 6

Matrix Multiplication D&C

6

π‘ˆ π‘œ = 8π‘ˆ π‘œ 2 + 4 π‘œ 2

2

π‘ˆ π‘œ = 8π‘ˆ π‘œ 2 + Θ(π‘œ2) 𝑏 = 8, 𝑐 = 2, 𝑔 π‘œ = π‘œ2 π‘œlog𝑐 𝑏 = π‘œlog2 8 = π‘œ3 Case 1! π‘ˆ π‘œ = Θ(π‘œ3) We can do better…

slide-7
SLIDE 7

Matrix Multiplication D&C

7

Multiply π‘œ Γ— π‘œ matrices (𝐡 and 𝐢)

𝐡 = 𝑏1 𝑏2 𝑏3 𝑏4 𝑏5 𝑏6 𝑏7 𝑏8 𝑏9 𝑏10 𝑏11 𝑏12 𝑏13 𝑏14 𝑏15 𝑏16

𝐡1,1 𝐡1,2 𝐡2,1 𝐡2,2

𝐡𝐢 = 𝐡1,1𝐢1,1 + 𝐡1,2𝐢2,1 𝐡1,1𝐢1,2 + 𝐡1,2𝐢2,2 𝐡2,1𝐢1,1 + 𝐡2,2𝐢2,1 𝐡2,1𝐢1,2 + 𝐡2,2𝐢2,2 𝐢 = 𝑐1 𝑐2 𝑐3 𝑐4 𝑐5 𝑐6 𝑐7 𝑐8 𝑐9 𝑐10 𝑐11 𝑐12 𝑐13 𝑐14 𝑐15 𝑐16

𝐢1,1 𝐢1,2 𝐢2,1 𝐢2,2 Idea: Use a Karatsuba-like technique on this

slide-8
SLIDE 8

Strassen’s Algorithm

8

Multiply π‘œ Γ— π‘œ matrices (𝐡 and 𝐢)

𝐡 = 𝑏1 𝑏2 𝑏3 𝑏4 𝑏5 𝑏6 𝑏7 𝑏8 𝑏9 𝑏10 𝑏11 𝑏12 𝑏13 𝑏14 𝑏15 𝑏16

𝐡1,1 𝐡1,2 𝐡2,1 𝐡2,2

𝐢 = 𝑐1 𝑐2 𝑐3 𝑐4 𝑐5 𝑐6 𝑐7 𝑐8 𝑐9 𝑐10 𝑐11 𝑐12 𝑐13 𝑐14 𝑐15 𝑐16

𝐢1,1 𝐢1,2 𝐢2,1 𝐢2,2 Calculate:

𝑅1 = 𝐡1,1 + 𝐡2,2 (𝐢1,1 + 𝐢2,2) 𝑅2 = 𝐡2,1 + 𝐡2,2 𝐢1,1 𝑅3 = 𝐡1,1(𝐢1,2 βˆ’ 𝐢2,2) 𝑅4 = 𝐡2,2(𝐢2,1 βˆ’ 𝐢1,1) 𝑅6 = 𝐡2,1 βˆ’ 𝐡1,1 (𝐢1,1 + 𝐢1,2) 𝑅5 = 𝐡1,1 + 𝐡1,2 𝐢2,2 𝑅7 = 𝐡1,2 βˆ’ 𝐡2,2 (𝐢2,1 + 𝐢2,2)

𝐡1,1𝐢1,1 + 𝐡1,2𝐢2,1 𝐡1,1𝐢1,2 + 𝐡1,2𝐢2,2 𝐡2,1𝐢1,1 + 𝐡2,2𝐢2,1 𝐡2,1𝐢1,2 + 𝐡2,2𝐢2,2 𝑅1 + 𝑅4 βˆ’ 𝑅5 + 𝑅7 𝑅3 + 𝑅5 𝑅2 + 𝑅4 𝑅1 βˆ’ 𝑅2 + 𝑅3 + 𝑅6

Find 𝐡𝐢:

Number Mults.: 7 Number Adds.: 18 π‘ˆ π‘œ = 7π‘ˆ π‘œ 2 + 9 2 π‘œ2

slide-9
SLIDE 9

Strassen’s Algorithm

9

π‘ˆ π‘œ = 7π‘ˆ π‘œ 2 + 9 2 π‘œ2 𝑏 = 7, 𝑐 = 2, 𝑔 π‘œ = 9 2 π‘œ2 π‘œlog𝑐 𝑏 = π‘œlog2 7 β‰ˆ π‘œ2.807 Case 1! π‘ˆ π‘œ = Θ π‘œlog2 7 β‰ˆ Θ(π‘œ2.807)

slide-10
SLIDE 10

10

π‘œ3 π‘œlog2 7

Strassen’s Algorithm

slide-11
SLIDE 11

Is this the fastest?

11

Best possible is unknown May not even exist!

slide-12
SLIDE 12

Recurrence Solving Techniques

12

Tree Guess/Check β€œCookbook” Substitution

?

slide-13
SLIDE 13

Observation

  • Divide: 𝐸(π‘œ) time,
  • Conquer: recurse on small problems, size 𝑑
  • Combine: C(π‘œ) time
  • Recurrence:

π‘ˆ π‘œ = 𝐸 π‘œ + π‘ˆ(𝑑) + 𝐷(π‘œ)

  • Many D&C recurrences are of form:

π‘ˆ π‘œ = π‘π‘ˆ π‘œ 𝑐 + 𝑔(π‘œ)

13

slide-14
SLIDE 14

Master Theorem

  • Case 1: if 𝑔 π‘œ = 𝑃(π‘œlog𝑐 𝑏 βˆ’πœ) for some constant 𝜁 > 0, then

π‘ˆ π‘œ = Θ(π‘œlog𝑐 𝑏)

  • Case 2: if 𝑔 π‘œ = Θ(π‘œlog𝑐 𝑏), then π‘ˆ π‘œ = Θ(π‘œlog𝑐 𝑏 log π‘œ)
  • Case 3: if 𝑔 π‘œ = Ξ©(π‘œlog𝑐 𝑏+𝜁) for some constant 𝜁 > 0, and if

𝑏𝑔

π‘œ 𝑐

≀ 𝑑𝑔(π‘œ) for some constant 𝑑 < 1 and all sufficiently large π‘œ, then π‘ˆ π‘œ = Θ(𝑔 π‘œ )

14

π‘ˆ π‘œ = π‘π‘ˆ π‘œ 𝑐 + 𝑔(π‘œ)

slide-15
SLIDE 15

Substitution Method

  • Idea: take a β€œdifficult” recurrence, re-express it such that one
  • f our other methods applies.
  • Example:

15

π‘ˆ π‘œ = 2π‘ˆ π‘œ + log2 π‘œ

slide-16
SLIDE 16

Tree method

16

π‘œ

π‘ˆ π‘œ = 2π‘ˆ( π‘œ) + log2 π‘œ

π‘œ π‘œ

π‘œ π‘œ π‘œ π‘œ

… … … …

2 2 2 … 2 2 2

log2 π‘œ 1 2 log2 π‘œ 1 2 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 1 1 1 1 1

log2 π‘œ log2 π‘œ log2 π‘œ log2 π‘œ + + + + + + + + +

log2 log2 π‘œ

π‘ˆ π‘œ = 𝑃(log2 π‘œ β‹… log2 log2 π‘œ)

log2 π‘œ1/2 = 1 2 log2 π‘œ

slide-17
SLIDE 17

Substitution Method

17

π‘ˆ π‘œ = 2π‘ˆ π‘œ + log2 π‘œ Let π‘œ = 2𝑛, i.e. 𝑛 = log2 π‘œ Let 𝑇 𝑛 = 2𝑇

𝑛 2

+ 𝑛 π‘ˆ 2𝑛 = 2π‘ˆ 2

𝑛 2

+ 𝑛 Rewrite in terms of exponent! Case 2! Let 𝑇 𝑛 = Θ(𝑛 log 𝑛) Substitute Back Let T π‘œ = Θ(log π‘œ log log π‘œ) π‘ˆ π‘œ = 2π‘ˆ π‘œ1/2 + log2 π‘œ

I don’t like the Β½ in the exponent Now the variable is in the exponent on both sides! S will operate exactly as T, just redefined in terms of the exponent 𝑇 𝑛 = π‘ˆ(2𝑛)

slide-18
SLIDE 18

Tree method

18

π‘œ

π‘ˆ π‘œ = 2π‘ˆ( π‘œ) + log2 π‘œ

π‘œ π‘œ

π‘œ π‘œ π‘œ π‘œ

… … … …

2 2 2 … 2 2 2

log2 π‘œ 1 2 log2 π‘œ 1 2 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 4 log2 π‘œ 1 1 1 1 1 1

log2 π‘œ log2 π‘œ log2 π‘œ log2 π‘œ + + + + + + + + +

log2 log2 π‘œ

π‘œ = 2𝑛 π‘ˆ 2𝑛 = 2π‘ˆ 2

𝑛 2

+ 𝑛

2𝑛 2𝑛/2 2𝑛/2 2𝑛/4 2𝑛/4 2𝑛/4 2𝑛/4

slide-19
SLIDE 19

Tree method

19

2𝑛

π‘ˆ 2𝑛 = 2π‘ˆ(2𝑛/2) + 𝑛

2𝑛/2 2𝑛/2 2𝑛/4 2𝑛/4 2𝑛/4 2𝑛/4

… … … …

21 21 21 … 21 21 21

𝑛 𝑛 2 𝑛 2 𝑛 4 𝑛 4 𝑛 4 𝑛 4 1 1 1 1 1 1

𝑛 𝑛 𝑛 𝑛 + + + + + + + + +

log2𝑛

π‘œ = 2𝑛

slide-20
SLIDE 20

Tree method

20

𝑛 𝑛/2 𝑛/2 𝑛/4 𝑛/4 𝑛/4 𝑛/4

… … … …

1 1 1 … 1 1 1

𝑛 𝑛 2 𝑛 2 𝑛 4 𝑛 4 𝑛 4 𝑛 4 1 1 1 1 1 1

𝑛 𝑛 𝑛 𝑛 + + + + + + + + +

log2𝑛

π‘ˆ π‘œ = 𝑃(𝑛 β‹… log2 𝑛) π‘œ = 2𝑛 π‘ˆ 2𝑛 = 𝑇(𝑛) 𝑇 𝑛 = 2𝑇 𝑛 2 + 𝑛 = 𝑃(log2 π‘œ β‹… log2 log2 π‘œ)