Divide and Conquer Chapter 4 1 Integer Multiplication 2 Integer - - PowerPoint PPT Presentation

β–Ά
divide and conquer
SMART_READER_LITE
LIVE PREVIEW

Divide and Conquer Chapter 4 1 Integer Multiplication 2 Integer - - PowerPoint PPT Presentation

Mar 12, 2023 186 likes β€’643 views

Divide and Conquer Chapter 4 1 Integer Multiplication 2 Integer Multiplication = Each with n bits x= 1 0 1 1 y= 1 1 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 Z= 1 0 0 0 0 1 0 0 3 Integer

slide-1
SLIDE 1

Divide and Conquer

Chapter 4

1

slide-2
SLIDE 2

Integer Multiplication

2

slide-3
SLIDE 3

Integer Multiplication

𝑨 = 𝑦 Γ— 𝑧

Each with n bits

x= 1 1 1 y= 1 1 1 1 1 1 1 1 Z= 1 1

3

slide-4
SLIDE 4

Integer Multiplication

1.

NaΓ―ve_IM(x, y)

2.

z = 0

3.

while y > 0

4.

if (y is odd) then z += x

5.

x *= 2

6.

y /= 2

7.

end

8.

return z

n bit operations

n iterations

π‘ˆ π‘œ = Θ π‘œ2

4

slide-5
SLIDE 5

D&C Integer Multiplication

𝑦1 𝑦 𝑦2 𝑧1 𝑧 𝑧2

π‘œ 2 𝑐𝑗𝑒𝑑 π‘œ 2 𝑐𝑗𝑒𝑑

5

slide-6
SLIDE 6

D&C Integer Multiplication

𝑦1 𝑦 𝑦2 𝑧1 𝑧 𝑧2

𝑨 = 𝑦12

π‘œ 2 + 𝑦2

𝑧12

π‘œ 2 + 𝑧2

z = 𝑦1𝑧1 2π‘œ + 𝑦1𝑧2 + 𝑦2𝑧1 2

π‘œ 2 + 𝑦2𝑧2

π‘ˆ π‘œ = 4π‘ˆ π‘œ 2 + π‘œ π‘ˆ π‘œ = Θ π‘œ2

𝑦2 Γ— 𝑧2 𝑦2 Γ— 𝑧1 𝑦1 Γ— 𝑧2 𝑦1 Γ— 𝑧1

6

slide-7
SLIDE 7

Karatsuba’s Algorithm

𝑨 = 𝑦𝑧 z = 𝑦1𝑧1 2π‘œ + 𝑦1𝑧2 + 𝑦2𝑧1 2

π‘œ 2 + 𝑦2𝑧2

𝑨 = 𝑦1𝑧1 2π‘œ + 𝑦1 βˆ’ 𝑦2 𝑧2 βˆ’ 𝑧1 + 𝑦1𝑧1 + 𝑦2𝑧2 2

π‘œ 2 + 𝑦2𝑧2

π‘ˆ π‘œ = 3π‘ˆ π‘œ 2 + π‘œ π‘ˆ π‘œ = Θ π‘œlg 3

𝑦1 𝑦 𝑦2 𝑧1 𝑧 𝑧2

7

slide-8
SLIDE 8

Example

𝑦 = 34 𝑦1 = 3 𝑦2 = 4 𝑧 = 53 𝑧1 = 5 𝑧2 = 3

π‘œ = 2 𝑨 = 𝑦𝑧 z = 𝑦1𝑧1 102 + ΰ΅« ΰ΅― 𝑦1 βˆ’ 𝑦2 𝑧2 βˆ’ 𝑧1 + 𝑦1𝑧1 + 𝑦2𝑧2 10

2 2 + 𝑦2𝑧2

𝑦 = 5 β‹… 3 100 + βˆ’1 β‹… βˆ’2 + 3 β‹… 5 + 4 β‹… 3 10 + 4 β‹… 3 = 1500 + 2 + 15 + 12 β‹… 10 + 12 = 1802

8

slide-9
SLIDE 9

Matrix Multiplication

Section 4.2

9

slide-10
SLIDE 10

Matrix Multiplication

= . 𝐷 = 𝐡. 𝐢 π‘‘π‘—π‘˜ = σ𝑙=1

𝑙=π‘œ 𝑏𝑗𝑙. π‘π‘™π‘˜, assuming A, B, and C are

square π‘œ Γ— π‘œ matrices

10

slide-11
SLIDE 11

Simple Matrix Multiplication

n n n

π‘ˆ π‘œ = Θ π‘œ3

11

slide-12
SLIDE 12

D&C Matrix Multiplication

𝐡 = 𝐡11 𝐡12 𝐡21 𝐡22 , 𝐢 = 𝐢11 𝐢12 𝐢21 𝐢22 , 𝐷 = 𝐷11 𝐷12 𝐷21 𝐷22 𝐷11 𝐷12 𝐷21 𝐷22 = 𝐡11 𝐡12 𝐡21 𝐡22 . 𝐢11 𝐢12 𝐢21 𝐢22 𝐷11 = 𝐡11. 𝐢11 + 𝐡12. 𝐢21 𝐷12 = 𝐡11. 𝐢12 + 𝐡12. 𝐢22 𝐷21 = 𝐡21. 𝐢11 + 𝐡22. 𝐢21 𝐷22 = 𝐡21. 𝐢12 + 𝐡22. 𝐢22

12

slide-13
SLIDE 13

D&C Matrix Multiplication

13

slide-14
SLIDE 14

Analysis of the D&C Algorithm

π‘ˆ π‘œ = 8π‘ˆ Ξ€ π‘œ 2 + Θ π‘œ2 By applying the Master Theorem 𝑏 = 8, 𝑐 = 2, 𝑔 π‘œ = Θ π‘œ2 π‘œlog𝑐 𝑏 = π‘œ3 𝑔 π‘œ = Θ π‘œ2 = 𝑃 π‘œ3βˆ’1 Case 1 applies π‘ˆ π‘œ = Θ π‘œlog𝑐 𝑏 = Θ π‘œ3

14

slide-15
SLIDE 15

Strassen’s Algorithm

𝐡 = 𝐡11 𝐡12 𝐡21 𝐡22 , 𝐢 = 𝐢11 𝐢12 𝐢21 𝐢22 , 𝐷 = 𝐷11 𝐷12 𝐷21 𝐷22 𝑇1 = 𝐢12 βˆ’ 𝐢22 𝑇6 = 𝐢11 + 𝐢22 𝑇2 = 𝐡11 + 𝐡12 𝑇7 = 𝐡12 βˆ’ 𝐡22 𝑇3 = 𝐡21 + 𝐡22 𝑇8 = 𝐢21 + 𝐢22 𝑇4 = 𝐢21 βˆ’ 𝐢11 𝑇9 = 𝐡11 βˆ’ 𝐡21 𝑇5 = 𝐡11 + 𝐡22 𝑇10 = 𝐢11 + 𝐢12

15

slide-16
SLIDE 16

Strassen’s Algorithm

𝑄

1 = 𝐡11. 𝑇1

𝑄2 = 𝑇2. 𝐢22 𝑄3 = 𝑇3. 𝐢11 𝑄

4 = 𝐡22. 𝑇4

𝑄5 = 𝑇5. 𝑇6 𝑄6 = 𝑇7. 𝑇8 𝑄7 = 𝑇9. 𝑇10 𝐷11 = 𝑄5 + 𝑄

4 βˆ’ 𝑄2 + 𝑄6

𝐷12 = 𝑄

1 + 𝑄2

𝐷21 = 𝑄3 + 𝑄

4

𝐷22 = 𝑄5 + 𝑄

1 βˆ’ 𝑄3 βˆ’ 𝑄7

16

slide-17
SLIDE 17

Strassen’s Algo Sketch

  • 1. Strassen(A, B, n)
  • 2. Split A, B into quadrants
  • 3. Compute S1, …, S10 (Matrix add/sub)
  • 4. Compute P1, …, P7 recursively
  • 5. Compute C1, …, C4 (Matrix add/sub)
  • 6. Return C

π‘ˆ π‘œ = 7π‘ˆ Ξ€ π‘œ 2 + Θ π‘œ2

17

slide-18
SLIDE 18

Analysis of Strassen’s Algorithm

π‘ˆ π‘œ = 7π‘ˆ Ξ€ π‘œ 2 + Θ π‘œ2 By applying the Master Theorem 𝑏 = 7, 𝑐 = 2, 𝑔 π‘œ = Θ π‘œ2 π‘œlog𝑐 𝑏 = π‘œlog2 7 = π‘œ2.80735… β‰… π‘œ2.8 𝑔 π‘œ = Θ π‘œ2 = 𝑃 π‘œlg 7βˆ’0.8 Case 1 applies π‘ˆ π‘œ = Θ π‘œlog𝑐 𝑏 = 𝑃 π‘œ2.81

18

slide-19
SLIDE 19

Linear-time Selection

Section 9.3

19

slide-20
SLIDE 20

Linear-time Selection

Given an array 𝐡 of π‘œ elements and an integer 1 ≀ 𝑙 ≀ π‘œ, find the π‘™π‘’β„Ž smallest element in 𝐡 NaΓ―ve algorithm, sort 𝐡 and pick the π‘™π‘’β„Ž element in the sorted array βž” Θ π‘œπ‘šπ‘• π‘œ Select and remove the smallest element k times βž” Θ π‘œπ‘™ Another quick-sort-like algorithm

Pick the first element (pivot) Place it in its position in the array Recursively process one subarray

20

slide-21
SLIDE 21

Quick-sort-like Algorithm

𝐡

p1

𝑙

21

slide-22
SLIDE 22

Quick-sort-like Algorithm

𝐡

p1

𝑙

p2

22

slide-23
SLIDE 23

Quick-sort-like Algorithm

𝐡

p1

𝑙

p2 p3

23

slide-24
SLIDE 24

Quick-sort-like Algorithm

𝐡

p1

𝑙

p2 p3 p4

24

slide-25
SLIDE 25

Quick-sort-like Algorithm

𝐡

p1

𝑙

p2 p3 p4

βœ“

π‘ˆ π‘œ = Θ π‘œ2

How to choose a good pivot?

25

slide-26
SLIDE 26

Median of Fives

A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45}

26

slide-27
SLIDE 27

Median of Fives

A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45}

  • 1. Partition into groups of 5

94 82 88 12 23 61 11 13 70 37 28 31 64 6 19 32 27 38 35 21 93 22 43 30 67 90 48 42 65 45 50 91 69 57 24

27

slide-28
SLIDE 28

Median of Fives

A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45}

  • 2. Sort each sublist

12 23 82 88 94 11 13 37 61 70 6 19 28 31 64 21 27 32 35 38 22 30 43 67 93 42 45 48 65 90 24 50 57 69 91

28

slide-29
SLIDE 29

Median of Fives

A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45}

  • 3. Find the median of each sublist

12 23 82 88 94 11 13 37 61 70 6 19 28 31 64 21 27 32 35 38 22 30 43 67 93 42 45 48 65 90 24 50 57 69 91 𝑁

29

slide-30
SLIDE 30

Median of Fives

A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45}

  • 4. Recursively find the median of the medians

𝑁 = 82,37,28,32,57,43,48 Median of medians (m*) = 43 Partition A around m* and recursively process

  • ne side

30

slide-31
SLIDE 31

Algorithm Pseudo-code

1.

SELECT(𝐡, π‘œ, 𝑙)

2.

if (π‘œ ≀ 5) then sort 𝐡 and return π‘™π‘’β„Ž element

3.

Partition 𝐡 into groups of 5

4.

𝑁  Find the median of each group

5.

π‘›βˆ— = SELECT(𝑁,

π‘œ 5 , π‘œ 10)

6.

Partition 𝐡 around π‘›βˆ—, let it be at position 𝑗

7.

if (𝑗 = 𝑙) then return π‘›βˆ—

8.

if (𝑗 > 𝑙) then return SELECT(𝐡[1, 𝑗 βˆ’ 1], 𝑗 βˆ’ 1, 𝑙)

9.

if (𝑗 < 𝑙) then return SELECT(𝐡[𝑗 + 1, π‘œ], π‘œ βˆ’ 𝑗, 𝑙 – 𝑗)

31

slide-32
SLIDE 32

π‘›βˆ—

Size of Sublist

32

π‘œ 5 group

𝑁 medians

  • f fives
slide-33
SLIDE 33

Size of Sublist

π‘›βˆ—

33

π‘œ 5 group

𝑁 medians

  • f fives

< <

slide-34
SLIDE 34

Size of Sublist

π‘›βˆ—

34

π‘œ 5 group

𝑁 medians

  • f fives

< < < < < < < < < < < < < < < <

slide-35
SLIDE 35

? ?

Size of Sublist

π‘›βˆ—

A π‘›βˆ—

35

slide-36
SLIDE 36

Size of Sublist

π‘œ 5 groups each of size 5

π‘›βˆ— is larger than half of them π‘›βˆ— is larger than π‘œ

10 groups

π‘›βˆ— is larger than at least 3 elements in each group Similarly, π‘›βˆ— is less than at least 3 elements in each group Size of each of the two sublists is [ 3

10 π‘œ, 7 10 π‘œ]

Worst-case scenario, we prune 3n/10 elements and recursively process 7n/10

36

slide-37
SLIDE 37

Recurrence Relation

π‘ˆ π‘œ = ቐ 𝑏 ; π‘œ ≀ 5 π‘ˆ

π‘œ 5 + π‘ˆ 7π‘œ 10 + π‘π‘œ

; π‘œ > 5 Can we apply the Master theorem? Let’s try recursive tree expansion

37

slide-38
SLIDE 38

Recurrence Tree

π‘ˆ π‘œ

38

slide-39
SLIDE 39

Recurrence Tree

π‘π‘œ π‘ˆ π‘œ 5 π‘ˆ 7π‘œ 10

39

slide-40
SLIDE 40

Recurrence Tree

π‘π‘œ π‘π‘œ 5 7π‘π‘œ 10 π‘ˆ π‘œ 25 π‘ˆ 7π‘œ 50 π‘ˆ 7π‘œ 50 π‘ˆ 49π‘œ 100

40

slide-41
SLIDE 41

Recurrence Tree

π‘π‘œ π‘π‘œ 5 7π‘π‘œ 10 π‘ˆ π‘œ 25 π‘ˆ 7π‘œ 50 π‘ˆ 7π‘œ 50 π‘ˆ 49π‘œ 100 π‘π‘œ 9π‘œ 10 81π‘œ 100

41

slide-42
SLIDE 42

Running Time

π‘ˆ π‘œ = σ𝑗=0

𝑗=𝑒 9 10 𝑗

π‘œ = π‘œ σ𝑗=0

𝑗=𝑒 9 10 𝑗

We do not know the depth 𝑒 of the tree π‘ˆ π‘œ ≀ π‘œ σ𝑗=0

𝑗=∞ 9 10 𝑗

≀ π‘œ

1 1βˆ’ 9

10

≀ 10π‘œ π‘ˆ π‘œ = Θ π‘œ

42

slide-43
SLIDE 43

Proof by Induction

We want to prove that π‘ˆ π‘œ = 𝑃 π‘œ π‘ˆ π‘œ ≀ π‘‘π‘œ, for 𝑑 > 0 and π‘œ β‰₯ π‘œ0 Base case: π‘ˆ π‘œ = 𝑏 for π‘œ ≀ 5 Setting 𝑑 β‰₯ 𝑏 satisfies the base case Assume that π‘ˆ π‘œ ≀ π‘‘π‘œ is true for all π‘œ ≀ 𝑛 We want to prove that it is true for π‘œ = 𝑛 + 1 π‘ˆ 𝑛 + 1 = π‘ˆ

𝑛+1 5

+ π‘ˆ

7 𝑛+1 10

+ 𝑏 𝑛 + 1 π‘ˆ 𝑛 + 1 ≀ 𝑑

𝑛+1 5

+ 𝑑

7 𝑛+1 10

+ 𝑏 𝑛 + 1

43

slide-44
SLIDE 44

Proof by Induction

We want to prove that

𝑑 𝑛 + 1 5 + 𝑑 7 𝑛 + 1 10 + 𝑏 𝑛 + 1 ≀ 𝑑 𝑛 + 1

𝑑 5 + 7𝑑 10 + 𝑏 ≀ 𝑑 𝑑 10 β‰₯ 𝑏

𝑑 β‰₯ 10𝑏 By setting 𝑑 β‰₯ 10𝑏, the inequality π‘ˆ 𝑛 + 1 ≀ 𝑑 𝑛 + 1 will be true π‘ˆ π‘œ = 𝑃 π‘œ

44

slide-45
SLIDE 45

Conclusion

Express problems as divide and conquer algorithms Analyze the running time of divide and conquer algorithms Optimize and improve the worst-case running time of divide and conquer algorithms Reading: Chapter 4, Section 9.3

45