CS141: Intermediate Data Structures and Algorithms Greedy - - PowerPoint PPT Presentation

β–Ά
cs141 intermediate data structures and algorithms greedy
SMART_READER_LITE
LIVE PREVIEW

CS141: Intermediate Data Structures and Algorithms Greedy - - PowerPoint PPT Presentation

Jan 18, 2023 380 likes β€’1.02k views

CS141: Intermediate Data Structures and Algorithms Greedy Algorithms Amr Magdy Activity Selection Problem Given a set of activities = { 1 , 2 , , } where each activity has a start time and a finish

slide-1
SLIDE 1

CS141: Intermediate Data Structures and Algorithms Greedy Algorithms

Amr Magdy

slide-2
SLIDE 2

Activity Selection Problem

Given a set of activities 𝑇 = {𝑏1, 𝑏2, … , π‘π‘œ} where each activity 𝑗 has a start time 𝑑𝑗 and a finish time 𝑔𝑗 , where 0 ≀ 𝑑𝑗 < 𝑔𝑗 < ∞. An activity 𝑏𝑗 happens in the half-open time interval [𝑑𝑗, 𝑔𝑗).

2

slide-3
SLIDE 3

Activity Selection Problem

Given a set of activities 𝑇 = {𝑏1, 𝑏2, … , π‘π‘œ} where each activity 𝑗 has a start time 𝑑𝑗 and a finish time 𝑔𝑗 , where 0 ≀ 𝑑𝑗 < 𝑔𝑗 < ∞. An activity 𝑏𝑗 happens in the half-open time interval [𝑑𝑗, 𝑔𝑗). Activities compete on a single resource, e.g., CPU

3

slide-4
SLIDE 4

Activity Selection Problem

Given a set of activities 𝑇 = {𝑏1, 𝑏2, … , π‘π‘œ} where each activity 𝑗 has a start time 𝑑𝑗 and a finish time 𝑔𝑗 , where 0 ≀ 𝑑𝑗 < 𝑔𝑗 < ∞. An activity 𝑏𝑗 happens in the half-open time interval [𝑑𝑗, 𝑔𝑗). Activities compete on a single resource, e.g., CPU Two activities are said to be compatible if they do not

  • verlap.

4

slide-5
SLIDE 5

Activity Selection Problem

Given a set of activities 𝑇 = {𝑏1, 𝑏2, … , π‘π‘œ} where each activity 𝑗 has a start time 𝑑𝑗 and a finish time 𝑔𝑗 , where 0 ≀ 𝑑𝑗 < 𝑔𝑗 < ∞. An activity 𝑏𝑗 happens in the half-open time interval [𝑑𝑗, 𝑔𝑗). Activities compete on a single resource, e.g., CPU Two activities are said to be compatible if they do not

  • verlap.

The problem is to find a maximum-size compatible subset, i.e., a one with the maximum number of activities.

5

slide-6
SLIDE 6

Example

6

slide-7
SLIDE 7

A Compatible Set

7

slide-8
SLIDE 8

A Better Compatible Set

8

slide-9
SLIDE 9

An Optimal Solution

9

slide-10
SLIDE 10

Another Optimal Solution

10

slide-11
SLIDE 11

Activity Selection Problem

Solution algorithm?

Brute force (naΓ―ve): all possible combinations οƒ  O(2n) Can we do better? Divide line for D&C is not clear

11

slide-12
SLIDE 12

Activity Selection Problem

Solution algorithm?

Brute force (naΓ―ve): all possible combinations οƒ  O(2n) Can we do better? Divide line for D&C is not clear

Does the problem have optimal substructure?

i.e., the optimal solution of a bigger problem has optimal solutions for subproblems

12

slide-13
SLIDE 13

Activity Selection Problem

Does the problem have optimal substructure?

i.e., the optimal solution of a bigger problem has optimal solutions for subproblems

Assume A is an optimal solution for S

Is A’ = A-{ai} an optimal solution for S’ = S-{ai and its incompatible activities}? If A’ is not an optimal solution, then there an optimal solution A’’ for S’ so that |A’’| > |A’| Then B=A’’ U {ai} is a solution for S, |B|=|A’’|+1, |A|=|A’|+1 Then |B| > |A|, i.e., |A| is not an optimal solution, contradiction Then A’ must be an optimal solution for S’

13

slide-14
SLIDE 14

Activity Selection Problem

Does the problem have optimal substructure?

i.e., the optimal solution of a bigger problem has optimal solutions for subproblems

Assume A is an optimal solution for S

Is A’ = A-{ai} an optimal solution for S’ = S-{ai and its incompatible activities}? If A’ is not an optimal solution, then there an optimal solution A’’ for S’ so that |A’’| > |A’| Then B=A’’ U {ai} is a solution for S, |B|=|A’’|+1, |A|=|A’|+1 Then |B| > |A|, i.e., |A| is not an optimal solution, contradiction Then A’ must be an optimal solution for S’

Proof by contradiction

Assume the opposite of your goal Given that prove a contradiction, then your goal is proved

14

slide-15
SLIDE 15

Activity Selection Problem

What does having optimal substructure means?

We can solve smaller problems, then expand to larger Similar to dynamic programming

15

slide-16
SLIDE 16

Activity Selection Problem

What does having optimal substructure means?

We can solve smaller problems, then expand to larger Similar to dynamic programming

Instead, can we a greedy choice?

i.e., take the best choice so far, reduce the problem size, and solve a subproblem later

16

slide-17
SLIDE 17

Activity Selection Problem

What does having optimal substructure means?

We can solve smaller problems, then expand to larger Similar to dynamic programming

Instead, can we a greedy choice?

i.e., take the best choice so far, reduce the problem size, and solve a subproblem later

Greedy choices

Longest first Shortest first Earliest start first Earliest finish first …?

17

slide-18
SLIDE 18

Activity Selection Problem

Greedy choice: earliest finish first

Why? It leaves as much resource as possible for other tasks

18

slide-19
SLIDE 19

Activity Selection Problem

Greedy choice: earliest finish first

Why? It leaves as much resource as possible for other tasks

Solution:

Include earliest finish activity am in solution A Remove all am’s incompatible activities Repeat for the remaining earliest finish activity

19

slide-20
SLIDE 20

Activity Selection Problem: Greedy Solution

20

slide-21
SLIDE 21

Activity Selection Problem: Greedy Solution

21

slide-22
SLIDE 22

Activity Selection Problem: Greedy Solution

22

slide-23
SLIDE 23

Activity Selection Problem: Greedy Solution

23

slide-24
SLIDE 24

Activity Selection Problem: Greedy Solution

24

slide-25
SLIDE 25

Activity Selection Problem: Greedy Solution

25

slide-26
SLIDE 26

Activity Selection Problem: Greedy Solution

26

slide-27
SLIDE 27

Activity Selection Problem: Greedy Solution

27

slide-28
SLIDE 28

Activity Selection Problem

Pseudo code?

28

slide-29
SLIDE 29

Activity Selection Problem

Pseudo code? findMaxSet(Array a, int n) {

  • Sort β€œa” based on earliest finish time
  • result οƒŸ {}
  • for i = 1 to n

validAi = true for j = 1 to result.size if (a[i] is incompatible with result[j]) validAi = false if (validAi) result οƒŸ result U a[i]

  • return result

}

29

slide-30
SLIDE 30

Activity Selection Problem

Is greedy choice is enough to get optimal solution?

30

slide-31
SLIDE 31

Activity Selection Problem

Is greedy choice is enough to get optimal solution? Greedy choice property

Prove that if am has the earliest finish time, it must be included in some optimal solution.

31

slide-32
SLIDE 32

Activity Selection Problem

Is greedy choice is enough to get optimal solution? Greedy choice property

Prove that if am has the earliest finish time, it must be included in some optimal solution.

Assume a set S and a solution set A, where am βˆ‰ A

Let aj is the activity with the earliest finish time in A (not in S) Compose another set A’ = A – {aj} U {am} A’ still have all activities disjoint (as am has the global earliest finish time and A activities are already disjoint), and |A’|=|A| Then A’ is an optimal solution Then am is always included in an optimal solution

32

slide-33
SLIDE 33

Elements of a Greedy Algorithm

1.

Optimal Substructure

2.

Greedy Choice Property

33

slide-34
SLIDE 34

Greedy vs. Dynamic Programming

Solving the bigger problem include One choice (greedy) vs Multiple possible choices

34

slide-35
SLIDE 35

Greedy vs. Dynamic Programming

Solving the bigger problem include One choice (greedy) vs Multiple possible choices One subproblem A lot of overlapping subproblems

35

slide-36
SLIDE 36

Greedy vs. Dynamic Programming

Solving the bigger problem include One choice (greedy) vs Multiple possible choices One subproblem A lot of overlapping subproblems Both have optimal substructure

36

slide-37
SLIDE 37

Greedy vs. Dynamic Programming

Solving the bigger problem include One choice (greedy) vs Multiple possible choices One subproblem A lot of overlapping subproblems Both have optimal substructure Elements:

37

Greedy DM Optimal substructure Optimal substructure Greedy choice property Overlapping subproblems

slide-38
SLIDE 38

Knapsack Problem

38

45

slide-39
SLIDE 39

Knapsack Problem

39

0-1 Knapsack: Each item either included or not Greedy choices:

Take the most valuable οƒ  Does not lead to optimal solution Take the most valuable per unit οƒ  Works in this example

45

slide-40
SLIDE 40

Knapsack Problem

40

0-1 Knapsack: Each item either included or not Greedy choices:

Take the most valuable οƒ  Does not lead to optimal solution Take the most valuable per unit οƒ  Does not work

30

slide-41
SLIDE 41

Knapsack Problem

41

Fractional Knapsack: Part of items can be included

30

slide-42
SLIDE 42

Knapsack Problem

42

Fractional Knapsack: Part of items can be included Greedy choices:

Take the most valuable οƒ  Does not lead to optimal solution Take the most valuable per unit οƒ  Does work

30

slide-43
SLIDE 43

Fractional Knapsack Problem

Greedy choice property: take the most valuable per weight unit

43

slide-44
SLIDE 44

Fractional Knapsack Problem

Greedy choice property: take the most valuable per weight unit Proof of optimality:

Given the set 𝑇 ordered by the value-per-weight, taking as much as possible π‘¦π‘˜ from the item π‘˜ with the highest value-per-weight will lead to an optimal solution π‘Œ Assume we have another optimal solution π‘Œ` where we take less amount of item π‘˜, say π‘¦π‘˜` < π‘¦π‘˜ . Since π‘¦π‘˜` < π‘¦π‘˜, there must be another item 𝑙 which was taken with a higher amount in π‘Œ`, i.e., 𝑦𝑙` > 𝑦𝑙. We create another solution π‘Œ`` by doing the following changes in π‘Œ` Reduce the amount of item 𝑙 by a value 𝑨 and increase the amount of item π‘˜ by a value 𝑨 The value of the new solution π‘Š`` = π‘Š` + 𝑨 π‘€π‘˜/π‘₯π‘˜ βˆ’ 𝑨 𝑀𝑙/π‘₯𝑙 = π‘Š` + 𝑨 (π‘€π‘˜/π‘₯π‘˜βˆ’π‘€π‘™/π‘₯𝑙) οƒ  π‘€π‘˜/π‘₯π‘˜βˆ’π‘€π‘™/π‘₯𝑙 β‰₯ 0 οƒ  π‘Š`` β‰₯ π‘Š`

44

slide-45
SLIDE 45

Fractional Knapsack Problem

Optimal substructure

45

slide-46
SLIDE 46

Fractional Knapsack Problem

Optimal substructure Given the problem 𝑇 with an optimal solution π‘Œ with value π‘Š, we want to prove that the solution π‘Œ` = π‘Œ βˆ’ π‘¦π‘˜ is

  • ptimal to the problem 𝑇` = 𝑇 - {π‘˜} and the knapsack

capacity 𝑋` = 𝑋 βˆ’ π‘¦π‘˜ Proof by contradiction

Assume that π‘Œ` is not optimal to 𝑇` There is another solution π‘Œ`` to 𝑇` that has a higher total value π‘Š`` > π‘Š` Then π‘Œ`` U {π‘¦π‘˜} is a solution to 𝑇 with value π‘Š``+ π‘¦π‘˜> π‘Š`+ π‘¦π‘˜ > π‘Š Contradiction as π‘Š is the optimal value

46

slide-47
SLIDE 47

Fractional Knapsack Problem

Fknapsack (W, S, v’s, w’s) {

  • Sort S based on vi/wi value
  • rw = W
  • result = { }
  • for each si in S

if(wi <= rw) result = result U si rw = rw-wi else result = result U rw/wi * si rw = 0

  • return result

}

47

slide-48
SLIDE 48

Huffman Codes

48

slide-49
SLIDE 49

Huffman Codes

Prefix Codes: No code is allowed to be a prefix of another code

Prefix codes give optimal data compression

49

slide-50
SLIDE 50

Huffman Codes

Prefix Codes: No code is allowed to be a prefix of another code

Prefix codes give optimal data compression

Example: Message β€˜JAVA’ a = β€œ0”, j = β€œ11”, v = β€œ10” Encoded message β€œ110100” Decoding β€œ110100”

50

slide-51
SLIDE 51

Huffman Codes

Prefix Codes: No code is allowed to be a prefix of another code

Prefix codes give optimal data compression

Example: Message β€˜JAVA’ a = β€œ0”, j = β€œ11”, v = β€œ10” Encoded message β€œ110100” Decoding β€œ110100” In the table: Encoding with fixed-length needs 300K bits Encoding with variable-length needs 224K bits

51

slide-52
SLIDE 52

Huffman Codes

Fixed-length tree Variable-length tree

52

slide-53
SLIDE 53

Huffman Codes

Fixed-length tree Variable-length tree We need an algorithm to build the optimal variable-length tree

53

slide-54
SLIDE 54

Huffman Codes: Tree Construction

54

slide-55
SLIDE 55

Huffman Codes: Tree Construction

55

slide-56
SLIDE 56

Huffman Codes: Tree Construction

56

slide-57
SLIDE 57

Huffman Codes: Tree Construction

57

slide-58
SLIDE 58

Huffman Codes: Tree Construction

58

slide-59
SLIDE 59

Huffman Codes: Tree Construction

59

slide-60
SLIDE 60

Huffman Codes: Tree Construction

60

slide-61
SLIDE 61

Huffman Codes

Details of optimal substructure and greedy choice property in the text book

61

slide-62
SLIDE 62

Book Readings and Credits

Book Readings:

16.1 – 16.3

Credits to:

  • Prof. Ahmed Eldawy notes

62