CSC373 Week 2: Greedy Algorithms Nisarg Shah 373F19 - Nisarg Shah - - PowerPoint PPT Presentation
CSC373 Week 2: Greedy Algorithms Nisarg Shah 373F19 - Nisarg Shah 1 Recap Divide & Conquer Master theorem Counting inversions in ( log ) Finding closest pair of points in 2 in log Fast
373F19 - Nisarg Shah 1
373F19 - Nisarg Shah 2
β’ Master theorem β’ Counting inversions in π(π log π) β’ Finding closest pair of points in β2 in π π log π β’ Fast integer multiplication in π πlog2 3 β’ Fast matrix multiplication in π πlog2 7 β’ Finding ππ’β smallest element (in particular, median) in
373F19 - Nisarg Shah 3
β’ We want to find a solution π¦ that maximizes some
β’ But the space of possible solutions π¦ is too large β’ The solution π¦ is typically composed of several parts (e.g.
β’ Instead of directly computing π¦β¦
(this needs to be defined carefully for each problem)
373F19 - Nisarg Shah 4
β’ Job π starts at time π‘
π and finishes at time π π
β’ Two jobs are compatible if they donβt overlap β’ Goal: find maximum-size subset of mutually compatible jobs
373F19 - Nisarg Shah 5
β’ Consider jobs in some βnaturalβ order β’ Take each job if itβs compatible with the ones already
β’ Earliest start time: ascending order of π‘
π
β’ Earliest finish time: ascending order of π
π
β’ Shortest interval: ascending order of π
π β π‘ π
β’ Fewest conflicts: ascending order of π
π, where π π is the
373F19 - Nisarg Shah 6
π
π
π β π‘ π
π, where π π is the number of
remaining jobs that conflict with π
373F19 - Nisarg Shah 7
373F19 - Nisarg Shah 8
β’ Sort jobs by finish time. Say π
1 β€ π 2 β€ β― β€ π π
β’ When deciding whether job π should be included, we
π β₯ π πβ, where πβ is the last added job
π β€ π πβ
added job
β’ Running time: π π log π
373F19 - Nisarg Shah 9
β’ Suppose for contradiction that greedy is not optimal β’ Say greedy selects jobs π1, π2, β¦ , ππ sorted by finish time β’ Consider the optimal solution π1, π2, β¦ , ππ (also sorted by
373F19 - Nisarg Shah 10
β’ Both ππ +1 and ππ +1 were compatible with the previous
β’ Consider the solution π1, π2, β¦ , ππ , ππ +1, ππ +2, β¦ , ππ
ππ +1 β€ π ππ +1)
Another standard method is induction
373F19 - Nisarg Shah 11
β’ Job π starts at time π‘
π and finishes at time π π
β’ Two jobs are compatible if they donβt overlap β’ Goal: group jobs into fewest partitions such that jobs in
β’ Find the maximum compatible set using the previous
β’ Doesnβt work (check by yourselves)
373F19 - Nisarg Shah 12
373F19 - Nisarg Shah 13
373F19 - Nisarg Shah 14
β’ Go through lectures in some βnaturalβ order β’ Assign each lecture to a compatible classroom (which?),
β’ Earliest start time: ascending order of π‘
π
β’ Earliest finish time: ascending order of π
π
β’ Shortest interval: ascending order of π
π β π‘ π
β’ Fewest conflicts: ascending order of π
π, where π π is the
373F19 - Nisarg Shah 15
373F19 - Nisarg Shah 16
373F19 - Nisarg Shah 17
β’ Key step: check if the next lecture can be scheduled at
β’ Store classrooms in a priority queue
β’ Is lecture π compatible with some classroom?
π at least as large as the minimum key?β
increase its key to π
π
π
β’ π(π) priority queue operations, π(π log π) time
373F19 - Nisarg Shah 18
β’ # classrooms needed β₯ maximum βdepthβ at any point
β’ We now show that our greedy algorithm uses only these
373F19 - Nisarg Shah 19
β’ Let π = # classrooms used by greedy β’ Classroom π was opened because there was a schedule π
β’ All these π lectures end after π‘
π
β’ Since we sorted by start time, they all start at/before π‘
π
β’ So at time π‘
π, we have π overlapping lectures
β’ Hence, depth β₯ π β’ So all schedules use β₯ π classrooms. β’ QED!
373F19 - Nisarg Shah 20
β’ Graph π» = (π, πΉ) β’ Vertices π = jobs/lectures β’ Edge π, π β πΉ if jobs π and π are incompatible
373F19 - Nisarg Shah 21
β’ Interval graphs = graphs which can be obtained from
β’ In fact, this holds even when we are not given an interval
β’ Yes! Chordal graphs
373F19 - Nisarg Shah 22
β’ We have a single machine β’ Each job π requires π’π units of time and is due by time ππ β’ If itβs scheduled to start at π‘
π, it will finish at π π = π‘ π + π’π
β’ Lateness: βπ = max 0, π
π β ππ
β’ Goal: minimize the maximum lateness, π = max
π
β’ We can decide the start time β’ All jobs must be scheduled on a single machine
373F19 - Nisarg Shah 23
373F19 - Nisarg Shah 24
β’ Consider jobs one-by-one in some βnaturalβ order β’ Schedule jobs in this order (nothing special to do here,
β’ Shortest processing time first: ascending order of
β’ Earliest deadline first: ascending order of due time ππ β’ Smallest slack first: ascending order of ππ β π’π
373F19 - Nisarg Shah 25
β’ Shortest processing time first
β’ Smallest slack first
373F19 - Nisarg Shah 26
373F19 - Nisarg Shah 27
β’ There is an optimal schedule with no idle time
373F19 - Nisarg Shah 28
β’ Earliest deadline first has no idle time
β’ π, π such that ππ < ππ but π is scheduled before π
β’ By definition, earliest deadline first has no inversions
β’ If a schedule with no idle time has an inversion, it has a
373F19 - Nisarg Shah 29
β’ Swapping adjacently scheduled inverted jobs doesnβt
β’ Let β and ββ² denote lateness before/after swap β’ Clearly, βπ = βπ
β² for all π β π, π
β’ Also, clearly, βπ
β² β€ βπ
373F19 - Nisarg Shah 30
β’ Swapping adjacently scheduled inverted jobs doesnβt
β’ βπ
β² = π π β² β ππ = π π β ππ β€ π π β ππ = βπ
β’ πβ² = max βπ
β², βπ β², max πβ π,π βπ β²
πβ π,π βπ β€ π
373F19 - Nisarg Shah 31
β’ Suppose for contradiction that itβs not optimal β’ Consider an optimal schedule πβ which has fewest inversions
stays optimal) and reduces the number of inversions by 1
schedules.
373F19 - Nisarg Shah 32
β’ We have a document that is written using π distinct labels β’ NaΓ―ve encoding: represent each label using π = log π bits β’ If the document has length π, this uses π log π bits β’ Say for English documents with no punctuations etc, we
373F19 - Nisarg Shah 33
β’ What if π, π, π , π‘ are much more frequent in the
β’ Can we assign shorter codes to more frequent letters?
β’ π = 0, π = 1, π = 01, β¦ β’ See a problem?
373F19 - Nisarg Shah 34
β’ Map each label π¦ to a bit-string π(π¦) such that for all
β’ Then itβs impossible to have a scenario like this
β’ So we can read left to right, find the first point where it
π(π¦) π(π§)
373F19 - Nisarg Shah 35
β’ Given π symbols and their frequencies (π₯1, β¦ , π₯π), find a
π
π
π₯π β βπ is the length of the compressed document
β’ (π₯π, π₯π, π₯π, π₯π, π₯π, π₯π) = (42,20,5,10,11,12) β’ No need to remember the numbers βΊ
373F19 - Nisarg Shah 36
373F19 - Nisarg Shah 37
β’ Build a priority queue by adding π¦, π₯π¦ for each symbol π¦ β’ While |queue|β₯ 2
373F19 - Nisarg Shah 38
373F19 - Nisarg Shah 39
373F19 - Nisarg Shah 40
373F19 - Nisarg Shah 41
373F19 - Nisarg Shah 42
373F19 - Nisarg Shah 43
373F19 - Nisarg Shah 44
β’ π(π log π) β’ Can be made π(π) if the labels are given to you sorted by
β’ Induction on the number of symbols π β’ Base case: For π = 2, there are only two possible
β’ Hypothesis: Assume it returns an optimal encoding with
373F19 - Nisarg Shah 45
β’ Consider the case of π symbols β’ Lemma 1: If π₯π¦ < π₯π§, then βπ¦ β₯ βπ§ in any optimal tree.
the overall length (exercise!).
β’ Lemma 2: There is an optimal tree π in which the two
length assigned to them. Then, if theyβre not siblings, chop and rearrange the tree to make them siblings (exercise!).
β’ Now, we can compare the tree πΌ produced by Huffman
373F19 - Nisarg Shah 46
β’ Let π¦ and π§ be the two least frequency symbols β’ In Huffman, we combine them in the first step into βxyβ β’ Let πΌβ² and πβ² be trees obtained from πΌ and π by treating
β’ Use induction hypothesis: πππππ’β πΌβ² β€ πππππ’β(πβ²) β’ πππππ’β πΌ = πππππ’β πΌβ² + π₯π¦ + π₯π§ β 1 β’ πππππ’β π = πππππ’β πβ² + π₯π¦ + π₯π§ β 1 β’ QED!
373F19 - Nisarg Shah 47
β’ Dijkstraβs shortest path algorithm β’ Kruskal and Primβs minimum spanning tree algorithms