General remarks Algorithms Algorithms Oliver Oliver Kullmann - PowerPoint PPT Presentation

CS 270 CS 270 General remarks Algorithms Algorithms Oliver Oliver Kullmann Kullmann Week 2 Growth of Growth of Functions Functions Divide-and- First we consider an important tool for the analysis of Divide-and- Divide and Conquer Conquer Conquer algorithms: Big-Oh . Min-Max- Min-Max- Problem Problem Then we introduce an important algorithmic paradigm: Tutorial Tutorial Divide-and-Conquer . Growth of Functions 1 We conclude by presenting and analysing a simple example. Divide-and-Conquer Reading from CLRS for week 2 2 Chapter 2, Section 3 Min-Max-Problem Chapter 3 Tutorial 3 CS 270 CS 270 Growth of Functions O -Notation Algorithms Algorithms Oliver Oliver Kullmann Kullmann � � g ( n ) is the set of all functions f ( n ) for which there are O positive constants c and n 0 such that Growth of Growth of A way to describe behaviour of functions in the limit. We Functions Functions Divide-and- Divide-and- are studying asymptotic efficiency. f ( n ) ≤ cg ( n ) for all n ≥ n 0 . Conquer Conquer Min-Max- Min-Max- Describe growth of functions. Problem Problem Tutorial Tutorial Focus on what’s important by abstracting away low-order cg ( n ) terms and constant factors. f ( n ) How we indicate running times of algorithms. A way to compare “sizes” of functions: O corresponds to ≤ n n 0 Ω corresponds to ≥ Θ corresponds to = g ( n ) is an asymptotic upper bound for f ( n ). ≥ 0 . We consider only functions f , g : N → R If f ( n ) ∈ O ( g ( n )), we write f ( n ) = O ( g ( n )) (we will precisely explain this soon)

CS 270 CS 270 O -Notation Examples Ω-Notation Algorithms Algorithms Oliver Oliver Kullmann Kullmann 2 n 2 = O ( n 3 ), with c = 1 and n 0 = 2. � � Ω g ( n ) is the set of all functions f ( n ) for which there are Growth of Growth of Functions Functions Example of functions in O ( n 2 ): positive constants c and n 0 such that Divide-and- Divide-and- Conquer Conquer n 2 Min-Max- Min-Max- f ( n ) ≥ cg ( n ) for all n ≥ n 0 . Problem Problem n 2 + n Tutorial Tutorial n 2 + 1000 n 1000 n 2 + 1000 n f ( n ) cg ( n ) Also n n n 0 n / 1000 n 1 . 999999 g ( n ) is an asymptotic lower bound for f ( n ). n 2 / lg lg lg n CS 270 CS 270 Ω-Notation Examples Θ-Notation Algorithms Algorithms √ n = Ω(lg n ), with c = 1 and n 0 = 16. Oliver Oliver Kullmann Kullmann � � Θ g ( n ) is the set of all functions f ( n ) for which there are Growth of Growth of Example of functions in Ω( n 2 ): Functions Functions positive constants c 1 , c 2 and n 0 such that Divide-and- Divide-and- n 2 Conquer Conquer Min-Max- Min-Max- c 1 g ( n ) ≤ f ( n ) ≤ c 2 g ( n ) for all n ≥ n 0 . n 2 + n Problem Problem Tutorial Tutorial n 2 − n 1000 n 2 + 1000 n c 2 g ( n ) 1000 n 2 − 1000 n f ( n ) c 1 g ( n ) Also n n 3 n 0 n 2 . 0000001 n 2 lg lg lg n g ( n ) is an asymptotic tight bound for f ( n ). 2 2 n

CS 270 CS 270 Θ-Notation (cont’d) Asymptotic notation in equations Algorithms Algorithms Oliver Oliver Kullmann Kullmann Growth of When on right-hand side Growth of Functions Functions Θ( n 2 ) stands for some anonymous function in the set Θ( n 2 ). Divide-and- Divide-and- Examples 1 Conquer Conquer 2 n 2 + 3 n + 1 = 2 n 2 + Θ( n ) means 2 n 2 + 3 n + 1 = 2 n 2 + f ( n ) Min-Max- Min-Max- n 2 / 2 − 2 n = Θ( n 2 ), with c 1 = 1 4 , c 2 = 1 Problem Problem 2 , and n 0 = 8. for some f ( n ) ∈ Θ( n ). In particular, f ( n ) = 3 n + 1. Tutorial Tutorial Theorem 2 When on left-hand side f ( n ) = Θ( g ( n )) if and only if f ( n ) = O ( g ( n )) and No matter how the anonymous functions are chosen on the f ( n ) = Ω( g ( n )) . left-hand side, there is a way to choose the anonymous functions on the right-hand side to make the equation valid. Interpret 2 n 2 + Θ( n ) = Θ( n 2 ) as meaning for all functions Leading constants and lower order terms do not matter. f ( n ) ∈ Θ( n ), there exists a function g ( n ) ∈ Θ( n 2 ) such that 2 n 2 + f ( n ) = g ( n ). CS 270 CS 270 Asymptotic notation chained together Example Analysis Algorithms Algorithms Oliver Oliver Kullmann Kullmann Insertion-Sort ( A ) 2 n 2 + 3 n + 1 = 2 n 2 + Θ( n ) = Θ( n 2 ) Growth of Growth of 1 for j = 2 to A . length Functions Functions Divide-and- 2 key = A [ j ] Divide-and- Interpretation: Conquer Conquer 3 / / Insert A [ j ] into sorted sequence A [1 . . j − 1]. Min-Max- Min-Max- Problem Problem First equation: There exists f ( n ) ∈ Θ( n ) such that 4 i = j − 1 Tutorial Tutorial 2 n 2 + 3 n + 1 = 2 n 2 + f ( n ). 5 while i > 0 and A [ i ] > key 6 A [ i +1] = A [ i ] Second equation: For all g ( n ) ∈ Θ( n ) (such as the f ( n ) 7 i = i − 1 used to make the first equation hold), there exists h ( n ) ∈ Θ( n 2 ) such that 2 n 2 + g ( n ) = h ( n ). 8 A [ i +1] = key Note The for -loop on line 1 is executed O ( n ) times; and each What has been said of “Θ” on this and the previous slide also statement costs constant time, except for the while -loop on applies to “ O ” and “Ω”. lines 5-7 which costs O ( n ). O ( n 2 ) . Thus overall runtime is: O ( n ) × O ( n ) = Note: In fact, as seen last week, worst-case runtime is Θ( n 2 ).

CS 270 CS 270 Divide-and-Conquer Approach Naive Min-Max Algorithms Algorithms Find minimum and maximum of a list A of n > 0 numbers. Oliver Oliver Kullmann Kullmann There are many ways to design algorithms. Growth of Growth of Naive-Min-Max ( A ) Functions Functions For example, insertion sort is incremental: having sorted Divide-and- Divide-and- 1 least = A [1] A [1 . . j − 1], place A [ j ] correctly, so that A [1 . . j ] is sorted. Conquer Conquer Min-Max- Min-Max- 2 for i = 2 to A . length Problem Problem 3 if A [ i ] < least Tutorial Tutorial Divide-and-Conquer is another common approach: 4 least = A [ i ] 5 greatest = A [1] Divide the problem into a number of subproblems that are 6 for i = 2 to A . length smaller instances of the same problem. 7 if A [ i ] > greatest Conquer the subproblems by solving them recursively. 8 greatest = A [ i ] Base case: If the subproblem are small enough, just solve 9 return ( least , greatest ) them by brute force. Combine the subproblem solutions to give a solution to the The for -loop on line 2 makes n − 1 comparisons, as does the original problem. for -loop on line 6, making a total of 2 n − 2 comparisons. Can we do better? Yes! CS 270 CS 270 Divide-and-Conquer Min-Max Divide-and-Conquer Min-Max Algorithm Algorithms Algorithms Initially called with Min-Max ( A , 1 , A . length ). Oliver Oliver Kullmann Kullmann As we are dealing with subproblems, we state each subproblem Growth of Min-Max ( A , p , q ) Growth of Functions Functions as computing minimum and maximum of a subarray A [ p . . q ]. 1 if p = q Divide-and- Divide-and- Initially, p = 1 and q = A . length , but these values change as we Conquer 2 return ( A [ p ] , A [ q ]) Conquer Min-Max- Min-Max- recurse through subproblems. 3 if p = q − 1 Problem Problem Tutorial 4 if A [ p ] < A [ q ] Tutorial To compute minimum and maximum of A [ p . . q ]: 5 return ( A [ p ] , A [ q ]) 6 else return ( A [ q ] , A [ p ]) Divide by splitting into two subarrays A [ p . . r ] and A [ r +1 . . q ], 7 r = ⌊ ( p + q ) / 2 ⌋ where r is the halfway point of A [ p . . q ]. 8 ( min 1 , max 1) = Min-Max ( A , p , r ) 9 ( min 2 , max 2) = Min-Max ( A , r +1 , q ) Conquer by recursively computing minimum and maximum of � � 10 min( min 1 , min 2) , max( max 1 , max 2) return the two subarrays A [ p . . r ] and A [ r +1 . . q ]. Combine by computing the overall minimum as the min of the two recursively computed minima, similar for the overall Note maximum. In line 7, r computes the halfway point of A [ p . . q ]. n = q − p + 1 is the number of elements from which we compute the min and max.