introduction to randomized algorithms quicksort
play

Introduction to Randomized Algorithms: QuickSort Lecture 2 January - PowerPoint PPT Presentation

Sep 18, 2023 •380 likes •1.51k views

CS 498ABD: Algorithms for Big Data, Spring 2019 Introduction to Randomized Algorithms: QuickSort Lecture 2 January 17, 2019 Chandra (UIUC) CS498ABD 1 Spring 2019 1 / 51 Outline Our goal Basics of randomization probability space,

  1. Union bound The probability of the union of two events, is no bigger than the probability of the sum of their probabilities. Lemma For any two events E and F , we have that � � � � � � Pr E ∪ F ≤ Pr E + Pr F . Proof. Consider E and F to be a collection of elmentery events (which they are). We have � � � Pr E ∪ F = Pr[ x ] x ∈ E ∪ F � � � � � � ≤ Pr[ x ] + Pr[ x ] = Pr E + Pr F . x ∈ E x ∈ F Chandra (UIUC) CS498ABD 15 Spring 2019 15 / 51

  2. Random Variables Definition Given a probability space (Ω , Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω → R . Chandra (UIUC) CS498ABD 16 Spring 2019 16 / 51

  3. Random Variables Definition Given a probability space (Ω , Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω → R . Definition Expectation For a random variable X over a probability space (Ω , Pr) the expectation of X is defined as � ω ∈ Ω Pr[ ω ] X ( ω ) . In other words, the expectation is the average value of X according to the probabilities given by Pr[ · ] . Chandra (UIUC) CS498ABD 16 Spring 2019 16 / 51

  4. Expectation Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . X : Ω → R where X ( i ) = i mod 2 . Then 1 E[ X ] = � 6 � 6 i =1 Pr[ i ] · X ( i ) = 1 i =1 X ( i ) = 1 / 2 . 6 Chandra (UIUC) CS498ABD 17 Spring 2019 17 / 51

  5. Expectation Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . X : Ω → R where X ( i ) = i mod 2 . Then 1 E[ X ] = � 6 � 6 i =1 Pr[ i ] · X ( i ) = 1 i =1 X ( i ) = 1 / 2 . 6 Y : Ω → R where Y ( i ) = i 2 . Then 2 6 · i 2 = 91 / 6 . E[ Y ] = � 6 1 i =1 Chandra (UIUC) CS498ABD 17 Spring 2019 17 / 51

  6. Expected number of vertices? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . Compute the expected number of vertices in H. (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Chandra (UIUC) CS498ABD 18 Spring 2019 18 / 51

  7. Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

  8. Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # vertices in H as per ω = # 1s in ω . Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

  9. Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # vertices in H as per ω = # 1s in ω . � E[ X ] = ω ∈ Ω Pr[ ω ] X ( ω ) = � ω ∈ Ω 1 / 2 n X ( ω ) 1 / 2 n � n � n � = k k =0 k 1 / 2 n (2 n n = 2 ) = n / 2 Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

  10. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Chandra (UIUC) CS498ABD 20 Spring 2019 20 / 51

  11. Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

  12. Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # edges present in H as per ω = ?? Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

  13. Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # edges present in H as per ω = ?? How to compute E[ X ] ? Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

  14. Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

  15. Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Special type of random variables that are quite useful. Definition Given a probability space (Ω , Pr) and an event A ⊆ Ω the indicator random variable X A is a binary random variable where X A ( ω ) = 1 if ω ∈ A and X A ( ω ) = 0 if ω �∈ A . Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

  16. Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Special type of random variables that are quite useful. Definition Given a probability space (Ω , Pr) and an event A ⊆ Ω the indicator random variable X A is a binary random variable where X A ( ω ) = 1 if ω ∈ A and X A ( ω ) = 0 if ω �∈ A . Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . Let A be the even that i is divisible by 3 . Then X A ( i ) = 1 if i = 3 , 6 and 0 otherwise. Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

  17. Expectation of indicator random variable Proposition For an indicator variable X A , E[ X A ] = Pr[ A ] . Proof. � E[ X A ] = X A ( ω ) Pr[ ω ] ω ∈ Ω � � = 1 · Pr[ ω ] + 0 · Pr[ ω ] ω ∈ A ω ∈ Ω \ A � = Pr[ ω ] ω ∈ A = Pr[ A ] . Chandra (UIUC) CS498ABD 23 Spring 2019 23 / 51

  18. Linearity of Expectation Lemma Let X , Y be two random variables (not necessarily independent) over a probability space (Ω , Pr) . Then E[ X + Y ] = E[ X ] + E[ Y ] . Proof. � E[ X + Y ] = Pr[ ω ] ( X ( ω ) + Y ( ω )) ω ∈ Ω � � = Pr[ ω ] X ( ω ) + Pr[ ω ] Y ( ω ) = E[ X ] + E[ Y ] . ω ∈ Ω ω ∈ Ω Chandra (UIUC) CS498ABD 24 Spring 2019 24 / 51

  19. Linearity of Expectation Lemma Let X , Y be two random variables (not necessarily independent) over a probability space (Ω , Pr) . Then E[ X + Y ] = E[ X ] + E[ Y ] . Proof. � E[ X + Y ] = Pr[ ω ] ( X ( ω ) + Y ( ω )) ω ∈ Ω � � = Pr[ ω ] X ( ω ) + Pr[ ω ] Y ( ω ) = E[ X ] + E[ Y ] . ω ∈ Ω ω ∈ Ω Corollary E[ a 1 X 1 + a 2 X 2 + . . . + a n X n ] = � n i =1 a i E[ X i ] . Chandra (UIUC) CS498ABD 24 Spring 2019 24 / 51

  20. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  21. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  22. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  23. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  24. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) �� � Pr[ A e ] = m � � E[ X ] = E = E[ X A e ] = X A e 4 e ∈ E e ∈ E e ∈ E Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  25. Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) �� � Pr[ A e ] = m � � E[ X ] = E = E[ X A e ] = X A e 4 e ∈ E e ∈ E e ∈ E It is important to setup random variables carefully. Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

  26. Expected number of triangles? Let G = ( V , E ) be a graph with n vertices and m edges. Assume G has t triangles (i.e., a triangle is a simple cycle with three vertices). Let H be the graph resulting from deleting independently each vertex of G with probability 1 / 2 . The expected number of triangles in H is (A) t/2. (B) t/4. (C) t/8. (D) t/16. (E) none of the above. Chandra (UIUC) CS498ABD 26 Spring 2019 26 / 51

  27. Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

  28. Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

  29. Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

  30. Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. X = # H , Y = # T . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

  31. Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. X = # H , Y = # T . Dependent. Why? Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

  32. Independent Randomized Variables Lemma If X and Y are independent then E[ XY ] = E[ X ] · E[ Y ] Proof. � E[ X · Y ] = Pr[ ω ] ( X ( ω ) · Y ( ω )) ω ∈ Ω � = Pr[ X = x ∧ Y = y ] ( x · y ) x , y ∈ R � = Pr[ X = x ] · Pr[ Y = y ] · x · y x , y ∈ R � � = ( Pr[ X = x ] x )( Pr[ Y = y ] y ) = E[ X ] E[ Y ] x ∈ R y ∈ R Chandra (UIUC) CS498ABD 28 Spring 2019 28 / 51

  33. Types of Randomized Algorithms Typically one encounters the following types: Las Vegas randomized algorithms: for a given input x 1 output of algorithm is always correct but the running time is a random variable . In this case we are interested in analyzing the expected running time. Chandra (UIUC) CS498ABD 29 Spring 2019 29 / 51

  34. Types of Randomized Algorithms Typically one encounters the following types: Las Vegas randomized algorithms: for a given input x 1 output of algorithm is always correct but the running time is a random variable . In this case we are interested in analyzing the expected running time. Monte Carlo randomized algorithms: for a given input x the 2 running time is deterministic but the output is random ; correct with some probability. In this case we are interested in analyzing the probability of the correct output (and also the running time). Algorithms whose running time and output may both be random. 3 Chandra (UIUC) CS498ABD 29 Spring 2019 29 / 51

  35. Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π : Let Q ( x ) be the time for Q to run on input x of length | x | . 1 Worst-case analysis: run time on worst input for a given size n . 2 T wc ( n ) = max x : | x | = n Q ( x ) . Chandra (UIUC) CS498ABD 30 Spring 2019 30 / 51

  36. Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π : Let Q ( x ) be the time for Q to run on input x of length | x | . 1 Worst-case analysis: run time on worst input for a given size n . 2 T wc ( n ) = max x : | x | = n Q ( x ) . Randomized algorithm R for a problem Π : Let R ( x ) be the time for Q to run on input x of length | x | . 1 R ( x ) is a random variable: depends on random bits used by R . 2 E[ R ( x )] is the expected running time for R on x 3 Worst-case analysis: expected time on worst input of size n 4 T rand − wc ( n ) = max x : | x | = n E[ R ( x )] . Chandra (UIUC) CS498ABD 30 Spring 2019 30 / 51

  37. Analyzing Monte Carlo Algorithms Randomized algorithm M for a problem Π : Let M ( x ) be the time for M to run on input x of length | x | . 1 For Monte Carlo, assumption is that run time is deterministic. Let Pr[ x ] be the probability that M is correct on x . 2 Pr[ x ] is a random variable: depends on random bits used by M . 3 Worst-case analysis: success probability on worst input 4 P rand − wc ( n ) = min x : | x | = n Pr[ x ] . Chandra (UIUC) CS498ABD 31 Spring 2019 31 / 51

  38. Part III Randomized Quick Sort Chandra (UIUC) CS498ABD 32 Spring 2019 32 / 51

  39. Randomized QuickSort Randomized QuickSort Pick a pivot element uniformly at random from the array. 1 Split array into 3 subarrays: those smaller than pivot, those 2 larger than pivot, and the pivot itself. Recursively sort the subarrays, and concatenate them. 3 Chandra (UIUC) CS498ABD 33 Spring 2019 33 / 51

  40. Analysis What events to count? Number of Comparisions. Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

  41. Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

  42. Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Too Big!! Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

  43. Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Too Big!! What random variables to define? What are the events of the algorithm? Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

  44. Analysis via Recurrence Given array A of size n , let Q ( A ) be number of comparisons of 1 randomized QuickSort on A . Note that Q ( A ) is a random variable. 2 Let A i left and A i right be the left and right arrays obtained if: 3 Let X i be indicator random variable, which is set to 1 if pivot is of rank i in A , else zero. n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i · right ) . i =1 Chandra (UIUC) CS498ABD 35 Spring 2019 35 / 51

  45. Analysis via Recurrence Given array A of size n , let Q ( A ) be number of comparisons of 1 randomized QuickSort on A . Note that Q ( A ) is a random variable. 2 Let A i left and A i right be the left and right arrays obtained if: 3 Let X i be indicator random variable, which is set to 1 if pivot is of rank i in A , else zero. n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i · right ) . i =1 Since each element of A has probability exactly of 1 / n of being chosen: E[ X i ] = Pr[ pivot has rank i ] = 1 / n . Chandra (UIUC) CS498ABD 35 Spring 2019 35 / 51

  46. Independence of Random Variables Lemma Random variables X i is independent of random variables Q ( A i left ) as well as Q ( A i right ) , i.e. � X i · Q ( A i � � Q ( A i � left ) = E[ X i ] E left ) E � � � � X i · Q ( A i Q ( A i right ) = E[ X i ] E right ) E Proof. This is because the algorithm, while recursing on Q ( A i left ) and Q ( A i right ) uses new random coin tosses that are independent of the coin tosses used to decide the first pivot. Only the latter decides value of X i . Chandra (UIUC) CS498ABD 36 Spring 2019 36 / 51

  47. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

  48. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

  49. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 By linearity of expectation, and independence random variables: n � � � � � � �� � Q ( A i Q ( A i Q ( A ) = n + E[ X i ] left ) + E right ) . E E i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

  50. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 By linearity of expectation, and independence random variables: n � � � � � � �� � Q ( A i Q ( A i Q ( A ) = n + E[ X i ] left ) + E right ) . E E i =1 n 1 � � � ⇒ Q ( A ) ≤ n + n ( T ( i − 1) + T ( n − i )) . E i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

  51. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . Chandra (UIUC) CS498ABD 38 Spring 2019 38 / 51

  52. Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We derived: n 1 � � � Q ( A ) ≤ n + n ( T ( i − 1) + T ( n − i )) . E i =1 Note that above holds for any A of size n . Therefore n 1 � A : | A | = n E[ Q ( A )] = T ( n ) ≤ n + max n ( T ( i − 1) + T ( n − i )) . i =1 Chandra (UIUC) CS498ABD 38 Spring 2019 38 / 51

  53. Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

  54. Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Lemma T ( n ) = O ( n log n ) . Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

  55. Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Lemma T ( n ) = O ( n log n ) . Proof. (Guess and) Verify by induction. Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

  56. Part IV Slick analysis of QuickSort Chandra (UIUC) CS498ABD 40 Spring 2019 40 / 51

  57. A Slick Analysis of QuickSort Let Q ( A ) be number of comparisons done on input array A : For 1 ≤ i < j < n let R ij be the event that rank i element is 1 compared with rank j element. X ij is the indicator random variable for R ij . That is, X ij = 1 if 2 rank i is compared with rank j element, otherwise 0 . Chandra (UIUC) CS498ABD 41 Spring 2019 41 / 51

  58. A Slick Analysis of QuickSort Let Q ( A ) be number of comparisons done on input array A : For 1 ≤ i < j < n let R ij be the event that rank i element is 1 compared with rank j element. X ij is the indicator random variable for R ij . That is, X ij = 1 if 2 rank i is compared with rank j element, otherwise 0 . � Q ( A ) = X ij 1 ≤ i < j ≤ n and hence by linearity of expectation, � � � � � � � � Q ( A ) = X ij = Pr R ij . E E 1 ≤ i < j ≤ n 1 ≤ i < j ≤ n Chandra (UIUC) CS498ABD 41 Spring 2019 41 / 51

  59. A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

  60. A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

  61. A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 As such, probability of comparing 5 to 8 is Pr[ R 4 , 7 ] . Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

  62. A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 If pivot too small (say 3 [rank 2]). Partition and call recursively: 1 7 5 9 1 3 4 8 6 1 3 7 5 9 4 8 6 = ⇒ Decision if to compare 5 to 8 is moved to subproblem. Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

  63. A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 If pivot too small (say 3 [rank 2]). Partition and call recursively: 1 7 5 9 1 3 4 8 6 1 3 7 5 9 4 8 6 = ⇒ Decision if to compare 5 to 8 is moved to subproblem. If pivot too large (say 9 [rank 8]): 2 7 5 9 1 3 4 8 6 7 5 9 1 3 4 8 6 7 5 1 3 4 8 6 9 = ⇒ Decision if to compare 5 to 8 moved to subproblem. Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

  64. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

  65. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ If pivot is 8 (rank 7). Bingo! 2 7 5 9 1 3 4 8 6 7 5 1 3 4 6 8 9 = ⇒ Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

  66. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ If pivot is 8 (rank 7). Bingo! 2 7 5 9 1 3 4 8 6 7 5 1 3 4 6 8 9 = ⇒ If pivot in between the two numbers (say 6 [rank 5]): 3 7 5 9 1 3 4 8 6 5 1 3 4 6 7 8 9 = ⇒ 5 and 8 will never be compared to each other. Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

  67. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? Conclusion: R i , j happens if and only if: i th or j th ranked element is the first pivot out of i th to j th ranked elements. How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. Equivalent to picking pivot uniformly at random 3 (as QuickSort do). Chandra (UIUC) CS498ABD 44 Spring 2019 44 / 51

  68. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. = ⇒ R i , j happens if either i or j have lowest priority out of elements rank i to j , Chandra (UIUC) CS498ABD 45 Spring 2019 45 / 51

  69. A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. = ⇒ R i , j happens if either i or j have lowest priority out of elements rank i to j , There are k = j − i + 1 relevant elements. = 2 2 � � Pr R i , j k = j − i + 1 . Chandra (UIUC) CS498ABD 45 Spring 2019 45 / 51

  70. A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

  71. A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Lemma � � 2 Pr R ij = j − i +1 . Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

  72. A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Lemma � � 2 Pr R ij = j − i +1 . Proof. Let a 1 , . . . , a i , . . . , a j , . . . , a n be elements of A in sorted order. Let S = { a i , a i +1 , . . . , a j } Observation: If pivot is chosen outside S then all of S either in left array or right array. Observation: a i and a j separated when a pivot is chosen from S for the first time. Once separated no comparison. Observation: a i is compared with a j if and only if either a i or a j is chosen as a pivot from S at separation... Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

  73. A Slick Analysis of QuickSort Continued... Lemma � � 2 Pr = j − i +1 . R ij Proof. Let a 1 , . . . , a i , . . . , a j , . . . , a n be sort of A . Let S = { a i , a i +1 , . . . , a j } Observation: a i is compared with a j if and only if either a i or a j is chosen as a pivot from S at separation. Observation: Given that pivot is chosen from S the probability that it is a i or a j is exactly 2 / | S | = 2 / ( j − i + 1) since the pivot is chosen uniformly at random from the array. Chandra (UIUC) CS498ABD 47 Spring 2019 47 / 51

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Introduction to Randomized Algorithms: QuickSort Lecture 2 August 27, 2020 Chandra (UIUC)

Introduction to Randomized Algorithms: QuickSort Lecture 2 August 27, 2020 Chandra (UIUC)

CS 498ABD: Algorithms for Big Data Introduction to Randomized Algorithms: QuickSort Lecture 2 August 27, 2020 Chandra (UIUC) CS498ABD 1 Fall 2020 1 / 33 Outline Today Randomized Algorithms Two types Las Vegas Monte Carlo

1.03k views • 71 slides
Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides courtesy of Charles

Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides courtesy of Charles

CMPS 2200 Fall 2014 Randomized Algorithms, Quicksort and Randomized Selection Carola Wenk Slides courtesy of Charles Leiserson with additions by Carola Wenk CMPS 2200 Intro. to Algorithms 1 Deterministic Algorithms Runtime for

765 views • 54 slides
Randomized algorithms Review basics from ``Think like the pros'' Recall QuickSort(low,

Randomized algorithms Review basics from ``Think like the pros'' Recall QuickSort(low,

Randomized algorithms Review basics from ``Think like the pros'' Recall QuickSort(low, high) { if (high-low 1) return; partition(low, high) and return split; QuickSort(low, split); QuickSort(split+1, high); } Partition

727 views • 36 slides
Randomized Algorithms Randomized Algorithms Two Types of Randomized Algorithms Two Types of

Randomized Algorithms Randomized Algorithms Two Types of Randomized Algorithms Two Types of

Randomized Algorithms Randomized Algorithms Two Types of Randomized Algorithms Two Types of Randomized Algorithms and and Some Complexity Classes Some Complexity Classes Speaker: Chuang-Chieh Lin Advisor: Professor Maw-Shang Chang National

910 views • 52 slides
Quicksort Quicksort In-place Merge Sort g T(n) = (n lg(n)) Not in-place CSE

Quicksort Quicksort In-place Merge Sort g T(n) = (n lg(n)) Not in-place CSE

Sorting Review g Introduction to Algorithms Introduction to Algorithms Insertion Sort T(n) = (n 2 ) Quicksort Quicksort In-place Merge Sort g T(n) = (n lg(n)) Not in-place CSE 680 Selection Sort (from

249 views • 9 slides
ECS231 Low-rank approximation revisited (Introduction to Randomized Algorithms) May 23, 2019

ECS231 Low-rank approximation revisited (Introduction to Randomized Algorithms) May 23, 2019

ECS231 Low-rank approximation revisited (Introduction to Randomized Algorithms) May 23, 2019 1 / 15 Outline 1. Review: low-rank approximation 2. Prototype randomized SVD algorithm 3. Accelerated randomized SVD algorithms 4. CUR

262 views • 15 slides
Data Structures and Algorithms Chapter 7: Sorting (Quicksort) Text: Read Weiss, 7.7 Izmir

Data Structures and Algorithms Chapter 7: Sorting (Quicksort) Text: Read Weiss, 7.7 Izmir

CE 221 Data Structures and Algorithms Chapter 7: Sorting (Quicksort) Text: Read Weiss, 7.7 Izmir University of Economics 1 Quicksort As the name implies quicksort is the fastest known sorting algorithm in practice. Average running

514 views • 18 slides
Randomized algorithms Quick-sort Closest pair of points Inge Li Grtz 1 2

Randomized algorithms Quick-sort Closest pair of points Inge Li Grtz 1 2

Randomized algorithms Today What are randomized algorithms? Properties of randomized algorithms Three examples: Median/Select. Randomized algorithms Quick-sort Closest pair of points Inge Li Grtz 1 2

358 views • 10 slides
ECS231 Least-squares problems (Introduction to Randomized Algorithms) May 21, 2019 1 / 12

ECS231 Least-squares problems (Introduction to Randomized Algorithms) May 21, 2019 1 / 12

ECS231 Least-squares problems (Introduction to Randomized Algorithms) May 21, 2019 1 / 12 Outline 1. linear least squares review 2. Solving LS by sampling 3. Solving LS by randomized preconditioning 4. Gradient-based optimization

259 views • 12 slides
Sorting Algorithms rules of the game shellsort mergesort quicksort animations

Sorting Algorithms rules of the game shellsort mergesort quicksort animations

Sorting Algorithms rules of the game shellsort mergesort quicksort animations Reference: Algorithms in Java, Chapters 6-8 1 Classic sorting algorithms Critical components in the worlds computational infrastructure.

704 views • 52 slides
CS 574: Randomized Algorithms Lecture 14. Introduction to Martingales October 8, 2015 Lecture

CS 574: Randomized Algorithms Lecture 14. Introduction to Martingales October 8, 2015 Lecture

CS 574: Randomized Algorithms Lecture 14. Introduction to Martingales October 8, 2015 Lecture 14. Introduction to Martingales CS 574: Randomized Algorithms Martingales Introduction For independent r.v.s X i we showed tight concentration of

256 views • 21 slides
Randomized Algorithms Randomized Algorithms Markov Chains and Random Walks Markov Chains and

Randomized Algorithms Randomized Algorithms Markov Chains and Random Walks Markov Chains and

Randomized Algorithms Randomized Algorithms Markov Chains and Random Walks Markov Chains and Random Walks Speaker: Chuang-Chieh Lin Advisor: Professor Maw-Shang Chang National Chung Cheng University 2007/4/25 2007/4/25 References

837 views • 79 slides
Randomness in Computing L ECTURE 7 Last time Randomized quicksort Markovs inequality

Randomness in Computing L ECTURE 7 Last time Randomized quicksort Markovs inequality

Randomness in Computing L ECTURE 7 Last time Randomized quicksort Markovs inequality Variance Today Variance, covariance Chebyshevs inequality Variance of Binomial and Geometric RVs 2/11/2020 Sofya

349 views • 11 slides
Introduction to Randomized Algorithms Arijit Bishnu ( arijit@isical.ac.in ) Advanced Computing

Introduction to Randomized Algorithms Arijit Bishnu ( arijit@isical.ac.in ) Advanced Computing

Introduction Quick Sort Minimum Enclosing Disk Min Cut Introduction to Randomized Algorithms Arijit Bishnu ( arijit@isical.ac.in ) Advanced Computing and Microelectronics Unit Indian Statistical Institute Kolkata 700108, India. Introduction

1.24k views • 93 slides
Divide and Conquer Algorithms Mergesort, Quicksort Strassens Algorithm CSE 421

Divide and Conquer Algorithms Mergesort, Quicksort Strassens Algorithm CSE 421

Divide and Conquer Algorithms Mergesort, Quicksort Strassens Algorithm CSE 421 Closest Pair Algorithm (2d) Algorithms Inversion counting Integer Multiplication (Karatsubas Algorithm) Richard Anderson FFT

235 views • 4 slides
CS 574: Randomized Algorithms Lecture 1. Introduction to Randomness August 25, 2015 Lecture 1.

CS 574: Randomized Algorithms Lecture 1. Introduction to Randomness August 25, 2015 Lecture 1.

Administrativia and Syllabus Why Randomness? Randomness in Computer Science CS 574: Randomized Algorithms Lecture 1. Introduction to Randomness August 25, 2015 Lecture 1. Introduction to Randomness CS 574: Randomized Algorithms

1.23k views • 61 slides
On the Resiliency of Randomized Routing Against Multiple Edge

On the Resiliency of Randomized Routing Against Multiple Edge

On the Resiliency of Randomized Routing Against Multiple Edge Failures Slobodan Mitrovi (EPFL) Joint work with M. Chiesa, A. Gurtov, A. Mdry, I.

861 views • 57 slides
Randomized sparse Kaczmarz methods Dirk Lorenz, joint with Frank Schpfer, Feb 9, 2018 Inverse

Randomized sparse Kaczmarz methods Dirk Lorenz, joint with Frank Schpfer, Feb 9, 2018 Inverse

Randomized sparse Kaczmarz methods Dirk Lorenz, joint with Frank Schpfer, Feb 9, 2018 Inverse Problems and Machine Learning, Caltech 2018 The Kaczmarz method Randomization Sparsity Split feasibility problems Convergence rates Feb 9, 2018

912 views • 68 slides
DieHard: Probabilistic Memory Safety for Unsafe Programming Languages Emery Berger Ben Zorn

DieHard: Probabilistic Memory Safety for Unsafe Programming Languages Emery Berger Ben Zorn

DieHard: Probabilistic Memory Safety for Unsafe Programming Languages Emery Berger Ben Zorn University of Massachusetts Microsoft Research Amherst Presented by: Brian Norris PLDI 2006 PLDI 2006 Frivolity Happy Leap Day! PLDI 2006

609 views • 35 slides
Approximate Spectral Clustering via Randomized Sketching Christos Boutsidis Yahoo! Labs, New

Approximate Spectral Clustering via Randomized Sketching Christos Boutsidis Yahoo! Labs, New

Approximate Spectral Clustering via Randomized Sketching Christos Boutsidis Yahoo! Labs, New York Joint work with Alex Gittens (Ebay), Anju Kambadur (IBM) The big picture: sketch and solve Tradeoff : Speed (depends on the size of A)

378 views • 22 slides
Introducing Random Search H YP ERPARAMETER TUN IN G IN P YTH ON Alex Scriven Data Scientist

Introducing Random Search H YP ERPARAMETER TUN IN G IN P YTH ON Alex Scriven Data Scientist

Introducing Random Search H YP ERPARAMETER TUN IN G IN P YTH ON Alex Scriven Data Scientist What you already know Very similar to grid search: Dene an estimator, which hyperparameters to tune and the range of values for each

482 views • 27 slides
Gov 2000: 2. Random Variables and Probability Distributions Matthew Blackwell Fall 2016 1 / 56

Gov 2000: 2. Random Variables and Probability Distributions Matthew Blackwell Fall 2016 1 / 56

Gov 2000: 2. Random Variables and Probability Distributions Matthew Blackwell Fall 2016 1 / 56 1. Random Variables 2. Probability Distributions 3. Cumulative Distribution Functions 4. Properties of Distributions 5. Famous distributions 6.

791 views • 56 slides
Causality: Explanation versus Prediction Department of Government London School of Economics and

Causality: Explanation versus Prediction Department of Government London School of Economics and

MT Causality Counterfactuals Randomized Experiments Causality: Explanation versus Prediction Department of Government London School of Economics and Political Science MT Causality Counterfactuals Randomized Experiments 1 Brief Review of

1.27k views • 77 slides
Ongoing developments in IEEE 802.11 WLAN standardisation A study group on randomized and changing

Ongoing developments in IEEE 802.11 WLAN standardisation A study group on randomized and changing

. . . . . . . . . . . . . . . . Ongoing developments in IEEE 802.11 WLAN standardisation A study group on randomized and changing MAC addresses Amelia Andersdotter (Chair, RCM TIG, IEEE 802.11) 1 HotPETS, Stockholm, July 2019 .

498 views • 16 slides

More recommend