advanced algorithms coms31900
play

Advanced Algorithms COMS31900 Probability recap. Rapha el - PowerPoint PPT Presentation

Aug 10, 2023 •362 likes •1.48k views

Advanced Algorithms COMS31900 Probability recap. Rapha el Clifford Slides by Markus Jalsenius Randomness and probability Probability The sample space S is the set of outcomes of an experiment. Probability The sample space S is the set

  1. Random variable A random variable (r.v.) Y over sample space S is a function S → R i.e. it maps each outcome x ∈ S to some real number Y ( x ) . Pr( The probability of Y taking value y is { x ∈ S st. Y ( x ) = y } E XAMPLE sum over all values of x such that Y ( x ) = y Two coin flips. S Y What is Pr( Y = 2) ? H H 2 Pr( x ) = 1 H T 1 � Pr( Y = 2) = 4+ T H 5 x ∈{ HH , TT } T T 2

  2. Random variable A random variable (r.v.) Y over sample space S is a function S → R i.e. it maps each outcome x ∈ S to some real number Y ( x ) . Pr( The probability of Y taking value y is { x ∈ S st. Y ( x ) = y } E XAMPLE sum over all values of x such that Y ( x ) = y Two coin flips. S Y What is Pr( Y = 2) ? H H 2 Pr( x ) = 1 H T 1 � Pr( Y = 2) = 4+ T H 5 x ∈{ HH , TT } T T 2 Pr( Y = 2) = 1 2

  3. Random variable A random variable (r.v.) Y over sample space S is a function S → R i.e. it maps each outcome x ∈ S to some real number Y ( x ) . Pr( The probability of Y taking value y is { x ∈ S st. Y ( x ) = y } E XAMPLE Two coin flips. S Y H H 2 H T 1 T H 5 T T 2 Pr( Y = 2) = 1 2

  4. Random variable A random variable (r.v.) Y over sample space S is a function S → R i.e. it maps each outcome x ∈ S to some real number Y ( x ) . Pr( The probability of Y taking value y is { x ∈ S st. Y ( x ) = y } E XAMPLE Two coin flips. The expected value (the mean) of a r.v. Y , denoted E ( Y ) , is S Y H H 2 E H T 1 T H 5 T T 2 Pr( Y = 2) = 1 2

  5. Random variable A random variable (r.v.) Y over sample space S is a function S → R i.e. it maps each outcome x ∈ S to some real number Y ( x ) . Pr( The probability of Y taking value y is { x ∈ S st. Y ( x ) = y } E XAMPLE Two coin flips. The expected value (the mean) of a r.v. Y , denoted E ( Y ) , is S Y H H 2 E H T 1 T H 5 T T 2 2 · 1 1 · 1 5 · 1 = 5 � � � � � � E ( Y ) = + + 2 4 4 2 Pr( Y = 2) = 1 2

  6. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1

  7. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.)

  8. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values.

  9. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. random variable

  10. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values.

  11. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ?

  12. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem)

  13. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem) The sample space S = { (1 , 1) , (1 , 2) , (1 , 3) . . . (6 , 6) } ( 36 outcomes)

  14. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem) The sample space S = { (1 , 1) , (1 , 2) , (1 , 3) . . . (6 , 6) } ( 36 outcomes) 1 E ( Y ) = � � x ∈ S Y ( x ) · Pr( x ) = x ∈ S Y ( x ) = 36

  15. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem) The sample space S = { (1 , 1) , (1 , 2) , (1 , 3) . . . (6 , 6) } ( 36 outcomes) 1 E ( Y ) = � � x ∈ S Y ( x ) · Pr( x ) = x ∈ S Y ( x ) = 36

  16. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem) The sample space S = { (1 , 1) , (1 , 2) , (1 , 3) . . . (6 , 6) } ( 36 outcomes) 1 E ( Y ) = � � x ∈ S Y ( x ) · Pr( x ) = x ∈ S Y ( x ) = 36 1 36 (1 · 2 + 2 · 3 + 3 · 4 + · · · + 1 · 12) = 7

  17. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 1: (without the theorem) The sample space S = { (1 , 1) , (1 , 2) , (1 , 3) . . . (6 , 6) } ( 36 outcomes) 1 E ( Y ) = � � x ∈ S Y ( x ) · Pr( x ) = x ∈ S Y ( x ) = 36 1 36 (1 · 2 + 2 · 3 + 3 · 4 + · · · + 1 · 12) = 7

  18. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ?

  19. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 2: (with the theorem)

  20. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 2: (with the theorem) Let the r.v. Y 1 be the value of the first die and Y 2 the value of the second

  21. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 2: (with the theorem) Let the r.v. Y 1 be the value of the first die and Y 2 the value of the second E ( Y 1 ) = E ( Y 2 ) = 3 . 5

  22. Linearity of expectation T HEOREM (Linearity of expectation) Let Y 1 , Y 2 , . . . , Y k be k random variables. Then k k � � � � = E ( Y i ) E Y i i =1 i =1 Linearity of expectation always holds, (regardless of whether the random variables are independent or not.) E XAMPLE Roll two dice. Let the r.v. Y be the sum of the values. What is E ( Y ) ? Approach 2: (with the theorem) Let the r.v. Y 1 be the value of the first die and Y 2 the value of the second E ( Y 1 ) = E ( Y 2 ) = 3 . 5 so E ( Y ) = E ( Y 1 + Y 2 ) = E ( Y 1 ) + E ( Y 2 ) = 7

  23. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I )

  24. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = 0 · Pr( I = 0) + 1 · Pr( I = 1) = Pr( I = 1) .

  25. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) .

  26. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise).

  27. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together!

  28. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times.

  29. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll?

  30. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise)

  31. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise) Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes)

  32. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise) Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes) n n n Pr( I j = 1) = ( n − 1) · 7 � � � � � = E ( I j ) = E I j 12 j =2 j =2 j =2

  33. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! Linearity of Expectation E XAMPLE Roll a die n times. Let Y 1 , Y 2 , . . . , Y k be k random variables. Then What is the expected number rolls that show a value that is at least the value of the previous roll? k k � � � � For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll = E ( Y i ) E Y i is at least the value of the previous roll (and I j = 0 otherwise) i =1 i =1 Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes) n n n Pr( I j = 1) = ( n − 1) · 7 � � � � � = E ( I j ) = E I j 12 j =2 j =2 j =2

  34. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise) Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes) n n n Pr( I j = 1) = ( n − 1) · 7 � � � � � = E ( I j ) = E I j 12 j =2 j =2 j =2

  35. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise) Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes) n n n Pr( I j = 1) = ( n − 1) · 7 � � � � � = E ( I j ) = E I j 12 j =2 j =2 j =2

  36. Indicator random variables An indicator random variable is a r.v. that can only be 0 or 1 . (usually referred to by the letter I ) Fact: E ( I ) = Pr( I = 1) . Often an indicator r.v. I is associated with an event such that I = 1 if the event happens (and I = 0 otherwise). Indicator random variables and linearity of expectation work great together! E XAMPLE Roll a die n times. What is the expected number rolls that show a value that is at least the value of the previous roll? For j ∈ { 2 , . . . , n } , let indicator r.v. I j = 1 if the value of the j th roll is at least the value of the previous roll (and I j = 0 otherwise) Pr( I j = 1) = 21 7 36 = 12 . (by counting the outcomes) n n n Pr( I j = 1) = ( n − 1) · 7 � � � � � = E ( I j ) = E I j 12 j =2 j =2 j =2

  37. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph.

  38. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most

  39. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most 1 2 of all cars drive at least 120 mph,

  40. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most 1 2 of all cars drive at least 120 mph, . . . otherwise the mean must be higher than 60 mph. (a contradiction)

  41. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most 2 3 of all cars drive at least 90 mph, . . . otherwise the mean must be higher than 60 mph. (a contradiction)

  42. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most 2 3 of all cars drive at least 90 mph, . . . otherwise the mean must be higher than 60 mph. (a contradiction) T HEOREM (Markov’s inequality) If X is a non-negative r.v., then for all a > 0 , Pr( X ≥ a ) ≤ E ( X ) . a

  43. Markov’s inequality E XAMPLE Suppose that the average (mean) speed on the motorway is 60 mph. It then follows that at most 2 3 of all cars drive at least 90 mph, . . . otherwise the mean must be higher than 60 mph. (a contradiction) T HEOREM (Markov’s inequality) If X is a non-negative r.v., then for all a > 0 , Pr( X ≥ a ) ≤ E ( X ) . a E XAMPLE From the example above: 120 = 1 60 � Pr( speed of a random car ≥ 120 mph ) ≤ 2 , � Pr( speed of a random car ≥ 90 mph ) ≤ 60 90 = 2 3 .

  44. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat.

  45. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat?

  46. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 .

  47. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E

  48. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . Fact: E ( I ) = Pr( I = 1) . By linearity of expectation. . . E

  49. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E

  50. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E By Markov’s inequality (recall: Pr( X ≥ a ) ≤ E ( X ) ), a

  51. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E By Markov’s inequality (recall: Pr( X ≥ a ) ≤ E ( X ) ), a Pr( 5 or more people leaving with their own hats ) ≤ 1 5 ,

  52. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E By Markov’s inequality (recall: Pr( X ≥ a ) ≤ E ( X ) ), a Pr( 5 or more people leaving with their own hats ) ≤ 1 5 , Pr( at least 1 person leaving with their own hat ) ≤ 1 1 = 1 .

  53. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E By Markov’s inequality (recall: Pr( X ≥ a ) ≤ E ( X ) ), a Pr( 5 or more people leaving with their own hats ) ≤ 1 5 , Pr( at least 1 person leaving with their own hat ) ≤ 1 1 = 1 . (sometimes Markov’s inequality is not particularly informative)

  54. Markov’s inequality E XAMPLE n people go to a party, leaving their hats at the door. Each person leaves with a random hat. How many people leave with their own hat? For j ∈ { 1 , . . . , n } , let indicator r.v. I j = 1 if the j th person gets their own hat, otherwise I j = 0 . By linearity of expectation. . . E By Markov’s inequality (recall: Pr( X ≥ a ) ≤ E ( X ) ), a Pr( 5 or more people leaving with their own hats ) ≤ 1 5 , Pr( at least 1 person leaving with their own hat ) ≤ 1 1 = 1 . (sometimes Markov’s inequality is not particularly informative) In fact, here it can be shown that as n → ∞ , the probability that at least one person leaves with their own hat is 1 − 1 e ≈ 0 . 632 .

  55. Markov’s inequality C OROLLARY If X is a non-negative r.v. that only takes integer values, then Pr( X > 0) = Pr( X ≥ 1) ≤ E ( X ) . For an indicator r.v. I , the bound is tight ( = ), as Pr( I > 0) = E ( I ) .

  56. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1

  57. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This is the probability at least one of the events happens

  58. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1

  59. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty)

  60. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF

  61. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF Define indicator r.v. I j to be 1 if event V j happens, otherwise I j = 0 .

  62. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF Define indicator r.v. I j to be 1 if event V j happens, otherwise I j = 0 . Let the r.v. X = � k j =1 I j be the number of events that happen.

  63. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF Define indicator r.v. I j to be 1 if event V j happens, otherwise I j = 0 . Let the r.v. X = � k j =1 I j be the number of events that happen. � � k = Pr( X > 0) ≤ E ( X ) = E ( � k j =1 I j ) = � k � Pr j =1 E ( I j ) j =1 V j = � k j =1 Pr( V j )

  64. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF Define indicator r.v. I j to be 1 if event V j happens, otherwise I j = 0 . Let the r.v. X = � k j =1 I j be the number of events that happen. � � k = Pr( X > 0) ≤ E ( X ) = E ( � k j =1 I j ) = � k � Pr j =1 E ( I j ) j =1 V j = � k j =1 Pr( V j ) by previous Markov corollary

  65. Union bound T HEOREM (union bound) Let V 1 , . . . , V k be k events. Then k k � � � � Pr ≤ Pr( V i ) . V i i =1 i =1 This bound is tight ( = ) when the events are all disjoint. ( V i and V j are disjoint iff V i ∩ V j is empty) P ROOF Define indicator r.v. I j to be 1 if event V j happens, otherwise I j = 0 . Let the r.v. X = � k j =1 I j be the number of events that happen. � � k = Pr( X > 0) ≤ E ( X ) = E ( � k j =1 I j ) = � k � Pr j =1 E ( I j ) j =1 V j = � k j =1 Pr( V j ) by previous Markov corollary Linearity of expectation

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Advanced Algorithms COMS31900 Approximation algorithms part two more constant factor

Advanced Algorithms COMS31900 Approximation algorithms part two more constant factor

Advanced Algorithms COMS31900 Approximation algorithms part two more constant factor approximations Rapha el Clifford Slides by Benjamin Sach Approximation Algorithms Recap An algorithm A is an -approximation algorithm for problem P

2.14k views • 175 slides
Advanced Algorithms COMS31900 Approximation algorithms part four Asymptotic Polynomial Time

Advanced Algorithms COMS31900 Approximation algorithms part four Asymptotic Polynomial Time

Advanced Algorithms COMS31900 Approximation algorithms part four Asymptotic Polynomial Time Approximation Schemes Rapha el Clifford Slides by Benjamin Sach Approximation Algorithms Recap A polynomial time algorithm A is an

1.52k views • 134 slides
Advanced Algorithms COMS31900 Approximation algorithms part three (Fully) Polynomial Time

Advanced Algorithms COMS31900 Approximation algorithms part three (Fully) Polynomial Time

Advanced Algorithms COMS31900 Approximation algorithms part three (Fully) Polynomial Time Approximation Schemes Rapha el Clifford Slides by Benjamin Sach Approximation Algorithms Recap An algorithm A is an -approximation algorithm for

1.72k views • 161 slides
Advanced Algorithms COMS31900 Lowest Common Ancestor (with a bit on on Range Minimum Queries)

Advanced Algorithms COMS31900 Lowest Common Ancestor (with a bit on on Range Minimum Queries)

Advanced Algorithms COMS31900 Lowest Common Ancestor (with a bit on on Range Minimum Queries) Rapha el Clifford Slides by Benjamin Sach Advanced Algorithms COMS31900 Lowest Common Ancestor (with a bit on on Range Minimum Queries)

1.9k views • 168 slides
Advanced Algorithms COMS31900 Hashing part one Chaining, true randomness and universal

Advanced Algorithms COMS31900 Hashing part one Chaining, true randomness and universal

Advanced Algorithms COMS31900 Hashing part one Chaining, true randomness and universal hashing Rapha el Clifford Slides by Benjamin Sach and Markus Jalsenius Dictionaries In a dictionary data structure we store ( key , value ) -pairs

1.56k views • 98 slides
Advanced Algorithms COMS31900 Hashing part three Cuckoo Hashing Rapha el Clifford Slides

Advanced Algorithms COMS31900 Hashing part three Cuckoo Hashing Rapha el Clifford Slides

Advanced Algorithms COMS31900 Hashing part three Cuckoo Hashing Rapha el Clifford Slides by Benjamin Sach Back to the start (again) A dynamic dictionary stores ( key , value ) -pairs and supports: add ( key , value ) , lookup ( key )

2.03k views • 163 slides
Advanced Algorithms COMS31900 Hashing part two Static Perfect Hashing Rapha el Clifford

Advanced Algorithms COMS31900 Hashing part two Static Perfect Hashing Rapha el Clifford

Advanced Algorithms COMS31900 Hashing part two Static Perfect Hashing Rapha el Clifford Slides by Benjamin Sach Dictionaries and Hashing recap A dynamic dictionary stores ( key , value ) -pairs and supports: add ( key , value ) ,

1.14k views • 99 slides
Exercise Sheet 1: Hashing and Bloom filters COMS31900 Advanced Algorithms 2019/2020 Please feel

Exercise Sheet 1: Hashing and Bloom filters COMS31900 Advanced Algorithms 2019/2020 Please feel

Exercise Sheet 1: Hashing and Bloom filters COMS31900 Advanced Algorithms 2019/2020 Please feel free to discuss these problems on the unit discussion board. If you would like to have your answers marked, please either hand them in in person at

361 views • 5 slides
Advanced Algorithms COMS31900 Orthogonal Range Searching Rapha el Clifford Slides by

Advanced Algorithms COMS31900 Orthogonal Range Searching Rapha el Clifford Slides by

Advanced Algorithms COMS31900 Orthogonal Range Searching Rapha el Clifford Slides by Benjamin Sach Orthogonal range searching A 2D range searching data structure stores n distinct ( x, y ) -pairs and supports: the lookup ( x 1 , x 2 ,

2.2k views • 187 slides
Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University Feb. 25, 2019 Advanced

Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University Feb. 25, 2019 Advanced

Advanced Algorithms (I) Chihao Zhang Shanghai Jiao Tong University Feb. 25, 2019 Advanced Algorithms (I) 1/14 Advanced Algorithms In the course, we will learn Approximation Algorithms linear programming, semi-definite programming spectral

646 views • 64 slides
Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for clustering Gaussian Mixture Models Often called EM-based clustering Kind of a misnomer in my opinion What distinguishes this algorithm

681 views • 21 slides
Advanced Algorithms (IV) Chihao Zhang Shanghai Jiao Tong University Mar. 18, 2019 Advanced

Advanced Algorithms (IV) Chihao Zhang Shanghai Jiao Tong University Mar. 18, 2019 Advanced

Advanced Algorithms (IV) Chihao Zhang Shanghai Jiao Tong University Mar. 18, 2019 Advanced Algorithms (IV) 1/11 Review n u v E w T u w v s.t. w u u max V w T u w u u V Advanced Algorithms (IV) e Vector Program MaxCut Integer

606 views • 39 slides
Divide and Conquer Algorithms: Advanced Sorting Prichard Ch. 10.2: Advanced Sorting Algorithms

Divide and Conquer Algorithms: Advanced Sorting Prichard Ch. 10.2: Advanced Sorting Algorithms

Divide and Conquer Algorithms: Advanced Sorting Prichard Ch. 10.2: Advanced Sorting Algorithms 1 Sorting Algorithm n Organize a collection of data into either ascending or descending order. n Internal sort q Collection of data fits entirely in

573 views • 42 slides
Advanced Algorithms LP-based Algorithms LP rounding: Relax the

Advanced Algorithms LP-based Algorithms LP rounding: Relax the

Advanced Algorithms LP-based Algorithms LP rounding: Relax the integer program to LP; round the optimal LP solution to a nearby feasible integral solution. The primal-dual schema: Find a pair of

653 views • 38 slides
Advanced Algorithms (II) Chihao Zhang Shanghai Jiao Tong University Mar. 4, 2019 Advanced

Advanced Algorithms (II) Chihao Zhang Shanghai Jiao Tong University Mar. 4, 2019 Advanced

Advanced Algorithms (II) Chihao Zhang Shanghai Jiao Tong University Mar. 4, 2019 Advanced Algorithms (II) 1/11 We show that this ratio can also be achieved via direct rounding. Recall that we have the following linear programming relaxation.

513 views • 50 slides
ADVANCED ALGORITHMS Lecture 14: randomized algorithms 1 ANNOUNCEMENTS HW 3 out; due

ADVANCED ALGORITHMS Lecture 14: randomized algorithms 1 ANNOUNCEMENTS HW 3 out; due

ADVANCED ALGORITHMS Lecture 14: randomized algorithms 1 ANNOUNCEMENTS HW 3 out; due Friday after fall break Look out for project topics! announcement in a little while Groups of 2-3 Give a list of >= 5 topics, top choice

900 views • 23 slides
Introduction to Natural Language Processing a course taught as B4M36NLP at Open Informatics by

Introduction to Natural Language Processing a course taught as B4M36NLP at Open Informatics by

Introduction to Natural Language Processing a course taught as B4M36NLP at Open Informatics by members of the Institute of Formal and Applied Linguistics Today: Week 1, lecture Todays topic: Introduction & Probability & Information

845 views • 57 slides
MA162: Finite mathematics . Jack Schmidt University of Kentucky April 10th, 2013 Schedule: HW

MA162: Finite mathematics . Jack Schmidt University of Kentucky April 10th, 2013 Schedule: HW

. MA162: Finite mathematics . Jack Schmidt University of Kentucky April 10th, 2013 Schedule: HW 1.1-1.4, 2.1-2.6, 3.1-3.3, 4.1, 5.1-5.3, 6A-6C (Late) HW 7A due Friday, Apr 12, 2013 HW 7B due Friday, Apr 19, 2013 HW 7C due Friday, Apr 26,

654 views • 49 slides
Foundations of Computer Science Lecture 18 Random Variables Measurable Outcomes Probability

Foundations of Computer Science Lecture 18 Random Variables Measurable Outcomes Probability

Foundations of Computer Science Lecture 18 Random Variables Measurable Outcomes Probability Distribution Function Bernoulli, Uniform, Binomial and Exponential Random Variables Last Time 1 Independence. Using independence to estimate

310 views • 13 slides
MachineLearning CMPT726 SimonFraserUniversity

MachineLearning CMPT726 SimonFraserUniversity

MachineLearning CMPT726 SimonFraserUniversity BinomialParameterEstimation Outline MaximumLikelihoodEstimation SmoothedFrequencies,LaplaceCorrection. BayesianApproach.

770 views • 22 slides
1. Probabilistic Models Andrej Bogdanov Alice Bob Can Alice and Bob make a connection? In

1. Probabilistic Models Andrej Bogdanov Alice Bob Can Alice and Bob make a connection? In

ENGG 2430 / ESTR 2004: Probability and Sta.s.cs Spring 2019 1. Probabilistic Models Andrej Bogdanov Alice Bob Can Alice and Bob make a connection? In uncertain situations we want a number saying how likely something is probability The

707 views • 55 slides
IN4080 2020 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lnning 2 Probabilities Tutorial,

IN4080 2020 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lnning 2 Probabilities Tutorial,

1 IN4080 2020 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lnning 2 Probabilities Tutorial, 18 Aug. Today Probability theory 3 Probability Random variable The benefits of statistics in NLP: 4 1. Part of the (learned) model:

703 views • 45 slides
Mathematics for Machine Learning Weinan Zhang Shanghai Jiao Tong University http://wnzhang.net

Mathematics for Machine Learning Weinan Zhang Shanghai Jiao Tong University http://wnzhang.net

2019 CS420 Machine Learning, Lecture 1A (Home Reading Materials) Mathematics for Machine Learning Weinan Zhang Shanghai Jiao Tong University http://wnzhang.net http://wnzhang.net/teaching/cs420/index.html Areas of Mathematics Essential to

1.21k views • 103 slides
Probability statistics So, understand some basic probability Chapters 4 & 5 Also,

Probability statistics So, understand some basic probability Chapters 4 & 5 Also,

4/27/2017 Overview IMGD 2905 Statistics important for What are some examples of game analysis probabilities needed for Probability important for game development? Probability statistics So, understand some basic probability

660 views • 10 slides

More recommend