Continuing Probability. Wrap up: Total Probability and Conditional - - PowerPoint PPT Presentation

▶
continuing probability
SMART_READER_LITE
LIVE PREVIEW

Continuing Probability. Wrap up: Total Probability and Conditional - - PowerPoint PPT Presentation

Nov 04, 2022 364 likes •2.11k views

Continuing Probability. Wrap up: Total Probability and Conditional Probability. Continuing Probability. Wrap up: Total Probability and Conditional Probability. Product Rule, Correlation, Independence, Bayes Rule, Total probability Assume

slide-1
SLIDE 1

Continuing Probability.

Wrap up: Total Probability and Conditional Probability.

slide-2
SLIDE 2

Continuing Probability.

Wrap up: Total Probability and Conditional Probability. Product Rule, Correlation, Independence, Bayes’ Rule,

slide-3
SLIDE 3

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN.

slide-4
SLIDE 4

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B].

slide-5
SLIDE 5

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B]. Indeed, B is the union of the disjoint sets An ∩B for n = 1,...,N.

slide-6
SLIDE 6

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B]. Indeed, B is the union of the disjoint sets An ∩B for n = 1,...,N.

slide-7
SLIDE 7

Conditional probability: example.

Two coin flips.

slide-8
SLIDE 8

Conditional probability: example.

Two coin flips. First flip is heads.

slide-9
SLIDE 9

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads?

slide-10
SLIDE 10

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT};

slide-11
SLIDE 11

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space.

slide-12
SLIDE 12

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads:

slide-13
SLIDE 13

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}.

slide-14
SLIDE 14

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}.

slide-15
SLIDE 15

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A;

slide-16
SLIDE 16

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still.

slide-17
SLIDE 17

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still.

slide-18
SLIDE 18

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still. Event B = two heads.

slide-19
SLIDE 19

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still. Event B = two heads. The probability of two heads if the first flip is heads.

slide-20
SLIDE 20

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still. Event B = two heads. The probability of two heads if the first flip is heads. The probability of B given A

slide-21
SLIDE 21

Conditional probability: example.

Two coin flips. First flip is heads. Probability of two heads? Ω = {HH,HT,TH,TT}; Uniform probability space. Event A = first flip is heads: A = {HH,HT}. New sample space: A; uniform still. Event B = two heads. The probability of two heads if the first flip is heads. The probability of B given A is 1/2.

slide-22
SLIDE 22

Conditional Probability.

Definition: The conditional probability of B given A is Pr[B|A] = Pr[A∩B] Pr[A]

slide-23
SLIDE 23

Emptiness..

Suppose I toss 3 balls into 3 bins.

slide-24
SLIDE 24

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”;

slide-25
SLIDE 25

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.”

slide-26
SLIDE 26

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]?

slide-27
SLIDE 27

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]?

slide-28
SLIDE 28

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B]

slide-29
SLIDE 29

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] =

slide-30
SLIDE 30

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] =

slide-31
SLIDE 31

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

slide-32
SLIDE 32

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B]

slide-33
SLIDE 33

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] =

slide-34
SLIDE 34

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

slide-35
SLIDE 35

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B]

slide-36
SLIDE 36

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B] = Pr[A∩B]

Pr[B]

slide-37
SLIDE 37

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B] = Pr[A∩B]

Pr[B]

= (1/27)

(8/27) = 1/8;

slide-38
SLIDE 38

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B] = Pr[A∩B]

Pr[B]

= (1/27)

(8/27) = 1/8; vs. Pr[A] = 8 27.

slide-39
SLIDE 39

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B] = Pr[A∩B]

Pr[B]

= (1/27)

(8/27) = 1/8; vs. Pr[A] = 8 27.

A is less likely given B:

slide-40
SLIDE 40

Emptiness..

Suppose I toss 3 balls into 3 bins. A =“1st bin empty”; B =“2nd bin empty.” What is Pr[A|B]? Pr[B] = Pr[{(a,b,c) | a,b,c ∈ {1,3}] = Pr[{1,3}3] = 8

27

Pr[A∩B] = Pr[(3,3,3)] = 1

27

Pr[A|B] = Pr[A∩B]

Pr[B]

= (1/27)

(8/27) = 1/8; vs. Pr[A] = 8 27.

A is less likely given B: If second bin is empty the first is more likely to have balls in it.

slide-41
SLIDE 41

Outline Conditional Probability

  • Mult. Rule

Bayes Rule Independence Takeaway Counting

Three Card Problem

Three cards: Red/Red, Red/Black, Black/Black. Pick one at random and place on the table. The upturned side is a

  • Red. What is the probability that the other side is Black?

Can’t be the BB card, so...prob should be 0.5, right? R: upturned card is Red; RB: the Red/Black card was selected. Want P(RB|R). What’s wrong with the reasoning that leads to 1

2?

P(RB|R) = P(RB ∩ R) P(R) =

1 3 1 2 1 3(1) + 1 3 1 2 + 1 3(0)

=

1 6 1 2

= 1 3 Once you are given R: it is twice as likely that the RR card was picked.

4

slide-42
SLIDE 42

Gambler’s fallacy.

Flip a fair coin 51 times.

slide-43
SLIDE 43

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads”

slide-44
SLIDE 44

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads”

slide-45
SLIDE 45

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ?

slide-46
SLIDE 46

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH}

slide-47
SLIDE 47

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH} B ∩A = {HH ···HH}

slide-48
SLIDE 48

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH} B ∩A = {HH ···HH} Uniform probability space.

slide-49
SLIDE 49

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH} B ∩A = {HH ···HH} Uniform probability space. Pr[B|A] = |B∩A|

|A|

= 1

2.

slide-50
SLIDE 50

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH} B ∩A = {HH ···HH} Uniform probability space. Pr[B|A] = |B∩A|

|A|

= 1

2.

Same as Pr[B].

slide-51
SLIDE 51

Gambler’s fallacy.

Flip a fair coin 51 times. A = “first 50 flips are heads” B = “the 51st is heads” Pr[B|A] ? A = {HH ···HT,HH ···HH} B ∩A = {HH ···HH} Uniform probability space. Pr[B|A] = |B∩A|

|A|

= 1

2.

Same as Pr[B].

The likelihood of 51st heads does not depend on the previous flips.

slide-52
SLIDE 52

Product Rule

Recall the definition:

slide-53
SLIDE 53

Product Rule

Recall the definition: Pr[B|A] = Pr[A∩B] Pr[A] .

slide-54
SLIDE 54

Product Rule

Recall the definition: Pr[B|A] = Pr[A∩B] Pr[A] . Hence, Pr[A∩B] = Pr[A]Pr[B|A].

slide-55
SLIDE 55

Product Rule

Recall the definition: Pr[B|A] = Pr[A∩B] Pr[A] . Hence, Pr[A∩B] = Pr[A]Pr[B|A]. Consequently, Pr[A∩B ∩C] = Pr[(A∩B)∩C]

slide-56
SLIDE 56

Product Rule

Recall the definition: Pr[B|A] = Pr[A∩B] Pr[A] . Hence, Pr[A∩B] = Pr[A]Pr[B|A]. Consequently, Pr[A∩B ∩C] = Pr[(A∩B)∩C] = Pr[A∩B]Pr[C|A∩B]

slide-57
SLIDE 57

Product Rule

Recall the definition: Pr[B|A] = Pr[A∩B] Pr[A] . Hence, Pr[A∩B] = Pr[A]Pr[B|A]. Consequently, Pr[A∩B ∩C] = Pr[(A∩B)∩C] = Pr[A∩B]Pr[C|A∩B] = Pr[A]Pr[B|A]Pr[C|A∩B].

slide-58
SLIDE 58

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then

slide-59
SLIDE 59

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1].

slide-60
SLIDE 60

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof:

slide-61
SLIDE 61

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction.

slide-62
SLIDE 62

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction. Assume the result is true for n.

slide-63
SLIDE 63

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction. Assume the result is true for n. (It holds for n = 2.)

slide-64
SLIDE 64

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction. Assume the result is true for n. (It holds for n = 2.) Then,

Pr[A1 ∩···∩An ∩An+1] = Pr[A1 ∩···∩An]Pr[An+1|A1 ∩···∩An]

slide-65
SLIDE 65

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction. Assume the result is true for n. (It holds for n = 2.) Then,

Pr[A1 ∩···∩An ∩An+1] = Pr[A1 ∩···∩An]Pr[An+1|A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]Pr[An+1|A1 ∩···∩An],

slide-66
SLIDE 66

Product Rule

Theorem Product Rule Let A1,A2,...,An be events. Then Pr[A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]. Proof: By induction. Assume the result is true for n. (It holds for n = 2.) Then,

Pr[A1 ∩···∩An ∩An+1] = Pr[A1 ∩···∩An]Pr[An+1|A1 ∩···∩An] = Pr[A1]Pr[A2|A1]···Pr[An|A1 ∩···∩An−1]Pr[An+1|A1 ∩···∩An],

so that the result holds for n +1.

slide-67
SLIDE 67

Correlation

An example.

slide-68
SLIDE 68

Correlation

An example. Random experiment: Pick a person at random.

slide-69
SLIDE 69

Correlation

An example. Random experiment: Pick a person at random. Event A: the person has lung cancer.

slide-70
SLIDE 70

Correlation

An example. Random experiment: Pick a person at random. Event A: the person has lung cancer. Event B: the person is a heavy smoker.

slide-71
SLIDE 71

Correlation

An example. Random experiment: Pick a person at random. Event A: the person has lung cancer. Event B: the person is a heavy smoker. Fact: Pr[A|B] = 1.17×Pr[A].

slide-72
SLIDE 72

Correlation

An example. Random experiment: Pick a person at random. Event A: the person has lung cancer. Event B: the person is a heavy smoker. Fact: Pr[A|B] = 1.17×Pr[A]. Conclusion:

◮ Smoking increases the probability of lung cancer by 17%.

slide-73
SLIDE 73

Correlation

An example. Random experiment: Pick a person at random. Event A: the person has lung cancer. Event B: the person is a heavy smoker. Fact: Pr[A|B] = 1.17×Pr[A]. Conclusion:

◮ Smoking increases the probability of lung cancer by 17%. ◮ Smoking causes lung cancer.

slide-74
SLIDE 74

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A].

slide-75
SLIDE 75

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look.

slide-76
SLIDE 76

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A]

slide-77
SLIDE 77

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A] ⇔ Pr[A∩B] = 1.17×Pr[A]Pr[B]

slide-78
SLIDE 78

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A] ⇔ Pr[A∩B] = 1.17×Pr[A]Pr[B] ⇔ Pr[B|A] = 1.17×Pr[B].

slide-79
SLIDE 79

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A] ⇔ Pr[A∩B] = 1.17×Pr[A]Pr[B] ⇔ Pr[B|A] = 1.17×Pr[B]. Conclusion:

◮ Lung cancer increases the probability of smoking by 17%.

slide-80
SLIDE 80

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A] ⇔ Pr[A∩B] = 1.17×Pr[A]Pr[B] ⇔ Pr[B|A] = 1.17×Pr[B]. Conclusion:

◮ Lung cancer increases the probability of smoking by 17%. ◮ Lung cancer causes smoking.

slide-81
SLIDE 81

Correlation

Event A: the person has lung cancer. Event B: the person is a heavy smoker. Pr[A|B] = 1.17×Pr[A]. A second look. Note that Pr[A|B] = 1.17×Pr[A] ⇔ Pr[A∩B] Pr[B] = 1.17×Pr[A] ⇔ Pr[A∩B] = 1.17×Pr[A]Pr[B] ⇔ Pr[B|A] = 1.17×Pr[B]. Conclusion:

◮ Lung cancer increases the probability of smoking by 17%. ◮ Lung cancer causes smoking. Really?

slide-82
SLIDE 82

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B].

slide-83
SLIDE 83

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.)

slide-84
SLIDE 84

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A.

slide-85
SLIDE 85

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich.

slide-86
SLIDE 86

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich. That does not

mean that poor people should buy a Tesla to get rich.

slide-87
SLIDE 87

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich. That does not

mean that poor people should buy a Tesla to get rich.

◮ People who go to the opera are more likely to have a good

career.

slide-88
SLIDE 88

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich. That does not

mean that poor people should buy a Tesla to get rich.

◮ People who go to the opera are more likely to have a good

  • career. That does not mean that going to the opera will

improve your career.

slide-89
SLIDE 89

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich. That does not

mean that poor people should buy a Tesla to get rich.

◮ People who go to the opera are more likely to have a good

  • career. That does not mean that going to the opera will

improve your career.

◮ Rabbits eat more carrots and do not wear glasses.

slide-90
SLIDE 90

Causality vs. Correlation

Events A and B are positively correlated if Pr[A∩B] > Pr[A]Pr[B]. (E.g., smoking and lung cancer.) A and B being positively correlated does not mean that A causes B or that B causes A. Other examples:

◮ Tesla owners are more likely to be rich. That does not

mean that poor people should buy a Tesla to get rich.

◮ People who go to the opera are more likely to have a good

  • career. That does not mean that going to the opera will

improve your career.

◮ Rabbits eat more carrots and do not wear glasses. Are

carrots good for eyesight?

slide-91
SLIDE 91

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN.

slide-92
SLIDE 92

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B].

slide-93
SLIDE 93

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B]. Indeed, B is the union of the disjoint sets An ∩B for n = 1,...,N.

slide-94
SLIDE 94

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Then, Pr[B] = Pr[A1 ∩B]+···+Pr[AN ∩B]. Indeed, B is the union of the disjoint sets An ∩B for n = 1,...,N. Thus, Pr[B] = Pr[A1]Pr[B|A1]+···+Pr[AN]Pr[B|AN].

slide-95
SLIDE 95

Total probability

Assume that Ω is the union of the disjoint sets A1,...,AN. Pr[B] = Pr[A1]Pr[B|A1]+···+Pr[AN]Pr[B|AN].

slide-96
SLIDE 96

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise.

slide-97
SLIDE 97

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads.

slide-98
SLIDE 98

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair?

slide-99
SLIDE 99

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis:

slide-100
SLIDE 100

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,

slide-101
SLIDE 101

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’

slide-102
SLIDE 102

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B].

slide-103
SLIDE 103

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] =

slide-104
SLIDE 104

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] =

slide-105
SLIDE 105

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,

slide-106
SLIDE 106

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] =

slide-107
SLIDE 107

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2

slide-108
SLIDE 108

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2 = Pr[¯ A]

slide-109
SLIDE 109

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2 = Pr[¯ A] Now, Pr[B] = Pr[A∩B]+Pr[¯ A∩B] =

slide-110
SLIDE 110

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2 = Pr[¯ A] Now, Pr[B] = Pr[A∩B]+Pr[¯ A∩B] = Pr[A]Pr[B|A]+Pr[¯ A]Pr[B|¯ A]

slide-111
SLIDE 111

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2 = Pr[¯ A] Now, Pr[B] = Pr[A∩B]+Pr[¯ A∩B] = Pr[A]Pr[B|A]+Pr[¯ A]Pr[B|¯ A] = (1/2)(1/2)+(1/2)0.6 = 0.55.

slide-112
SLIDE 112

Is your coin loaded?

Your coin is fair w.p. 1/2 or such that Pr[H] = 0.6, otherwise. You flip your coin and it yields heads. What is the probability that it is fair? Analysis: A = ‘coin is fair’,B = ‘outcome is heads’ We want to calculate P[A|B]. We know P[B|A] = 1/2,P[B|¯ A] = 0.6,Pr[A] = 1/2 = Pr[¯ A] Now, Pr[B] = Pr[A∩B]+Pr[¯ A∩B] = Pr[A]Pr[B|A]+Pr[¯ A]Pr[B|¯ A] = (1/2)(1/2)+(1/2)0.6 = 0.55. Thus, Pr[A|B] = Pr[A]Pr[B|A] Pr[B] = (1/2)(1/2) (1/2)(1/2)+(1/2)0.6 ≈ 0.45.

slide-113
SLIDE 113

Is your coin loaded?

A picture:

slide-114
SLIDE 114

Is your coin loaded?

A picture:

slide-115
SLIDE 115

Is your coin loaded?

A picture: Imagine 100 situations, among which m := 100(1/2)(1/2) are such that A and B occur and n := 100(1/2)(0.6) are such that ¯ A and B occur.

slide-116
SLIDE 116

Is your coin loaded?

A picture: Imagine 100 situations, among which m := 100(1/2)(1/2) are such that A and B occur and n := 100(1/2)(0.6) are such that ¯ A and B occur. Thus, among the m +n situations where B occurred, there are m where A occurred.

slide-117
SLIDE 117

Is your coin loaded?

A picture: Imagine 100 situations, among which m := 100(1/2)(1/2) are such that A and B occur and n := 100(1/2)(0.6) are such that ¯ A and B occur. Thus, among the m +n situations where B occurred, there are m where A occurred. Hence, Pr[A|B] = m m +n = (1/2)(1/2) (1/2)(1/2)+(1/2)0.6.

slide-118
SLIDE 118

Independence

Definition: Two events A and B are independent if

slide-119
SLIDE 119

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B].

slide-120
SLIDE 120

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

slide-121
SLIDE 121

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are

slide-122
SLIDE 122

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

slide-123
SLIDE 123

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are

slide-124
SLIDE 124

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are not independent;

slide-125
SLIDE 125

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are not independent;

◮ When flipping coins, A = coin 1 yields heads and B = coin

2 yields tails are

slide-126
SLIDE 126

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are not independent;

◮ When flipping coins, A = coin 1 yields heads and B = coin

2 yields tails are independent;

slide-127
SLIDE 127

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are not independent;

◮ When flipping coins, A = coin 1 yields heads and B = coin

2 yields tails are independent;

◮ When throwing 3 balls into 3 bins, A = bin 1 is empty and

B = bin 2 is empty are

slide-128
SLIDE 128

Independence

Definition: Two events A and B are independent if Pr[A∩B] = Pr[A]Pr[B]. Examples:

◮ When rolling two dice, A = sum is 7 and B = red die is 1

are independent;

◮ When rolling two dice, A = sum is 3 and B = red die is 1

are not independent;

◮ When flipping coins, A = coin 1 yields heads and B = coin

2 yields tails are independent;

◮ When throwing 3 balls into 3 bins, A = bin 1 is empty and

B = bin 2 is empty are not independent;

slide-129
SLIDE 129

Independence and conditional probability

Fact: Two events A and B are independent if and only if

slide-130
SLIDE 130

Independence and conditional probability

Fact: Two events A and B are independent if and only if Pr[A|B] = Pr[A].

slide-131
SLIDE 131

Independence and conditional probability

Fact: Two events A and B are independent if and only if Pr[A|B] = Pr[A]. Indeed:

slide-132
SLIDE 132

Independence and conditional probability

Fact: Two events A and B are independent if and only if Pr[A|B] = Pr[A]. Indeed: Pr[A|B] = Pr[A∩B]

Pr[B] , so that

slide-133
SLIDE 133

Independence and conditional probability

Fact: Two events A and B are independent if and only if Pr[A|B] = Pr[A]. Indeed: Pr[A|B] = Pr[A∩B]

Pr[B] , so that

Pr[A|B] = Pr[A] ⇔ Pr[A∩B] Pr[B] = Pr[A]

slide-134
SLIDE 134

Independence and conditional probability

Fact: Two events A and B are independent if and only if Pr[A|B] = Pr[A]. Indeed: Pr[A|B] = Pr[A∩B]

Pr[B] , so that

Pr[A|B] = Pr[A] ⇔ Pr[A∩B] Pr[B] = Pr[A] ⇔ Pr[A∩B] = Pr[A]Pr[B].

slide-135
SLIDE 135

Bayes Rule

Another picture: We imagine that there are N possible causes A1,...,AN.

slide-136
SLIDE 136

Bayes Rule

Another picture: We imagine that there are N possible causes A1,...,AN.

slide-137
SLIDE 137

Bayes Rule

Another picture: We imagine that there are N possible causes A1,...,AN. Imagine 100 situations, among which 100pnqn are such that An and B occur, for n = 1,...,N. Thus, among the 100∑m pmqm situations where B occurred, there are 100pnqn where An occurred.

slide-138
SLIDE 138

Bayes Rule

Another picture: We imagine that there are N possible causes A1,...,AN. Imagine 100 situations, among which 100pnqn are such that An and B occur, for n = 1,...,N. Thus, among the 100∑m pmqm situations where B occurred, there are 100pnqn where An occurred. Hence, Pr[An|B] = pnqn ∑m pmqm .

slide-139
SLIDE 139

Why do you have a fever?

slide-140
SLIDE 140

Why do you have a fever?

Using Bayes’ rule, we find

slide-141
SLIDE 141

Why do you have a fever?

Using Bayes’ rule, we find Pr[Flu|High Fever] = 0.15×0.80 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.58

slide-142
SLIDE 142

Why do you have a fever?

Using Bayes’ rule, we find Pr[Flu|High Fever] = 0.15×0.80 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.58 Pr[Ebola|High Fever] = 10−8 ×1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 5×10−8

slide-143
SLIDE 143

Why do you have a fever?

Using Bayes’ rule, we find Pr[Flu|High Fever] = 0.15×0.80 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.58 Pr[Ebola|High Fever] = 10−8 ×1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 5×10−8 Pr[Other|High Fever] = 0.85×0.1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.42

slide-144
SLIDE 144

Why do you have a fever?

Using Bayes’ rule, we find Pr[Flu|High Fever] = 0.15×0.80 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.58 Pr[Ebola|High Fever] = 10−8 ×1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 5×10−8 Pr[Other|High Fever] = 0.85×0.1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.42 These are the posterior probabilities.

slide-145
SLIDE 145

Why do you have a fever?

Using Bayes’ rule, we find Pr[Flu|High Fever] = 0.15×0.80 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.58 Pr[Ebola|High Fever] = 10−8 ×1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 5×10−8 Pr[Other|High Fever] = 0.85×0.1 0.15×0.80+10−8 ×1+0.85×0.1 ≈ 0.42 These are the posterior probabilities. One says that ‘Flu’ is the Most Likely a Posteriori (MAP) cause of the high fever.

slide-146
SLIDE 146

Bayes’ Rule Operations

slide-147
SLIDE 147

Bayes’ Rule Operations

slide-148
SLIDE 148

Bayes’ Rule Operations

Bayes’ Rule is the canonical example of how information changes our opinions.

slide-149
SLIDE 149

Thomas Bayes

Source: Wikipedia.

slide-150
SLIDE 150

Thomas Bayes

A Bayesian picture of Thomas Bayes.

slide-151
SLIDE 151

Testing for disease.

Let’s watch TV!!

slide-152
SLIDE 152

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male.

slide-153
SLIDE 153

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease)

slide-154
SLIDE 154

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease) A - prostate cancer. B - positive PSA test.

slide-155
SLIDE 155

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease) A - prostate cancer. B - positive PSA test.

◮ Pr[A] = 0.0016, (.16 % of the male population is affected.) ◮ Pr[B|A] = 0.80 (80% chance of positive test with disease.) ◮ Pr[B|A] = 0.10 (10% chance of positive test without

disease.)

slide-156
SLIDE 156

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease) A - prostate cancer. B - positive PSA test.

◮ Pr[A] = 0.0016, (.16 % of the male population is affected.) ◮ Pr[B|A] = 0.80 (80% chance of positive test with disease.) ◮ Pr[B|A] = 0.10 (10% chance of positive test without

disease.) From http://www.cpcn.org/01 psa tests.htm and http://seer.cancer.gov/statfacts/html/prost.html (10/12/2011.)

slide-157
SLIDE 157

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease) A - prostate cancer. B - positive PSA test.

◮ Pr[A] = 0.0016, (.16 % of the male population is affected.) ◮ Pr[B|A] = 0.80 (80% chance of positive test with disease.) ◮ Pr[B|A] = 0.10 (10% chance of positive test without

disease.) From http://www.cpcn.org/01 psa tests.htm and http://seer.cancer.gov/statfacts/html/prost.html (10/12/2011.) Positive PSA test (B). Do I have disease?

slide-158
SLIDE 158

Testing for disease.

Let’s watch TV!! Random Experiment: Pick a random male. Outcomes: (test,disease) A - prostate cancer. B - positive PSA test.

◮ Pr[A] = 0.0016, (.16 % of the male population is affected.) ◮ Pr[B|A] = 0.80 (80% chance of positive test with disease.) ◮ Pr[B|A] = 0.10 (10% chance of positive test without

disease.) From http://www.cpcn.org/01 psa tests.htm and http://seer.cancer.gov/statfacts/html/prost.html (10/12/2011.) Positive PSA test (B). Do I have disease? Pr[A|B]???

slide-159
SLIDE 159

Bayes Rule.

slide-160
SLIDE 160

Bayes Rule.

Using Bayes’ rule, we find

slide-161
SLIDE 161

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10

slide-162
SLIDE 162

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013.

slide-163
SLIDE 163

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013. A 1.3% chance of prostate cancer with a positive PSA test.

slide-164
SLIDE 164

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013. A 1.3% chance of prostate cancer with a positive PSA test. Surgery anyone?

slide-165
SLIDE 165

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013. A 1.3% chance of prostate cancer with a positive PSA test. Surgery anyone? Impotence...

slide-166
SLIDE 166

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013. A 1.3% chance of prostate cancer with a positive PSA test. Surgery anyone? Impotence... Incontinence..

slide-167
SLIDE 167

Bayes Rule.

Using Bayes’ rule, we find P[A|B] = 0.0016×0.80 0.0016×0.80+0.9984×0.10 = .013. A 1.3% chance of prostate cancer with a positive PSA test. Surgery anyone? Impotence... Incontinence.. Death.

slide-168
SLIDE 168

Summary

Events, Conditional Probability, Independence, Bayes’ Rule

slide-169
SLIDE 169

Summary

Events, Conditional Probability, Independence, Bayes’ Rule Key Ideas:

◮ Conditional Probability:

Pr[A|B] = Pr[A∩B]

Pr[B]

slide-170
SLIDE 170

Summary

Events, Conditional Probability, Independence, Bayes’ Rule Key Ideas:

◮ Conditional Probability:

Pr[A|B] = Pr[A∩B]

Pr[B] ◮ Independence: Pr[A∩B] = Pr[A]Pr[B].

slide-171
SLIDE 171

Summary

Events, Conditional Probability, Independence, Bayes’ Rule Key Ideas:

◮ Conditional Probability:

Pr[A|B] = Pr[A∩B]

Pr[B] ◮ Independence: Pr[A∩B] = Pr[A]Pr[B]. ◮ Bayes’ Rule:

Pr[An|B] = Pr[An]Pr[B|An] ∑m Pr[Am]Pr[B|Am].

slide-172
SLIDE 172

Summary

Events, Conditional Probability, Independence, Bayes’ Rule Key Ideas:

◮ Conditional Probability:

Pr[A|B] = Pr[A∩B]

Pr[B] ◮ Independence: Pr[A∩B] = Pr[A]Pr[B]. ◮ Bayes’ Rule:

Pr[An|B] = Pr[An]Pr[B|An] ∑m Pr[Am]Pr[B|Am]. Pr[An|B] = posterior probability;Pr[An] = prior probability .

slide-173
SLIDE 173

Summary

Events, Conditional Probability, Independence, Bayes’ Rule Key Ideas:

◮ Conditional Probability:

Pr[A|B] = Pr[A∩B]

Pr[B] ◮ Independence: Pr[A∩B] = Pr[A]Pr[B]. ◮ Bayes’ Rule:

Pr[An|B] = Pr[An]Pr[B|An] ∑m Pr[Am]Pr[B|Am]. Pr[An|B] = posterior probability;Pr[An] = prior probability .

◮ All these are possible:

Pr[A|B] < Pr[A];Pr[A|B] > Pr[A];Pr[A|B] = Pr[A].