Conditional Probabilities Anders Ringgaard Kristensen Department of - - PowerPoint PPT Presentation

Department of Veterinary and Animal Sciences

Conditional Probabilities

Anders Ringgaard Kristensen

Outline

Probabilities Conditional probabilities Bayes’ theorem

Department of Veterinary and Animal Sciences

Slide 2

Probabilities: Basic concepts

The probability concept is used in daily language. What do we mean when we say:

• The probability of the outcome ”5” when rolling a dice

is 1/6?

• The probability that cow no. 543 is pregnant is 0.40?
• The probability that USA will attack North Korea within

5 years is 0.05?

Department of Veterinary and Animal Sciences

Slide 3

Interpretations of probabilities

At least 3 different interpretations are observed:

• A “frequentist” interpretation:
• The probability expresses how frequent we will observe a

given outcome if exactly the same experiment is repeated a “large” number of times. The value is rather

• bjective.
• An objective belief interpretation:
• The probability expresses our belief in a certain

(unobservable) state or event. The belief may be based

• n an underlying frequentist interpretation of similar

cases and thus be rather objective.

• A subjective belief interpretation:
• The probability expresses our belief in a certain

unobservable (or not yet observed) event.

Department of Veterinary and Animal Sciences

Slide 4

”Experiments”

An experiment may be anything creating an

• utcome we can observe.

The sample space, S, is the set of all possible

• utcomes.

An event, A, is a subset of S, i.e. A ⊆ S Two events A1 and A2 are called disjoint, if they have no common outcomes, i.e. if A1 ∩ A2 = ∅

Department of Veterinary and Animal Sciences

Slide 5

Example of experiment

Rolling a dice:

• The sample space is S = {1, 2, 3, 4, 5, 6}
• Examples of events:
• A1 = {1}
• A2 = {1, 5}
• A3 = {4, 5, 6}
• Since A1 ∩ A3 = ∅, A1 and A3 are disjoint.
• A1 and A2 are not disjoint, because A1 ∩ A2 = {1}

Department of Veterinary and Animal Sciences

Slide 6

A simplified definition

Let S be the sample space of an experiment. A probability distribution P on S is a function, so that

• P(S) = 1.
• For any event A ⊆ S, 0 ≤ P(A) ≤ 1
• For any two disjoint events A1 and A2 ,
• P(A1 ∪ A2) = P(A1) + P(A2)

Department of Veterinary and Animal Sciences

Slide 7

Example: Rolling a dice

Like before: S = {1, 2, 3, 4, 5, 6} A valid probability function on S is, for A ⊆ S:

• P(A) = |A|/6 where |A| is the size of A (i.e. the

number of elements it contains)

• P({1}) = P({2}) = P({3}) = P({4}) = P({5}) =

P({6}) = 1/6

• P({1, 5}) = 2/6 = 1/3
• P({1, 2, 3}) = 3/6 = 1/2

Notice, that many other valid probability functions could be defined (even though the one above is the only one that makes sense from a frequentist point of view).

Department of Veterinary and Animal Sciences

Slide 8

Independence

If two events A and B are independent, then

• P(A ∩ B) = P(A)P(B).

Example: Rolling two dices

• S = {(1, 1), (1, 2),…, (1, 6),…, (6, 6)}
• For any A ⊆ S: P(A) = |A|/36
• A = {(6, 1), (6, 2), …, (6, 6)} ⇒ P(A) = 6/36 = 1/6
• B = {(1, 6), (2, 6), …, (6, 6)} ⇒ P(B) = 6/36 = 1/6
• A ∩ B = {(6, 6)} and P(A ∩ B) = (1/6)(1/6) = 1/36

Department of Veterinary and Animal Sciences

Slide 9

Conditional probabilities Let A and B be two events, where P(B) > 0 The conditional probability of A given B is written as P(A|B), and it is by definition

Department of Veterinary and Animal Sciences

Slide 10

Example: Rolling a dice

Again, let S = {1, 2, 3, 4, 5, 6}, and P(A) = |A|/6. Define B = {1, 2, 3}, and A = {2}. Then A ∩ B = {2}, and The logical result: If you know the

• utcome is 1, 2 or 3, it is reasonable to

assume that all 3 values are equally probable.

Department of Veterinary and Animal Sciences

Slide 11

Conditional sum rule Let A1, A2, …An be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then

Department of Veterinary and Animal Sciences

Slide 12

Sum rule: Dice example

Define the 3 disjoint events A1 = {1, 2}, A2 = {3, 4}, A3 = {5, 6} Thus A1 ∪ A2 ∪ A3 = S Define B = {1, 3, 5} (we know that P(B) = ½) P(B| A1) = P(B ∩ A1)/P(A1) = (1/6)/(1/3) = ½ P(B| A2) = P(B ∩ A2)/P(A2) = (1/6)/(1/3) = ½ P(B| A3) = P(B ∩ A3)/P(A3) = (1/6)/(1/3) = ½ Thus

Department of Veterinary and Animal Sciences

Slide 13

Bayes’ theorem Let A1, A2, …An be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then Bayes’ theorem is extremely important in all kinds of reasoning under uncertainty. Updating of belief.

Department of Veterinary and Animal Sciences

Slide 14

Updating of belief, I

In a dairy herd, the conception rate is known to be 0.40. Define M as the event ”mating” for a cow. Define Π+ as the event ”pregnant” for the same cow, and Π- as the event ”not pregnant”. Thus P(Π+ | M) = 0.40 is a conditional probability. Given that the cow has been mated, the probability of pregnancy is 0.40. Accordingly, P(Π- | M) = 0.60 After 3 weeks the farmer observes the cow for heat. The farmer’s heat detection rate is 0.55. Define H+ as the event that the farmer detects heat. Thus, P(H+ | Π-) = 0.55, and P(H- | Π-) = 0.45 There is a slight risk that the farmer erroneously observes a pregnant cow to be in heat. We assume, that P(H+ | Π+) = 0.01 Notice, that all probabilities are figures that makes sense and are estimated on a routine basis (except P(H+ | Π+) which is a guess)

Department of Veterinary and Animal Sciences

Slide 15

Updating of belief, II

Now, let us assume that the farmer observes the cow, and concludes, that it is not in heat. Thus, we have observed the event H- and we would like to know the probability, that the cow is pregnant, i.e. we wish to calculate P(Π+ | H-) We apply Bayes’ theorem: We know all probabilities in the formula, and get In other words, our belief in the event ”pregnant” increases from 0.40 to 0.59 based on a negative heat observation result

Department of Veterinary and Animal Sciences

Slide 16

Summary of probabilities

Probabilities may be interpreted

• As frequencies
• As objective or subjective beliefs in certain events

The belief interpretation enables us to represent uncertain knowledge in a concise way. Bayes’ theorem lets us update our belief (knowledge) as new

• bservations are done.

Department of Veterinary and Animal Sciences

Slide 17