8/30/2018 Department of Veterinary and Animal Sciences Advanced - - PDF document

8/30/2018 1

Department of Veterinary and Animal Sciences

Advanced Quantitative Methods in Herd Management Probabilities and distributions

Anders Ringgaard Kristensen

Outline

Probabilities Conditional probabilities Bayes’ theorem Distributions

• Discrete
• Continuous

Distribution functions Sampling from distributions

• Estimation
• Hypotheses
• Confidence intervals

Department of Veterinary and Animal Sciences Slide 2

Probabilities: Basic concepts

The probability concept is used in daily language. What do we mean when we say:

• The probability of the outcome ”5” when rolling a dice

is 1/6?

• The probability that cow no. 543 is pregnant is 0.40?
• The probability that USA will attack North Korea within

5 years is 0.05?

Department of Veterinary and Animal Sciences Slide 3

8/30/2018 2

Interpretations of probabilities

At least 3 different interpretations are observed:

• A “frequentist” interpretation:
• The probability expresses how frequent we will observe a

given outcome if exactly the same experiment is repeated a “large” number of times. The value is rather

• bjective.
• An objective belief interpretation:
• The probability expresses our belief in a certain

(unobservable) state or event. The belief may be based

• n an underlying frequentist interpretation of similar

cases and thus be rather objective.

• A subjective belief interpretation:
• The probability expresses our belief in a certain

unobservable (or not yet observed) event.

Department of Veterinary and Animal Sciences Slide 4

”Experiments”

An experiment may be anything creating an

• utcome we can observe.

The sample space, S, is the set of all possible

• utcomes.

An event, A, is a subset of S, i.e. A ⊆ S Two events A1 and A2 are called disjoint, if they have no common outcomes, i.e. if A1 ∩ A2 = ∅

Department of Veterinary and Animal Sciences Slide 5

Example of experiment

Rolling a dice:

• The sample space is S = {1, 2, 3, 4, 5, 6}
• Examples of events:
• A1 = {1}
• A2 = {1, 5}
• A3 = {4, 5, 6}
• Since A1 ∩ A3 = ∅, A1 and A3 are disjoint.
• A1 and A2 are not disjoint, because A1 ∩ A2 = {1}

Department of Veterinary and Animal Sciences Slide 6

8/30/2018 3

A simplified definition

Let S be the sample space of an experiment. A probability distribution P on S is a function, so that

• P(S) = 1.
• For any event A ⊆ S, 0 ≤ P(A) ≤ 1
• For any two disjoint events A1 and A2 ,
• P(A1 ∪ A2) = P(A1) + P(A2)

Department of Veterinary and Animal Sciences Slide 7

Example: Rolling a dice

Like before: S = {1, 2, 3, 4, 5, 6} A valid probability function on S is, for A ⊆ S:

• P(A) = |A|/6 where |A| is the size of A (i.e. the

number of elements it contains)

• P({1}) = P({2}) = P({3}) = P({4}) = P({5}) =

P({6}) = 1/6

• P({1, 5}) = 2/6 = 1/3
• P({1, 2, 3}) = 3/6 = 1/2

Notice, that many other valid probability functions could be defined (even though the one above is the only one that makes sense from a frequentist point of view).

Department of Veterinary and Animal Sciences Slide 8

Independence

If two events A and B are independent, then

• P(A ∩ B) = P(A)P(B).

Example: Rolling two dices

• S = {(1, 1), (1, 2),…, (1, 6),…, (6, 6)}
• For any A ⊆ S: P(A) = |A|/36
• A = {(6, 1), (6, 2), …, (6, 6)} ⇒ P(A) = 6/36 = 1/6
• B = {(1, 6), (2, 6), …, (6, 6)} ⇒ P(B) = 6/36 = 1/6
• A ∩ B = {(6, 6)} and P(A ∩ B) = (1/6)(1/6) = 1/36

Department of Veterinary and Animal Sciences Slide 9

8/30/2018 4

Conditional probabilities Let A and B be two events, where P(B) > 0 The conditional probability of A given B is written as P(A|B), and it is by definition

Department of Veterinary and Animal Sciences Slide 10

Example: Rolling a dice

Again, let S = {1, 2, 3, 4, 5, 6}, and P(A) = |A|/6. Define B = {1, 2, 3}, and A = {2}. Then A ∩ B = {2}, and The logical result: If you know the

• utcome is 1, 2 or 3, it is reasonable to

assume that all 3 values are equally probable.

Department of Veterinary and Animal Sciences Slide 11

Conditional sum rule Let A1, A2, …An be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then

Department of Veterinary and Animal Sciences Slide 12

8/30/2018 5

Sum rule: Dice example

Define the 3 disjoint events A1 = {1, 2}, A2 = {3, 4}, A3 = {5, 6} Thus A1 ∪ A2 ∪ A3 = S Define B = {1, 3, 5} (we know that P(B) = ½) P(B| A1) = P(B ∩ A1)/P(A1) = (1/6)/(1/3) = ½ P(B| A2) = P(B ∩ A2)/P(A2) = (1/6)/(1/3) = ½ P(B| A3) = P(B ∩ A3)/P(A3) = (1/6)/(1/3) = ½ Thus

Department of Veterinary and Animal Sciences Slide 13

Bayes’ theorem Let A1, A2, …An be pair wise disjoint events so that Let B be an event so that P(B) > 0. Then Bayes’ theorem is extremely important in all kinds of reasoning under uncertainty. Updating of belief.

Department of Veterinary and Animal Sciences Slide 14

Updating of belief, I

In a dairy herd, the conception rate is known to be 0.40. Define M as the event ”mating” for a cow. Define Π+ as the event ”pregnant” for the same cow, and Π- as the event ”not pregnant”. Thus P(Π+ | M) = 0.40 is a conditional probability. Given that the cow has been mated, the probability of pregnancy is 0.40. Accordingly, P(Π- | M) = 0.60 After 3 weeks the farmer observes the cow for heat. The farmer’s heat detection rate is 0.55. Define H+ as the event that the farmer detects heat. Thus, P(H+ | Π-) = 0.55, and P(H- | Π-) = 0.45 There is a slight risk that the farmer erroneously observes a pregnant cow to be in heat. We assume, that P(H+ | Π+) = 0.01 Notice, that all probabilities are figures that makes sense and are estimated on a routine basis (except P(H+ | Π+) which is a guess)

Department of Veterinary and Animal Sciences Slide 15

8/30/2018 6

Updating of belief, II

Now, let us assume that the farmer observes the cow, and concludes, that it is not in heat. Thus, we have observed the event H- and we would like to know the probability, that the cow is pregnant, i.e. we wish to calculate P(Π+ | H-) We apply Bayes’ theorem: We know all probabilities in the formula, and get In other words, our belief in the event ”pregnant” increases from 0.40 to 0.59 based on a negative heat observation result

Department of Veterinary and Animal Sciences Slide 16

Summary of probabilities

Probabilities may be interpreted

• As frequencies
• As objective or subjective beliefs in certain events

The belief interpretation enables us to represent uncertain knowledge in a concise way. Bayes’ theorem lets us update our belief (knowledge) as new

• bservations are done.

Department of Veterinary and Animal Sciences Slide 17

Discrete distributions

In some cases the probability is defined by a certain function defined over the sample space. In those cases, we say that the outcome is drawn from a standard distribution. There exist standard distributions for many natural phenomena. If the sample space is a countable set, we denote the corresponding distribution as discrete.

Department of Veterinary and Animal Sciences Slide 18

8/30/2018 7

Discrete distributions If X is the random variable representing the outcome, the expected value of a discrete distribution is defined as The variance is defined as We shall look at two important discrete distributions:

• The binomial distribution
• The Poisson distribution.

Department of Veterinary and Animal Sciences Slide 19

The binomial distribution I

Consider an experiment with binary outcomes: Success (s) or failure (f)

• Mating of a sow → Pregnant (s), not pregnant (f)
• Tossing a coin → Heads (s), tails (f)
• Testing for a disease → Present (s), not present (f)

Assume that the probability of success is p and that the experiment is repeated n times. Let X be the total number of successes observed in the n experiments. The sample space of the compound n experiments is S = {0, 1, 2, …, n} The random variable X is then said to be binomially distributed with parameters p and n.

Department of Veterinary and Animal Sciences Slide 20

The binomial distribution II The probability function P(X = k) is (by

• bjective frequentist interpretation) given

by where is the binomial coefficient which may be calculated or looked up in a table.

Department of Veterinary and Animal Sciences Slide 21

8/30/2018 8

The binomial distribution III The mean (expected value) of a binomial distribution is simply E(X) = np. The variance is Var(X) = np(1-p) The binomial distribution is one of the most frequently used distribution for natural phenomena.

Department of Veterinary and Animal Sciences Slide 22

The binomial distribution IV

Three binomial distributions with n = 10 0,05 0,1 0,15 0,2 0,25 0,3 0,35 1 2 3 4 5 6 7 8 9 10 k P(k ) 0,2 0,5 0,8

Three binomial distributions, where n = 10, and p = 0.2, 0.5 and 0.8, respectively.

The Poisson distribution I

If a certain phenomenon occurs a random with a constant intensity (but independently of each

• thers) the total number of occurrences X in a

time interval of a given length (or in a space of a given area) is Poisson distributed with parameter λ Examples:

• Number of (non-infectious) disease cases per

month

• Number of feeding system failures per year
• Number of labor incidents per year

Department of Veterinary and Animal Sciences Slide 24

8/30/2018 9

The Poisson distribution II

The sample space for Y is S = {0, 1, 2, … } The probability function P(X = k) is (by objective frequentist interpretation) given by The expected value is E(X) = λ The variance is Var(X) = λ The Poisson distribution may be used as an approximation for a binomial distribution with ”small” p and ”large” n

Department of Veterinary and Animal Sciences Slide 25

The Poisson distribution III

Three Poisson distributions with λ = 2, 6 and 12, respectively.

Three poisson distributions

0,05 0,1 0,15 0,2 0,25 0,3 5 10 15 20 25 k P(k ) 2 6 12

Continuous distributions

In some cases, the sample space S of a distribution is not countable. If, furthermore, S is an interval on R, the random variable X taking values in S is said to have a continuous distribution. For any x ∈ S, we have P(X = x) = 0. Thus, no probability function exists for a continuous distribution. Instead, the distribution is characterized by a density function f(x).

Department of Veterinary and Animal Sciences Slide 27

8/30/2018 10

Density functions The density function f has the following properties (for a, b ∈ R and a ≤ b) Thus, for a continuous distribution, f can

• nly be interpreted as a probability when

integrated over an interval.

Department of Veterinary and Animal Sciences Slide 28

Continuous distributions

For a continuous distribution, the expected value E(X) is defined as And the variance is (just like the discrete case) We shall here look at 3 important distributions:

• The uniform distribution
• The normal distribution
• The exponential distributions

Department of Veterinary and Animal Sciences Slide 29

The uniform distribution

If S = [a; b], and the random variable X has a uniform distribution on S, then the density function is The expected value and the variance are

Uniform

0,2 0,4 0,6 0,8 1 0,5 1 1,5 2 x f(x)

Department of Veterinary and Animal Sciences Slide 30

8/30/2018 11

The normal distribution I If S = R, and the random variable X has a normal distribution on S, then the density function is The expected value and the variance simply turn out to be E(X) = µ, and Var(X) = σ2 We say that X is N(µ, σ2), or X ~ N(µ, σ2)

Department of Veterinary and Animal Sciences Slide 31

The normal distribution II

The normal distribution may be used to represent almost all kinds of random outcome on the continuous scale in the real world. Exceptions are phenomena that are bounded in some sense (e.g. the waiting time to be served in a queue cannot be negative) It can be showed (central limit theorems) that if X1, X2, …, Xn are random variables of (more or less) any kind, then the sum Yn = X1 + X2 + …+ Xn is normally distributed for n sufficiently large. The normal distribution is the cornerstone among statistical distributions.

Department of Veterinary and Animal Sciences Slide 32

Normal distributions III

Three normal distributions with mean m and standard deviation s

Three normal distributions

0,1 0,2 0,3 0,4 0,5

• 10
• 5

5 10 x f(x) m=0, s=3 m=-5, s=1 m=0, s=1

Department of Veterinary and Animal Sciences Slide 33

8/30/2018 12

Normal distributions IV

The normal distribution with µ = 0, and σ = 1 is called the standard normal distribution. A random variable being standard normally distributed is often denoted as Z The density function of the standard normal distribution is often denoted as φ. It follows that

Department of Veterinary and Animal Sciences Slide 34

Normal distributions V Let X1 ~ N(µ1, σ1

2), X2 ~ N(µ2, σ2 2), and

X1 and X2 are independent. Define Y1 = X1 + X2 and Y1 = X1 − X2. Then

• Y1 ~ N(µ1 + µ2, σ1

2 + σ2 2)

• Y2 ~ N(µ1 − µ2, σ1

2 + σ2 2)

Let a and b be arbitrary real numbers, and let X ~ N(µ, σ2). Define Y =aX + b. Then, Y ~ N(aµ + b, a2σ2)

Department of Veterinary and Animal Sciences Slide 35

Normal distributions VI From the previous slide it follows in particular, that if X ~ N(µ, σ2), then So, if f is the density function of X ~ N(µ, σ2), then Thus, we can calculate the value of any density function for a normal distribution from the density distribution of the standard normal distribution.

Department of Veterinary and Animal Sciences Slide 36

8/30/2018 13

Exponential distribution I If S = R+ = ]0; 1[, and the random variable X has an exponential distribution on S, then the density function is The expected value and the variance are E(X) = λ-1, and Var(X) = λ-2 We say that X is exponentially distributed with parameter λ.

Department of Veterinary and Animal Sciences Slide 37

Exponential distribution II

The exponential distribution is in many ways complimentary to the Poisson distribution. If something happens at random at constant intensity, the number of events within a fixed time interval is Poisson distributed, and the waiting time between two events is exponentially distributed. Less frequently used in herd management.

Department of Veterinary and Animal Sciences Slide 38

Exponential distribution III

Three exponential distributions

0,2 0,4 0,6 0,8 1 2 4 6 8 10 x f(x) 1 0,5 0,2

Three exponential distributions with mean 1, 2 and 5, respectively.

8/30/2018 14

Distribution functions I The distributions presented have all been defined by their probability functions (discrete distributions) and density functions (continuous distributions). We might just as well have used the distribution function F, which is defined in the same way for both classes of distributions:

• F(x) = P(X ≤ x)

Department of Veterinary and Animal Sciences Slide 40

Distribution functions II Even though the definition is the same, the value of the distribution function is calculated in different ways for the two classes of distributions.

• For discrete distributions
• For continuous distributions

Department of Veterinary and Animal Sciences Slide 41

Distribution functions III

It follows directly, that for a continuous distribution, F’(x) = f(x) The distribution function of the standard normal distribution is often denoted as Φ, and naturally Φ’(z) = φ(z) . No closed form (formula) exists for Φ, it must be looked up in tables. For discrete distributions, the distribution function most often doesn’t have a closed form, so it must be looked up in tables.

Department of Veterinary and Animal Sciences Slide 42

8/30/2018 15

Distribution functions IV Any distribution function F has the following two properties:

• F(x) → 0 for x → -∞
• F(x) → 1 for x → ∞

Department of Veterinary and Animal Sciences Slide 43

Distribution function, Binomial

Three binomial distributions with n = 10 0,05 0,1 0,15 0,2 0,25 0,3 0,35 1 2 3 4 5 6 7 8 9 10 k P(k ) 0,2 0,5 0,8 Three binomial distributions with n = 10 0,2 0,4 0,6 0,8 1 1,2 1 2 3 4 5 6 7 8 9 10 k P(k ) 0,2 0,5 0,8

Probability functions to the left, distribution functions to the right.

Distribution function, Poisson

Three poisson distributions 0,05 0,1 0,15 0,2 0,25 0,3 5 10 15 20 25 k P(k ) 2 6 12 Three poisson distributions 0,2 0,4 0,6 0,8 1 1,2 5 10 15 20 25 k P(k ) 2 6 12

Probability function Distribution function

8/30/2018 16

Distribution function, uniform

Uniform

0,2 0,4 0,6 0,8 1 0,5 1 1,5 2 x f(x)

Uniform

0,2 0,4 0,6 0,8 1 0,5 1 1,5 2 x f(x)

Density function to the left Distribution function to the right Distribution function, normal

Three normal distributions 0,1 0,2 0,3 0,4 0,5

• 10
• 5

5 10 x f(x) m=0, s=3 m=-5, s=1 m=0, s=1 Three normal distributions 0,2 0,4 0,6 0,8 1

• 10
• 5

5 10 x f(x) m=0, s=3 m=-5, s=1 m=0, s=1

Density function to the left Distribution function to the right Distribution function, exponential

Three exponential distributions 0,2 0,4 0,6 0,8 1 2 4 6 8 10 x f(x) 1 0,5 0,2 Three exponential distributions 0,2 0,4 0,6 0,8 1 2 4 6 8 10 x f(x) 1 0,5 0,2

Density function to the left Distribution function to the right

8/30/2018 17

Sampling from a distribution

Assume that X1, X2, …, Xn are sampled independently from the same distribution having the known expectation µ and the known standard deviation σ Then the mean of the sample has the expected value µ and the standard deviation In particular, if the Xi’s are N(µ, σ2) then the sample mean is N(µ, σ2/n)

Department of Veterinary and Animal Sciences Slide 49

Sampling from a normal distribution

Assume that X1, X2, …, Xn are sampled independently from the same normal distribution N(µ, σ2) where µ is unknown and σ is known. For some reason we expect (hope) that µ has a certain value µ0, and we would therefore like to test the following hypothesis:

• H0: µ = µ0

How can we do that? Well, we know that the sample mean is N(µ, σ2/n)

Department of Veterinary and Animal Sciences Slide 50

Hypothesis testing, normal dist. I

A normal distribution with standard deviation 3 0,1 0,2

• 1
• 9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

x f(x) m=0, s=3 A normal distribution with standard deviation 3 0,2 0,4 0,6 0,8 1

• 1
• 9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 10

x f(x) m=0, s=3

Observations close to the mean are far more likely than distant

• bservations.

From the distribution function we can calculate the likelihood that an

• bservation falls within the interval µ ± σ

The likelihood that an observation falls within the interval µ ± 2σ Rule of thumb: 2/3 of the observations falls within ±σ and 95% within ±2σ

8/30/2018 18

Hypothesis testing, normal dist. II

We can test our hypothesis H0 for instance by calculating a confidence interval for the mean. A 95% confidence interval for the sample mean (distributed as N(µ, σ2/n)) under H0 is calculated as If the sample mean is included in the interval, we accept H0,

• therwise we reject.

If neither µ nor σ are known, the sample mean becomes student-t distributed (with n-1 degrees of freedom) instead. Then the confidence interval becomes wider as consequence

• f the uncertainty on σ. For large n the student-t distribution

converges towards a standard normal distribution.

Department of Veterinary and Animal Sciences Slide 52

Hypothesis testing, binomial

Assume that we have observed the outcome of X successes out of n in a binomial trial. We would like to test the hypothesis:

• H0: p = p0

Under H0, the expected number of successes is E0(X) = np0 and the variance is Var0 = np0(1-p0) How likely is it that the observed value of X is drawn from a binomial distribution with parameters p0 and n? Basically two approaches may be used:

• Approximate with the normal distribution N(np0,Var0). This

is a reasonable approach if n is big. Remember that n now has a different meaning! We have only one observation from the distribution

• Use the distribution function of the binomial distribution
• directly. Only valid approach for small n.

Department of Veterinary and Animal Sciences Slide 53

Other distributions

Used as hyper distributions for parameters of other distributions in order to represent uncertainty:

• The Gamma distribution (hyper distribution for the mean

and variance of a poisson)

• The Beta distribution (hyper distribution for the p parameter
• f a binomial distribution)

Will be discussed briefly under advanced topics. Distributions for statistical tests:

• The χ2 distribution.
• The student-t distribution
• The F distribution

Those distributions will not be discussed very much in this course. Many other distributions are described in literature …

Department of Veterinary and Animal Sciences Slide 54

8/30/2018 19

What distribution …

… can I use to represent:

• Litter size in sheep?
• Litter size in sows?
• Number of cows/sows conceiving after first

service.

• Time to first estrus?
• Milk yield of dairy cows?
• Daily gain of slaughter pigs?

Department of Veterinary and Animal Sciences Slide 55