06-10-2014 Institut for Produktionsdyr og Heste Optimal replacement - - PDF document

06-10-2014 1

Optimal replacement policies for dairy cows based on daily yield measurements

Katarina Nielsen Dominiak, Ph.D Student Advanced Quantitative Methods in Herd Management

Institut for Produktionsdyr og Heste AQMHM 06-10-2014 Dias 1

In all types of life stock production the decision of when to replace or keep an animal is mulitifactorial For dairy cows some factors are

• Current milk yield
• Expectations to future milk yield
• Illness
• Replacement heifers available
• Reproduction
• Goal of the farmer/producer
• Market prices
• Milk
• Heifers
• Feed
• Etc

Institut for Produktionsdyr og Heste AQMHM 06-10-2014 Dias 2

Background Always a new situation

Every day you know more - so when is the optimal time to make decisions?

AQMHM 06-10-2014 Dias 3 Institut for Produktionsdyr og Heste

06-10-2014 2

Markov Decision Process - MDP

Observation of state -> Decision -> Reward and probabilistic evolution of the system to a new state Has been used for replacement policies for decades Often calculating economic impact of factors – used by academics and experts Stage lengths in previous models from 1 month to 1 year Not suitable for the farmer or for day-to-day decisions

AQMHM 06-10-2014 Dias 4 Institut for Produktionsdyr og Heste

Hierarchial Markov Decision Process - HMDP

A series of MDP built together into one MDP Stages of the process expanded to a child process – multiple times if needed AMS with biosensors provide detailed information on a daily basis But no dairy cow replacement models based on daily time steps exist This paper describes the first step

• f developing such a model

AQMHM 06-10-2014 Dias 5 Institut for Produktionsdyr og Heste

Structure of the model

AQMHM 06-10-2014 Dias 6 Institut for Produktionsdyr og Heste

06-10-2014 3

Biological Functions used in the model

• Milk Yield
• Growth
• Feeding and Energy
• Reproduction
• Involuntary Culling (IC)

AQMHM 06-10-2014 Dias 7 Institut for Produktionsdyr og Heste

Daily milk yield

Daily milk yield measurements on individual level available t = day in lactation (measured as days from calving) j = lactation number µ = average daily milk yield (herd level) M = milk yield in kilograms (cow level) A = production potential (cow level) ~ N(0,σ2

A)

X = local production effect (cow level) v = random error term ~ N(0, σ2

v) AQMHM 06-10-2014 Dias 8 Institut for Produktionsdyr og Heste

Daily milk yield at lactation 3

AQMHM 06-10-2014 Dias 9 Institut for Produktionsdyr og Heste

06-10-2014 4

Substracting average daily milk yield on both sides gives the residual milk yield – how does the yield differ from expected:

Residual milk yield

AQMHM 06-10-2014 Dias 10 Institut for Produktionsdyr og Heste

Milk yield modeled as a State Space Model (SSM)

SSM combines relevant prior knowledge and current information through an observation equation and a system equation Residual daily milk yield (observation equation) Can be written using matrix notation A is constant over time hence expressing a permanent trait for the cow

AQMHM 06-10-2014 Dias 11 Institut for Produktionsdyr og Heste

Milk yield modeled as a State Space Model (SSM)

The system equation describes the relationship between variables A and X from time t – 1 to time t W is the covariance matrix to random variable wt,j and ρ is an autocorelation factor

AQMHM 06-10-2014 Dias 12 Institut for Produktionsdyr og Heste

06-10-2014 5

Milk yield modeled as a State Space Model (SSM)

Assumption to is that the prior is normal distributed with mean and covariance matrix:

AQMHM 06-10-2014 Dias 13 Institut for Produktionsdyr og Heste

Milk yield modeled as a State Space Model (SSM)

AQMHM 06-10-2014 Dias 14 Institut for Produktionsdyr og Heste

Milk yield modeled as a State Space Model (SSM)

For j > 1 the expected production potential is the same as estimated during the last lactation An estimation of the production potential of 5 means that the cow should produce 5 kg above herd average every day

AQMHM 06-10-2014 Dias 15 Institut for Produktionsdyr og Heste

06-10-2014 6

Growth

Total body weight (BW) was estimated using Gompertz growth curve and a BCS curve yielding the BCS during lactation

AQMHM 06-10-2014 Dias 16 Institut for Produktionsdyr og Heste

Feeding and Energy

Measured in the net energy of Scandinavian Feed Units (SFU) Total amount of energy needed during one cycle is the sum of energy needed for:

• Maintenance
• Fetus growth
• Milk yield
• Standardized BW gain

AQMHM 06-10-2014 Dias 17 Institut for Produktionsdyr og Heste

Reproduction

A model simulating the estrus cycle was constructed based on a continuous Markov chain. The model could calculate the probability of a positive pregnancy test

AQMHM 06-10-2014 Dias 18 Institut for Produktionsdyr og Heste 35 days from calving till insemination 40 days from insemination till pregnancy test Repeatedly inseminated till day 250 if not tested pregnant 193 days lactation from positive pregnancy test till drying off 49 days from drying off till calving

06-10-2014 7

Involuntary culling (IC)

Dead cows or cows slaughtered because of other reasons than milk yield or failure to concieve IC is typically influenced by factors like

• Lactation stage (max 10 in this model)
• Disease incidence
• Age of the cow

AQMHM 06-10-2014 Dias 19 Institut for Produktionsdyr og Heste

Model structure

AQMHM 06-10-2014 Dias 20 Institut for Produktionsdyr og Heste

Stage length, States, and Decisions

Level 0: State – dummy, representing insertion of new cow Stage – defined by child process, life span of a cow (level 1) Level 1: State – Expected production potential of the cow (Ât,j) 13 levels Stage – defined by child process, lactation (level 2) maximum 10 stages Level 2: State – defined by a combination of and drying off week ŵ Moreover an IC state was added Stage – one day (except drying off week stage which was defined as one week) Action – keep or replace

AQMHM 06-10-2014 Dias 21 Institut for Produktionsdyr og Heste

06-10-2014 8

Transition Probabilities

A child process at level 2 representing lactation j = a state at stage t = milk yield = drying off week is either described through or the IC state

AQMHM 06-10-2014 Dias 22 Institut for Produktionsdyr og Heste

Transition Probabilities

Given state , unknown pregnancy status and decision keep the transition probabilities are = the probability of IC = the probability of a positive pregnancy test = the probability of a transition from to

AQMHM 06-10-2014 Dias 23 Institut for Produktionsdyr og Heste

Transition Probabilities

Given state , known pregnancy status and decision keep the transition probabilities are

AQMHM 06-10-2014 Dias 24 Institut for Produktionsdyr og Heste

06-10-2014 9

Transition Probabilities

If decision replace is made the process returned to level 0 with probability 1 If the cow entered the IC state the process returned to level 0 with probability 1 At the end of drying off week the process returned to level 1 (next lactation)

AQMHM 06-10-2014 Dias 25 Institut for Produktionsdyr og Heste

Rewards

Rewards were equal to the expected economic net revenue (NPV) Revenues from

• Milk
• Calves
• Carcasses

Minus costs of

• Feed
• New heifer

AQMHM 06-10-2014 Dias 26 Institut for Produktionsdyr og Heste

Optimizing the model

The objective was to maximize the NPV using a specific discount rate The optimizing was done by using a combination of

• Value iteration

and

• Policy iteration

AQMHM 06-10-2014 Dias 27 Institut for Produktionsdyr og Heste

06-10-2014 10

Results

AQMHM 06-10-2014 Dias 28 Institut for Produktionsdyr og Heste

Results

Retention pay off (RPO) = NPV(keep) – NPV(replace)

AQMHM 06-10-2014 Dias 29 Institut for Produktionsdyr og Heste AQMHM 06-10-2014 Dias 30 Institut for Produktionsdyr og Heste

Results

06-10-2014 11

Results

AQMHM 06-10-2014 Dias 31 Institut for Produktionsdyr og Heste

Results

AQMHM 06-10-2014 Dias 32 Institut for Produktionsdyr og Heste

Summary

The HMDP model updates on a daily basis Herd specific SSM parameters (lactation curves, revenues and expenditures) make the model useable for the farmer Real life validation is expensive and difficult All models are validated through model validation techniques (plots, input data validation, sensitivity analysis and comparison to other models) Seasonal effects are not included Decisions on insemination and treatment not included This HMDP model is a step in the direction of implementation in real herds

AQMHM 06-10-2014 Dias 33 Institut for Produktionsdyr og Heste