Markov Decision Processes: Biosens II E. Jrgensen & Lars R. - - PDF document

SLIDE 1

Markov Decision Processes: Biosens II

• E. Jørgensen & Lars R. Nielsen

Department of Genetics and Biotechnology Faculty of Agricultural Sciences, University of Århus

10/10 2008

SLIDE 2

Background: Markov Decision Processes

Examples

• Dairy Replacement models
• Sow Replacement models
• Delivery strategy
• . . .

Background: Biosens project

• Electronic ID and recording at each milking
• Robot milking
• Yield, temperature
• In-line laboratory
• Lactate DeHydrogenase (LDH)
• Electric Conductivity (EC)
• Somatic cell counts
• Progesterone
• . . .

Background: Biosens project

Methods

• Kalman-filter and DLM/State Space Models
• Multiprocess: outlier and trend model
• Decisions via heuristic decision rules
• Implemented in Herd navigator

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

SLIDE 3

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

Biosens II

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 2 / 13

☞ Biosens II: Improved monitoring and management of dairy production based on on-farm biosensors ☞ Goal: Better detection of oestrus and illnesses ☞ Focus on biomarkers in milk (progesterone, LDH, yield, etc.) ☞ Commercial partner Lattec I/S (FOSS A/S and DeLaval AB) ☞ Five year project (2007-2011). Budget ≈5 mill EUR ☞ Commercial product Herd NavigatorTM based on Biosens project (www.herdnavigator.com)

Use decision theoretical for optimizing of decisions

BioSens II

• Base decisions on herd specific parameters
• Optimize treatment strategies on cow level (economic

point of view).

• Need model for predicting milk yield, degree of infection,
• estrus → state space models (SSMs).
• Need model for calculating the optimal strategy →

Markov decision processes (MDPs).

Use decision theoretical for optimizing of decisions

BioSens II

• Base decisions on herd specific parameters
• Optimize treatment strategies on cow level (economic

point of view).

• Need model for predicting milk yield, degree of infection,
• estrus → state space models (SSMs).
• Need model for calculating the optimal strategy →

Markov decision processes (MDPs).

Project Phases

Phases

• Extend existing Dairy replacement models.
• Include decisions and registrations related to reproduction
• Include decisions and registrations related to disease

(Mastitis)

SLIDE 4

Model components

Phase 1

• Daily yield
• Reproduction
• Involuntary culling
• Growth and economic parameters

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis.

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis.

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis.

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis.

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis.

SLIDE 5

MDPs applied to dairy

Background

→ Biosens II → Subproject 2.3 → MDPs in dairy

MDP intro Dairy HMDP Model OR50 – Sep. 11’th 2008 – 4 / 13

☞ Many papers about the dairy cow replacement problem but limited use in pratice. Reasons could be:

• Often large and complex models with no friendly user

interface.

• Parameters in the model must be estimated, i.e. data

collection frameworks at herd level must exist.

• Stage length: one month up to a year → no assistance when

to inseminate, treat or cull the cow in the current month. ☞ Bio-sensors and cow specific traits/interventions exists in modern dairy herds → parameters can be estimated on a daily basis. Develop MDP with daily stages based on daily yield measurements.

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4 S1 S2 S3 S4

s1,1 s2,1 s3,1 s1,2 s2,2 s3,2 s1,3 s2,3 s3,3 s1,4 s2,4 s3,4

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4 S1 S2 S3 S4

s1,1 s2,1 s3,1 s1,2 s2,2 s3,2 s1,3 s2,3 s3,3 s1,4 s2,4 s3,4 r p

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4 S1 S2 S3 S4

s1,1 s2,1 s3,1 s1,2 s2,2 s3,2 s1,3 s2,3 s3,3 s1,4 s2,4 s3,4

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4 S1 S2 S3 S4

s1,1 s2,1 s3,1 s1,2 s2,2 s3,2 s1,3 s2,3 s3,3 s1,4 s2,4 s3,4

SLIDE 6

What is an MDP?

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 5 / 13

1 2 3 4 S1 S2 S3 S4

s1,1 s2,1 s3,1 s1,2 s2,2 s3,2 s1,3 s2,3 s3,3 s1,4 s2,4 s3,4

Hierarchical MDP (HMDP)

Background MDP intro

→ What is an MDP? → Hierarchical MDP

Dairy HMDP Model OR50 – Sep. 11’th 2008 – 6 / 13

child process child process child process child process child processes

Level 0 Level 1 Level 2

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

SLIDE 7

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

Dairy HMDP properties

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 7 / 13

☞ Hierarchical MDP (HMDP) based on lactation cycles of the cow. ☞ Daily stages during the lactation except for the dry period (49 days). ☞ 3 levels and running over an infinite time-horizon. ☞ State variables are dry week and state variables related to the milk yield SSM embedded into the HMDP + IC state. ☞ Decisions Replace, Keep and Dry. ☞ Maximize the net present reward of the cow.

Hierarchical overview

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 8 / 13

Level cow 1 cow 2 cow 3 cow 4 cow 5 1 parity 1 parity 2 parity 3 parity 12 2 r e p l a c e r e p l a c e d r y new cow new cow insemination starts insemination ends calving calving

Tilstande for hver tidstrin

Background Markov beslutningsprocesser MDP cow model

→ MDP for malkekoen → Hierarkisk ordning → Milk yield model → Tilstande for hver

tidstrin Netto indtægt Sandsynligheder Fremtiden BFG – April 4’th 2008 – 8 / 13

m(1), π(1) m(i), π(i) m(q), π(q) m(1), π(1) m(i), π(i) m(q), π(q)

invol invol replaced replaced

time t time t + 1

replace replaced (return to founder process) keep

π beskriver tidspunkt hvor skal goldes (-1 hvis tidspunkt ikke kendt

endnu). I den nuværende model ca. 11 mill tilstande totalt.

Rewards

For each state and decision we can calculate the net reward

• Replace: Slaughter value of cow
• Keep: Value of milk - feeding costs etc.
• Dry - off : Price of calv - feeding costs etc.

Lactation yield

100 200 300 400 10 20 30 40 50 60 70

time from calving, d

• Obs. yield

SLIDE 8

Lactation yield

censorering

100 200 300 400 10 20 30 40 50 60 70

time from calving, d

• Obs. yield
• Start and end of registrations
• Culling
• Drying-off

Lactation yield

Characteristics

100 200 300 400 10 20 30 40 50 60 70

time from calving, d

• Obs. yield
• Mean lactation curve
• Effect of cow
• Short term fluctuations
• . . .

Milk yield SSM

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 9 / 13

Observed milk yield intensity

Mtc = µt + Ac + Xtc + νtc

Subtract herd effect (remove index c)

Yt = Mt − µt = Fθt + νt = ( 1 1 ) A Xt

• + νt

θt = Gθt−1 + ωt = 1 ρ A Xt−1

• +

ǫt

• where

(θt | Y0, . . . , Yt) ∼ N(mt, Ct)

t milk yield 10 20 30 40 50 60 50 100 150 200 250 1 50 100 150 200 250 2 50 100 150 200 250 3 M t µt t residual milk yield −20 −10 10 20 30 50 100 150 200 250 1 50 100 150 200 250 2 50 100 150 200 250 3 Y t E(Y t | D t−1) E(A 3 | D t−1)

Lactation yield

Estimation

The State Space model can be formulated as a linear normal mixed model and estimated e.g. via R or PROC MIXED in SAS. A spline function is used to estimate the mean lactation curve.

Lactation yield

Estimation

The State Space model can be formulated as a linear normal mixed model and estimated e.g. via R or PROC MIXED in SAS. A spline function is used to estimate the mean lactation curve. Complications !

Lactation yield

DFC mean yield (kg)

10 20 30 40 100 200 300 400 500

1 2 3+

SLIDE 9

Milk yield SSM

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 9 / 13

Observed milk yield intensity

Mtc = µt + Ac + Xtc + νtc

Subtract herd effect (remove index c)

Yt = Mt − µt = Fθt + νt = ( 1 1 ) A Xt

• + νt

θt = Gθt−1 + ωt = 1 ρ A Xt−1

• +

ǫt

• where

(θt | Y0, . . . , Yt) ∼ N(mt, Ct)

t milk yield 10 20 30 40 50 60 50 100 150 200 250 1 50 100 150 200 250 2 50 100 150 200 250 3 M t µt t residual milk yield −20 −10 10 20 30 50 100 150 200 250 1 50 100 150 200 250 2 50 100 150 200 250 3 Y t E(Y t | D t−1) E(A 3 | D t−1)

Embedding the SSM into a MDP

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 10 / 13

☞ Can find P(mt+1 | mt) if store the mean mt and variance Ct in each state. ☞ Discrete states → discretize mt with { ˜

m(1), . . . , ˜ m(q)} and

calculate P( ˜

m(i)

t+1 | ˜

m(j)

t )

☞ Discretization can be done non-uniform (mt = (E(At), E(Xt))).

Embedding the SSM into a MDP

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 10 / 13

☞ Can find P(mt+1 | mt) if store the mean mt and variance Ct in each state. ☞ Discrete states → discretize mt with { ˜

m(1), . . . , ˜ m(q)} and

calculate P( ˜

m(i)

t+1 | ˜

m(j)

t )

☞ Discretization can be done non-uniform (mt = (E(At), E(Xt))).

Embedding the SSM into a MDP

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 10 / 13

☞ Can find P(mt+1 | mt) if store the mean mt and variance Ct in each state. ☞ Discrete states → discretize mt with { ˜

m(1), . . . , ˜ m(q)} and

calculate P( ˜

m(i)

t+1 | ˜

m(j)

t )

☞ Discretization can be done non-uniform (mt = (E(At), E(Xt))).

Embedding the SSM into a MDP

Background MDP intro Dairy HMDP Model

→ HMDP properties → Hierarchical overview → Yield SSM → Embed the SSM → Prelim. results 1 → Prelim. results 2 → Future work

OR50 – Sep. 11’th 2008 – 10 / 13

☞ Can find P(mt+1 | mt) if store the mean mt and variance Ct in each state. ☞ Discrete states → discretize mt with { ˜

m(1), . . . , ˜ m(q)} and

calculate P( ˜

m(i)

t+1 | ˜

m(j)

t )

☞ Discretization can be done non-uniform (mt = (E(At), E(Xt))).

5 10 15 2 4 6 8

uniform non-uniform

Discretization, mt state

−10 −5 5 10 −10 −5 5 10 (−6.5) 1 (2.47) 2 (4.93) 3 (−8.97) 4 (0) 5 (2.46) 6 (−17.97) 7 (−9) 8 (−6.54) 9 (11.45) 10 (0.02) 11 (−2.62) 12 (−14.04) 13 (−1.29) 14 (11.44) 15 (3.9) 16 (−8.84) 17 (9) 18 (−5.08) 19 (−5.06) 20 (8.97) 21 (8.84) 22 (0.01) 23 (−7.54) 24 (1.29) 25 (17.97) 26 (−14.06) 27 (−0.02) 28 (3.93) 29 (14.04) 30 (−11.45) 31 (14.06) 32 (6.54) 33 (−3.93) 34 (6.5) 35 (−3.9) 36 (−11.44) 37 (7.54) 38 (2.62) 39 (−0.01) 40 (−4.93) 41 (5.06) 42 (−2.46) 43 (5.08) 44 (−2.47) m1 m2

SLIDE 10

Reproduction model component (πt state)

• Pregnancy defines end of lactation period
• Decision: Inseminate when oestrus is observed
• Observe heat/oestrus
• Quality of insemination
• Observe pregnancy

. .

(1) Induction Weaning CL regression Puberty (2) Oestrogen peak (3) LH peak (4) Ovulation Start (1) CL regression (2) Oestrogen Peak (7) Follicle rupture (5) Oestrus Start Observable Oestrus (6) Oestrus End Pregnancy recognition

Figure: Outline of events in the oestrus cycle. NB! drawing not scaled according to time.

. .

S2 S3 S′

3

S4 P C S1

Figure: The oestrus-cycle as a semi-Markov Process

Example: Pregnancy rate ∈ {0.5, 0.6, 0.7}

Days from calving to drying off

320 340 360 380 400 420 440 0.000 0.005 0.010 0.015 0.020 0.025

Days from Calving Density

Example: Pregnancy rate ∈ {0.5, 0.6, 0.7}

Days from calving to drying off

320 340 360 380 400 420 440 0.0 0.2 0.4 0.6 0.8 1.0

Days from Calving Probability

Example: Pregnancy rate ∈ {0.5, 0.6, 0.7}

Days from calving to drying off

320 340 360 380 400 420 440 0.00 0.01 0.02 0.03 0.04 0.05

Days from Calving

• Marg. Probability

SLIDE 11

Model component: Involuntary culling

• Calving to start of insemination (Voluntary Waiting

Period)

• From start of insemination until either pregnancy is

confirmed or end of insemination

• Pregnancy confirmation to dry-off
• End of insemination to culling

Model component: Involuntary culling

DFC Daily prob. of involuntary culling.

0.0004 0.0006 0.0008 0.0010 0.0012 0.0014 100 200 300

1 2 3 4 5+ DFC Accumulated prob. of involuntary culling.

0.00 0.05 0.10 0.15 0.20 0.25 100 200 300

Other model components

Based on approach in SIMHERD

dfc Total body weight

580 600 620 640 660 680 100 200 300 400

1 2 3 4 5 6 7 8 9 10 11 12

Other model components

Based on approach in SIMHERD

dfc Total body weight

580 600 620 640 660 680 100 200 300 400

1 2 3 4 5 6 7 8 9 10 11 12

Current state of model

• Model running (Linux)
• Manuscript in prep
• Parameters may need fine-tuning
• Verification of results

Examples

50 100 150 200 250 300 −15 −10 −5 5 10 15 days from start residual yield

• 50

100 150 200 250 300 20 22 24 26 28 30 days from start

• ave. yield
• 50

100 150 200 250 300 2000 4000 6000 8000 days from start RPO −10 −5 5 10 −10 −5 5 10 m1 m2

• Cow 2314803003 lac = 1 dfc = 11 dry = −1

SLIDE 12

• Prelim. results: Examples
• Start of lactation vs. milk yield
• Value of calf
• Replacement vs recovery

Implementation

• R-package(s)
• MLHMP Java library
• C++ program parts
• Linux (large model)

Challenges in final part of project

• Number of state variables → Complexity
• Observed vs. latent variables
• Insemination known / imperfect pregnancy test
• Latent disease state
• Information from other herds.