Markov Decision Processes Case example sow replacement Anders - - PowerPoint PPT Presentation

Markov Decision Processes

Case example – sow replacement Anders Ringgaard Kristensen Presented by Leonardo de Knegt

Department of Large Animal Sciences

A sow replacement model

At every weaning or return to oestrus it is decided whether or not to cull the sow. The decision is based on a prediction of the future performance of the sow. The prediction is compared to the expected performance

• f a gilt.

Only based on existing registrations.

Toft, N. & E. Jørgensen. 2002. Estimation of farm specific parameters in a longitudinal model for litter size with variance components and random dropout. Livestock Production Science 77, 175-185. Kristensen, A.R. & T.A. Søllested. 2004. A sow replacement model using Bayesian updating in a three-level hierarchic Markov process I. Biological model. Livestock Production Science 87, 13-24. Kristensen, A.R. & T.A. Søllested. 2004. A sow replacement model using Bayesian updating in a three-level hierarchic Markov process II. Optimization model. Livestock Production Science 87, 25-36.

Prediction of sow performance

Based on

• Individual conditions
• Age
• Litter size results (all)
• Re-matings
• Herd specific conditions
• Litter size profile of the herd
• Level of involuntary replacement
• Piglet mortality
• Prices
• Feed intake
• Standard conditions
• Weight of sows

What do we know about litter size in sows?

The profile for a herd:

• It is lower for a gilt than for a second parity sow
• A maximum is reached at parity 3, 4, 5
• It is decreasing for older sows
• Some herds produce at a higher level than others

Within herd:

• Some sows produce at a higher level than others
• The repeatability over parities is rather low
• Large random variation

Litter size profiles of sows in different herds

Department of Veterinary and Animal Sciences

Slide 5

Litter size profiles

Estimated by Toft & Jørgensen (2002). Censored data:

• Only the best sows are kept.
• The results for high parities are unknown for sows that have already

been culled.

• Must be taken into account by the estimation procedure.
• If ignored, the expected litter size is over-estimated for high parities.

A litter size model

Based on Toft & Jørgensen (2002), Bono et al. (2012) suggested the following litter size model meeting the demands mentioned on the previous slide: Yit = µt + Mi(t) + εit , where

• Yit is the litter size of sow i at parity t
• µt = is the litter size profile of the herd (average litter size at parity t)

described by 5 parameters, θ1, θ2, θ3, θ4, θ5

• Mi(t) ~ N(0, σ2) is the effect of sow i at parity t
• εit ~ N(0, τ2) is random variation (noise)

The sow effect is auto correlated over parities:

• Cov(Mi(t), Mi(t+u)) = exp(-uα)σ2, in other words
• Mi(t) = ρMi(t-1) + ηit , where
• ρ = exp(-α)
• ηit ~ N(0, (1-ρ2)σ2)

Described by 8 herd specific parameters: θ1, θ2, θ3, θ4, θ5, τ, σ, α

Department of Veterinary and Animal Sciences

Slide 7

8 10 12 14 16 1 2 3 4 5 6 7 8 9 10 11 12 Litter size Parity

Litter size profile for a specific herd: 5 parameters

Slope = θ5 θ1 θ2 θ3 θ4

Litter size profile for a specific herd

7 9 11 13 15 1 2 3 4 5 6 7 8 9 10 11 12 Parity Litter size

Estimated from herd data Herd level productivity Censoring:

• Only the best sows are kept
• Profile will be biased if simple

averages are used

• A maximum likelihood

estimation technique is used

Litter size profile for a specific herd

7 9 11 13 15 1 2 3 4 5 6 7 8 9 10 11 12 Parity Litter size

Estimated from herd data Herd level productivity Individual sows are compared to that

• Sow 1 is below average
• Sow 2 is above average

Sow 1 Sow 2

Prediction for an individual sow

Based on:

• Litter size profile of the herd
• Individual deviations from the litter size profile
• All previous litter sizes are taken into account
• Risk of return to oestrus
• Previous matings

Uncertainty about the prediction is taken into account

• Not just “average”

Department of Veterinary and Animal Sciences

Slide 11

The decision influences the future

5 6 1 1 1 2 2 1 3 7 8 1 1 1 2 Present sow is a parity 5 sow. Depending on the decision made, we may “next time” have a:

• 6th parity sow (if we keep)
• 1st parity sow (if we replace)

Looking two cycles forward, we may have a:

• 7th parity sow (if we still keep)
• 2nd parity sow (if we replaced last time)
• 1st parity sow (if we replaced this time)

Looking three cycles forward, we may have an 8th parity sow, a 1st parity sow, a 2nd parity sow

• r a 3rd parity sow.

Right now: Choose between

• The red tree
• The green tree

Department of Veterinary and Animal Sciences

Slide 12

1 1 1 2 2 1 3 5 6 7 8 1 1 1 2

Why is it difficult?

What we observe is not what we wish to know:

• Litter size versus productivity potential
• The selection problem

Many traits must be considered Combinatorical explosion Herd differences

Department of Veterinary and Animal Sciences

Slide 13

Illustration of the hierarchy

Founder Child 1

Sow 1 Sow 2 Sow 3 1 2 3 4 1 2 3 4 1 2 No state nor action Action: Mating method

Child 2

M G S M G S M G S M G S M G S M G S M G S M G S M G S M G S

Department of Veterinary and Animal Sciences

Slide 14

Stage length: lifespan of a sow Stage length: Reproductive cycle (parity) Stage length: Duration of “Mating”, “Gestation” and “Suckling” Actions Mating: allow 1…5 matings Gestations: none Suckling: keep or cull after weaning

Model structure: follow with page 28 of paper II

Founder level

• Stage: Lifespan of a sow (variable)
• No state and action

Child 1

• Stage: Reproductive cycle (weaning)
• State space:
• 1st parity: None
• 2nd parity: Litter size + culled
• 3rd parity (and older): Updated estimate for M(t) + culled
• Action: Mating method

Child 2:

• Stage: Mating, gestation, suckling
• State space:
• Mating: Healthy/Diseased
• Gestation: Pregnant/Open/Diseased
• Suckling Litter size + diseased
• Actions
• Mating: Allow 1,…,5 matings
• Gestation: None
• Suckling: Keep/Cull

Department of Veterinary and Animal Sciences

Slide 15

Summary, state space

A sow is described by:

• Parity
• Reproductive state (mating/gestation/suckling)
• Estimated value of M(t) based on all litter size results
• Present litter size (if suckling)

A herd is described by

• Litter size parameters
• Mortality
• Involuntary culling
• Conception rates
• Prices

Department of Veterinary and Animal Sciences

Slide 16

Decisions

Number of (re-)matings before culling Keep/Replace

• Binary decision
• Economic value for ranking

Department of Veterinary and Animal Sciences

Slide 17

Calibration of biological parameters

Department of Veterinary and Animal Sciences

Slide 18