1 Simple example of simulation Slide 4 Evaluation of expected - - PDF document

1

(Monte Carlo) Simulation

(Monte Carlo) Simulation

What is simulation? Simulation is an attempt to model a real world system in order to:

• Obtain a better understanding of the system

(including interactions)

• Physiological models
• Herd models
• Study the effects of various (complex)

decision strategies

• Herd models

Different kinds of simulation

Deterministic <– forget it in Herd Management

• All calculations based on average (fixed) values
• Same input -> same output
• System comprehension
• Ex: physiological models, differential equations,

Probabilistic model

• All calculations based on distributions
• Same input -> same output
• System comprehension, decision strategies
• Ex: Markov chains

Stochastic (Monte Carlo) models

• All random events are simulated by random

number generation

• Same input -> different output -> need many

replications

• System comprehension, decision strategies
• Herd constraints

2

Simple example of simulation

Evaluation of expected value of production E(V)

Deterministic: Simple stochastic:

• Simulate 100 items with µ = 10, repeat 1000

times

• E(V) = 435.60 SD = 18.73

Stochastic

• Draw µ from N(10,1), simulate 100 items
• Repeat the above 10.000 times
• E(V) = 426.89 SD = 73.86

Consequence of uncertainty

Remember: magenta line illustrates what we know!

3

Different kinds of simulation (cont’d)

Hierarchy

• Mechanistic
• A system is modeled by its elements (sub-systems)
• Ex: A farm consists of buildings and a herd, the herd

consists of individual cows, etc.

• Empirical
• Output directly modeled from the input

Time

• Dynamic
• Ex: A herd is followed over years to evaluate the

effect of a change in replacement policy, time is modeled as a variable

• Static
• Ex: Simulating a milking parlor with respect to

capacity for a single milking

When to use simulation? When other methods fail

• Because of complexity

The TV-show Quiz example

To help formalize the concepts, we will reintroduce the old example of the TV-show Quiz with the three doors:

• The host will show you 3 doors. Behind one of the doors

a prize is hidden. You just have to choose the correct door to claim the prize. However, after use you choose a door, the host will open one of the other doors, always

• ne where the treasure is not hidden. You now have the

choice between : 1) keeping your initial selection; 2) changing your selection to the other unopened door.

• What is the optimal strategy?

You have solved this a couple of times already, but here we try with simulation.

4

Model the quiz

Opened True Choice 1 Choice 2 Gain Assume: equal probability for each door hiding the prize

Model the quiz – identifying components Variables:

• True place: ‘True’ = {1,2,3}
• First choice: ‘Choice 1’ = {1,2,3}
• Door opened: ‘Open’ = {1,2,3} <- Do we

need that?

• Second choice: ‘Choice 2’ =

{‘keep’;’change’}

• Reward: ‘Reward’ = {0,1000}

Decision strategies:

• ‘Choice 1’: Random; ‘Choice 2’=‘keep’
• ‘Choice 1’: Random; ‘Choice 2’=‘change’
• (‘Choice 1’ = 1; ‘Choice 2’ = ‘keep’ if ‘Open’

= 3 else ‘Choice 2’ = ‘change’)

• (And the list goes on and on and on)

Simulating the quiz – What do we need?

We need a method for random sampling from the set {1,2,3}. A method: place the same number of green, yellow and magenta marbles in a brown paperback. Assign a number to each color and start drawing with replacement. Other methods: use a fair dice, flip a coin or … But:

• “The generation of random numbers is too important to

be left to chance”

Use pseudo-random numbers generated by computers, so that we can reproduce the results. Motivation for that: to ensure that changes in output are truly due to changes in the model and not random variation

5

Simulate the quiz – step-by-step

• First randomly draw a door for ‘True’ e.g. 2
• Then randomly select ‘Choice 1’ e.g. 1
• Evaluate decision strategy:
• ‘Choice 2’ = ‘keep’ then ‘Reward’ = 0
• ‘Choice 2’ = ‘Change’ then ‘Reward’ = 1000
• Repeat this 1000 times for each decision

strategy

• Calculate the average gain of each
• Select the best!
• And we are done… (Show Demo + R)

Formal concepts The system consists of three parts:

• Model input:
• A set of decision rules Θ
• A set of parameters Φ
• A set of output variables Ω

The parameter set can be further divided into Φ = {Φ0;ΦS}

• Φ0: ‘state-of-nature’ – initial settings
• ΦS: parameters that change during simulation
• ΦS = [Φs1, Φs2,…, ΦsT] where T is the number of

stages in the planning horizon

Ω is often a subset of Φ

Formal concepts – The TV show Quiz Model input:

• A set of decision rules Θ
• {‘choice 2’ = ‘keep’; ‘choice 2’ = ’change’}
• A set of parameters Φ = {Φ0;ΦS}
• Φ0: ‘state-of-nature’ – initial settings
• p’true’ = (1/3;1/3;1/3)
• p’choice 1’ = (1/3;1/3;1/3)
• ΦS: parameters that change during simulation
• ‘true’, ‘choice 1’, ‘opened’, ’gain’
• A set of output variables Ω
• ‘gain’

So, here Ω is a subset of Φ

6

Purpose of the simulation Calculate the expected utility E(U(θ,Φ)) for each decision rule θ \ Θ, select the

• ne that maximizes the expected utility.
• TV-Show: Expected utility ( or reward) for

each of the two decision rules

Expected utility:

U is the utility function, but it can in general be any function of the output

• variables. The f-functions are density functions

TV-show again – Integral expression The components:

• Φ = {p’true’;p’choice 1’; ‘true’, ‘choice 1’,

‘opened’, ’gain’)

• So one sample for θ1 = ‘change’ could be:
• φi = (p’true’;p’choice 1’;1,1,2,0)
• U(θ1,φi) = 0

The integral is horrible, but we can easily sample from the joint distribution and evaluate the utility function

Finally – we introduce the Monte Carlo part

• Even for TV show problem the integral is more hassle

to solve than it is worth.

• However, sampling from the distribution followed by

evaluation of the utility function is not that hard

• So the trick: Sample a lot (e.g K) samples from the

joint distribution for (θ,Φ) calculate the expected utility as:

This is the Monte Carlo part – a method for numerical integration

7

State of nature: Favorite doors So far we assumed all doors were equal

• Now, assume that host places reward behind

door 1 with 80% and 10% behind each of the others

• Why does it not matter?
• P(succes) = P(true=1)P(choice=1) +

P(true=2)P(choice=2) + P(true=3)P(choice=3) = 0.8*1/3+0.1*1/3+0.1*1/3 = 1/3

• What if you also have door 1 as favorite,

50% vs. 25% for each of the others

• P(succes) = P(true=1)P(choice=1) +

P(true=2)P(choice=2) + P(true=3)P(choice=3) = 0.8*0.5+0.1*0.25+0.1*0.25

State of nature in general State of nature will influence the optimal decision rule as well as the expected result What is the state of nature in a livestock simulation model:

• Average growth rate
• Herd mortality rate
• Average milk yield

Is the true state of nature known (with certainty)

• Does it matter?
• Yes, remember the simple example

Consequence of uncertainty

Remember: magenta line illustrates what we know!

8

State of nature, livestock models

• In livestock models we never know the true state of

nature

• We need to represent the uncertainty of the state of

nature of our herd

• We have some ideas: belief in the true value
• The belief may be represented as a (hyper) distribution

i.e. wi ~ N(µ,22) with µ ~ N(10,12)

• Representing the uncertainty of the state of nature is

essential, but often ignored the consequence is that models should be seen more as research tools than means of optimizing decision rules at a specific farm

A fresh look at the integral

Simulation procedure Select a set of decision rules Θ to be tested For each decision rule θ \ Θ

• Draw n states of nature Φ0

distribution of state of nature pΦ0

• For each state of nature Φ0

• Run the simulation m times
• Calculate the average result Ui over the m

simulations (Monte Carlo integration of the inner integral)

• Calculate the average value of U1, U2, Un

(Monte Carlo integration of the outer integral)

Select the best performing decision rule