Computational Statistical Modeling of Dynamic Socioeconomic, - - PowerPoint PPT Presentation

Computational Statistical Modeling

• f Dynamic Socioeconomic,

Geopolitical and Financial Systems

NYU Courant Institute of Mathematical Sciences Applied Mathematics Advanced Topics Course

Michael Kwak

April 3, 2012

Empirical Study of gamma–Poisson (mixture) distribution under various sampling schemes.

• Today’s Objectives:

– Introduce @risk, a Monte Carlo simulation software package. – Use simulation to study the gamma–Poisson distribution and its relationship to other distributions. – Use simulation to study the impact of various sampling schemes on the gamma–Poisson.

Introduction to Monte Carlo Simulation

Monte Carlo Simulation

• The term Monte Carlo Method was coined by S. Ulam and

Nicholas Metropolis in reference to games of chance, a popular attraction in Monte Carlo, Monaco. The random behavior in games of chance is similar to how Monte Carlo simulation selects variable values at random to simulate a

• model. For example, when you roll a fair six sided die, you

know that either a 1, 2, 3, 4, 5, or 6 will come up, but you don't know which for any particular roll. It's the same with the variables that have a known range of values but an uncertain value for any particular time or event.

Monte Carlo Simulation

• Monte Carlo simulations are based on the use of random

numbers, (1 through 6 for a die) and probability statistics (multinomial distribution with pi=1/6) to investigate problems. You can find Monte Carlo simulations used in everything from economics and finance to nuclear physics. While applications vary widely from field to field, all Monte Carlo simulations use random numbers to examine some problem. (e.g. interest rates, staffing needs, stock prices, inventory, phone calls per minute).

Monte Carlo Simulation

• Monte Carlo simulation is categorized as a sampling

method because the inputs are randomly generated from probability distributions to simulate the process of sampling from an actual population.

• We choose a distribution for the inputs that most closely

matches data we already have, or best represents our current state of knowledge.

• Can be used to understand how random variation, lack of

knowledge, or error affects the sensitivity, performance, or reliability of the system that is being modeled.

Monte Carlo Simulation

• The data generated from the simulation can be represented as probability

distributions (or histograms) or converted to error bars, reliability predictions, tolerance zones, and confidence intervals.

Monte Carlo Simulation

Step 1: Create a (parametric) model*, y = f(x1, x2, ..., xq). Step 2: Generate a set of random inputs, xi1, xi2, ..., xiq. Step 3: Evaluate the model and store the results as yi. Step 4: Repeat steps 2 and 3 for i = 1 to n. Step 5: Analyze the results using histograms, summary statistics, confidence intervals, etc

*(the initial model does not need to be parametric)

gamma–Poisson (mixture) distribution

gamma–Poisson (mixture) distribution

What we have seen before:

gamma–Poisson (mixture) distribution is a continuous mixture of Poisson distributions where the mixing distribution of the Poisson rate is a gamma distribution. a Poisson(λ) distribution, where λ is itself a random variable, distributed according to Gamma(r, p/(1 − p)) is equivalent to a Negative Binomial.

gamma–Poisson (mixture) distribution from Lecture 1

From Page 3 of the Cook Paper:

• Suppose X |Λ is a Poisson random variable and Λ is a

gamma(α, β) random variable. We create a new kind of random variable by starting with a Poisson but making it more variable by allowing the mean parameter to itself be random.

From Page 3 of the Cook Paper:

• Therefore the marginal distribution of X is

negative binomial with r = α and p = 1/(β + 1).

• Is this true? Lets test this assertion through

simulation.

Is the marginal distribution of gamma–Poisson a negative binomial with r = α and p = 1/(β + 1)

• So from lecture 1, we have a Poisson(λ) distribution,

where λ is itself a random variable, distributed according to Gamma(r, p/(1 − p))

• So suppose p = .75 and r=2.0
• Lets simulate the gamma‐Poisson

Lets simulate the gamma‐Poisson

Is the marginal distribution of gamma–Poisson a negative binomial with r = α and p = 1/(β + 1)

• Remember that according to the lecture

slides we have a Gamma (2, 3)

• According to Cook, Since β =3 and r=2 we

expect that we should have a neg bin with r=2 and p = 1/(β + 1) or p=.25

• Why is this different than the .75 we started

with?

It is important to always remember what you are trying to model

• In the lecture slides, p denotes the probability
• f success.
• In the Cook paper, where p denotes the

probability of a failure, not of a success. To convert formulas between this definition and the one used in the lecture we must, replace “p” with “1 − p”.

• So is this what we see from our simulation?

Results of 10,000 iterations gamma– Poisson with Replacement

Statistics Percentile Minimum 0.00 5% 0.00 Maximum 39.00 10% 1.00 Mean 6.00 15% 1.00 Std Dev 4.92 20% 2.00 Variance 24.18180918 25% 2.00 Skewness 1.459589057 30% 3.00 Kurtosis 6.102301293 35% 3.00 Median 5.00 40% 4.00 Mode 4.00 45% 4.00 Left X 0.00 50% 5.00 Left P 5% 55% 5.00 Right X 16.00 60% 6.00 Right P 95% 65% 7.00 Diff X 16.00 70% 8.00 Diff P 90% 75% 8.00 #Errors 80% 9.00 Filter Min Off 85% 11.00 Filter Max Off 90% 12.00 #Filtered 95% 16.00

Summary Statistics for Poisson(?)

gamma–Poisson distribution relationship to other distributions.

gamma–Poisson distribution relationship to other distributions: Poisson

• From the Cook paper section 4 paragraph 4,
• Let’s see if this is true
• How can we set this up and what do we expect to

see?

gamma–Poisson distribution relationship to other distributions: Poisson

gamma–Poisson distribution relationship to other distributions: Geometric

How can we set this up and what do we expect to see? From the Cook paper section 5.

gamma–Poisson distribution relationship to other distributions: Geometric

gamma–Poisson distribution relationship to other distributions: Geometric

gamma–Poisson distribution relationship to other distributions: Neg Binomial

How can we set this up and what do we expect to see? From the Cook paper section 5.

gamma–Poisson distribution relationship to other distributions: Neg Binomial

gamma–Poisson distribution relationship to other distributions: Neg Binomial

gamma–Poisson distribution relationship to other distributions: Binomial

How can we set this up and what do we expect to see? From the Cook paper section 5.

gamma–Poisson distribution relationship to other distributions: Binomial

gamma–Poisson distribution relationship to other distributions: Gamma

How can we set this up and what do we expect to see? From the Cook paper section 5.

gamma–Poisson distribution relationship to other distributions: Gamma

impact of various sampling schemes

• n the gamma–Poisson

Remember our Sampling Schemes

Sampling Without Replacement

• Before we look at the simulation, what do we

expect to see with regard to p?

• What impact do we expect this to have on our

gamma–Poisson mixture?

Sampling Without Replacement

Sampling Without Replacement

Replacement Plus 1 if True

• Before we look at the simulation, what do we

expect to see with regard to p?

• What impact do we expect this to have on our

gamma–Poisson mixture?

Replacement Plus 1 if True

Replacement Plus 1 if True

Replacement Plus One: “Chinese Restaurant Process”

• For simplicity we will consider the 1

dimensional case, which can be modeled with the beta‐binomial.

• What do we expect to see with regard to p?
• What impact do we expect this to have on our

gamma–Poisson mixture?

Replacement Plus One: “Chinese Restaurant Process”

Replacement Plus One: “Chinese Restaurant Process”