Computational Statistical Modeling of Dynamic Socioeconomic, - PowerPoint PPT Presentation

Computational Statistical Modeling of Dynamic Socioeconomic, Geopolitical and Financial Systems NYU Courant Institute of Mathematical Sciences Applied Mathematics Advanced Topics Course Michael Kwak April 3, 2012

Empirical Study of gamma–Poisson (mixture) distribution under various sampling schemes. • Today’s Objectives: Introduce @risk, a Monte Carlo simulation – software package. Use simulation to study the gamma–Poisson – distribution and its relationship to other distributions. Use simulation to study the impact of various – sampling schemes on the gamma–Poisson.

Introduction to Monte Carlo Simulation

Monte Carlo Simulation • The term Monte Carlo Method was coined by S. Ulam and Nicholas Metropolis in reference to games of chance, a popular attraction in Monte Carlo, Monaco. The random behavior in games of chance is similar to how Monte Carlo simulation selects variable values at random to simulate a model. For example, when you roll a fair six sided die, you know that either a 1, 2, 3, 4, 5, or 6 will come up, but you don't know which for any particular roll. It's the same with the variables that have a known range of values but an uncertain value for any particular time or event.

Monte Carlo Simulation • Monte Carlo simulations are based on the use of random numbers, (1 through 6 for a die) and probability statistics (multinomial distribution with p i =1/6) to investigate problems. You can find Monte Carlo simulations used in everything from economics and finance to nuclear physics. While applications vary widely from field to field, all Monte Carlo simulations use random numbers to examine some problem. (e.g. interest rates, staffing needs, stock prices, inventory, phone calls per minute).

Monte Carlo Simulation • Monte Carlo simulation is categorized as a sampling method because the inputs are randomly generated from probability distributions to simulate the process of sampling from an actual population . • We choose a distribution for the inputs that most closely matches data we already have , or best represents our current state of knowledge . • Can be used to understand how random variation , lack of knowledge , or error affects the sensitivity , performance , or reliability of the system that is being modeled.

Monte Carlo Simulation The data generated from the simulation can be represented as probability • distributions (or histograms) or converted to error bars , reliability predictions , tolerance zones , and confidence intervals .

Monte Carlo Simulation Step 1: Create a (parametric) model* , y = f (x 1 , x 2 , ..., x q ). Step 2: Generate a set of random inputs , x i 1 , x i 2 , ..., x iq . Step 3: Evaluate the model and store the results as y i . Step 4: Repeat steps 2 and 3 for i = 1 to n . Step 5: Analyze the results using histograms, summary statistics, confidence intervals, etc *(the initial model does not need to be parametric)

gamma–Poisson (mixture) distribution

gamma–Poisson (mixture) distribution What we have seen before: gamma–Poisson (mixture) distribution is a continuous mixture of Poisson distributions where the mixing distribution of the Poisson rate is a gamma distribution. a Poisson( λ ) distribution, where λ is itself a random variable, distributed according to Gamma(r, p/(1 − p)) is equivalent to a Negative Binomial.

gamma–Poisson (mixture) distribution from Lecture 1

From Page 3 of the Cook Paper: • Suppose X | Λ is a Poisson random variable and Λ is a gamma( α , β ) random variable. We create a new kind of random variable by starting with a Poisson but making it more variable by allowing the mean parameter to itself be random.

From Page 3 of the Cook Paper: • Therefore the marginal distribution of X is negative binomial with r = α and p = 1/( β + 1). • Is this true? Lets test this assertion through simulation.

Is the marginal distribution of gamma–Poisson a negative binomial with r = α and p = 1/( β + 1) • So from lecture 1, we have a Poisson( λ ) distribution, where λ is itself a random variable, distributed according to Gamma(r, p/(1 − p)) • So suppose p = .75 and r=2.0 • Lets simulate the gamma ‐ Poisson

Lets simulate the gamma ‐ Poisson

Is the marginal distribution of gamma–Poisson a negative binomial with r = α and p = 1/( β + 1) • Remember that according to the lecture slides we have a Gamma (2, 3) • According to Cook, Since β =3 and r=2 we expect that we should have a neg bin with r=2 and p = 1/( β + 1) or p=.25 • Why is this different than the .75 we started with?

It is important to always remember what you are trying to model • In the lecture slides, p denotes the probability of success. • In the Cook paper, where p denotes the probability of a failure , not of a success. To convert formulas between this definition and the one used in the lecture we must, replace “ p ” with “1 − p ”. • So is this what we see from our simulation?

Results of 10,000 iterations gamma– Poisson with Replacement Summary Statistics for Poisson(?) Statistics Percentile Minimum 0.00 5% 0.00 Maximum 39.00 10% 1.00 Mean 6.00 15% 1.00 Std Dev 4.92 20% 2.00 Variance 24.18180918 25% 2.00 Skewness 1.459589057 30% 3.00 Kurtosis 6.102301293 35% 3.00 Median 5.00 40% 4.00 Mode 4.00 45% 4.00 Left X 0.00 50% 5.00 Left P 5% 55% 5.00 Right X 16.00 60% 6.00 Right P 95% 65% 7.00 Diff X 16.00 70% 8.00 Diff P 90% 75% 8.00 #Errors 0 80% 9.00 Filter Min Off 85% 11.00 Filter Max Off 90% 12.00 #Filtered 0 95% 16.00

gamma–Poisson distribution relationship to other distributions.

gamma–Poisson distribution relationship to other distributions : Poisson • From the Cook paper section 4 paragraph 4, • Let’s see if this is true • How can we set this up and what do we expect to see?

gamma–Poisson distribution relationship to other distributions: Poisson

gamma–Poisson distribution relationship to other distributions: Geometric From the Cook paper section 5. How can we set this up and what do we expect to see?

gamma–Poisson distribution relationship to other distributions: Geometric

gamma–Poisson distribution relationship to other distributions: Geometric

gamma–Poisson distribution relationship to other distributions: Neg Binomial From the Cook paper section 5. How can we set this up and what do we expect to see?

gamma–Poisson distribution relationship to other distributions: Neg Binomial

gamma–Poisson distribution relationship to other distributions: Neg Binomial

gamma–Poisson distribution relationship to other distributions: Binomial From the Cook paper section 5. How can we set this up and what do we expect to see?

gamma–Poisson distribution relationship to other distributions: Binomial

gamma–Poisson distribution relationship to other distributions: Gamma From the Cook paper section 5. How can we set this up and what do we expect to see?

gamma–Poisson distribution relationship to other distributions: Gamma

impact of various sampling schemes on the gamma–Poisson

Remember our Sampling Schemes

Sampling Without Replacement • Before we look at the simulation, what do we expect to see with regard to p? • What impact do we expect this to have on our gamma–Poisson mixture?

Sampling Without Replacement

Sampling Without Replacement

Replacement Plus 1 if True • Before we look at the simulation, what do we expect to see with regard to p? • What impact do we expect this to have on our gamma–Poisson mixture?

Replacement Plus 1 if True

Replacement Plus 1 if True

Replacement Plus One: “Chinese Restaurant Process” • For simplicity we will consider the 1 dimensional case, which can be modeled with the beta ‐ binomial. • What do we expect to see with regard to p? • What impact do we expect this to have on our gamma–Poisson mixture?

Replacement Plus One: “Chinese Restaurant Process”

Replacement Plus One: “Chinese Restaurant Process”