Stochastic Simulation Generation of random variables Discrete - PowerPoint PPT Presentation

Stochastic Simulation Generation of random variables Discrete sample space Bo Friis Nielsen Applied Mathematics and Computer Science Technical University of Denmark 2800 Kgs. Lyngby – Denmark Email: bfn@imm.dtu.dk

Plan W1.1-2 Plan W1.1-2 Random number generation Independent, uniformly distributed RN Distribution Independent Random variable Model Output DTU 02443 – lecture 3 2

Random variables Random variables Aim • The scope is the generation of independent random variables X 1 , X 2 , ... X n with a given distribution , F x ( x ) , (or probability density function [pdf]). • We assume we have access to a supply ( U i ) of random numbers, independent samples from the uniform distribution on ]0 , 1[ . • Our task is to transform U i into X i . DTU 02443 – lecture 3 3

Uniform distribution I Uniform distribution I Our norm distribution or building block, U (0 , 1) f ( x ) = 1 F ( x ) = x for 0 ≤ x ≤ 1 E ( X ) = 1 1 Var ( X ) = 2 12 1 0 1 DTU 02443 – lecture 3 4

Coin Coin or uniform distribution ����������� ����������� 1 ����������� ����������� 0 ����������� ����������� 1 ����������� ����������� ����������� ����������� 1/2 0 1 X = 0 , 1 1/2 0 P ( X = i ) = 1 0 1 2 1/2 � � U > 1 X := X = ⌊ (2 U ) ⌋ 2 0 0 1 DTU 02443 – lecture 3 5

Bernoulli trial Bernoulli trial Toss a coin with P ( X = 1) = p and P ( X = 0) = 1 − p . F ( x ) = P ( X ≤ x ) X = U > 1 − p PDF X = 0 0 ≤ U ≤ 1 − p p 1−p X = 1 1 − p < U ≤ 1 0 1 0 1 1 CDF ������������� ������������� ������������� ������������� 1−p 0 1 ������������� ������������� ������������� ������������� ������������� ������������� 0 1 0 1−p 0 1 DTU 02443 – lecture 3 6

A fair die A fair die or uniform distribution 1 ���� ���� ����� ����� ����� ����� ���� ���� ����� ����� ����� ����� 1 ���� ���� 2 3 ����� ����� 5 ����� ����� 6 4 ���� ���� ����� ����� ����� ����� ���� ���� ����� ����� ����� ����� 0 1/6 1 1/6 X = 1 , 2 , ... 6 0 1 2 3 4 5 6 P ( X = i ) = 1 / 6 1/6 X = ⌊ (6 U ) ⌋ + 1 0 1 2 3 4 5 6 Can be generalized 6 → k . DTU 02443 – lecture 3 7

Discrete distribution - direct (crude) method Discrete distribution - direct (crude) method Suppose X can take k distinct values x 1 < x 2 < ... x k with p i = P ( X = x i ) , i = 1 , 2 , ... , k Then X takes the value x i with probability p i if U falls in an interval with length p i . That is if i − 1 i � � p j < U ≤ p j j =1 j =1 or X = x i if F ( x i − 1 ) < U ≤ F ( x i ) DTU 02443 – lecture 3 8

Geometric distribution, NB (1 , p ) Geometric distribution, NB (1 , p ) The discrete time version of waiting time. Memory-less. P ( X = n ) = (1 − p ) n − 1 p n = 1 , 2 , ... f ( n ) = P ( X ≤ n ) = 1 − (1 − p ) n F ( n ) = 1 − (1 − p ) n − 1 < U ≤ 1 − (1 − p ) n X = n if F ( n − 1) < U ≤ F ( n ) n − 1 < log (1 − U ) log (1 − p ) ≤ n �� �� log ( U ) X = + 1 log (1 − p ) DTU 02443 – lecture 3 9

Discrete distribution II Discrete distribution II ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� 1 2 3 4 5 6 7 ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� ������� ������� ������� ������� ���� ���� 1 0 P1 P2 1. Generate U 2. Find the interval i which U belong to. P i − 1 < U ≤ P i 3. output x i • Linear search (E ( X ) ) • Rearrangement of intervals • Binary search • Indexed search DTU 02443 – lecture 3 10

Rejection Method Rejection Method Simple rejection More optimistic: acceptance method. Assume P ( X = i ) = p i for i = 1 , 2 , ... k . p(i) Let c ≥ p i (then p i /c ≤ 1 ). C ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� 1. I = ⌊ ( k ∗ U 1 ) ⌋ + 1 ��� ��� ��� ��� ��� ��� ��� ��� ��� ��� �������������������� �������������������� ��� ��� 2. if U 2 ≤ p I /c output: I ��� ��� ��� ��� �� �� ��� ��� ��� ��� �� �� ��� ��� ��� ��� �� �� ��� ��� ��� ��� �� �� ��� ��� ��� ��� Else goto 1. �� �� �� �� ��� ��� ��� ��� �� �� �� �� ��� ��� ��� ��� �� �� �� �� ��� ��� ��� ��� �� �� �� �� ��� ��� ��� ��� �� �� �� �� ��� ��� ��� ��� 1 2 3 4 1 p i k c frequency for i : = p i p j � k 1 j =1 k c DTU 02443 – lecture 3 11

Alias method Alias method • A method for generating discrete random variates of general type • From discrete uniform to general discrete • Generate one random number • One comparison • Potentially one table look-up • Drawback: Complex set-up procedure 02443 02443 – lecture 3 12