Stratified Monte Carlo Integration and Applications R. El Haddad, - - PowerPoint PPT Presentation

Stratified sampling Integration of indicator functions Numerical results

Stratified Monte Carlo Integration and Applications

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran

Universit´ e Saint-Joseph, Beirut, Lebanon & Universit´ e de Savoie, Le Bourget-du-Lac, France & Birla Institute of Technology and Science, Pilani, India

MCQMC 2012 February 13 – 17, 2012, Sydney, Australia

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Plan of the talk

1

Stratified sampling

2

Integration of indicator functions

3

Numerical results

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

1

Stratified sampling

2

Integration of indicator functions

3

Numerical results

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Numerical integration

I := [0, 1), for s ≥ 1, λs is the s-dimensional Lebesgue measure, g : I s → R is square-integrable. We want to approximate J :=

• I s g(x)dλs(x).
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Monte Carlo approximation

{U1, . . . , UN} i.i.d. random variables uniformly distributed

• ver I s,

X := 1 N

N

• ℓ=1

g ◦ Uℓ. E[X] = J and Var(X) = σ2(g) N , where σ2(g) :=

• I s
• g(x)

2dλs(x) −

I s g(x)dλs(x)

2 .

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Stratified sampling

{D1, . . . , Dp} a partition of I s, N1, . . . , Np integers, {V k

1 , . . . , V k Nk} i.i.d. random variables uniformly distributed

• ver Dk.

Tk := 1 Nk

Nk

• ℓ=1

g ◦ V k

ℓ

T :=

p

• k=1

λs(Dk)Tk. E[T] = J and (proportional allocation 1) Var(T) ≤ Var(X).

1G.S. Fishman, Monte Carlo, Springer, New York (1996)

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling

{D1, . . . , DN} a partition of I s, λs(D1) = · · · = λs(DN) = 1 N , {V1, . . . , VN} independent random variables, Vℓ uniformly distributed over Dℓ. Y := 1 N

N

• ℓ=1

g ◦ Vℓ. E[Y ] = J and 2 3 Var(Y ) ≤ Var(X)

• 2S. Haber, A modified Monte Carlo quadrature, Math. Comput. 20, 361–368

(1966)

3R.C.H. Cheng, T. Davenport, The problem of dimensionality in stratified

sampling, Manage. Sci. 35, 1278–1296 (1989)

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling

1 1

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Latin hypercube sampling

Iℓ := [(ℓ − 1)/N, ℓ/N), 1 ≤ ℓ ≤ N {V i

1, . . . , V i N} independent random variables,

V i

ℓ uniformly distributed over Iℓ

{π1, . . . , πs} independent random permutations of {1, . . . , N}. Wℓ := (V 1

π1(ℓ), . . . , V s πs(ℓ))

Z := 1 N

N

• ℓ=1

g ◦ Wℓ. E[Z] = J and 4 Var(Z) ≤ Var(X)

4M.D. McKay, R.J. Beckman, W.J. Conover, A comparison of three

methods for selecting values of input variables in the analysis of output from a computer code, Technometrics, 21, 239–245 (1979)

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

1

Stratified sampling

2

Integration of indicator functions

3

Numerical results

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Monte Carlo approximation

A ⊂ I s, g := 1A, J = λs(A), {U1, . . . , UN} i.i.d. random variables uniformly distributed

• ver I s,

X := 1 N

N

• ℓ=1

1A ◦ Uℓ. Var(X) = 1 N λs(A)

• 1 − λs(A)
• ≤ 1

4N .

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

ε-collars

x∞ := max

1≤i≤s |xi|.

For ε > 0, A−ε := {x ∈ I s : ∀y ∈ I s \ A x − y∞ ≥ ε}, Aε := {x ∈ I s : ∃y ∈ A x − y∞ < ε}. A−ε ⊂ A ⊂ Aε. Suppose there exists a nondecreasing γ : [0, +∞) → [0, +∞) such that ∀ε > 0 max

• λs(Aε \ A), λs(A \ A−ε)
• ≤ γ(ε),
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

ε-collars

1 1

A

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

ε-collars

1 1

A A-ε

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

ε-collars

1 1

A A-ε Aε

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling

N = ns ; for k = (k1, . . . , ks) with 1 ≤ ki ≤ n, Ck :=

s

• i=1

ki − 1 n , ki n

• ,

{Vk : 1 ≤ ki ≤ n} independent random variables, Vk uniformly distributed over Ck. Y := 1 N

• k

1A ◦ Vk.

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling : Jordan measurable set

Proposition 1. If ∀ε > 0 max

• λs(Aε \ A), λs(A \ A−ε)
• ≤ γ(ε),

then Var(Y ) ≤ 1 2N γ

• 1

N1/s

• .
• Proof. We have

Var(Y ) ≤ 1 4N2 #{k : Ck ∩ A = ∅ and Ck ⊂ A}. Since

• Ck∩A=∅ and Ck⊂A

Ck ⊂ A1/n \ A−1/n, we have 1 N #{k : Ck ∩ A = ∅ and Ck ⊂ A} ≤ 2γ 1 n

• .

for linear γ : Var(Y ) ≤ O

• 1

N1+1/s

• .
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Under the hypersurface

f : I

s−1 → I and Af := {(x′, xs) ∈ I s : xs < f (x′)}.

1 1

Af

We want to approximate I :=

• I s−1 f (x′)dλs−1(x′) =
• I s 1Af (x)dλs(x).
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simple stratified sampling : under the hypersurface

Proposition 2. If f is of bounded variation V (f ) on I

s−1, then

Var(Y ) ≤ s − 1 4 V (f ) + 1 2

• 1

N1+1/s .

• Proof. For k = (k1, . . . , ks) with 1 ≤ ki ≤ n,

k′ = (k1, . . . , ks−1) and C ′

k′ := s−1

• i=1

ki − 1 n , ki n

• .

Var(Y ) ≤ 1 4N2

• k′

#{ks : C(k′,ks) ∩ Af = ∅ and C(k′,ks) ⊂ Af } if C(k′,ks) ∩ Af = ∅ then ∃x′

k′ ∈ C ′ k′ such that ks < nf (x′ k′) + 1,

if C(k′,ks) ⊂ Af then ∃y′

k′ ∈ C ′ k′ such that nf (y′ k′) < ks.

Var(Y ) ≤ 1 4N2

• k′
• n
• f (x′

k′) − f (y′ k′)

• + 2
• .
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Variation

The result follows from Lemma 1. Lemma 1. 5 Let f be a function of bounded variation V (f ) on I

s.

Let n1, . . . , ns be integers. For k = (k1, . . . , ks) with 1 ≤ ki ≤ ni, denote Ck := s

i=1

• ki−1

ni , ki ni

• and xk, yk ∈ Ck. Then
• k

|f (xk) − f (yk)| ≤ V (f )

s

• i=1

ni

s

• i=1

1 ni .

• 5C. L´

ecot, Error bounds for quasi-Monte Carlo integration with nets. Math.

• Comput. 65, 179–187 (1996)
• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

1

Stratified sampling

2

Integration of indicator functions

3

Numerical results

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Special latin hypercube sampling

N = ns Wℓ := (W 1

ℓ , . . . , W s ℓ )

W i

ℓ := ℓi − 1

n + πi( ℓi) − 1 N + Ui N ℓ := (ℓ1, . . . , ℓs)

• ℓi := (ℓ1, . . . , ℓi−1, ℓi+1, . . . , ℓs)

π1, . . . , πs independent random bijections {1, . . . , n}s−1 → {1, . . . , N/n} U = (U1, . . . , Us) random variable uniformly distributed over I s

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Special latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Special latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Special latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Special latin hypercube sampling

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Measure of the unit ball

Q := {x ∈ I s : x2 < 1} then λs(Q) = πs/2 2sΓ( s

2 + 1).

We compare MC, stratified MC and LHS. s = 2 : N = 102, 202, 302, . . . , 4002 = 160 000 points, s = 3 : N = 103, 203, 303, . . . , 2003 = 8 000 000 points, s = 4 : N = 104, 124, 144, . . . , 404 = 2 560 000 points. In order to estimate the variance, we replicate the quadrature independently M times and compute the sample variance, We use M = 100, 200, . . . , 1 000.

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Measure of the unit ball : variance order

Assuming Var = O(N−β), linear regression (M = 1 000) gives

Table: Order β of the variance

dimension MC stratified MC LHS 2 0.99 1.49 1.50 3 1.00 1.33 1.32 4 1.00 1.25 1.25

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Simulation of diffusion

We solve the pure initial-value problem (here +∞

−∞ c0(x)dx = 1).

   ∂c ∂t (x, t) = a∂2c ∂x2 (x, t), x ∈ R, t > 0, c(x, 0) = c0(x), x ∈ R Random walk Sample X0,1, . . . , X0,N from c0, e.g., Xℓ,N := C −1

0 ( 2ℓ−1 2N ),

Choose ∆t, let tn := n∆t, If Xn,1, . . . , Xn,N are known, Xn+1,ℓ = Xn,ℓ + f (Un,ℓ) where f (u) := √ 4a∆t erf−1(2u − 1) Un,ℓ i.i.d. random variables uniformly distributed over I

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Random walk using special LHS

Sample ˜ X0,1, . . . , ˜ X0,N from c0, e.g., ˜ Xℓ,N := C −1

0 ( 2ℓ−1 2N ),

Choose ∆t, let tn := n∆t, If ˜ Xn,1, . . . , ˜ Xn,N are known,

1

Sort : ˜ Xn,1 ≤ . . . ≤ ˜ Xn,N,

2

˜ Xn+1,⌊NW 1

n,ℓ⌋ = ˜

Xn,⌊NW 1

n,ℓ⌋ + f (W 2

n,ℓ)

where f (u) := √ 4a∆t erf−1(2u − 1) (W 1

n,1, W 2 n,1), . . . , (W 1 n,N, W 2 n,N) independent special LHS in I 2

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Test problem

   ∂c ∂t (x, t) = a∂2c ∂x2 (x, t), x ∈ R, t > 0, c(x, 0) = c0(x), x ∈ R Take c0(x) =

1 √ 2πe−x2

Compute a

0 c(x, T)dx for a = 4, T = 1 (with ∆t = 1/100)

Compare MC, stratified MC and LHS, with N = 102, . . . , 2002 particles, Compute the sample variance (M = 1 000 replications) Assuming Var = O(N−γ), linear regression gives MC stratified MC LHS γ 1.00 1.44 1.43

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Random walk for 1-D diffusion : computation time

Figure: Computation time for the sample variance of 100 independent copies of the approximation of a

0 c(x, t)dx as a function of N (from 102

to 2002). MC (+) stratified MC () and LHS (⋆).

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications

Stratified sampling Integration of indicator functions Numerical results

Thank You

• R. El Haddad, R. Fakhereddine, C. L´

ecot, G. Venkiteswaran Stratified Monte Carlo Integration and Applications