[PPT] - Imprecise random variables, random sets, and Monte Carlo simulation PowerPoint Presentation

SLIDE 1

Imprecise random variables, random sets, and Monte Carlo simulation

Thomas Fetz, Michael Oberguggenberger

Unit for Engineering Mathematics University of Innsbruck, Austria Thomas.Fetz@uibk.ac.at Michael.Oberguggenberger@uibk.ac.at

ISIPTA’15, Pescara, Italia

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 1 / 10

SLIDE 2

Research team Unit of Engineering Mathematics Faculty of Engineering Sciences University of Innsbruck, Austria

Research team “imprecise probabilities”:

Michael Oberguggenberger (head of unit) Jelena Nedeljkovic Thomas Fetz Martin Schwarz

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 2 / 10

SLIDE 3

Problem Given Expensive input-output map g : Rn → R : x → g(x). E.g. finite element computations (minutes or hours per computation). Family {Xλ }λ∈Λ of random variables modelling the uncertainty of variable x. Aim Upper/lower probabilities that g(x) ∈ B. Upper/lower probabilities that g(x) ≤ y (upper/lower cumulative distribution functions). Upper/lower probabilities that g(x) ≤ 0 (upper/lower probability of failure). Two approaches Monte-Carlo simulation of {g(Xλ )}λ∈Λ. Monte-Carlo simulation of the random set X generated by {g(Xλ )}λ∈Λ.

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 3 / 10

SLIDE 4

Two approaches 1 Family {Xλ }λ∈Λ of random variables Probability space (Ω,Σ,m). Family {Xλ }λ∈Λ of random variables Xλ : Ω → R : ω → Xλ (ω). Probability P(Xλ ∈ B) for fixed Xλ : P(Xλ ∈ B) =

Ω ✶Xλ (ω)∈B dm(ω).

(for initial analysis we drop the map g)

2 Random set X based on {Xλ }λ∈Λ Set-valued map X : Ω → R defined by X(ω) = {Xλ (ω) : λ ∈ Λ}. X is a random set, if upper/lower inverses X−(B) = {ω ∈ Ω : X(ω)∩B = ∅}, X−(B) = {ω ∈ Ω : X(ω) ⊆ B} are measurable subsets of Ω.

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 4 / 10

SLIDE 5

Two approaches 1 Family {Xλ }λ∈Λ of random variables Probability space (Ω,Σ,m). Family {Xλ }λ∈Λ of random variables Xλ : Ω → R : ω → Xλ (ω). Probability P(Xλ ∈ B) for fixed Xλ : P(Xλ ∈ B) =

Ω ✶Xλ (ω)∈B dm(ω).

(for initial analysis we drop the map g)

2 Random set X based on {Xλ }λ∈Λ Set-valued map X : Ω → R defined by X(ω) = {Xλ (ω) : λ ∈ Λ}. X is a random set, if upper/lower inverses X−(B) = {ω ∈ Ω : X(ω)∩B = ∅}, X−(B) = {ω ∈ Ω : X(ω) ⊆ B} are measurable subsets of Ω. Lower/upper probabilities for {Xλ }λ∈Λ

P(B) = inf

λ∈ΛP(Xλ ∈ B) = inf λ∈Λ

Ω ✶Xλ (ω)∈B dm(ω)

P(B) = sup

λ∈Λ

P(Xλ ∈ B) = sup

λ∈Λ

Ω ✶Xλ (ω)∈B dm(ω)

Lower/upper probabilities for X

P

(B) = m(X−(B)) =
Ω ✶X(ω)⊆B dm(ω)
P(B) = m(X−(B)) =
Ω ✶X(ω)∩B=∅ dm(ω)

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 4 / 10

SLIDE 6

Two approaches 1 Family {Xλ }λ∈Λ of random variables Probability space (Ω,Σ,m). Family {Xλ }λ∈Λ of random variables Xλ : Ω → R : ω → Xλ (ω). Probability P(Xλ ∈ B) for fixed Xλ : P(Xλ ∈ B) =

Ω ✶Xλ (ω)∈B dm(ω).

(for initial analysis we drop the map g)

2 Random set X based on {Xλ }λ∈Λ Set-valued map X : Ω → R defined by X(ω) = {Xλ (ω) : λ ∈ Λ}. X is a random set, if upper/lower inverses X−(B) = {ω ∈ Ω : X(ω)∩B = ∅}, X−(B) = {ω ∈ Ω : X(ω) ⊆ B} are measurable subsets of Ω. Lower/upper probabilities for {Xλ }λ∈Λ

P(B) = inf

λ∈ΛP(Xλ ∈ B) = inf λ∈Λ

Ω ✶Xλ (ω)∈B dm(ω)

P(B) = sup

λ∈Λ

P(Xλ ∈ B) = sup

λ∈Λ

Ω ✶Xλ (ω)∈B dm(ω)

Lower/upper probabilities for X

P

(B) = m(X−(B)) =
Ω ✶X(ω)⊆B dm(ω)
P(B) = m(X−(B)) =
Ω ✶X(ω)∩B=∅ dm(ω)

Theorem

P

≤ P ≤ P ≤

P

X is more imprecise than {Xλ }λ∈Λ!

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 4 / 10

SLIDE 7

Example Probability space: (Ω,Σ,m) = (R,B(R),m), m(B)=

R✶ω∈B

1 √ 2π e−ω2/2 dω. Family {X(µ,σ)}(µ,σ)∈Λ: X(µ,σ)(ω) = σω + µ = ⇒ X(µ,σ) ∼ N(µ,σ2). Λ = [µ,µ]×[σ,σ] = [−0.5,2]×[1,2], B = [1,2.5]. X(ω) = {Xλ (ω) : λ ∈ Λ} = [X(ω),X(ω)] X(ω) = inf

µ∈[µ,µ] σ∈[σ,σ]

X(µ,σ)(ω) =

σω + µ

ω < 0 σω + µ ω ≥ 0 X(ω) = sup

µ∈[µ,µ] σ∈[σ,σ]

X(µ,σ)(ω) =

σω + µ

ω < 0 σω + µ ω ≥ 0 P(B) = inf

(µ,σ)∈ΛP(X(µ,σ) ∈ B) = P(X(−0.5,1) ∈ B)

= 0.0655 P(B) = sup

(µ,σ)∈Λ

P(X(µ,σ) ∈ B) = P(X(1.75,1) ∈ B) = 0.5467 P

(B) = m(X−(B)) = m(∅) = 0.0000
P(B) = m(X−(B)) = m([−1,3])

= Φ(3)−Φ(−1) = 0.8400

1 1 1 1 1 1 1 1 1 1 1 1

4 X X(ω) X(ω = 1) X X X(1.5,1.3) −1 3 2 4 X B X(1.75,1) X(− 1

2 ,1)

X(ω) ω −1 3 1 2.5

1 1 1 1 1 1

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 5 / 10

SLIDE 8

Simulation of a family {Xλ }λ∈Λ of random variables 1 Basic sample x1,...,xNsamp Generate a sample x1,...,xNsamp which is distributed as a basic random variable X∗. Distribution of X∗ should cover a greater range than a distribution of a single Xλ does. 2 Nsamp function evaluations g(xk), k = 1,...,Nsamp We compute g(xk) either using g directly or a cost saving surrogate model ˜ g. 3 Approximation of P(g(Xλ ) ≤ y) Probability P(g(Xλ ) ≤ y) for fixed λ is computed by reweighting the original sample. Weights wk(λ) depending on parameters λ for reweighting the sample x1,...,xNsamp according to the distribution of Xλ : wk(λ) = fXλ (xk) fX∗(xk) 1 Nsamp = fnew(xk) fold(xk) 1 Nsamp where fXλ and fX∗ are strictly positive densities. P(g(Xλ ) ≤ y) for different Xλ without additional function evaluations of g: P(g(Xλ ) ≤ y) =

Ω ✶g(Xλ (ω))≤y dm(ω) ≈

Nsamp

∑

k=1

✶g(Xλ (ωk))≤y ·wk(λ) =

Nsamp

∑

k=1

✶g(xk)≤y ·wk(λ).

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 6 / 10

SLIDE 9

Simulation of a family {Xλ }λ∈Λ of random variables 4 Approximation of P(g ≤ y) and P(g ≤ y) For the computation of the upper/lower probabilities P(g ≤ y) and P(g ≤ y) we use a grid of representative parameter values λi, estimate the probabilities P(g(Xλi) ≤ y) at the grid points λi by means of MC simulation and take the maximum/minimum value: P(g ≤ y) = sup

λ∈Λ

P(g(Xλ ) ≤ y) ≈ max

i=1,...,Ngrid

P(g(Xλi) ≤ y) ≈ max

i=1,...,Ngrid Nsamp

∑

k=1

✶g(xk)≤y ·wk(λi), P(g ≤ y) ≈ min

i=1,...,Ngrid Nsamp

∑

k=1

✶g(xk)≤y ·wk(λi). Effort: Ngrid ·Nsamp reweightings, Nsamp expensive function evaluations of g.

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 7 / 10

SLIDE 10

Simulation of a random set X 1 Propagation of a random set through g G(ω) = g(X(ω)) = {g(Xλ (ω))) : λ ∈ Λ} G(ω) = [G(ω),G(ω)] random interval G(ω) = ming(X(ω)), G(ω) = maxg(X(ω)) 2 Cumulative distribution functions F(y) = P(g ≤ y), F(y) = P

(g ≤ y)

F(y)=P

(−∞,y]∩[G,G] = ∅
=P
G ≤ y
=FG(y)

F(y)=P

[G,G] ⊂ (−∞,y]
=P
G ≤ y
=FG(y)

3 Algorithm for computing F(y) Generate ω1,...,ωNsamp distributed as m. For each ωn, estimate G(ωn) ≈ min

i g(Xλi(ωn)) using grid points λ1,...,λNgrid on Λ.

F(y) ≈

Nsamp

∑

k=1

✶G(ωk)≤0 ·

1 Nsamp .

Effort: Ngrid ·Nsamp expensive evaluations of g.

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 8 / 10

SLIDE 11

Simulation of a random set X 4 Cost saving methods, approximation of g by a surrogate model g Starting point: Collocation points xj, j = 1,...,Ncoll, in Rn and Ncoll evaluations yj = g(xj). Two levels are at hand: Ω

Xλ

− → Rn

g

− → R. A Surrogate model g of the map g : Rn → R: To obtain the lower bound G we replace g by g: G(ωn) ≈ min

i=1,...,Ngrid

g(Xλi(ωn)).

Effort: One surrogate model g, Ngrid ·Nsamp cheap evaluations of g and Ncoll expensive evaluations of g. B Surrogate models gi of maps Ω → g◦Xλi: Collocation points xj are pulled back to Ω. For each λi and xj, we get a collocation point ωij = X−1

λi (xj) in Ω.

Clearly, yj = g(Xλi(ωij)) = g(xj) for every i. Then G(ωn) ≈ min

i=1,...,Ngrid

gi(ωn).

Effort: Ngrid surrogate models gi, Nsamp cheap evaluations of gi for each i and Ncoll expensive evaluations of g. Advantage of surrogate models gi on Ω: Use of orthogonal polynomials with respect to the measure m. In the Gaussian case it means Hermite expansion.

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 9 / 10

SLIDE 12

Poster

Please visit our poster for more details and numerical examples! Thank you for your attention!

Imprecise random variables, random sets, and Monte Carlo simulation

Th. Fetz, M. Oberguggenberger, Unit for Engineering Mathematics,

University of Innsbruck, Austria

Given: Expensive input-output map g : Rn → R : x → g(x) and family {Xλ}λ∈Λ of random variables. Aim: Upper/lower probabilities that g(x) ∈ B where the uncertainty of x is modelled by {Xλ}λ∈Λ. Method: Monte-Carlo simulation of {g(Xλ)}λ∈Λ or of the random set X generated by {g(Xλ)}λ∈Λ. Problem

Probability space (Ω,Σ,m).
Family {Xλ}λ∈Λ of random variables

Xλ : Ω → R : ω → Xλ(ω).

Probability P(Xλ ∈ B) for fixed Xλ:

P(Xλ ∈ B) =

Ω

1Xλ (ω)∈B dm(ω).

(for initial analysis we drop the map g)

Family {Xλ}λ∈Λ of random variables

Set-valued map X : Ω → R defined by

X(ω) = {Xλ(ω) : λ ∈ Λ}.

X is a random set, if upper/lower inverses

X−(B) = {ω ∈ Ω : X(ω)∩B = ∅}, X−(B) = {ω ∈ Ω : X(ω) ⊆ B} are measurable subsets of Ω. Random set X based on {Xλ}λ∈Λ P(B) = inf

λ∈ΛP(Xλ ∈ B) = inf λ∈Λ

Ω

1Xλ (ω)∈B dm(ω)

P(B) = sup

λ∈Λ

P(Xλ ∈ B) = sup

λ∈Λ

Ω

1Xλ (ω)∈B dm(ω)

Lower/upper probabilities for {Xλ}λ∈Λ P

(B) = m(X−(B)) =
Ω

1X(ω)⊆B dm(ω)

P(B) = m(X−(B)) =
Ω

1X(ω)∩B=∅ dm(ω)

Lower/upper probabilities for X

P

≤ P ≤ P ≤

P

Theorem Assumptions: g : Rn → R is a continuous function, Λ is a compact subset of a metric space and the maps λ → Xλ(ω) are continuous for each fixed ω ∈ Ω. P

(g ≤ y), P(g ≤ y), P(g ≤ y),

P(g ≤ y) Goal: Approximation of P(g(Xλ) ≤ y), P(g ≤ y) and P(g ≤ y) by means of Monte Carlo simulation using

nly one sample for all random variables Xλ, λ ∈ Λ.

Simulation of a family of random variables

We generate a sample x1,...,xNsamp which dis-

tributed as a basic random variable X∗.

The distribution of X∗ should cover a greater

range than a distribution of a single Xλ does. 1 Basic sample x1,...,xNsamp For all k = 1,...,Nsamp we compute g(xk) either using g directly or a cost saving surrogate model ˜ g. 2 Nsamp function evaluations g(xk) Probability P(g(Xλ) ≤ y) for fixed λ is computed by reweighting the original sample.

Weights wk(λ) depending on parameters λ for

reweighting the sample x1,...,xNsamp according to the distribution of Xλ: wk(λ) = fXλ (xk) fX∗(xk) 1 Nsamp where fXλ and fX∗ are strictly positive densities.

Approximation of P(g(Xλ) ≤ y) for different ran-

dom variables Xλ without additional function evaluations of g: P(g(Xλ) ≤ y) =

Ω

1g(X(µ,σ)(ω))≤y dm(ω)

≈ Nsamp

∑

k=1 1g(X(µ,σ)(ωk))≤y ·wk(µ,σ) =

Nsamp

∑

k=1

1g(xk)≤y ·wk(λ).

3 Approximation of P(g(Xλ) ≤ y) For the computation of the upper/lower probabilities P(g ≤ y) and P(g ≤ y) we

use a grid of representative parameter values λi,
estimate the probabilities P(g(Xλi) ≤ y) at the

grid points λi by means of MC simulation

and take the maximum/minimum value:

P(g ≤ y) = sup

λ∈Λ

P(g(Xλ) ≤ y) ≈ max

i=1,...,Ngrid

P(g(Xλi) ≤ y) ≈ max

i=1,...,Ngrid Nsamp

∑

k=1

1g(xk)≤y ·wk(λi),

P(g ≤ y) ≈ min

i=1,...,Ngrid Nsamp

∑

k=1

1g(xk)≤y ·wk(λi).

Effort: Ngrid ·Nsamp reweightings, Nsamp function evaluations of g. 4 Approximation of P(g ≤ y) and P(g ≤ y) Goal: Approximation of P

(g ≤ y) and

P(g ≤ y) by means of Monte Carlo simulation. Simulation of a random set

G(ω) = g(X(ω)) = {g(Xλ(ω))) : λ ∈ Λ}
G(ω) = [G(ω),G(ω)] random interval
G(ω) = ming(X(ω)), G(ω) = maxg(X(ω))

1 Propagation of a random set through g

F(y) =

P(g ≤ y), F(y) = P

(g ≤ y)
F(y)=P
(−∞,y]∩[G,G] = ∅
=P
G ≤ y
=FG(y)
F(y)=P
[G,G] ⊂ (−∞,y]
=P
G ≤ y
=FG(y)

2 Cumulative distribution functions

Generate ω1,...,ωNsamp distributed as m.
For each ωn, estimate G(ωn) ≈ min

i g(Xλi(ωn)) using grid points λ1,...,λNgrid on Λ.

F(y) ≈

Nsamp

∑

k=1

1G(ωk)≤0 ·

1 Nsamp . Effort: Ngrid ·Nsamp evaluations of g.

3 Algorithm for computing F(y) Approximation of g by a surrogate model g. Starting point: Collocation points xj, j = 1,...,Ncoll in Rn and Ncoll function evaluations yj = g(xj). Two levels are at hand: Ω

Xλ

− → Rn

g

− → R. A Surrogate model g of the map g : Rn → R: To obtain the lower bound G in the above algorithm we replace g by g through points (xj,yj), G(ωn) ≈ min

i=1,...,Ngrid

g(Xλi(ωn)).

Effort: 1 surrogate model g, Ngrid ·Nsamp cheap evaluations of g and Ncoll evaluations of g. B Surrogate models gi of maps Ω → g ◦ Xλi: Collocation points xj are pulled back to Ω. For each λi and xj, we get a collocation point ωij = X−1 λi (xj) in Ω. Clearly, yj = g(Xλi(ωij)) = g(xj) for every i. Then G(ωn) ≈ min i=1,...,Ngrid

gi(ωn).

Effort: Ngrid surrogate models gi, Nsamp cheap evaluations of gi, i = 1,...,Ngrid, and Ncoll expensive evaluations of g. 4 Cost saving methods One may use orthogonal polynomials with respect to the measure m. In the Gaussian case it means Hermite expansion. 5 Advantage of surrogate models gi on Ω

Probability space:

(Ω,Σ,m) = (R,B(R),m), m(B)=

R

1ω∈B

1 √ 2π e−ω2/2dω.

Family {X(µ,σ)}(µ,σ)∈Λ:

X(µ,σ)(ω) = σω + µ = ⇒ X(µ,σ) ∼ N(µ,σ2).

Λ = [µ,µ]× [σ,σ] = [−0.5,2]× [1,2],

B = [1,2.5]. X(ω) = {Xλ(ω) : λ ∈ Λ} = [X(ω),X(ω)] X(ω) = inf

µ∈[µ,µ] σ∈[σ,σ]

X(µ,σ)(ω) =

σω + µ

ω < 0 σω + µ ω ≥ 0 X(ω) = sup

µ∈[µ,µ] σ∈[σ,σ]

X(µ,σ)(ω) =

σω + µ

ω < 0 σω + µ ω ≥ 0 P(B) = inf

(µ,σ)∈ΛP(X(µ,σ) ∈ B) = P(X(−0.5,1) ∈ B)

= 0.065457 P(B) = sup

(µ,σ)∈Λ

P(X(µ,σ) ∈ B) = P(X(1.75,1) ∈ B) = 0.546745 P

(B) = m(X−(B)) = m(∅) = 0
P(B) = m(X−(B)) = m([−1,3])

= Φ(3)− Φ(−1) = 0.839994 X X(ω) X(ω = 1) X X X(1.5,1.3) −1 3 2 4 X B X(1.75,1) X(− 1

2 ,1)

X(ω) ω −1 3 1 2.5 Example limit state function g g(x) spring constant x 15 25 35 45 −2 2

Given: Limit state function g and {X(µ,σ)}(µ,σ)∈Λ for spring constant x as in

the above example, but here with Λ = [µ,µ]× [σ,σ] = [20,30]× [0.5,3].

Goal: Upper/lower probabilities of failure.

Example: Beam bedded on spring with uncertain spring constant x

Grid points (µi,σj) with µi = 20,21,...,30 and σj = 0.5,1,1.5,...,3 on

set Λ = [µ,µ]× [σ,σ] = [20,30]× [0.5,3].

Focal set [G(ω),G(ω)] of the random set G at ω is approximated by

G(ω) ≈ min

i,j g(X(µi,σj)(ω)),

G(ω) ≈ max

i,j g(X(µi,σj)(ω)).

Approximation of the upper probability of failure of the beam by means of

Monte Carlo simulation:

P(g ≤ 0) = F(0) =
R

1G(ω)∩(−∞,0]=∅ dm(ω) =

R

1G(ω)≤0 dm(ω)

≈

Nsamp

∑

k=1

1G(ωk)≤0 ·

1 Nsamp = 0.358. with standard normally distributed sample ω1,...,ωNsamp, Nsamp = 100000.

Evaluations of g: Nsamp ·Ngrid = 100000 ·(11 ·6) = 6600000.

G g ◦ X(24,2) G G g(X(µi,σj)(ωk)) ω −4 −2 2 4 −1 1 2 X(ωk) G G G g(X(µi,σj)(ωk)) ω −4 −2 2 4 −1 1 2 Simulation of a random set

Failure probability P(g(X(µ,σ)) ≤ 0) of the beam for a fixed pair (µ,σ) ∈ Λ:

P(g(X(µ,σ)) ≤ 0) =

R

1g(X(µ,σ)(ω))≤0 dm(ω)

≈

Nsamp

∑

k=1

1g(X(µ,σ)(ωk))≤0 ·wk(µ,σ) ≈

Nsamp

∑

k=1

1g(xk)≤0 ·wk(µ,σ)

with X(µ,σ)(ωk) = σωk + µ = xk and weights wk(µ,σ) = fX(µ,σ)(xk) fX∗(xk) 1 Nsamp .

Basic sample x1,...,xNsamp, Nsamp =100000, distributed as X∗ ∼ N(25,62).
The upper probability of failure is approximated by

P(g ≤ 0) = sup

(µ,σ)∈Λ

P(g(X(µ,σ)) ≤ 0) ≈ max

i,j P(g(X(µi,σj)) ≤ 0) ≈ 0.221 using grid points (µi,σj) with µi = 20,21,...,30 and σj = 0.5,1,1.5,...,3.

Evaluations of g: Nsamp = 100000.

P(g ≤ 0) ≈ 0.221 P(g(X(µ,σ)) ≤ 0) σ µ 20 25 30 1 2 3 0.2 0.4 Simulation of a family of random variables

Th.Fetz, M. Oberguggenberger Imprecise random variables, random sets, and Monte Carlo simulation ISIPTA’15, Pescara, Italia 10 / 10