Convergence results for the Bayesian inversion theory Hanna Katriina - - PowerPoint PPT Presentation

Convergence results for the Bayesian inversion theory

Hanna Katriina Pikkarainen

Johann Radon Institute for Computational and Applied Mathematics Austrian Academy of Sciences

Workshop on Inverse and Partial Information Problems Linz, Austria, October 27-31, 2008

in collaboration with Prof. Andreas Neubauer (Johannes Kepler University Linz)

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 1 / 20

Overview

1

Introduction

2

Convergence rates for the finite-dimensional problem with normality assumption

3

Convergence issues in the infinite-dimensional setting

4

References

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 2 / 20

Overview

1

Introduction

2

Convergence rates for the finite-dimensional problem with normality assumption

3

Convergence issues in the infinite-dimensional setting

4

References

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 3 / 20

Bayes formula

Assumptions: a probability space (Ω, F, P) X and Y random variables with values in Rn and Rm, respectively Given: the prior probability density πpr(x) of X the likelihood function π(y | x) Posterior (conditional) probability density of X : πpost(x) = π(x | y) = πpr(x)π(y | x)

π(y)

if π(y) = 0.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 4 / 20

Bayesian inversion theory

Inverse problem: y = F(x) x ∈ X unknown y ∈ Y exact measurement F : X → Y operator with discontinuous inverse Inverse problem in the Bayesian framework: Given a noisy measurement Y = ydata, find the posterior distribution of X .

the model of an inverse problem a noise model a noise distribution a prior distribution 9 > > = > > ;

Bayes

− − − − − − →

formula

the posterior distribution

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 5 / 20

Linear inverse problem with additive noise

Linear model for indirect measurements: Y = AX + E Assumptions: X normal random variable with mean x0 and covariance matrix Γ E normal random variable with mean 0 and covariance matrix Σ X and E mutually independent

πpost(x) ∝ πpr(x)πnoise(ydata − Ax) ∝ exp ` − 1

2(x − xpost)TΓ−1 post(x − xpost)

´

where xpost = (Γ−1 + ATΣ−1A)−1(ATΣ−1ydata + Γ−1x0) and Γpost = (Γ−1 + ATΣ−1A)−1.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 6 / 20

Overview

1

Introduction

2

Convergence rates for the finite-dimensional problem with normality assumption

3

Convergence issues in the infinite-dimensional setting

4

References

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 7 / 20

Posterior distribution as a random variable

The data ydata is a realization of the random variable y + E. = ⇒ The posterior mean is a realization of the random variable Xpost(ω) = (Γ−1 + ATΣ−1A)−1(ATΣ−1(y + E(ω)) + Γ−1x0). (1) = ⇒ The posterior distribution µpost is a realization of the random variable Mpost : (Ω, F, P) → (M(Rn), ρP), ω → N (Xpost(ω), Γpost). (2) M(Rn) is the set of all Borel measures and ρP is the Prokhorov metric.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 8 / 20

Prokhorov and Ky Fan metrics

Definition

Let µ1 and µ2 be Borel measures in a metric space (X, ρX ). The distance between µ1 and µ1 in the Prokhorov metric is defined by ρP(µ1, µ1) := inf {ε > 0 : µ1(B) ≤ µ2 (Bε) + ε ∀ B ∈ B(X)} where B(X) is the Borel σ-algebra in X. The set Bε is the ε-neighbourhood of B, i.e., Bε := {x ∈ X : infz∈B ρX (x, z) < ε}.

Definition

Let ξ1 and ξ2 be random variables in a probability space (Ω, F, P) with values in a metric space (X, ρX ). The distance between ξ1 and ξ2 in the Ky Fan metric is defined by ρK(ξ1, ξ1) := inf {ε > 0 : P (ρX (ξ1(ω), ξ2(ω)) > ε) < ε} .

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 9 / 20

Lifting theorem

Theorem

Let ξ1, ξ2 and η1, η2 be random variables on metric spaces (X, ρX ) and (Y, ρY),

• respectively. Let

ρX (ξ1(ω), ξ2(ω)) ≤ Φ(ρY(η1(ω), η2(ω))) for almost all ω ∈ Ω where Φ is a monotonically increasing right-continuous

• function. Then

ρK(ξ1, ξ2) ≤ max {ρK(η1, η2), Φ(ρK(η1, η2))} .

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 10 / 20

x0-minimum norm least squares solution

Σ = σ2 ˆ Σ with ˆ Σ = 1 xQ = (x TQ−1x)1/2 for a positive definite symmetric matrix Q x † is the x0-minimum norm least squares solution: x † minimizes the residual Ax − yˆ

Σ and among all minimizers it then minimizes x − x0Γ.

(λ2

i , vi) is an orthonormal eigensystem of Γ1/2AT ˆ

Σ−1AΓ1/2 λ1 ≥ . . . ≥ λp > λp+1 = . . . = λn = 0 V1 = (v1, . . . , vp) and V2 = (vp+1, . . . , vn) µx † denotes the normal distribution N (x †, Γ1/2V T

2 V2Γ1/2). If the null space of A

is trivial, µx † := δx †.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 11 / 20

Convergence rates

Theorem

Let Xpost and Mpost be defined by (1) and (2), respectively. Then ρK(E, 0) = O

• σ
• 1 + | log σ|
• .

Furthermore, ρK(Xpost, x †) = O

• σ
• 1 + | log σ|
• and

ρK(Mpost, µx †) = O

• σ
• 1 + | log σ|
• .

= ⇒ (order optimal) convergence rates with the same order as the noise

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 12 / 20

Overview

1

Introduction

2

Convergence rates for the finite-dimensional problem with normality assumption

3

Convergence issues in the infinite-dimensional setting

4

References

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 13 / 20

Regularization with projection

Tx = y = ⇒ Tnx = Qny, Tn = QnT x ∈ X real separable Hilbert space y ∈ Y real separable Hilbert space T ∈ L(X, Y) {Yn} finite dimensional subspaces of R(T) Qn : Y → Yn orthogonal projector limn→∞ (I − Qn)y = 0 for all y ∈ R(T) Prior information: X ∼ N (x0, Γ) Wanted: stable solutions in the space Xn = ΓT ∗Yn

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 14 / 20

Bayesian approach

Additive noise: En ∼ N (0, σ2Qn) Posterior mean: Xpost,n(ω) = (σ2Γ−1 + T ∗QnT)−1(T ∗Qn(y + En(ω)) + σ2Γ−1Pnx0) T ∗ adjoint of T : X → Y XΓ = R(Γ1/2) and x1, x2Γ = Γ−1/2x1, Γ−1/2x2 Pn : XΓ → Xn orthogonal projector Posterior distribution: Mpost,n : (Ω, F, P) → (M(X), ρP) ω → N (Xpost,n(ω), Γpost,n) where Γpost,n = σ2(σ2Γ−1 + T ∗QnT)−1 Assumption: σncn ≥ 1 for some c > 1 = ⇒ ρK(En, 0) = O(σn

1 2 ) H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 15 / 20

Weighted Bayesian approach

Weighted posterior mean: X α

post,n(ω) = (αI + T #QnT)−1(T #Qn(y + En(ω)) + αPnx0)

(3) XΓ = R(Γ1/2) and x1, x2Γ = Γ−1/2x1, Γ−1/2x2 T # = ΓT ∗ adjoint of T : XΓ → Y Pn : XΓ → Xn orthogonal projector Xn,Γ space Xn equipped with the XΓ-norm Weighted posterior distribution: M α

post,n : (Ω, F, P) → (M(Xn,Γ), ρP)

ω → N (X α

post,n(ω), Γα post,n)

(4) where Γα

post,n = σ2(αI + T #QnT)−1

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 16 / 20

Convergence result

Theorem

Let X α

post,n and M α post,n be defined by (3) and (4), respectively, and suppose that

T is compact. If σn1/2 → 0 and σncn ≥ 1 for some c > 1 and if α → 0 and σ2n/α → 0 as σ → 0 and n → ∞, ρK(X α

post,n, x †) = O((I − Pn)x †Γ) + o(1)

where x † = T †y. Moreover, ρK(M α

post,n, δPnx †) = o(1)

where δPnx † is the point measure at Pnx †.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 17 / 20

Convergence rates

Theorem

Let X α

post,n and M α post,n be defined by (3) and (4), respectively, and suppose that

T is compact. Moreover, assume that x0 ∈ XΓ fullfills (I − P)x0 − x † = (T #T)µv, v ∈ N(T)⊥, µ ∈ (0, 1] where P is the orthogonal projector from XΓ onto N(T) ⊂ XΓ and x † = T †y. If σn1/2 → 0 and σncn ≥ 1 for some c > 1 and if α ∼ (σn1/2)2/(2µ+1) as σ → 0 and n → ∞, ρK(X α

post,n, x †) = O((I − Pn)x †Γ) + O(γ2µ n + (σn1/2)2µ/(2µ+1))

where γn = (I − Qn)TXΓ→Y. Moreover, ρK(M α

post,n, δPnx †) = O(γ2µ n + (σn1/2)2µ/(2µ+1))

where δPnx † is the point measure at Pnx †.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 18 / 20

Overview

1

Introduction

2

Convergence rates for the finite-dimensional problem with normality assumption

3

Convergence issues in the infinite-dimensional setting

4

References

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 19 / 20

References

• A. Hofinger and H. K. Pikkarainen. Convergence rate for the Bayesian

approach to linear inverse problems. Inverse Problems, 23:2469–2484, 2007.

• A. Hofinger and H. K. Pikkarainen. Convergence rates for linear inverse

problems in the presence of an additive normal noise. Stochastic Analysis and

• Applications. accepted.
• A. Neubauer and H. K. Pikkarainen. Convergence rate for the Bayesian

inversion theory. Proceedings in Applied Mathematics and Mechanics, 7:1080103-1080104, 2007.

• A. Neubauer and H. K. Pikkarainen. Convergence results for the Bayesian

inversion theory. Journal of Inverse and Ill-posed Problems, 16:601-613, 2008.

H.K. Pikkarainen (RICAM) Convergence results for the Bayesian inversion theory 20 / 20