The Geometry of Imprecise Inference Mik elis Bickis University of - - PowerPoint PPT Presentation

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

The Geometry of Imprecise Inference

Mik ¸elis Bickis

University of Saskatchewan

ISIPTA’15 20 July 2015 Pescara, Italy Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

◮ I am Professor Emeritus in the Department of Mathematics

and Statistics at the in Saskatoon

◮ My recent graduates:

◮ Osama Bataineh, PhD 2012 ◮ Chel Hee Lee, PhD 2014

◮ Two current students ◮ Research supported by the Natural Sciences and Engineering

Research Council of Canada

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Where is Saskatoon?

• Bickis

U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: P(A|B) = P(B|A)P(A) P(B)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: P(A|B) = P(B|A)P(A) P(B) log P(B|A) P(A) = log P(B|A) − log P(B)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: P(A|B) = P(B|A)P(A) P(B) log P(B|A) P(A) = log P(B|A) − log P(B) log P(A|B) P(Ac|B) = log P(B|A) P(B|Ac) + log P(A) P(Ac)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: P(A|B) = P(B|A)P(A) P(B) log P(B|A) P(A) = log P(B|A) − log P(B) log P(A|B) P(Ac|B) = log P(B|A) P(B|Ac) + log P(A) P(Ac) log P(A|B) P(A) = log P(B|A) P(B|Ac) + log P(Ac|B) P(Ac)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: log P(A|B) P(A) = log P(B|A) P(B|Ac) + log P(Ac|B) P(Ac)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: log P(A|B) P(A) = log P(B|A) P(B|Ac) + log P(Ac|B) P(Ac) log dΠy dΠ0 (θ) = log dPθ dP0 (y) + something else

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Bayes’ theorem: log P(A|B) P(A) = log P(B|A) P(B|Ac) + log P(Ac|B) P(Ac) log dΠy dΠ0 (θ) = log dPθ dP0 (y) + something else log dΠy dΠ0 (θ) = θTv(y) − I(P0|Pθ) − ψ(y)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

An exponential family of distributions can be represented in terms of the vector space of minimal sufficient statistics (i.e. functions on the observation space.) The manifold of distributions M maps uniquely unto a tangent space L. spanned by the minimal sufficient statistics.

I(P0jP3)

P3 L M

3v

P0

log dPθ dP0 = θv − I(P0|Pθ)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

A prior distribution is a distribution on L. An exponential family of priors can be expressed in terms of the space of linear functions

• n L.

This dual space will include the evaluation functional v → v(y), and thus will include all possible posteriors from any prior in the family. log dΠy dΠ0 (v) = v(y) − I(P0|Pv) − ψ(y)

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

The lower envelope theorem says that a lower prevision can be expressed as the infimum over a set of linear previsions. This means that an imprecise prior can represented by a set of precise priors. In the expression log dΠy dΠ0 (θ) = θTv(y) − I(P0|Pθ) − ψ(y) the first two terms on the right do not depend on the prior, and thus would translate all priors in the same way. The third term does not depend on the model parameter θ.

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

−2 2 η0 η1 Precise infrence

prior posterior

−2 2 η0 η1 Imprecise infrence

prior posterior

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Normal family with known variance.

20

• 1

1 2 3 4 5 6 7 8 21

• 10
• 8
• 6
• 4
• 2

2 4 6 8 10

Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Binomial model with normal prior of logit

• 2
• 1

expectation contours of logit-normal 21 1 2 0.5 1 1.5 20 0.2 0.15 0.1 0.05 2 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples

Censored exponential model with gamma prior

21

• 1

1 2 3 4 5 22

• 1

1 2 3 4 5

t=3 t=2 t=1

prior censored at 2 death at 3

Bickis U of S The Geometry of Imprecise Inference