Bayesian inference Marcel Lthi Graphics and Vision Research Group - - PowerPoint PPT Presentation

β–Ά
bayesian inference
SMART_READER_LITE
LIVE PREVIEW

Bayesian inference Marcel Lthi Graphics and Vision Research Group - - PowerPoint PPT Presentation

Oct 07, 2023 1.1k likes β€’1.29k views

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE Bayesian inference Marcel Lthi Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel University of Basel > DEPARTMENT OF MATHEMATICS AND

slide-1
SLIDE 1

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

Bayesian inference

Marcel LΓΌthi

Graphics and Vision Research Group Department of Mathematics and Computer Science University of Basel

slide-2
SLIDE 2

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Probabilities: What are they?

Four possible interpretations:

  • 1. Long-term frequencies
  • Relative frequency of an event over time
  • 2. Physical tendencies (propensities)
  • Arguments about a physical situation (causes of relative frequencies)
  • 3. Degree of belief (Bayesian probabilities)
  • Subjective beliefs about events/hypothesis/facts
  • 4. Logic
  • Degree of logical support for a particular hypothesis
slide-3
SLIDE 3

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Degree of belief: An Example

  • Dentist example: Does the patient have a cavity?

Bu But t the the patien tient t eith either r has as a a cavi vity or

  • r does
  • es not
  • t
  • There is no 80% cavity!
  • Having a cavity should not depend on whether the patient has a toothache or gum problems

These statements do not contradict each other, they summarize the dentist’s knowledge about the patient

3

𝑄 cavity = 0.1 𝑄 cavity toothache) = 0.8 𝑄 cavity toothache, gum problems) = 0.4

AIMA: Russell & Norvig, Artificial Intelligence. A Modern Approach, 3rd edition,

slide-4
SLIDE 4

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Uncertainty: Bayesian Probability

  • Bayesian probabilities rely on a subjective perspective:
  • Probabilities express our current knowledge.
  • Can change when we learn or see more
  • More data -> more certain about our result.
  • Subjective != Arbitrary
  • Given belief, conclusions follow by laws of probability calculus

4

Subjectivity: There is no single, real underlying distribution. A probability distribution expresses our knowledge – It is different in different situations and for different observers since they have different knowledge.

slide-5
SLIDE 5

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Belief Updates

Model Face distribution Ob Observ rvatio ion Concrete points Possibly uncertain Pos

  • sterior

Face distribution consistent with observation Prior belief More knowledge Posterior belief Consistency: Laws of probability calculus!

slide-6
SLIDE 6

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Two important rules

Marginal

Distribution of certain points only

Conditional

Distribution of points conditioned on known values of others

Probabilistic model: joint distribution of points

𝑄 𝑦1|𝑦2 = 𝑄 𝑦1, 𝑦2 𝑄 𝑦2 𝑄 𝑦1 = ෍

𝑦2

𝑄(𝑦1, 𝑦2)

𝑄 𝑦1, 𝑦2

Product rule: 𝑄 𝑦1, 𝑦2 = π‘ž 𝑦1 𝑦2 π‘ž(𝑦2)

slide-7
SLIDE 7

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Certain Observation

  • Observations are known values
  • Distribution of π‘Œ after observing

𝑧1, … , 𝑧𝑂: 𝑄 π‘Œ|𝑧1 … 𝑧𝑂

  • Conditional probability

𝑄 π‘Œ|𝑧1 … 𝑧𝑂 = 𝑄 π‘Œ, 𝑧, … , 𝑧𝑂 𝑄 𝑧1, … , 𝑧𝑂

X y1 yi yN

slide-8
SLIDE 8

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Towards Bayesian Inference

  • Update belief about π‘Œ by observing 𝑧1, … , 𝑧𝑂

𝑄 π‘Œ β†’ 𝑄 π‘Œ 𝑧1, … , 𝑧𝑂

  • Factorize joint distribution

𝑄 π‘Œ, 𝑧1, … , 𝑧𝑂 = 𝑄 𝑧1, … , 𝑧𝑂|π‘Œ 𝑄 π‘Œ

  • Rewrite conditional distribution

𝑄 π‘Œ|𝑧1, … , 𝑧𝑂 = 𝑄 π‘Œ, 𝑧1, … , 𝑧𝑂 𝑄 𝑧1, … , 𝑧𝑂 = 𝑄 𝑧1, … , 𝑧𝑂|π‘Œ 𝑄 π‘Œ 𝑄 𝑧1, … , 𝑧𝑂

More generally: distribution of model points π‘Œ given data 𝑍:

𝑄 π‘Œ|𝑍 = 𝑄 π‘Œ, 𝑍 𝑄 𝑍 = 𝑄 𝑍|π‘Œ 𝑄 π‘Œ 𝑄 𝑍

slide-9
SLIDE 9

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Uncertain Observation

  • Observations with uncertainty

Model needs to describe how observations are distributed with joint distribution 𝑄 π‘Œ, 𝑍

  • Still conditional probability

But joint distribution is more complex

  • Joint distribution factorized

𝑄 π‘Œ, 𝑍 = 𝑄 𝑍|π‘Œ 𝑄 π‘Œ

  • Likelihood 𝑄 𝑍|π‘Œ
  • Prior 𝑄 π‘Œ

X y1 + 𝜁 yi + 𝜁 yN + 𝜁

slide-10
SLIDE 10

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Likelihood

𝑄 π‘Œ, 𝑍 = 𝑄 𝑍|π‘Œ 𝑄 π‘Œ

  • Likelihood x prior: factorization is more flexible than full joint
  • Prior: distribution of core model without observation
  • Likelihood: describes how observations are distributed

Prio rior Lik Likelih ihood Join Joint

slide-11
SLIDE 11

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Bayesian Inference

  • Conditional/Bayes rule: method to update beliefs

𝑄 π‘Œ|𝑍 = 𝑄 𝑍|π‘Œ 𝑄 π‘Œ 𝑄 𝑍

  • Each observation updates our belief (changes knowledge!)

𝑄 π‘Œ β†’ 𝑄 π‘Œ 𝑍 β†’ 𝑄 π‘Œ 𝑍, π‘Ž β†’ 𝑄 π‘Œ 𝑍, π‘Ž, 𝑋 β†’ β‹―

  • Bayesian Inference: How beliefs evolve with observation
  • Recursive: Posterior becomes prior of next inference step

Prio rior Lik Likelih ihood Pos

  • sterior

Mar argin inal l Lik Likelih ihood

slide-12
SLIDE 12

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Marginalization

  • Models contain irrelevant/hidden variables

e.g. points on chin when nose is queried

  • Marginalize over hidden variables (Z)

𝑄 π‘Œ 𝑍 = ෍

𝐼

𝑄 π‘Œ, π‘Ž 𝑍 = ෍

𝐼

𝑄 𝑍, π‘Ž|π‘Œ 𝑄 π‘Œ 𝑄 𝑍, π‘Ž

slide-13
SLIDE 13

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

General Bayesian Inference

  • Observation of additional variables
  • Common case, e.g. image intensities, surrogate measures (size, sex, …)
  • Coupled to core model via likelihood factorization
  • General Bayesian inference case:
  • Distribution of data 𝑍
  • Parameters πœ„

𝑄 πœ„|𝑍 = 𝑄 𝑍|πœ„ 𝑄 πœ„ 𝑄 𝑍 = 𝑄 𝑍|πœ„ 𝑄 πœ„ ∫ 𝑄 𝑍|πœ„ 𝑄 πœ„ π‘’πœ„ 𝑄 πœ„|𝑍 ∝ 𝑄 𝑍|πœ„ 𝑄 πœ„

slide-14
SLIDE 14

> DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE

University of Basel

Summary: Bayesian Inference

  • Belief: formal expression of an observer’s knowledge
  • Subjective state of knowledge about the world
  • Beliefs are expressed as probability distributions
  • Formally not arbitrary: Consistency requires laws of probability
  • Observations change knowledge and thus beliefs
  • Bayesian inference formally updates prior beliefs to posteriors
  • Conditional Probability
  • Integration of observation via likelihood x prior factorization

𝑄 πœ„|𝑍 = 𝑄 𝑍|πœ„ 𝑄 πœ„ 𝑄 𝑍

18