Gaussian Process Regression with Perry Groot , Peter Lucas Censored - - PowerPoint PPT Presentation

▶
gaussian process regression with
SMART_READER_LITE
LIVE PREVIEW

Gaussian Process Regression with Perry Groot , Peter Lucas Censored - - PowerPoint PPT Presentation

Feb 19, 2024 373 likes •559 views

GP Regression with Censored Data using EP Gaussian Process Regression with Perry Groot , Peter Lucas Censored Data Using Expectation Introduction Propagation Problem Setting Bayesian Framework Perry Groot , Peter Lucas Prior - GPs

slide-1
SLIDE 1

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Gaussian Process Regression with Censored Data Using Expectation Propagation

Perry Groot, Peter Lucas

Radboud University Nijmegen {perry, peterl}@cs.ru.nl

6th European Workshop on Probabilistic Graphical Models PGM 2012

slide-2
SLIDE 2

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Introduction

1 2 3 4 5 6 7 8 9 10 0.05 0.1 0.15 0.2 0.25 0.3 0.35

slide-3
SLIDE 3

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Introduction - Truncation

1 2 3 4 5 6 7 8 9 10

slide-4
SLIDE 4

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Introduction - Censoring

1 2 3 4 5 6 7 8 9 10

slide-5
SLIDE 5

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Introduction

Both represent a limitation. Truncation: the population from which data is drawn Censoring: the variable of interest Key difference is in the explanatory variable Truncation: missing Censoring: fully observable Examples: Survival analysis, reliability testing

slide-6
SLIDE 6

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Problem Setting

Learn a function f : RD → R given a set of observations D = {(x1, y1), . . . , (xn, yn)} where y is a censored version of y∗: y =    l if y∗ ≤ l y∗ if l < y∗ < u u if y∗ ≥ u We are interested in the posterior p(f|D) = p(f)p(D|f) p(D)

slide-7
SLIDE 7

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Gaussian Processes

A Gaussian process (GP) is collection of random variables {fi} with the property that the joint distribution of any finite subset has a joint Gaussian distribution. A GP specifies a probability distribution over functions f(x) ∼ GP(m(x), k(x, x′)) and is fully specified by its mean function m(x) and covariance (or kernel) function k(x, x′). Typically m(x) = 0, which gives {f(x1), . . . , f(xI)} ∼ N(0, K) with Kij = k(xi, xj)

slide-8
SLIDE 8

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Gaussian Processes - Posterior process

A priori, given data D = {X, y} with y = f(X) and test points X ∗ we have f(X) f(X ∗)

  • ∼ N
  • 0,

K(X, X) K(X, X ∗) k(X ∗, X) K(X ∗, X ∗)

  • and after conditioning

f(X ∗)|X ∗, X, y ∼ N(µ, Σ) with µ = K(X ∗, X)K(X, X)−1y Σ = K(X ∗, X ∗) − K(X ∗, X) K(X, X)−1

  • O(n3)

K(X, X ∗)

slide-9
SLIDE 9

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Gaussian Processes - 1D demo

2 4 6 8 10 −10 −5 5 10 2 4 6 8 10 −10 −5 5 10 2 4 6 8 10 −10 −5 5 10 2 4 6 8 10 −10 −5 5 10

slide-10
SLIDE 10

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Likelihood

Assume that latent function values are contaminated with Gaussian noise with zero mean and unknown variance. Likelihood becomes a mixture of Gaussian and probit likelihood terms: L =

n

  • i=1

p(yi|fi) =

  • yi=l
  • 1 − Φ

fi − l σ

  • l<yi<u

1 σφ yi − fi σ

  • yi=u
  • Φ

fi − u σ

  • which is well-known as the Tobit likelihood.
slide-11
SLIDE 11

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Expectation Propagation

The posterior p(f|D) = p(f)p(D|f)

p(D)

is intractable EP approximates the likelihood by a Gaussian distribution making the posterior tractable Local likelihood approximations p(yi|fi) ≃ ti(fi|˜ Zi, ˜ µi, ˜ σ2

i ) = ˜

ZiN(fi|˜ µi, ˜ σ2

i )

Approximation is iteratively updated In the Gaussian case the update step turns out to be the same as moment matching The zeroth, first, and second moments of the Tobit likelihood can be computed analytically

slide-12
SLIDE 12

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Experiments

0.2 0.4 0.6 0.8 1 −10 −5 5 10 15 20

slide-13
SLIDE 13

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Experiments

0.2 0.4 0.6 0.8 1 −10 −5 5 10 15 20

slide-14
SLIDE 14

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Experiments

0.2 0.4 0.6 0.8 1 −10 −5 5 10 15 20

slide-15
SLIDE 15

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Experiments

u u u u u u (1) l l l (4) (2) (3)

Concordance index: c(D, G, f) = 1 |E|

  • Eij

1f(xi)<f(xj) where G = (X, E) order graph with edges E according to (1)–(4) Fraction of all pairs of inputs whose predicted values are correctly

  • rdered among all inputs that can

be ordered

slide-16
SLIDE 16

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Experiments

Housing data: 506 observations on 14 real-valued variables median value greater than $50.000 appear as $50.000 16 observations were censored (3.2% of the data)

Table: Concordance results housing data (mean c-index and standard deviation).

method c-index GP 0.866 ± 0.003 Tobit-GP (LA) 0.879 ± 0.008 Tobit-GP (EP) 0.892 ± 0.007

slide-17
SLIDE 17

GP Regression with Censored Data using EP Perry Groot, Peter Lucas Introduction Problem Setting Bayesian Framework

Prior - GPs Likelihood Inference

Experiments Conclusions

Conclusions

GPs provide a flexible, non-parametric Bayesian framework that can be extended to censored

  • bservations

The intractable posterior in case of a Tobit likelihood can be approximated with EP using analytic update steps leading to a stable algorithm