[PPT] - Efficient Inference in Fully Connected CRFs with Gaussian Edge PowerPoint Presentation

SLIDE 1

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

Philipp Kr¨ ahenb¨ uhl Vladlen Koltun

philkr@stanford.edu vladlen@stanford.edu Department of Computer Science, Stanford University

December 14, 2011

SLIDE 2

Multi-class image segmentation

Assign a class label to each pixel in the image table chair background

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 2 / 29

SLIDE 3

CRF models in multi-class image segmentation

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

MAP inference in conditional random field Unary term

◮ From classifier ◮ TextonBoost [Shotton et al. 09]

Pairwise term

◮ Consistent labeling

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 3 / 29

SLIDE 4

CRF models in multi-class image segmentation

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

MAP inference in conditional random field Unary term

◮ From classifier ◮ TextonBoost [Shotton et al. 09]

Pairwise term

◮ Consistent labeling

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 3 / 29

SLIDE 5

CRF models in multi-class image segmentation

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

MAP inference in conditional random field Unary term

◮ From classifier ◮ TextonBoost [Shotton et al. 09]

Pairwise term

◮ Consistent labeling

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 3 / 29

SLIDE 6

Adjacency CRF models

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

Pairwise term

◮ Neighboring pixels ◮ Color-sensitive Potts model

ψp(xi, xj) = 1[xi =xj ]

w (1) exp
−|Ii − Ij|2

2θ2

β

+ w (2)
P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 4 / 29

SLIDE 7

Adjacency CRF models

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

sky tree grass

grid crf

Efficient inference

◮ 1 second for 50′000 variables

Limited expressive power Only local interactions Excessive smoothing of object boundaries

◮ Shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 5 / 29

SLIDE 8

Adjacency CRF models

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

sky tree grass

grid crf

Efficient inference

◮ 1 second for 50′000 variables

Limited expressive power Only local interactions Excessive smoothing of object boundaries

◮ Shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 5 / 29

SLIDE 9

Adjacency CRF models

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

Efficient inference

◮ 1 second for 50′000 variables

Limited expressive power Only local interactions Excessive smoothing of object boundaries

◮ Shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 5 / 29

SLIDE 10

Adjacency CRF models

E(x) =

i

ψu(xi)

unary term

+

i
j∈Ni

ψp(xi, xj)

pairwise term

sky tree grass

grid crf

Efficient inference

◮ 1 second for 50′000 variables

Limited expressive power Only local interactions Excessive smoothing of object boundaries

◮ Shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 5 / 29

SLIDE 11

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Every node is connected to every other node

◮ Connections weighted differently

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 6 / 29

SLIDE 12

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Every node is connected to every other node

◮ Connections weighted differently

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 6 / 29

SLIDE 13

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

sky tree grass

fully connected

Long-range interactions No more shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 7 / 29

SLIDE 14

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Long-range interactions No more shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 7 / 29

SLIDE 15

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

sky tree grass

fully connected

Long-range interactions No more shrinking bias

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 7 / 29

SLIDE 16

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

sky tree grass

fully connected

Region-based [Rabinovich et al. 07, Galleguillos et al. 08, Toyoda & Hasegawa 08, Payet & Todorovic 10]

◮ Tractable up to hundreds of variables

Pixel-based

◮ Tens of thousands of variables ⋆ Billions of edges ◮ Computationally expensive

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 8 / 29

SLIDE 17

Fully connected CRF

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

sky tree grass

fully connected

Region-based [Rabinovich et al. 07, Galleguillos et al. 08, Toyoda & Hasegawa 08, Payet & Todorovic 10]

◮ Tractable up to hundreds of variables

Pixel-based

◮ Tens of thousands of variables ⋆ Billions of edges ◮ Computationally expensive

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 8 / 29

SLIDE 18

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Inference in 0.2 seconds

◮ 50′000 variables ◮ MCMC inference: 36 hrs

Pairwise potentials: linear combinations of Gaussians

bench road grass tree

fully connected

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 9 / 29

SLIDE 19

Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials Inference in 0.2 seconds

◮ 50′000 variables ◮ MCMC inference: 36 hrs

Pairwise potentials: linear combinations of Gaussians

bench road grass tree

fully connected

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 9 / 29

SLIDE 20

Model definition

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Gaussian edge potentials ψp(xi, xj) = µ(xi, xj)

K

m=1

w(m)k(m)(fi, fj) Label compatibility function µ Linear combination of Gaussian kernels k(m)(fi, fj) = exp(−1 2(fi − fj)Σ(m)(fi − fj)) Arbitrary feature space fi

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 10 / 29

SLIDE 21

Model definition

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Gaussian edge potentials ψp(xi, xj) = µ(xi, xj)

K

m=1

w(m)k(m)(fi, fj) Label compatibility function µ Linear combination of Gaussian kernels k(m)(fi, fj) = exp(−1 2(fi − fj)Σ(m)(fi − fj)) Arbitrary feature space fi

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 10 / 29

SLIDE 22

Model definition

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Gaussian edge potentials ψp(xi, xj) = µ(xi, xj)

K

m=1

w(m)k(m)(fi, fj) Label compatibility function µ Linear combination of Gaussian kernels k(m)(fi, fj) = exp(−1 2(fi − fj)Σ(m)(fi − fj)) Arbitrary feature space fi

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 10 / 29

SLIDE 23

Model definition

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Gaussian edge potentials ψp(xi, xj) = µ(xi, xj)

K

m=1

w(m)k(m)(fi, fj) Label compatibility function µ Linear combination of Gaussian kernels k(m)(fi, fj) = exp(−1 2(fi − fj)Σ(m)(fi − fj)) Arbitrary feature space fi

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 10 / 29

SLIDE 24

Model definition

E(x) =

i

ψu(xi)

unary term

+

i
j>i

ψp(xi, xj)

pairwise term

Gaussian edge potentials ψp(xi, xj) = µ(xi, xj)

K

m=1

w(m)k(m)( fi , fj ) Label compatibility function µ Linear combination of Gaussian kernels k(m)(fi, fj) = exp(−1 2(fi − fj)Σ(m)(fi − fj)) Arbitrary feature space fi

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 10 / 29

SLIDE 25

Detailed model definition

ψp(xi, xj) = µ(xi, xj)

w(1) exp(−|pi − pj|

2θ2

α

− |Ii − Ij| 2θ2

β

) + w(2) exp(−|pi − pj| 2θ2

γ

)

Label compatibility

◮ Potts model: µ(xi, xj) = 1[xi=xj] ◮ Semi-metric model: µ(xi, xj) learned from data

Appearance kernel

◮ Color-sensitive model

Local smoothness

◮ Discourages pixel level noise

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 11 / 29

SLIDE 26

Detailed model definition

ψp(xi, xj) = µ(xi, xj)

w(1) exp(−|pi − pj|

2θ2

α

− |Ii − Ij| 2θ2

β

) + w(2) exp(−|pi − pj| 2θ2

γ

)

Label compatibility

◮ Potts model: µ(xi, xj) = 1[xi=xj] ◮ Semi-metric model: µ(xi, xj) learned from data

Appearance kernel

◮ Color-sensitive model

Local smoothness

◮ Discourages pixel level noise

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 11 / 29

SLIDE 27

Detailed model definition

ψp(xi, xj) = µ(xi, xj)

w(1) exp(−|pi − pj|

2θ2

α

− |Ii − Ij| 2θ2

β

) + w(2) exp(−|pi − pj| 2θ2

γ

)

Label compatibility

◮ Potts model: µ(xi, xj) = 1[xi=xj] ◮ Semi-metric model: µ(xi, xj) learned from data

Appearance kernel

◮ Color-sensitive model

Local smoothness

◮ Discourages pixel level noise

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 11 / 29

SLIDE 28

Detailed model definition

ψp(xi, xj) = µ(xi, xj)

w(1) exp(−|pi − pj|

2θ2

α

− |Ii − Ij| 2θ2

β

) + w(2) exp(−|pi − pj| 2θ2

γ

)

Label compatibility

◮ Potts model: µ(xi, xj) = 1[xi=xj] ◮ Semi-metric model: µ(xi, xj) learned from data

Appearance kernel

◮ Color-sensitive model

Local smoothness

◮ Discourages pixel level noise

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 11 / 29

SLIDE 29

Detailed model definition

ψp(xi, xj) = µ(xi, xj)

w(1) exp(−|pi − pj|

2θ2

α

− |Ii − Ij| 2θ2

β

) + w(2) exp(−|pi − pj| 2θ2

γ

)

Label compatibility

◮ Potts model: µ(xi, xj) = 1[xi=xj] ◮ Semi-metric model: µ(xi, xj) learned from data

Appearance kernel

◮ Color-sensitive model

Local smoothness

◮ Discourages pixel level noise

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 11 / 29

SLIDE 30

Inference

Find the most likely assignment (MAP) ˆ x = argmax

x

P(x) where P(x)=exp(−E(x)) Mean field approximation Find Q(x) =

i Q(xi) close to P(x) in terms of KL-divergence

D(QP) ˆ xi ≈ argmaxxi Q(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 12 / 29

SLIDE 31

Inference

Find the most likely assignment (MAP) ˆ x = argmax

x

P(x) where P(x)=exp(−E(x)) Mean field approximation Find Q(x) =

i Q(xi) close to P(x) in terms of KL-divergence

D(QP) ˆ xi ≈ argmaxxi Q(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 12 / 29

SLIDE 32

Inference

Find the most likely assignment (MAP) ˆ x = argmax

x

P(x) where P(x)=exp(−E(x)) Mean field approximation Find Q(x) =

i Q(xi) close to P(x) in terms of KL-divergence

D(QP) ˆ xi ≈ argmaxxi Q(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 12 / 29

SLIDE 33

Inference

Find the most likely assignment (MAP) ˆ x = argmax

x

P(x) where P(x)=exp(−E(x)) Mean field approximation Find Q(x) =

i Q(xi) close to P(x) in terms of KL-divergence

D(QP) ˆ xi ≈ argmaxxi Q(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 12 / 29

SLIDE 34

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 35

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 36

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 37

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 38

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 39

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 40

Mean field approximation

Qi(xi = l) = 1 Zi exp   −ψu(xi) −

l′∈L

µ(l, l′)

K

m=1

w(m)

j=i

k(m)(fi, fj)Qj(l′)    Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

while not converged

◮ Message passing: ˜

Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

◮ Compatibility transform: ˆ

Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l)

◮ Local update: Qi(xi) ← exp{−ψu(xi) − ˆ

Qi(xi)}

◮ Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 13 / 29

SLIDE 41

Mean field approximation

Runtime analysis for N variables O(N) Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

∼10 while not converged

O(N2) Message passing: ˜ Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

O(N) Compatibility transform: ˆ Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l) O(N) Local update: Qi(xi) ← exp{−ψu(xi) − ˆ Qi(xi)} O(N) Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 14 / 29

SLIDE 42

Mean field approximation

Runtime analysis for N variables O(N) Initialize Qi(xi) ← 1

Zi exp{−φu(xi)}

∼10 while not converged

O(N2) Message passing: ✞ ✝ ☎ ✆ ˜ Q(m)

i

(l) ←

j=i k(m)(fi, fj)Qj(l)

O(N) Compatibility transform: ˆ Qi(xi) ←

l∈L µ(m)(xi, l) m w (m) ˜

Q(m)

i

(l) O(N) Local update: Qi(xi) ← exp{−ψu(xi) − ˆ Qi(xi)} O(N) Normalize: Qi(xi)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 14 / 29

SLIDE 43

Efficient message passing using high-dimensional filtering

Update all ˜ Q(m)

i

(l) simultaneously ˜ Q(m)

i

(l) =

j=i

k(m)(fi, fj)Qj(l) Efficiently computed using a cross-bilateral filter [Paris & Durand 09, Adams et al. 09, Adams et al. 10]

◮ Permutohedral lattice [Adams et al. 10]

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 15 / 29

SLIDE 44

High-dimensional filtering [Paris & Durand 09]

Q

(m) i

(l) =

j∈V

exp 1 2(f(m)

i

− f(m)

j

)2

Qj(l)

High-dimensional input signal Qj(l) Gaussian convolution Q

(m) i

(l) = G ⊗ Qj(l)

◮ Band-limited, smooth function

Can be reconstructed from a number of samples

◮ Nyquist theorem

f Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 16 / 29

SLIDE 45

High-dimensional filtering [Paris & Durand 09]

Q

(m) i

(l) =

j∈V

exp 1 2(f(m)

i

− f(m)

j

)2

Qj(l)

High-dimensional input signal Qj(l) Gaussian convolution Q

(m) i

(l) = G ⊗ Qj(l)

◮ Band-limited, smooth function

Can be reconstructed from a number of samples

◮ Nyquist theorem

f ¯ Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 16 / 29

SLIDE 46

High-dimensional filtering [Paris & Durand 09]

Q

(m) i

(l) =

j∈V

exp 1 2(f(m)

i

− f(m)

j

)2

Qj(l)

High-dimensional input signal Qj(l) Gaussian convolution Q

(m) i

(l) = G ⊗ Qj(l)

◮ Band-limited, smooth function

Can be reconstructed from a number of samples

◮ Nyquist theorem

f ¯ Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 16 / 29

SLIDE 47

High-dimensional filtering [Paris & Durand 09]

Downsample input signal Qj(l) Blur the sampled signal Upsample to reconstruct the filtered signal Qj(l) f Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 17 / 29

SLIDE 48

High-dimensional filtering [Paris & Durand 09]

Downsample input signal Qj(l) Blur the sampled signal Upsample to reconstruct the filtered signal Qj(l) f Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 17 / 29

SLIDE 49

High-dimensional filtering [Paris & Durand 09]

Downsample input signal Qj(l) Blur the sampled signal Upsample to reconstruct the filtered signal Qj(l) f ¯ Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 17 / 29

SLIDE 50

High-dimensional filtering [Paris & Durand 09]

Downsample input signal Qj(l) Blur the sampled signal Upsample to reconstruct the filtered signal Qj(l) f ¯ Q(f)

P. Kr¨

ahenb¨ uhl (Stanford) Efficient Inference in Fully Connected CRFs December 2011 17 / 29

SLIDE 51