[PPT] - Fast Edge Detection Using Structured Forests by Piotr Dollar, C. PowerPoint Presentation

SLIDE 1

Fast Edge Detection Using Structured Forests

by Piotr Dollar, C. Lawrence Zitnick (PAMI 2015)

Vojtˇ ech Cvrˇ cek

Reading Group Presentation

June 6, 2018

Fast edge detection June 6, 2018 1 / 22

SLIDE 2

Problem Definition

given a set of training images and corresponding segmentation masks

Fast edge detection June 6, 2018 2 / 22

SLIDE 3

Problem Definition

given a set of training images and corresponding segmentation masks predict an edge map for a novel input image

Fast edge detection June 6, 2018 2 / 22

SLIDE 4

Edge structure

Fast edge detection June 6, 2018 3 / 22

SLIDE 5

Random Forest - Single Tree

ft : X → Y L R hθj : X → {L, R} x ∈ X

Fast edge detection June 6, 2018 4 / 22

SLIDE 6

Random Forest - Single Tree

ft : X → Y L R h1

θ1

j (x) = x(k) < τ

θ1

j = (k, τ)

h2

θ2

j (x) = x(k1) − x(k2) < τ

θ2

j = (k1, k2, τ)

hθj(x) = δh1

θ1

j (x) + (1 − δ)h2

θ2

j (x)

δ ∈ {0, 1} x ∈ X ft : X → Y

Fast edge detection June 6, 2018 4 / 22

SLIDE 7

Random Forest - Single Tree

x ∈ X ft : X → Y

Fast edge detection June 6, 2018 4 / 22

SLIDE 8

Random Forest - Single Tree

x ∈ X ft : X → Y

Fast edge detection June 6, 2018 4 / 22

SLIDE 9

Random Forest - Single Tree

y1 y2 y3 y4 y5 y6 y7 x ∈ X ft : X → Y

Fast edge detection June 6, 2018 4 / 22

SLIDE 10

Training

each tree is trained independently in a recursive manner

Fast edge detection June 6, 2018 5 / 22

SLIDE 11

Training

each tree is trained independently in a recursive manner for a given node j and training set Sj ⊂ X × Y, randomly sample parameters θj from parameters space bad split

Fast edge detection June 6, 2018 5 / 22

SLIDE 12

Training

each tree is trained independently in a recursive manner for a given node j and training set Sj ⊂ X × Y, randomly sample parameters θj from parameters space Select θj resulting in a ’good’ split of the data good split

Fast edge detection June 6, 2018 5 / 22

SLIDE 13

Training - Information Gain Criterion

information gain criterion: Ij = I(Sj, SL

j , SR j ),

(1) where SL

j = {(x, y) ∈ Sj | h(x, θ) = 0}, SR j = Sj \ SL j are splits

Fast edge detection June 6, 2018 6 / 22

SLIDE 14

Training - Information Gain Criterion

information gain criterion: Ij = I(Sj, SL

j , SR j ),

(1) where SL

j = {(x, y) ∈ Sj | h(x, θ) = 0}, SR j = Sj \ SL j are splits

θj = argmaxθ Ij(Sj, θ)

Fast edge detection June 6, 2018 6 / 22

SLIDE 15

Training - Information Gain Criterion

information gain criterion: Ij = I(Sj, SL

j , SR j ),

(1) where SL

j = {(x, y) ∈ Sj | h(x, θ) = 0}, SR j = Sj \ SL j are splits

θj = argmaxθ Ij(Sj, θ) for multiclass classification (Y ⊂ Z) the standard definition of information gain is: Ij = H(Sj) −

k∈{L,R}

|Sk

j |

|Sj| H(Sk

j )

(2)

Fast edge detection June 6, 2018 6 / 22

SLIDE 16

Training - Information Gain Criterion

information gain criterion: Ij = I(Sj, SL

j , SR j ),

(1) where SL

j = {(x, y) ∈ Sj | h(x, θ) = 0}, SR j = Sj \ SL j are splits

θj = argmaxθ Ij(Sj, θ) for multiclass classification (Y ⊂ Z) the standard definition of information gain is: Ij = H(Sj) −

k∈{L,R}

|Sk

j |

|Sj| H(Sk

j )

(2) where H(S) is either the Shannon entropy (H(S) = −

y pylog(py))

r alternatively the Gini impurity (H(S) = −

y py(1 − py))

Fast edge detection June 6, 2018 6 / 22

SLIDE 17

Training

training stops when a maximum depth is reached or if information gain or training set size fall below fixed threshold

Fast edge detection June 6, 2018 7 / 22

SLIDE 18

Training

training stops when a maximum depth is reached or if information gain or training set size fall below fixed threshold single output y ∈ Y is assigned to a leaf node based on a problem specific ensemble model

Fast edge detection June 6, 2018 7 / 22

SLIDE 19

Testing

combining results from multiple trees depends on a problem specific ensemble

Fast edge detection June 6, 2018 8 / 22

SLIDE 20

Testing

combining results from multiple trees depends on a problem specific ensemble classification → majority voting

Fast edge detection June 6, 2018 8 / 22

SLIDE 21

Testing

combining results from multiple trees depends on a problem specific ensemble classification → majority voting regression → averaging

Fast edge detection June 6, 2018 8 / 22

SLIDE 22

Structured Forests

structured output space Y, e.g.:

Fast edge detection June 6, 2018 9 / 22

SLIDE 23

Structured Forests

structured output space Y, e.g.: use of structured outputs in random forests presents following challenges:

Fast edge detection June 6, 2018 9 / 22

SLIDE 24

Structured Forests

structured output space Y, e.g.: use of structured outputs in random forests presents following challenges:

computing splits for structured output spaces of high dimensions/complexity is time consuming

Fast edge detection June 6, 2018 9 / 22

SLIDE 25

Structured Forests

structured output space Y, e.g.: use of structured outputs in random forests presents following challenges:

computing splits for structured output spaces of high dimensions/complexity is time consuming it is unclear how to define information gain

?

Fast edge detection June 6, 2018 9 / 22

SLIDE 26

Structured Forests

structured output space Y, e.g.: use of structured outputs in random forests presents following challenges:

computing splits for structured output spaces of high dimensions/complexity is time consuming it is unclear how to define information gain

solution: mapping (somehow) structured output space Y into multiclass space C = {1, ..., k} Ij - multiclass case

Fast edge detection June 6, 2018 9 / 22

SLIDE 27

Structured Edges Clustering

How to efficiently compute splits?

Fast edge detection June 6, 2018 10 / 22

SLIDE 28

Structured Edges Clustering

How to efficiently compute splits? Can the task be transformed into a multiclass problem? Y

?

− → C = {1, ..., k}

Fast edge detection June 6, 2018 10 / 22

SLIDE 29

Structured Edges Clustering

How to efficiently compute splits? Can the task be transformed into a multiclass problem? Y

?

− → C = {1, ..., k} start with an intermediate mapping: Π : Y → Z (3)

Fast edge detection June 6, 2018 10 / 22

SLIDE 30

Structured Edges Clustering

How to efficiently compute splits? Can the task be transformed into a multiclass problem? Y

?

− → C = {1, ..., k} start with an intermediate mapping: Π : Y → Z (3) z = Π(y) is a long binary vector, which encodes whether every pair of pixels in the y belongs to the same or different segment

Fast edge detection June 6, 2018 10 / 22

SLIDE 31

Structured Edges Clustering

dimension of vectors z ∈ Z is reduced by PCA to m = 5, and clustering (k-means) splits Z into k = 2 clusters

Fast edge detection June 6, 2018 11 / 22

SLIDE 32

Structured Edges Clustering

dimension of vectors z ∈ Z is reduced by PCA to m = 5, and clustering (k-means) splits Z into k = 2 clusters Y

pairs

− − − → Z

PCA, k-means

− − − − − − − − → C

Fast edge detection June 6, 2018 11 / 22

SLIDE 33

Structured Edges Clustering

dimension of vectors z ∈ Z is reduced by PCA to m = 5, and clustering (k-means) splits Z into k = 2 clusters Y

pairs

− − − → Z

PCA, k-means

− − − − − − − − → C Ij - multiclass case

Fast edge detection June 6, 2018 11 / 22

SLIDE 34

Structured Edges Clustering

since the elements of Y are of size 16 x 16, the dimension of Z is 256

2

Fast edge detection

June 6, 2018 12 / 22

SLIDE 35

Structured Edges Clustering

since the elements of Y are of size 16 x 16, the dimension of Z is 256

2

too expensive → randomly sample m = 256 dimensions of Z

Fast edge detection June 6, 2018 12 / 22

SLIDE 36

Structured Edges Clustering

since the elements of Y are of size 16 x 16, the dimension of Z is 256

2

too expensive → randomly sample m = 256 dimensions of Z

Y

sampled pairs

− − − − − − − − → Z

PCA, k-means

− − − − − − − − → C

Fast edge detection June 6, 2018 12 / 22

SLIDE 37

Ensemble model

during training, we need to assign a single prediction to a leaf node

Fast edge detection June 6, 2018 13 / 22

SLIDE 38

Ensemble model

during training, we need to assign a single prediction to a leaf node during testing, we need to combine multiple predictions into one

Fast edge detection June 6, 2018 13 / 22

SLIDE 39

Ensemble model

during training, we need to assign a single prediction to a leaf node during testing, we need to combine multiple predictions into one to select a single output from a set y1, ..., yk ∈ Y:

Fast edge detection June 6, 2018 13 / 22

SLIDE 40

Ensemble model

during training, we need to assign a single prediction to a leaf node during testing, we need to combine multiple predictions into one to select a single output from a set y1, ..., yk ∈ Y:

compute zi = Πyi

Fast edge detection June 6, 2018 13 / 22

SLIDE 41

Ensemble model

during training, we need to assign a single prediction to a leaf node during testing, we need to combine multiple predictions into one to select a single output from a set y1, ..., yk ∈ Y:

compute zi = Πyi select yk∗ such that k∗ = argmink

i,j(zk,j − zi,j)2 (medoid)

Fast edge detection June 6, 2018 13 / 22

SLIDE 42

Ensemble model

during training, we need to assign a single prediction to a leaf node during testing, we need to combine multiple predictions into one to select a single output from a set y1, ..., yk ∈ Y:

compute zi = Πyi select yk∗ such that k∗ = argmink

i,j(zk,j − zi,j)2 (medoid)

a domain specific ensemble model (for edge map): y

′

k∗ = E[y

′

i ]

Fast edge detection June 6, 2018 13 / 22

SLIDE 43