Regularizing Part Geometry Instructor - Simon Lucey 16-623 - - - PowerPoint PPT Presentation

Regularizing Part Geometry

Instructor - Simon Lucey

16-623 - Designing Computer Vision Apps

Today

• Parts Based Registration
• Regularizing Parts (Heuristic)
• Regularizing Parts (Learned)

2

What is an Object?

Face Body

What is an Object?

Face Body

What is an Object?

Face Body

Left Eye Right Eye Nose Mouth Right Lower Arm Head Left Upper Arm Left Lower Arm Left Thigh Left Calf Right Upper Arm Torso Right Thigh Right Calf

What is a Part?

Face

Left Eye Left Eye Not Left Eye

When is a Collection of Parts an Object?

When is a Collection of Parts an Object?

Face

When is a Collection of Parts an Object?

Face Face

When is a Collection of Parts an Object?

Face Not a Face Face

What is an Object?

“A collection of semantically meaningful components with geometrical constraints on their spatial configuration’’

Data

Registration

Registration

This is an image of a body.

Registration

This is an image of a body. Show me the...

Registration

This is an image of a body.

Right Lower Arm? Head? Left Upper Arm? Left Lower Arm? Left Thigh? Left Calf? Right Upper Arm? Torso? Right Thigh? Right Calf?

Show me the...

Parts Based Registration

...

Right Lower Arm Head

Parts Based Registration

...

Right Lower Arm Head

Parts Based Registration

...

Right Lower Arm Head

Parts Based Registration

...

Right Lower Arm Head

min

x N

X

i=1

Di(xi) + λ R(x)

...

Right Lower Arm Head

Parts Based Registration

min

x N

X

i=1

Di(xi) + λ R(x)

...

Right Lower Arm Head

Parts Based Registration

min

x N

X

i=1

Di(xi) + λ R(x) Does the image at look like the part?

xi

ith

...

Right Lower Arm Head

Parts Based Registration

min

x N

X

i=1

Di(xi) + λ R(x) Does the image at look like the part?

xi

ith

Do the joint locations of the parts match the object?

Today

• Parts Based Registration
• Regularizing Parts (Heuristic)
• Regularizing Parts (Learned)

10

Why Regularize?

Why Regularize?

Appearance Variability

Why Regularize?

Appearance Variability

[7] Gross et al.’08

Why Regularize?

Appearance Variability

[7] Gross et al.’08

False Negatives

Why Regularise?

Why Regularise?

Appearance Ambiguity

Why Regularise?

? ?

[7] Gross et al.’08

Appearance Ambiguity

Why Regularise?

? ?

[7] Gross et al.’08

Appearance Ambiguity False Positives

Why Regularise?

Head Right Shoulder

[1] Felzenszwalb et al.’09

Left Lip Corner Right Eye Corner

What is a Regulariser?

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x)

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x)

Likelihood

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x)

Likelihood Prior

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x) max

x

p(I|x) p(x)

Likelihood Prior

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x) max

x

p(I|x) p(x) min

x

− log {p(I|x) p(x)}

Likelihood Prior

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x) max

x

p(I|x) p(x) min

x

− log {p(I|x)} − log {p(x)}

=

min

x

− log {p(I|x) p(x)}

Likelihood Prior

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x) max

x

p(I|x) p(x) min

x

− log {p(I|x)} − log {p(x)}

= =

min

x

X

i

D(xi) + λR(x) min

x

− log {p(I|x) p(x)}

Likelihood Prior

What is a Regulariser?

p(x|I) ∝ p(I|x)p(x) max

x

p(I|x) p(x) min

x

− log {p(I|x)} − log {p(x)}

= =

min

x

X

i

D(xi) + λR(x) R(x) ∝ − log{p(x)} min

x

− log {p(I|x) p(x)}

Likelihood Prior

What is a Regulariser?

Face Not a Face Face

What is a Regulariser?

Face Not a Face Face Regularisation helps disambiguate candidates locations of parts by enforcing geometric dependencies between parts!

What is a Regulariser?

Face Not a Face Face Regularisation helps disambiguate candidates locations of parts by enforcing geometric dependencies between parts!

p(I|x1)

p(I|x2)

p(x|I)

θ θ

θ

[22] Saragih et al.’10

What is a Regulariser?

Face Not a Face Face Regularisation helps disambiguate candidates locations of parts by enforcing geometric dependencies between parts!

p(I|x1)

p(I|x2)

p(x|I)

θ θ

θ

[22] Saragih et al.’10

What is a Regulariser?

Face Not a Face Face Regularisation helps disambiguate candidates locations of parts by enforcing geometric dependencies between parts!

p(I|x1)

p(I|x2)

p(x|I)

θ θ

θ

[22] Saragih et al.’10

Heuristic Regularisation

Heuristic Regularisation

Heuristic Regularisation

Heuristic Regularisation

Heuristic Regularisation

u v R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

Heuristic Regularisation

R(x) = k 5 uk2 + k 5 vk2

[2] Horn and Schunck’81

• Regularisation enforces a notion of

smoothness

• Points in close spatial proximity should

move in similar ways

• Not always true, but quite effective when

we know nothing else about the object of interest

• Extension using the L1-penalty considered

state of the art in heuristic regularisation

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 x = [x1; . . . ; xn]

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 x = [x1; . . . ; xn]

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

x = [x1; . . . ; xn]

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

=

• x − x0

2

L

x = [x1; . . . ; xn]

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

=

• x − x0

2

L

x = [x1; . . . ; xn]

Graph Laplacian

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

=

• x − x0

2

L

x = [x1; . . . ; xn] Lij =      |Ni| if i = j −1 if j ∈ Ni

• therwise

Graph Laplacian

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

=

• x − x0

2

L

L = ΦΛΦT x = [x1; . . . ; xn] Lij =      |Ni| if i = j −1 if j ∈ Ni

• therwise

Graph Laplacian

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

= X

i

λiΦiΦT

i

=

• x − x0

2

L

L = ΦΛΦT x = [x1; . . . ; xn] Lij =      |Ni| if i = j −1 if j ∈ Ni

• therwise

Graph Laplacian

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = k 5 uk2 = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

= X

i

λiΦiΦT

i

R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

=

• x − x0

2

L

L = ΦΛΦT x = [x1; . . . ; xn] Lij =      |Ni| if i = j −1 if j ∈ Ni

• therwise

Graph Laplacian

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

Φi

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

Φi

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φi

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    i λi x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    i λi Φ0 x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    i λi Φ0 Φi x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    i λi Φ0 Φi Φn−1 x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Φi

Laplacian Regularisation

Λ =    λ1 ... λn    i λi Φ0 Φi Φn−1 Φn x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) = X

i

λi

• ΦT

i (x − x0)

• 2

x − x0

Φ

T i

• x

− x

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 R(x) =

• x − x0

2

L

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

R(x) =

• x − x0

2

L

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 R(x) =

• x − x0

2

L

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 R(x) =

• x − x0

2

L

Σ ∝ L−1

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 = ΦΛ−1ΦT R(x) =

• x − x0

2

L

Σ ∝ L−1

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 = ΦΛ−1ΦT = ΨΩΨT R(x) =

• x − x0

2

L

Σ ∝ L−1

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 = ΦΛ−1ΦT = ΨΩΨT i σi Ω =    σ1 ... σn    R(x) =

• x − x0

2

L

Σ ∝ L−1

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 = ΦΛ−1ΦT = ΨΩΨT i σi Ω =    σ1 ... σn    = X

i

σiΨiΨT

i

R(x) =

• x − x0

2

L

Σ ∝ L−1

Laplacian Regularisation

x1 x2 x3 x4 x5 x6 x7 x8 x9 N(x) / exp ⇢ 1 2kx µk2

Σ−1

• Gaussian Distribution:

µ = x0 = ΦΛ−1ΦT = ΨΩΨT i σi Ω =    σ1 ... σn    = X

i

σiΨiΨT

i

Ψ0 Ψ1 Ψn R(x) =

• x − x0

2

L

Σ ∝ L−1

Example: Optical Flow

arg min

∆x ||I0(x) + ∂I0(x)

∂xT ∆x − I1(x)||2

2

∆x =    ∆x1 . . . ∆xN   

N = no. of pixels

General Topology

x0 6= What if grid?

General Topology

x0 6= What if grid?

General Topology

x0 6= What if grid? = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

R(x)

General Topology

x0 6= What if grid? = X

i

X

j∈Ni

[(xi − x0

i ) − (xj − x0 j)]2

R(x) x = x0 + Ψα

Smooth Deformation Basis

Ψ0 Ψ1 Ψ2 Ψ3 Ψ25 Ψ50

~Frequency x y z

Smooth Deformation Basis

Ψ0 Ψ1 Ψ2 Ψ3 Ψ25 Ψ50

~Frequency x y z

Heuristic Regularisation: Recap

• The concept of frequency that is penalised can be specialised

to the topology of the object though defining specialised graph- laplacian

• Regularisation is important because image measurements are not enough
• Priors model the space of valid instances of object’s geometry
• Regularisers penalise object geometry outside the space of valid instances.
• The smoothness the assumption is a good heuristic
• Laplacian regularisers enforce smoothness by penalising high frequency

variations more heavily than lower frequency variations Prior Regulariser

• But... is that the best we can do?

Today

• Parts Based Registration
• Regularizing Parts (Heuristic)
• Regularizing Parts (Learned)

30

Data Driven (Learned) Regularisers

What if we have annotated data?

[3] Huang et al.’07

Topology of Samples vs. Parts

Samples Data Parts

<3d

Topology of Samples vs. Parts

Samples Data Parts Sample Topology

<d <3d

Topology of Samples vs. Parts

Samples Data Parts Part Topology Sample Topology

Exhaustive Search (N > 1)

33

Exhaustive Search (N > 1)

33

φ{I}

φ{T1(0)}

φ{T2(0)}

I

φ{TN(0)}

Exhaustive Search (N > 1)

34

∗

∗ ∗

Feature Extraction Local Search ............

D2 D1 DN

............ Part Responses

Color Encoding of Filter Responses

Felzenszwalb, Girshick, McAllester & Ramanan, 2010

Exhaustive Search (N > 1)

34

D2

D1 DN

............

D2

D1 DN

............

p∗

1

p∗

2

p∗

N

O(M N) p∗ = arg min

p N

X

i=1

Di(pi) + λ R(p) p∗ = [pT

1 , . . . , pT N]T

Computational Cost

35

D2

D1 DN

............

p∗

1

p∗

2

p∗

N

O(M N) p∗ = arg min

p N

X

i=1

Di(pi) + λ R(p) p∗ = [pT

1 , . . . , pT N]T

Computational Cost

35

D2

D1 DN

............

p∗

N = arg min pN DN(pN)

p∗

2 = arg min p2 D2(p2)

p∗

1 = arg min p1 D1(p1)

Computational Cost

36

D2

D1 DN

............

O(M) p∗

1

p∗

2

p∗

N

p∗

N = arg min pN DN(pN)

p∗

2 = arg min p2 D2(p2)

p∗

1 = arg min p1 D1(p1)

Computational Cost

36

O(M) O(M)

p1 p2 p3 p4 p5

Exhaustive Search

37

O(M N)

“We can do much better than this if the graph is sparse.”

p1 p2 p3 p4 p5 O(NM 2)

Exhaustive Search

37

“We can do much better than this if the graph is sparse.”

Types of Graphs

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

Unconnected R(x) = X

i

φi(xi)

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

General Graph

R(x) = X

i

φi(xi) + X

i,j∈E

ψij(xi, xj)

Unconnected R(x) = X

i

φi(xi)

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

General Graph

R(x) = X

i

φi(xi) + X

i,j∈E

ψij(xi, xj)

Unconnected R(x) = X

i

φi(xi) Optimal Sparse Graph

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

General Graph

R(x) = X

i

φi(xi) + X

i,j∈E

ψij(xi, xj)

Unconnected R(x) = X

i

φi(xi) Optimal Sparse Graph LASSO

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

General Graph

R(x) = X

i

φi(xi) + X

i,j∈E

ψij(xi, xj)

Unconnected R(x) = X

i

φi(xi) Optimal Sparse Graph LASSO

min

θj

X

i

⇥xij X

k6=j

θjkxik⇥2 + λ |θj|

[24] Gu et al’07

Types of Graphs

Maximally Connected

R(x) = R(x1, . . . , xn)

General Graph

R(x) = X

i

φi(xi) + X

i,j∈E

ψij(xi, xj)

Unconnected R(x) = X

i

φi(xi) Optimal Sparse Graph LASSO

min

θj

X

i

⇥xij X

k6=j

θjkxik⇥2 + λ |θj|

Loopy... but often converges to good solutions!

[24] Gu et al’07

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

θ1 θ2

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

θ1 θ2

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

s.t. kθk2  γ min

θ

kY Xθk2

Ridge Regression:

θ1 θ2

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

s.t. kθk2  γ min

θ

kY Xθk2

Ridge Regression:

θ1 θ2

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

s.t. kθk2  γ min

θ

kY Xθk2

Ridge Regression:

s.t. |θ| ≤ γ min

θ

kY Xθk2

Lasso:

θ1 θ2

Lasso

min

θ

kY Xθk2 + λkθkc

c

min

θ

kY Xθk2 s.t.kθkc

c  γ

s.t. kθk2  γ min

θ

kY Xθk2

Ridge Regression:

s.t. |θ| ≤ γ min

θ

kY Xθk2

Lasso:

[14] Tibshirani’96

Optimal Sparse Graphs

[24] Gu et al.’07

Mean Procrustes Correlation Sparse Graph Face B

• d

y Hand

Optimal Sparse Graphs

Maximally Connected Star Unconnected Sparse

[24] Gu et al.’07

Tree Regularization

• Sparse graph of particular interest is a tree,

42

Felzenszwalb & Huttenlocher, 2005

Tree Regularization

• Sparse graph of particular interest is a tree,

42

Felzenszwalb & Huttenlocher, 2005

Dynamic Programming

• Globally optimal solution to any tree graph can be found

using “Dynamic Programming”.

43

scorej(pj) = Dj(pj) + P

k∈kids(j) mk(pj)

Dynamic Programming

• Globally optimal solution to any tree graph can be found

using “Dynamic Programming”.

43

scorej(pj) = Dj(pj) + P

k∈kids(j) mk(pj)

Dynamic Programming

• Globally optimal solution to any tree graph can be found

using “Dynamic Programming”.

43

p1 p2 p3 p4 p5

scorej(pj) = Dj(pj) + P

k∈kids(j) mk(pj)

Dynamic Programming

• Globally optimal solution to any tree graph can be found

using “Dynamic Programming”.

43

p1 p2 p3 p4 p5

“parent”

scorej(pj) = Dj(pj) + P

k∈kids(j) mk(pj)

Dynamic Programming

• Globally optimal solution to any tree graph can be found

using “Dynamic Programming”.

43

p1 p2 p3 p4 p5

“kids” “kids” “kids” “kids”

Tree-Structured Graphs

Tree-Structured Graphs

Forwards Backwards

Tree-Structured Graphs

Star Spanning Tree Forwards Backwards

Tree-Structured Graphs

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

V∗ = {i}

Initialise:

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

V∗ = {i}

Initialise:

V∗ = V

While :

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

V∗ = {i}

Initialise:

V∗ = V

While :

j = arg min

j

Wij

s.t i ∈ V∗ , j / ∈ V∗

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

Wij = ( if i = j P

k xi.xj

√

x2

i +x2 j

• therwise

V∗ = {i}

Initialise:

V∗ = V

While :

j = arg min

j

Wij

s.t i ∈ V∗ , j / ∈ V∗

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

Wij = ( if i = j P

k xi.xj

√

x2

i +x2 j

• therwise

V∗ ← V∗ ∪ {j}

V∗ = {i}

Initialise:

V∗ = V

While :

j = arg min

j

Wij

s.t i ∈ V∗ , j / ∈ V∗

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

Maximum Spanning Tree Prim’s Algorithm

Wij = ( if i = j P

k xi.xj

√

x2

i +x2 j

• therwise

V∗ ← V∗ ∪ {j}

V∗ = {i}

Initialise:

V∗ = V

While :

j = arg min

j

Wij

s.t i ∈ V∗ , j / ∈ V∗

[17] Prim’57

Star Spanning Tree

Rc(xj) = φj(xj) + X

i∈Cj

ψj(xj, xi) + X

k

Rc(xk)

R(x) = Rc(xroot)

Forwards Backwards

Tree-Structured Graphs

[1] Felzenszwalb et al.’09

Graph Potentials

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

ψij(xi, xj) = wT

ij [dx; d2 x; dy; d2 y]

[dx; dy] = xi − xj

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

ψij(xi, xj) = wT

ij [dx; d2 x; dy; d2 y]

[dx; dy] = xi − xj = ([xi; xj] − µij)T Σ−1

ij ([xi; xj] − µij)

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

ψij(xi, xj) = wT

ij [dx; d2 x; dy; d2 y]

[dx; dy] = xi − xj = ([xi; xj] − µij)T Σ−1

ij ([xi; xj] − µij)

R(x) = (x − µ)T Σ−1(x − µ)

Sparse Gaussian Prior:

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

ψij(xi, xj) = wT

ij [dx; d2 x; dy; d2 y]

[dx; dy] = xi − xj = ([xi; xj] − µij)T Σ−1

ij ([xi; xj] − µij)

R(x) = (x − µ)T Σ−1(x − µ)

Sparse Gaussian Prior:

Σ−1

Graph Potentials

Spring Models:

[1] Felzenszwalb et al.’09 [25] Yang and Ramanan’11

ψij(xi, xj) : encodes knowledge of distribution of relative spatial location of parts

ψij(xi, xj) = wT

ij [dx; d2 x; dy; d2 y]

[dx; dy] = xi − xj = ([xi; xj] − µij)T Σ−1

ij ([xi; xj] − µij)

R(x) = (x − µ)T Σ−1(x − µ)

Sparse Gaussian Prior:

Σ−1

Sparsity structure follows a tree topology!

Learning Sample Topology

<3d

Sample Topology

Procrustes Analysis

x = [x1; . . . ; xn; y1; . . . ; yn]

Procrustes Analysis

x = [x1; . . . ; xn; y1; . . . ; yn]

[4] Matthews et al.’07

Procrustes Analysis

x = [x1; . . . ; xn; y1; . . . ; yn]

Procrustes

[4] Matthews et al.’07

Procrustes Analysis

x = [x1; . . . ; xn; y1; . . . ; yn]

Procrustes Learning

[4] Matthews et al.’07