SLIDE 1
Introduction
- Image labeling
– Classifying every image patch into a finite set of classes
- Typical issues
Multiscale Conditional Random Fields for Image Labeling Xuming He, - - PowerPoint PPT Presentation
Multiscale Conditional Random Fields for Image Labeling Xuming He, - - PowerPoint PPT Presentation
Multiscale Conditional Random Fields for Image Labeling Xuming He, Richard Zemel and Miguel A. Carreira-Perpinan Department of Computer Science University of Toronto Introduction Image labeling Classifying every image patch into a
SLIDE 2
SLIDE 3
Random Field Framework
- Generative Markov Random Fields (MRFs)
- Modeling:
- Labeling:
Label (L) Image (X)
i
x
i
l
j
l
∏ ∏
=
i j i j i i i
q Z P , P
,
) , ( 1 ) | ( ) ( l l l x X L ) ( / ) , ( ) | ( X X L X L P P P =
SLIDE 4
Motivation
- Capturing context in different scales
– Local constraints vs. Global configurations – Representing context dependency
- Building conditional model
– Saving modeling resources – Training model discriminatively
- Learning context from data
– Adapting context representation to images and labels
SLIDE 5
Overview
- Multiscale Conditional Random Fields (mCRF)
) | ( X L
C
P ) (L
R
P ) (L
G
P
Regional label features Global label features
) ( ) ( ) | ( 1 ) | ( L L X L X L
G R C
P P P Z P =
SLIDE 6
Examples of Global Features
Rhino/hippo Polar bear Water Snow Vegetation Ground Sky
SLIDE 7
Examples of Local Features
Sky Vegetation Marking Road Building Street Obj. Car
SLIDE 8
Labeling Images
- Inferring Labels L from Image X
- Mode of Posterior Marginals Criterion
- Approximate Inference
– Block Gibbs sampling – Comparable to other methods: Loopy BP, Mean Field.
) | ( max arg
*
X l l
l i i
P
i
=
SLIDE 9
Parameter Estimation
- Supervised Learning with Dataset
- Conditional Maximum Likelihood
- Gradient Ascent using MCMC
- Supervised Contrastive Divergence
∑
=
t t t
P )) ; | ( log( max θ θ
θ
X L )
∑
∂ ∂ − ∂ ∂ ∝ ∆
t t X L P k k t k X L P k k t k k
h h
t
) | ( ) | (
) ; | ( log ) ; | ( log
θ
θ θ θ θ θ X L X L
{ }
N t
t t
, , 1 ), , ( K = X L
SLIDE 10
Experiment Setup
- Data set:
– Subset of Corel database: African and Arctic wildlife natural
- scenes. (Training: 60, Testing: 40, 7 label classes)
– Sowerby database: Rural and suburban scenes. (Training: 60, Testing: 44, 7 label classes)
- Performance comparison:
– Generative MRF – Local classifier
SLIDE 11
Results on Corel Image Data
Image Classifier MRF mCRF mCRF confidence
Rhino/hippo Polar bear Water Snow Vegetation Ground Sky
Classification rate 66.9% 66.2% 80.0%
SLIDE 12
Results on Sowerby Image Data
Image Classifier MRF mCRF mCRF confidence
Sky Vegetation Marking Road Building Street Obj. Car
Classification rate 82.4% 81.8% 89.5%
SLIDE 13
Summary and Future Work
- Summary
– Multiscale representation of contextual information – Learning features from image data – Conditional RF model trained discriminatively
- Future Work