Model Fitting " - - PowerPoint PPT Presentation

Model Fitting

ע הנבנש רועיש לע ססובמ" לט ירנסה

תורוקמ

• דומילה רפס ינפ לע רזופמ...

Fitting: Motivation

• We’ve learned how to

detect edges, corners,

• blobs. Now what?
• We would like to form a

higher-level, more compact representation of the features in the image by grouping multiple features according to a simple model

9300 Harris Corners Pkwy, Charlotte, NC

Source: K. Grauman

Fitting

• Choose a parametric model to represent a

set of features

simple model: lines simple model: circles complicated model: car

Fitting

• Choose a parametric model to represent a

set of features

• Line, ellipse, spline, etc.
• Three main questions:
• What model represents this set of features best?
• Which of several model instances gets which feature?
• How many model instances are there?
• Computational complexity is important
• It is infeasible to examine every possible set of parameters

and every possible combination of features

Fitting: Issues

• Noise in the measured feature locations
• Extraneous data: clutter (outliers), multiple lines
• Missing data: occlusions

Case study: Line detection

Fitting: Issues

• If we know which points belong to the line,

how do we find the “optimal” line parameters?

• Least squares
• What if there are outliers?
• RANSAC
• What if there are many lines?
• Voting methods: Hough transform
• What if we’re not even sure it’s a line?
• Model selection

תודוקנל וק תמאתה"תחת-שער"

Least squares line fitting

Data: (x1, y1), …, (xn, yn) Line equation: yi = m xi + b Find (m, b) to minimize

 

  

n i i i

b x m y E

1 2

) (

(xi, yi) y=mx+b

Least squares line fitting

Data: (x1, y1), …, (xn, yn) Line equation: yi = m xi + b Find (m, b) to minimize

2 2    Y X XB X dB dE

T T

 

) ( ) ( ) ( 2 ) ( ) ( 1 1 1

2 2 1 1 1 2

XB XB Y XB Y Y XB Y XB Y XB Y b m x x y y b m x y E

T T T T n n n i i i

                                                        Normal equations: least squares solution to XB=Y

 

  

n i i i

b x m y E

1 2

) (

(xi, yi) y=mx+b

Y X XB X

T T



ב- MATLAB

• תואוושמה תכרעמל ןורתפה

XB=Y : B = X\Y;

Problem with “vertical” least squares

• Not rotation-invariant
• Fails completely for vertical lines

Total least squares

Distance between point (xi, yi) and line ax+by=d (a2+b2=1): |axi + byi – d|

 

  

n i i i

d y b x a E

1 2

) ( (xi, yi)

ax+by=d

Unit normal: N=(a, b)

Total least squares

Distance between point (xi, yi) and line ax+by=d (a2+b2=1): |axi + byi – d|

Proof: (from wikipedia)

 

  

n i i i

d y b x a E

1 2

) ( (xi, yi)

ax+by=d

Unit normal: N=(a, b)

Total least squares

Distance between point (xi, yi) and line ax+by=d (a2+b2=1): |axi + byi – d| Find (a, b, d) to minimize the sum of squared perpendicular distances

 

  

n i i i

d y b x a E

1 2

) ( (xi, yi)

ax+by=d

 

  

n i i i

d y b x a E

1 2

) (

Unit normal: N=(a, b)

Total least squares

Distance between point (xi, yi) and line ax+by=d (a2+b2=1): |axi + byi – d| Find (a, b, d) to minimize the sum of squared perpendicular distances

 

  

n i i i

d y b x a E

1 2

) ( (xi, yi)

ax+by=d

 

  

n i i i

d y b x a E

1 2

) (

Unit normal: N=(a, b)

) ( 2

1

      

 

n i i i

d y b x a d E

1 1 n n i i i i

a b d x y ax by n n

 

   

 

) ( ) ( )) ( ) ( (

2 1 1 1 2

UN UN b a y y x x y y x x y y b x x a E

T n n n i i i

                             ) ( 2   N U U dN dE

T

Solution to (UTU)N = 0, subject to ||N||2 = 1: eigenvector of UTU associated with the smallest eigenvalue (least squares solution to homogeneous linear system UN = 0)

Total least squares

               y y x x y y x x U

n n

 

1 1

                  

   

    n i i n i i i n i i i n i i T

y y y y x x y y x x x x U U

1 2 1 1 1 2

) ( ) )( ( ) )( ( ) (

second moment matrix

Total least squares

               y y x x y y x x U

n n

 

1 1

                  

   

    n i i n i i i n i i i n i i T

y y y y x x y y x x x x U U

1 2 1 1 1 2

) ( ) )( ( ) )( ( ) (

) , ( y x

N = (a, b)

second moment matrix

) , ( y y x x

i i

 

Least squares: Robustness to noise

Least squares fit to the red points:

Least squares: Robustness to noise

Least squares fit with an outlier:

Problem: squared error heavily penalizes outliers

תוינוציח תודוקנ שישכ הרוק המ?

RANSAC

• Random sample consensus (RANSAC):

Very general framework for model fitting in the presence of outliers

• M. A. Fischler, R. C. Bolles. Random Sample Consensus: A Paradigm for Model

Fitting with Applications to Image Analysis and Automated Cartography. Comm. of the ACM, Vol 24, pp 381-395, 1981.

Fitting a Line

Least squares fit

RANSAC

• Select sample of m

points at random

RANSAC

• Select sample of m points

at random

• Calculate model

parameters that fit the data in the sample

RANSAC

• Select sample of m points

at random

• Calculate model

parameters that fit the data in the sample

• Calculate error function

for each data point

RANSAC

• Select sample of m points

at random

• Calculate model

parameters that fit the data in the sample

• Calculate error function

for each data point

• Select data that support

current hypothesis

RANSAC

• Select sample of m points

at random

• Calculate model

parameters that fit the data in the sample

• Calculate error function

for each data point

• Select data that support

current hypothesis

RANSAC for line fitting

Repeat N times:

• Draw s points uniformly at random
• Fit line to these s points
• Find inliers to this line among the remaining

points (i.e., points whose distance from the line is less than t)

• If there are d or more inliers, accept the line

and refit using all inliers

Choosing the parameters

• Initial number of points s
• Typically minimum number needed to fit the model
• Distance threshold t
• Choose t so probability for inlier is p (e.g. 0.95)
• Zero-mean Gaussian noise with std. dev. σ: t2=3.84σ2
• Number of iterations N
• Choose N so that, with probability p, at least one random

sample is free from outliers (e.g. p=0.99) (outlier ratio: e)

proportion of outliers e

s 5% 10% 20% 25% 30% 40% 50% 2 2 3 5 6 7 11 17 3 3 4 7 9 11 19 35 4 3 5 9 13 17 34 72 5 4 6 12 17 26 57 146 6 4 7 16 24 37 97 293 7 4 8 20 33 54 163 588 8 5 9 26 44 78 272 1177

Source: M. Pollefeys

Choosing the parameters

• Initial number of points s
• Typically minimum number needed to fit the model
• Distance threshold t
• Choose t so probability for inlier is p (e.g. 0.95)
• Zero-mean Gaussian noise with std. dev. σ: t2=3.84σ2
• Number of iterations N
• Choose N so that, with probability p, at least one random

sample is free from outliers (e.g. p=0.99) (outlier ratio: e)

• Consensus set size d
• Should match expected inlier ratio

Source: M. Pollefeys

RANSAC pros and cons

• Pros
• Simple and general
• Applicable to many different problems
• Often works well in practice
• Cons
• Lots of parameters to tune
• Can’t always get a good initialization of the model based on

the minimum number of samples

• Sometimes too many iterations are required
• Can fail for extremely low inlier ratios
• We can often do better than brute-force sampling

דחא וקמ רתוי שישכ הרוק המ?

Voting schemes

• Let each feature vote for all the models that

are compatible with it

• Hopefully the noise features will not vote

consistently for any single model

• Missing data doesn’t matter as long as there

are enough features remaining to agree on a good model

Hough transform

• An early type of voting scheme
• General outline:
• Discretize parameter space into bins
• For each feature point in the image, put a vote in every bin in

the parameter space that could have generated this point

• Find bins that have the most votes

P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc.

• Int. Conf. High Energy Accelerators and Instrumentation, 1959

Image space Hough parameter space

Parameter space representation

• A line in the image corresponds to a point in

Hough space

Image space Hough parameter space

Source: S. Seitz

Parameter space representation

• What does a point (x0, y0) in the image space

map to in the Hough space?

Image space Hough parameter space

Parameter space representation

• What does a point (x0, y0) in the image space

map to in the Hough space?

Image space Hough parameter space

Parameter space representation

• What does a point (x0, y0) in the image space

map to in the Hough space?

• Answer: the solutions of b = –x0m + y0
• This is a line in Hough space

Image space Hough parameter space

Parameter space representation

• Where is the line that contains both (x0, y0)

and (x1, y1)?

Image space Hough parameter space

(x0, y0) (x1, y1) b = –x1m + y1

Parameter space representation

• Where is the line that contains both (x0, y0)

and (x1, y1)?

• It is the intersection of the lines b = –x0m + y0 and

b = –x1m + y1 Image space Hough parameter space

(x0, y0) (x1, y1) b = –x1m + y1

Line Detection by Hough Transform

y x

) , ( c m

Parameter Space

1 1 1 1 1 1 2 1 1 1 1 1 1

) , ( c m A

Algorithm:

• Quantize Parameter Space
• Create Accumulator Array
• Set
• For each image edge increment:
• If lies on the line:
• Find local maxima in

) , ( c m ) , ( c m A c m c m A , ) , (   ) , (

i i y

x 1 ) , ( ) , (   c m A c m A ) , ( c m ) , ( c m A

i i

y m x c   

• Problems with the (m,b) space:
• Unbounded parameter domain
• Vertical lines require infinite m
• Quick solution?

Parameter space representation

• Problems with the (m,b) space:
• Unbounded parameter domain
• Vertical lines require infinite m
• Alternative: polar representation

Parameter space representation

       sin cos y x

Each point will add a sinusoid in the (,) parameter space

Algorithm outline

• Initialize accumulator H

to all zeros

• For each edge point (x,y)

in the image For θ = 0 to 180 ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end end

• Find the value(s) of (θ, ρ) where H(θ, ρ) is a

local maximum

• The detected line in the image is given by

ρ = x cos θ + y sin θ ρ θ

features votes

Basic illustration

Other shapes

Square Circle

Other shapes

Several lines

A more complicated image

http://ostatic.com/files/images/ss_hough.jpg

ב- MATLAB

[H,T,R] = hough(BW); % The following function finds no more than 5 peaks in the Hough matrix H P = houghpeaks(H,5); % Extracts the line segments (not a part of the algorithm described, but useful) lines = houghlines(BW,T,R,P); % Each line i can then be plotted by: for i=1:numel(lines) xy = [lines(i).point1; lines(i).point2]; plot(xy(:,1),xy(:,2)); end

features votes

Effect of noise

features votes

Effect of noise

Peak gets fuzzy and hard to locate

Effect of noise

• Number of votes for a line of 20 points with

increasing noise:

Random points

Uniform noise can lead to spurious peaks in the array

features votes

Random points

• As the level of uniform noise increases, the

maximum number of votes increases too:

Dealing with noise

• Choose a good grid / discretization
• Too coarse: large votes obtained when too many different

lines correspond to a single bucket

• Too fine: miss lines because some points that are not

exactly collinear cast votes for different buckets

• Increment neighboring bins (smoothing in

accumulator array)

• Try to get rid of irrelevant features
• Take only edge points with significant gradient magnitude

Incorporating image gradients

• Recall: when we detect an

edge point, we also know its gradient direction

• But this means that the line

is uniquely determined!

• Modified Hough transform:

For each edge point (x,y) θ = gradient orientation at (x,y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end

Finding Circles by Hough Transform

Equation of Circle:

2 2 2

) ( ) ( r b y a x

i i

   

If radius is known:

) , ( b a A

Accumulator Array (2D Hough Space)

סוידרה םא R עודי...

Equation of Circle:

2 2 2

) ( ) ( r b y a x

i i

   

If radius is not known: 3D Hough Space! Use Accumulator array

) , , ( r b a A

What is the surface in the hough space?

Finding Circles by Hough Transform

עודי אל סוידרה םא...

• לדג םירטמרפה בחרמ :

a, b, R

b a r

Using Gradient Information

• Gradient information can save lot of computation:

Edge Location Edge Direction Need to increment only one point in Accumulator!!

i



) , (

i i y

x

Assume radius is known:

  sin cos r y b r x a    

תורוצל רבעמ"תוטושפ"

Generalized Hough transform

• We want to find a shape defined by its boundary

points and a reference point

• D. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes,

Pattern Recognition 13(2), 1981, pp. 111-122. a

Generalized Hough Transform

• Model Shape NOT described by equation

Generalized Hough Transform

• Model Shape NOT described by equation

Generalized Hough Transform

Find Object Center given edges Create Accumulator Array Initialize: For each edge point For each entry in table, compute: Increment Accumulator: Find Local Maxima in

) , (

c c y

x A ) , ( ) , (

c c c c

y x y x A   ) , , (

i i i y

x  1 ) , ( ) , (  

c c c c

y x A y x A ) , (

c c y

x A

i k i k i c i k i k i c

r y y r x x   sin cos    

i k

r

) , (

c c y

x ) , , (

i i i y

x 

Generalized Hough transform

• Assumption: translation is the only

transformation here, i.e., orientation and scale are fixed.

• How can we generalize the idea to a model

that can be rotated or scaled?

Source: K. Grauman

Hough transform: Discussion

• Pros
• Can deal with non-locality and occlusion
• Can detect multiple instances of a model
• Some robustness to noise: noise points unlikely to contribute

consistently to any single bin

• Cons
• Complexity of search time increases exponentially with the

number of model parameters

• Non-target shapes can produce spurious peaks in parameter

space

• It’s hard to pick a good grid size
• Hough transform vs. RANSAC

תוצובקל תודוקנ תמאתה

תוצובקל תודוקנ תמאתה

• Clustering
• Segmentation
• K-means

הנומתה תא קלחל הצרנ עבצ יפל םירוזאל

היצביטומ

תואצות

רוקמ K=5 K=10

K-means clustering

• Want to minimize sum of squared Euclidean

distances between points xi and their nearest cluster centers mk Algorithm:

• Randomly initialize K cluster centers
• Iterate until convergence:
• Assign each data point to the nearest center
• Recompute each cluster center as the mean of all points

assigned to it

 

 

k k i k i

m x M X D

cluster cluster in point 2

) ( ) , (

PUTTING IT ALL TOGETHER המגוד :תוינוכמ רותיאל תכרעמ

Implicit shape models: Training

• 1. Build codebook of patches around extracted

interest points using k-means clustering

• B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and

Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004

Implicit shape models: Training

• 1. Build codebook of patches around extracted

interest points using k-means clustering

• 2. Map the patch around each interest point to

closest codebook entry

Implicit shape models: Training

• 1. Build codebook of patches around extracted

interest points using k-means clustering

• 2. Map the patch around each interest point to

closest codebook entry

• 3. For each codebook entry, store all positions

it was found, relative to object center

Implicit shape models: Testing

1. Given test image, extract patches, match to codebook entry 2. Cast votes for possible positions of object center 3. Search for maxima in voting space 4. Extract weighted segmentation mask based on stored masks for the codebook occurrences

םויה וניאר המ זא?

םוכיס

• רשיל תודוקנ תמאתה

– םיתוחפ םיעוביר– יתש תואסרג – RANSAC –ףאאה

• ףאאה תויללכ תורוצלו םילגעמל
• תוצובקל תודוקנ ץוביק

k-means clustering

• תוינוכמ רותיאל םתירוגלאל המגוד

םיפקשל תורוקמ

• שרופמב וניוצש ולא דבלמ , לע םיססובמ םיפקש

ש ולא:

• Svetlana Lazebnik
• Ondřej Chum
• Jason Lawrence
• Szymon Rusinkiewicz
• Harvey Rhody
• Utkarsh Sinha