Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays - - PDF document

▶
computer vision
SMART_READER_LITE
LIVE PREVIEW

Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays - - PDF document

Aug 14, 2022 467 likes •800 views

9/6/2015 Computer Vision Dr. Danna Gurari September 8, 2015 1 Todays Outline Aligning two images Chamfer distance Applications Analyze binary images Thresholding Morphological operators Connected

slide-1
SLIDE 1

9/6/2015 1

1

Computer Vision

  • Dr. Danna Gurari

September 8, 2015

Today’s Outline

▫ Aligning two images ▫ Chamfer distance ▫ Applications ▫ Analyze binary images ▫ Thresholding ▫ Morphological operators ▫ Connected components ▫ Region properties ▫ Applications

Today’s Outline

▫ Aligning two images ▫ Chamfer distance ▫ Applications ▫ Analyze binary images ▫ Thresholding ▫ Morphological operators ▫ Connected components ▫ Region properties ▫ Applications

slide-2
SLIDE 2

9/6/2015 2

Aligning Two Images: Idea

  • Figure from Belongie et al.
  • Pixel level similarity? Different.
  • Shape similarity? Similar!

5

Chamfer distance

Set of points in image Set of points on template Minimum distance between point t and some point in I

Chamfer distance

I:

  • Average matching distance to nearest

feature

Minimum distance between point t and some point in I

slide-3
SLIDE 3

9/6/2015 3

Chamfer distance

  • Average matching distance to nearest

feature

Edge image

How is the measure different than just filtering with a mask having the shape points? How expensive is a naïve implementation?

Source: Yuri Boykov

3 4 2 3 2 3 5 4 4 2 2 3 1 1 2 2 1 1 2 1 1 1 2 1 1 2 3 2 1 1 1 1 2 3 3 2 1 1 1 1 1 2 1 1 2 3 4 3 2 1 1 2 2 Distance Transform Image features (2D)

Distance Transform is a function that for each image pixel p assigns a non-negative number corresponding to distance from p to the nearest feature in the image I

Features could be edge points, foreground points,…

Distance transform Distance transform

  • riginal

distance transform edges

Value at (x,y) tells how far that position is from the nearest edge point (or other binary mage structure)

>> help bwdist

slide-4
SLIDE 4

9/6/2015 4

Distance transform (1D)

Adapted from D. Huttenlocher

// 0 if j is in P, infinity otherwise

Distance Transform (2D)

Adapted from D. Huttenlocher

Chamfer distance: Example

1. 2. 3. INPUT OUTPUT

slide-5
SLIDE 5

9/6/2015 5

Chamfer distance: Example

  • Average distance to nearest feature

Edge image Distance transform image

Chamfer distance: Recognition

Fig from D. Gavrila, DAGM 1999

Edge image Distance transform image

Chamfer distance: properties

  • Sensitive to scale and rotation
  • Tolerant of small shape changes, clutter
  • Need large number of template shapes
  • Inexpensive way to match shapes
slide-6
SLIDE 6

9/6/2015 6

Chamfer matching system

  • Gavrila et al.

http://gavrila.net/Research/Chamfer_System/chamfer_system.html

QuickTime™ and a Microsoft Video 1 decompressor are needed to see this picture. QuickTime™ and a Microsoft Video 1 decompressor are needed to see this picture.

Chamfer matching system

  • Gavrila et al.

http://gavrila.net/Research/Chamfer_System/chamfer_system.html

QuickTime™ and a Microsoft Video 1 decompressor are needed to see this picture. QuickTime™ and a Microsoft Video 1 decompressor are needed to see this picture.

Chamfer matching system

QuickTime™ and a Microsoft Video 1 decompressor are needed to see this picture.

  • Gavrila et al.

http://gavrila.net/Research/Chamfer_System/chamfer_system.html

Prototypes Final Shapes

slide-7
SLIDE 7

9/6/2015 7

Today’s Outline

▫ Aligning two images ▫ Chamfer distance ▫ Applications ▫ Analyze binary images ▫ Thresholding ▫ Morphological operators ▫ Connected components ▫ Region properties ▫ Applications

Binary images

Kristen Grauman, UT-Austin

Binary image analysis: basic steps

  • Convert the image into binary form

– Thresholding

  • Clean up the thresholded image

– Morphological operators

  • Extract separate blobs

– Connected components

  • Describe the blobs with region properties

Kristen Grauman, UT-Austin

slide-8
SLIDE 8

9/6/2015 8 Binary images

  • Two pixel values

– Foreground and background – Mark region(s) of interest

Thresholding

  • Grayscale -> binary mask
  • Useful if object of interest’s intensity distribution is

distinct from background

  • Example

http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FITZGIBBON/si mplebinary.html

Thresholding

  • Given a grayscale image or an intermediate

matrix threshold to create a binary

  • utput.

Gradient magnitude

Looking for pixels where gradient is strong.

fg_pix = find(gradient_mag > t);

Example: edge detection

Kristen Grauman, UT-Austin

slide-9
SLIDE 9

9/6/2015 9

=

  • Thresholding
  • Given a grayscale image or an intermediate

matrix threshold to create a binary

  • utput.

Example: background subtraction

Looking for pixels that differ significantly from the “empty” background.

fg_pix = find(diff > t);

Kristen Grauman, UT-Austin

Thresholding

  • Given a grayscale image or an intermediate

matrix threshold to create a binary

  • utput.

Example: intensity-based detection

Looking for dark pixels

fg_pix = find(im < 65);

Kristen Grauman, UT-Austin

Thresholding

  • Given a grayscale image or an intermediate

matrix threshold to create a binary

  • utput.

Example: color-based detection

Looking for pixels within a certain hue range.

fg_pix = find(hue > t1 & hue < t2);

Kristen Grauman, UT-Austin

slide-10
SLIDE 10

9/6/2015 10 A nice case: bimodal intensity histograms

Ideal histogram, light object on dark background Actual observed histogram with noise

Images: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT2/node3.html

Not so nice cases

Shapiro and Stockman

Issues

  • What to do with “noisy” binary
  • utputs?

– Holes – Extra small fragments

  • How to demarcate multiple

regions of interest?

– Count objects – Compute further features per object

Kristen Grauman, UT-Austin

slide-11
SLIDE 11

9/6/2015 11

Morphological operators

  • Change the shape of the foreground regions via

intersection/union operations between a scanning structuring element and binary image.

  • Useful to clean up result from thresholding
  • Basic operators are:

– Dilation – Erosion

Dilation

  • Expands connected components
  • Grow features
  • Fill holes

Before dilation After dilation

Kristen Grauman, UT-Austin

Erosion

  • Erode connected components
  • Shrink features
  • Remove bridges, branches, noise

Before erosion After erosion

Kristen Grauman, UT-Austin

slide-12
SLIDE 12

9/6/2015 12

Structuring elements

  • Masks of varying shapes and sizes used to

perform morphology, for example:

  • Scan mask across foreground pixels to transform

the binary image

  • >> help strel

Kristen Grauman, UT-Austin

Dilation vs. Erosion

  • At each position:
  • Dilation: if current pixel is foreground, OR the

structuring element with the input image.

Example for Dilation (1D)

1 1 1 1 1 1

Input image Structuring Element

1 1

Output Image

1 1 1

Adapted from T. Moeslund

slide-13
SLIDE 13

9/6/2015 13

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1

Output Image

1 1 1

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1

Output Image

1 1 1

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1

Output Image

1 1 1

slide-14
SLIDE 14

9/6/2015 14

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1 1 1 1

Output Image

1 1 1

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1 1 1 1 1

Output Image

1 1 1

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1 1 1 1 1 1

Output Image

1 1 1

slide-15
SLIDE 15

9/6/2015 15

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1 1 1 1 1 1

Output Image

1 1 1

Example for Dilation

1 1 1 1 1 1

Input image Structuring Element

1 1 1 1 1 1 1 1 1

Output Image

1 1 1

Note that the object gets bigger and holes are filled.

>> help imdilate

2D example for dilation

Shapiro & Stockman

slide-16
SLIDE 16

9/6/2015 16

Dilation vs. Erosion

At each position:

  • Dilation: if current pixel is foreground, OR the

structuring element with the input image.

  • Erosion: if every pixel under the structuring

element’s nonzero entries is foreground, OR the current pixel with S.

Example for Erosion (1D)

1 1 1 1 1 1

Input image Structuring Element Output Image

1 1 1

_

Example for Erosion (1D)

1 1 1 1 1 1

Input image Structuring Element Output Image

1 1 1

_

slide-17
SLIDE 17

9/6/2015 17

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element Output Image

1 1 1

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element Output Image

1 1 1

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element Output Image

1 1 1

slide-18
SLIDE 18

9/6/2015 18

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element

1

Output Image

1 1 1

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element

1

Output Image

1 1 1

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element

1

Output Image

1 1 1

slide-19
SLIDE 19

9/6/2015 19

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element

1

Output Image

1 1 1

Example for Erosion

1 1 1 1 1 1

Input image Structuring Element

1 1

Output Image

1 1 1

Note that the object gets smaller

>> help imerode

2D example for erosion

Shapiro & Stockman

slide-20
SLIDE 20

9/6/2015 20

Opening

  • Erode, then dilate
  • Remove small objects, keep original shape

Before opening After opening

Closing

  • Dilate, then erode
  • Fill holes, but keep original shape

Before closing After closing Applet: http://bigwww.epfl.ch/demo/jmorpho/start.php

Morphology operators on grayscale images

  • Dilation and erosion typically performed on binary

images.

  • If image is grayscale: for dilation take the

neighborhood max, for erosion take the min.

  • riginal

dilated eroded

Kristen Grauman, UT-Austin

slide-21
SLIDE 21

9/6/2015 21

Issues

  • What to do with “noisy” binary
  • utputs?

– Holes – Extra small fragments

  • How to demarcate multiple

regions of interest?

– Count objects – Compute further features per object

Kristen Grauman, UT-Austin

Connected components

  • Various algorithms to compute

– Recursive (in memory) – Two rows at a time (image not necessarily in memory) – Parallel propagation strategy

Recursive connected components

  • Demo http://www.cosc.canterbury.ac.nz/mukundan/covn/Label.html
  • Find an unlabeled pixel, assign it a new

label

  • Search to find its neighbors, and

recursively repeat to find their neighbors til there are no more

  • Repeat
slide-22
SLIDE 22

9/6/2015 22

Connected components

  • Identify distinct regions of “connected pixels”

Shapiro and Stockman

Connectedness

  • Defining which pixels are considered neighbors

4-connected 8-connected

Source: Chaitanya Chandra

Connected components

  • We’ll consider a

sequential algorithm that requires only 2 passes over the image.

  • Input: binary image
  • Output: “label” image,

where pixels are numbered per their t

slide-23
SLIDE 23

9/6/2015 23

Sequential connected components

Adapted from J. Neira

Sequential connected components Sequential connected components

slide-24
SLIDE 24

9/6/2015 24

Sequential connected components Connected components

Slide credit: Pinar Duygulu

Region properties

  • Given connected components, can compute

simple features per blob, such as: – Area (num pixels in the region) – Centroid (average x and y position of pixels in the region) – Bounding box (min and max coordinates) – Circularity (ratio of mean dist. to centroid over std)

A1=200 A2=170

Kristen Grauman, UT-Austin

slide-25
SLIDE 25

9/6/2015 25

Circularity

Shapiro & Stockman [Haralick]

Binary image analysis: basic steps (recap)

  • Convert the image into binary form

– Thresholding

  • Clean up the thresholded image

– Morphological operators

  • Extract separate blobs

– Connected components

  • Describe the blobs with region properties

Matlab

  • N = hist(Y,M)
  • L = bwlabel (BW,N);
  • STATS =

regionprops(L,PROPERTIES) ;

  • 'Area'
  • 'Centroid'
  • 'BoundingBox'
  • 'Orientation‘, …
  • IM2 = imerode(IM,SE);
  • IM2 = imdilate(IM,SE);
  • IM2 = imclose(IM, SE);
  • IM2 = imopen(IM, SE);
slide-26
SLIDE 26

9/6/2015 26 Example using binary image analysis: OCR

[Luis von Ahn et al. http://recaptcha.net/learnmore.html]

Example using binary image analysis: segmentation of a liver

Slide credit: Li Shen

Example using binary image analysis: Bg subtraction + blob detection

…

slide-27
SLIDE 27

9/6/2015 27

University of Southern California http://iris.usc.edu/~icohen/projects/vace/detection.htm

Example using binary image analysis: Bg subtraction + blob detection

Binary images

  • Pros

– Can be fast to compute, easy to store – Simple processing techniques available – Lead to some useful compact shape descriptors

  • Cons

– Hard to get “clean” silhouettes – Noise common in realistic scenarios – Can be too coarse of a representation – Not 3d

Kristen Grauman, UT-Austin

Today’s Outline (Completed)

▫ Aligning two images ▫ Chamfer distance ▫ Applications ▫ Analyze binary images ▫ Thresholding ▫ Morphological operators ▫ Connected components ▫ Region properties ▫ Applications

slide-28
SLIDE 28

9/6/2015 28

Slide credit: Birgi Tamersoy

Background subtraction

  • Simple techniques can do ok with static camera
  • …But hard to do perfectly
  • Widely used:

– Traffic monitoring (counting vehicles, detecting & tracking vehicles, pedestrians), – Human action recognition (run, walk, jump, squat), – Human-computer interaction – Object tracking

Slide credit: Birgi Tamersoy

slide-29
SLIDE 29

9/6/2015 29

Slide credit: Birgi Tamersoy Slide credit: Birgi Tamersoy Slide credit: Birgi Tamersoy

slide-30
SLIDE 30

9/6/2015 30 Frame differences

  • vs. background subtraction
  • Toyama et al. 1999

Slide credit: Birgi Tamersoy

Average/Median Image

Alyosha Efros, CMU

slide-31
SLIDE 31

9/6/2015 31

Background Subtraction

  • =

Alyosha Efros, CMU

Pros and cons

  • Advantages:
  • Extremely easy to implement and use!
  • All pretty fast.
  • Corresponding background models need

not be constant, they change over time.

  • Disadvantages:
  • Accuracy of frame differencing depends on
  • bject speed and frame rate
  • Median background model: relatively high

memory requirements.

  • Setting global threshold Th

When will this basic approach fail?

Slide credit: Birgi Tamersoy

Background mixture models

  • Adaptive Background Mixture Models for Real-Time Tracking, Chris Stauer & W.E.L. Gr

Idea: model each background pixel with a mixture of Gaussians; update its parameters over time.

slide-32
SLIDE 32

9/6/2015 32

Background subtraction with depth

How can we select foreground pixels based on depth information?