CS 4495 Computer Vision Linear Filtering 2: Templates, Edges Aaron - PowerPoint PPT Presentation

Templates/Edges CS 4495 Computer Vision – A. Bobick CS 4495 Computer Vision Linear Filtering 2: Templates, Edges Aaron Bobick School of Interactive Computing

Templates/Edges CS 4495 Computer Vision – A. Bobick Last time: Convolution • Convolution: • Flip the filter in both dimensions (right to left, bottom to top) • Then apply cross-correlation = ∗ G H F H* H* F Notation for convolution operator K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Convolution vs. correlation Convolution = ∗ G H F (Cross-)correlation • When H is symmetric, no difference. We tend to use the terms interchangeably. • Convolution with an impulse (centered at 0,0) is the identity K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Filters for features • Previously, thinking of filtering as a way to remove or reduce noise • Now, consider how filters will allow us to abstract higher- level “ features ”. • Map raw pixels to an intermediate representation that will be used for subsequent processing • Goal: reduce amount of data, discard redundancy, preserve what’s useful K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching • Filters as templates : Note that filters look like the effects they are intended to find --- “matched filters” • Use ( normalized ) cross-correlation score to find a given pattern (template) in the image. • Normalization needed to control for relative brightness. More in problem sets. K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching Template (mask) Scene A toy example K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching Template Detected template K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching Detected template Correlation map K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Where’s Waldo? Template Scene K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Where’s Waldo? Template Detected template K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template demo… • In directory C:\Bobick\matlab\CS4495\Filter • echodemo waldotemplate

Templates/Edges CS 4495 Computer Vision – A. Bobick Where’s Waldo? Detected template Correlation map K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching Template Scene What if the template is not identical to some subimage in the scene? K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Template matching Template Detected template Match can be meaningful, if scale, orientation, and general appearance is right. K. Grauman

Templates/Edges CS 4495 Computer Vision – A. Bobick Generic features… • When looking for a specific object or pattern, the features can be defined for that pattern – we will do this later in the course for specific object recognition. • But for generic images, what would be good features? What are the parts or properties of the image that encode its “meaning” for human (or other biological) observers? • Some examples of greatly reduced images…

Templates/Edges CS 4495 Computer Vision – A. Bobick Edges seem to be important…

Templates/Edges CS 4495 Computer Vision – A. Bobick Origin of Edges surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity • Edges are caused by a variety of factors • Information theory view: edges encode change, change is what is hard to predict, therefore edges efficiently encode an image

Templates/Edges CS 4495 Computer Vision – A. Bobick In a real image Depth discontinuity: Reflectance change: object boundary appearance information, texture Cast shadows Change in surface orientation: shape

Templates/Edges CS 4495 Computer Vision – A. Bobick Contrast and invariance

Templates/Edges CS 4495 Computer Vision – A. Bobick Edge detection • Convert a 2D image into a set of curves • Extracts salient features of the scene • More compact than pixels

Templates/Edges CS 4495 Computer Vision – A. Bobick Edge detection • How can you tell that a pixel is on an edge?

Templates/Edges CS 4495 Computer Vision – A. Bobick Images as functions… • Edges look like steep cliffs

Templates/Edges CS 4495 Computer Vision – A. Bobick Edge Detection Basic idea: look for a neighborhood with strong signs of change. Problems: 81 82 26 24 82 33 25 25 • neighborhood size 81 82 26 24 • how to detect change

Templates/Edges CS 4495 Computer Vision – A. Bobick Derivatives and edges An edge is a place of rapid change in the image intensity function. intensity function (along horizontal scanline) first derivative image edges correspond to extrema of derivative Source: L. Lazebnik

Templates/Edges CS 4495 Computer Vision – A. Bobick Differential Operators • Differential operators – here we mean some operation that when applied to the image returns some derivatives. • We will model these “operators” as masks/kernels which when applied to the image yields a new function that is the image gradient function . • We will then threshold the this gradient function to select the edge pixels. • Which brings us to the question: What’s a gradient?

Templates/Edges CS 4495 Computer Vision – A. Bobick Image gradient The gradient of an image: The gradient points in the direction of most rapid increase in intensity The gradient direction is given by: • how does this relate to the direction of the edge? The edge strength is given by the gradient magnitude

Templates/Edges CS 4495 Computer Vision – A. Bobick Discrete gradient For 2D function, f(x,y), the partial derivative is: ∂ + ε − f ( x , y ) f ( x , y ) f ( x , y ) = lim ∂ ε ε → x 0 For discrete data, we can approximate using finite differences: ∂ + − f ( x , y ) f ( x 1 , y ) f ( x , y ) ≈ ∂ x 1 ≈ + − f x ( 1, ) y f x y ( , ) “right derivative” But is it???

Templates/Edges CS 4495 Computer Vision – A. Bobick Finite differences Computer Vision - A Modern Approach Set: Linear Filters Slides by D.A. Forsyth

Templates/Edges CS 4495 Computer Vision – A. Bobick Partial derivatives of an image ∂ ∂ f ( x , y ) f ( x , y ) ∂ ∂ x y ? -1 1 or -1 1 1 -1 Which shows changes with respect to x? (showing correlation filters)

Templates/Edges CS 4495 Computer Vision – A. Bobick Differentiation and convolution • For 2D function, f(x,y), the partial derivative is: ∂ + ε − f ( x , y ) f ( x , y ) f ( x , y ) = lim ∂ ε ε → x 0 • For discrete data, we can approximate using finite differences: ∂ + − f ( x , y ) f ( x 1 , y ) f ( x , y ) ≈ ∂ x 1 • To implement above as convolution, what would be the associated filter?

Templates/Edges CS 4495 Computer Vision – A. Bobick The discrete gradient • We want an “operator” (mask/kernel) that we can apply to the image that implements: ∂ + ε − f ( x , y ) f ( x , y ) f ( x , y ) = lim ∂ ε ε → x 0 How would you implement this as a cross-correlation? (not flipped) Average of Not symmetric “left” and 0 0 0 0 0 around image “right” -1/2 0 +1/2 -1 +1 point; which is derivative . 0 0 0 0 0 “middle” pixel? See?

Templates/Edges CS 4495 Computer Vision – A. Bobick Example: Sobel operator -1 0 1 1 2 1 -2 0 2 0 0 0 -1 0 1 -1 -2 -1 On a pixel of the image I •Let g x be the response to mask S x (sometimes * 1/8) •Let g y be the response to mask S y What is the gradient? (Sobel) Gradient is ∇ I = [g x g y ] T 2 + g y 2 ) 1/2 is the gradient magnitude. g = (g x θ = atan2(g y , g x ) is the gradient direction.

Templates/Edges CS 4495 Computer Vision – A. Bobick Sobel Operator on Blocks Image original image gradient thresholded magnitude gradient magnitude

Templates/Edges CS 4495 Computer Vision – A. Bobick Some Well-Known Masks for Computing Gradients Sx Sy -1 0 1 1 2 1 -2 0 2 0 0 0 • Sobel: -1 0 1 -1 -2 -1 -1 0 1 1 1 1 • Prewitt: -1 0 1 0 0 0 -1 0 1 -1 -1 -1 • Roberts 0 1 1 0 -1 0 0 -1

Templates/Edges CS 4495 Computer Vision – A. Bobick Matlab does edges >> My = fspecial(‘sobel’); >> outim = imfilter(double(im), My); >> imagesc(outim); >> colormap gray;

Templates/Edges CS 4495 Computer Vision – A. Bobick But… • Consider a single row or column of the image • Plotting intensity as a function of x • Apply derivative operator…. Uh, where’s the edge?

Templates/Edges CS 4495 Computer Vision – A. Bobick Finite differences responding to noise Increasing noise -> (this is zero mean additive gaussian noise) D. Forsyth

Templates/Edges CS 4495 Computer Vision – A. Bobick Solution: smooth first f h ∗ h f ∂ ∗ x h ( f ) ∂ ∂ ∗ x h ( f ) Where is the edge? Look for peaks in ∂