Segmentation and Contour Detection Image segmentation is the process - PowerPoint PPT Presentation

Segmentation and Contour Detection Image segmentation is the process of   assigning a label to every pixel in an image such that pixels with the same label share certain visual characteristics. Contours are the lines separating Contours are the lines separating   different image regions. => Segmentation and Contour detection   are closely related.

What is segmentation? Dividing image into its constituent regions or objects

Problem Statement

Berkeley Segmentation Dataset Berkeley Segmentation dataset  (BSDS500). 500 Manually segmented images.  Manual Segmentations

Hard Edges vs. Soft Edges Hard Edges vs. Soft Edges

λ= λ= λ=3 0 8 2

Consistency A Perceptual organization forms a tree: Image BG BG L-bird L-bird R-bird R-bird B C grass bush far Two segmentations are beak body beak body consistent when they can be eye head eye head explained by the same • A,C are refinements of B segmentation tree (i.e. they • A,C are mutual refinements could be derived from a single • A,B,C represent the same percept perceptual organization). • Attention accounts for differences

Combining Segmentations Combining Segmentations

How To Combine How To Combine Segmentations Segmentations? ? Option A: Spectral Clustering Choose N nearest neighbors for each pixel. 1. Calculate sparse affinity matrix. 2. Use spectral clustering to get K segments. 3. Find contours. Find contours. 4. 4. 2 3 4 1 . . .

How To Combine How To Combine Segmentations Segmentations? ? Option B: Directly combine contours of segmentations Find contour of a single segmentation 1. Combine all contours. 2. 1 1 2 . . .

Segmentation Discontinuity Gray value detection Similarity    Thresholding  Thresholding Detection of: Detection of: Isolated points    Region growing    Lines   Region splitting and merging Edges

Dealing with textures Similarity in Color Similarity in Texture 16

Why supervised? 17

Dimensionality Reduction Color & Texture Descriptor  Reduced noise  Curse of dimensionality  Less computational load 18

Converting textures to “color” channels 19

Local Color Histogram Measures color mixtures in small windows (11x11)  Clustering the histograms of the image into 10 groups.  Assigning each pixel to the centroid of its cluster.  20

Gabor Filters – Cont. 21

Gabor filters Zsa Zsa Gabor Dennis Gabor 6 February 1917, Budapest, Hungary 5 June 1900, Budapest, Hungary

Gabor Filters

Gabor Filter Bank Example Scale σ σ σ 3 1 2 θ 1 Orientation θ 2 θ 3 θ 4 Example Gabor filter bank with 3 scale values and 4  orientation values:

Gabor filter responses

Standard way to handle texture  Apply Gabor filter bank and generate a  texture vector for each pixel.  Perform Kmeans/GMM/pca on the texture  vector space, each group centroid is called vector space, each group centroid is called texton  For each pixel(texton) neighborhood,  calculate texton histogram    The resulting histogram values are the  texton vector space.

Chi square distance between texton histograms Chi-square i   0.1 0.1 j j  k 0.8  2 K [ h ( m ) h ( m )] 1  χ  i j 2 ( h , h )  i j 2 h ( m ) h ( m )  m 1 i j

Scale-Space Basics