Review- Image segmentation BBM 413 Fundamentals of • Goal: identify groups of pixels that go together Image Processing Erkut Erdem Dept. of Computer Engineering Hacettepe University Segmentation – Part 2 Slide credit: S. Seitz, K. Grauman Review- The goals of segmentation Review- What is segmentation? • Separate image into coherent “objects” • Clustering image elements that “belong together” – Partitioning • Divide into regions/sequences with coherent internal properties – Grouping • Identify sets of coherent tokens in image image human segmentation http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/ Slide credit: S. Lazebnik Slide credit: Fei-Fei Li
Review- K-means clustering Review - K-means: pros and cons • Basic idea: randomly initialize the k cluster centers, and Pros iterate between the two steps we just saw. • Simple, fast to compute 1. Randomly initialize the cluster centers, c 1 , ..., c K • Converges to local minimum of within-cluster squared error 2. Given cluster centers, determine points in each cluster • For each point p, find the closest c i . Put p into cluster i 3. Given points in each cluster, solve for c i Cons/issues • Set c i to be the mean of points in cluster i 4. If c i have changed, repeat Step 2 • Setting k? Properties • Sensitive to initial centers • Will always converge to some solution • Sensitive to outliers • Can be a “local minimum” • does not always find the global minimum of objective function: • Detects spherical clusters • Assuming means can be computed Slide credit: S. Seitz Slide credit: K Grauman Mean shift clustering and Segmentation methods segmentation • Segment foreground from background • An advanced and versatile technique for clustering-based • Histogram-based segmentation segmentation • Segmentation as clustering – K-means clustering – Mean-shift segmentation • Graph-theoretic segmentation – Min cut – Normalized cuts • Interactive segmentation http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature Space Analysis, PAMI 2002. Slide credit: S. Lazebnik
Mean shift algorithm Finding Modes in a Histogram • The mean shift algorithm seeks modes or local maxima of density in the feature space Feature space image (L*u*v* color values) • How Many Modes Are There? – Easy to see, hard to compute Slide credit: S. Seitz Slide credit: S. Lazebnik Mean shift algorithm Mean shift Mean Shift Algorithm Search 1. Choose a search window size. window 2. Choose the initial location of the search window. 3. Compute the mean location (centroid of the data) in the search window. Center of 4. Center the search window at the mean location computed in Step 3. mass 5. Repeat Steps 3 and 4 until convergence. The mean shift algorithm seeks the � mode � or point of highest density of a data distribution: Two issues: (1) Kernel to interpolate density based on sample positions. (2) Gradient ascent to mode. Mean Shift vector Slide credit: Y. Ukrainitz & B. Sarel Slide credit: B. Freeman and A. Torralba
Mean shift Mean shift Search Search window window Center of Center of mass mass Mean Shift Mean Shift vector vector Slide credit: Y. Ukrainitz & B. Sarel Slide credit: Y. Ukrainitz & B. Sarel Mean shift Mean shift Search Search window window Center of Center of mass mass Mean Shift Mean Shift vector vector Slide credit: Y. Ukrainitz & B. Sarel Slide credit: Y. Ukrainitz & B. Sarel
Mean shift Mean shift Search Search window window Center of Center of mass mass Mean Shift vector Slide credit: Y. Ukrainitz & B. Sarel Slide credit: Y. Ukrainitz & B. Sarel Mean shift clustering/segmentation Mean shift clustering • Find features (color, gradients, texture, etc) • Initialize windows at individual feature points • Cluster: all data points in the attraction basin of a mode • Perform mean shift for each window until convergence • Attraction basin: the region for which all trajectories lead to the • Merge windows that end up near the same “peak” or mode same mode Slide credit: Y. Ukrainitz & B. Sarel Slide credit: S. Lazebnik
Apply mean shift jointly in the image Window in image domain (left col.) and range (right col.) domains 1 Intensities of pixels within image domain window 2 0 1 Center of mass of pixels within 3 both image and range domain 0 1 windows Window in range domain 4 Center of mass of pixels within both image and range domain windows 5 6 0 1 7 Comaniciu and Meer, IEEE PAMI vol. 24, no. 5, 2002 0 1 Slide credit: B. Freeman and A. Torralba Slide credit: B. Freeman and A. Torralba Mean shift segmentation results More results http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html Slide credit: S. Lazebnik Slide credit: S. Lazebnik
More results Mean shift pros and cons • Pros – Does not assume spherical clusters – Just a single parameter (window size) – Finds variable number of modes – Robust to outliers • Cons – Output depends on window size – Computationally expensive – Does not scale well with dimension of feature space Slide credit: S. Lazebnik Slide credit: S. Lazebnik Graph-Theoretic Image Segmentation Segmentation methods • Segment foreground from background Build a weighted graph G=(V,E) from image • Histogram-based segmentation V: image pixels • Segmentation as clustering – K-means clustering E: connections between – Mean-shift segmentation pairs of nearby pixels • Graph-theoretic segmentation • Min cut W : probabilit y that i & j • Normalized cuts ij • Interactive Segmentation belong to the same region Segmentation = graph partition Slide credit: B. Freeman and A. Torralba
A Weighted Graph and its Graphs Representations Representation Affinity Matrix a b c d e 0 1 0 0 1 1 . 1 . 3 0 0 a ⎡ ⎤ ⎡ ⎤ a a ⎢ ⎥ ⎢ ⎥ b b b 1 0 0 0 0 . 1 1 . 4 0 . 2 ⎢ ⎥ ⎢ ⎥ c 0 0 0 0 1 . 3 . 4 1 . 6 . 7 ⎢ ⎥ ⎢ ⎥ c W = ⎢ ⎥ ⎢ ⎥ c d 0 0 0 0 1 0 0 . 6 1 1 e e ⎢ ⎥ ⎢ ⎥ 6 ⎢ 1 0 1 1 0 ⎥ ⎢ 0 . 2 . 7 1 1 ⎥ e ⎣ ⎦ ⎣ ⎦ d W : probabilit y that i & j d ij Adjacency Matrix belong to the same region Slide credit: B. Freeman and A. Torralba Slide credit: B. Freeman and A. Torralba * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003 Segmentation by graph partitioning Affinity between pixels Similarities among pixel descriptors W ij = exp(-|| z i – z j || 2 / σ 2 ) j w ij σ = Scale factor… it will hunt us later i A B C • Break graph into segments – Delete links that cross between segments – Easiest to break links that have low affinity • similar pixels should be in the same segments • dissimilar pixels should be in different segments Slide credit: B. Freeman and A. Torralba Slide credit: S. Seitz
Affinity between pixels Scale affects affinity Small σ : group only nearby points • Similarities among pixel descriptors W ij = exp(-|| z i – z j || 2 / σ 2 ) Large σ : group far-away points • σ = Scale factor… Interleaving edges it will hunt us later W ij = 1 - max Pb Line between i and j With Pb = probability of boundary Slide credit: B. Freeman and A. Torralba Slide credit: S. Lazebnik Example eigenvector British Machine Vision Conference, pp. 103-108, 1990 points W ij = exp(-|| z i – z j || 2 / σ 2 ) With an appropriate σ eigenvector W= The eigenvectors of W are: Affinity matrix Three points in feature space The first 2 eigenvectors group the points as desired… Slide credit: B. Freeman and A. Torralba Slide credit: B. Freeman and A. Torralba
Example eigenvector Graph cut B A points • Set of edges whose removal makes a graph disconnected eigenvector • Cost of a cut: sum of weights of cut edges • A graph cut gives us a segmentation – What is a “good” graph cut and how do we find one? Affinity matrix Slide credit: B. Freeman and A. Torralba Slide credit: S. Seitz Segmentation methods Minimum cut • Segment foreground from background • Histogram-based segmentation A cut of a graph G is the set of edges S such that removal of S from G disconnects G . • Segmentation as clustering – K-means clustering – Mean-shift segmentation Cut : sum of the weight of the cut edges: • Graph-theoretic segmentation ∑ • Min cut cut (A,B) = W( u , v ), • Normalized cuts u ∈ A, v ∈ B with A ∩ B = ∅ • Interactive segmentation Slide credit: B. Freeman and A. Torralba
Recommend
More recommend
