VIDEO SIGNALS Segmentation
WHAT IS SEGMENTATION WHAT IS SEGMENTATION Segmentation is a fundamental low level operation on Segmentation is a fundamental low-level operation on images. If an image is already partitioned into segments If an image is already partitioned into segments, where each segment is a “homogeneous” region, then a number of subsequent image processing tasks become easier. b i A homogeneous region refers to a group of connected pixels in the image that share a common feature This pixels in the image that share a common feature. This feature could be brightness, color, texture, motion, etc.
DIFFERENT KINDS OF SEGMENTATION DIFFERENT KINDS OF SEGMENTATION Brightness segmentation Color segmentation
DIFFERENT KINDS OF SEGMENTATION DIFFERENT KINDS OF SEGMENTATION Motion segmentation
DIFFERENT KINDS OF SEGMENTATION DIFFERENT KINDS OF SEGMENTATION Segmentation based on texture g
LUMINANCE BASED SEGMENTATION LUMINANCE-BASED SEGMENTATION
ERROR PROBABILITY FOR THRESHOLDING ERROR PROBABILITY FOR THRESHOLDING
OPTIMAL SUPERVISED THRESHOLDING OPTIMAL SUPERVISED THRESHOLDING
UNSUPERVISED THRESHOLDING UNSUPERVISED THRESHOLDING
UNSUPERVISED THRESHOLDING UNSUPERVISED THRESHOLDING
CHROMA KEYING CHROMA KEYING Color is more powerful for pixel-wise segmentation: 3-d vs. 1-d C l i f l f i l i t ti 3 d 1 d space Take picture in front of a blue screen (or green or orange) Take picture in front of a blue screen (or green, or orange)
SOFT CHROMA KEYING SOFT CHROMA KEYING
LANDSAT IMAGE PROCESSING LANDSAT IMAGE PROCESSING
MULTIDIMENSIONAL MAP DETECTOR MULTIDIMENSIONAL MAP DETECTOR Label categories in L b l t g i i training set by hand Subdivide n dimensional Subdivide n-dimensional space into small bins Count frequency of Count frequency of occurrence for each bin and class in training set g For test data: identify bin, detect the more , probable category
MAP DETECTOR IN RGB SPACE MAP DETECTOR IN RGB-SPACE
LINEAR DISCRIMINANT FUNCTION LINEAR DISCRIMINANT FUNCTION
SELF SUPERVISED ROAD DETECTION SELF-SUPERVISED ROAD DETECTION
REGIONS VS BOUNDARIES REGIONS VS. BOUNDARIES Boundary detection is the dual goal of image segmentation. B d d t ti i th d l l f i t ti If the boundaries between segments are specified then it is equivalent to identifying the individual segments q y g g themselves. BUT BUT In the process of image segmentation, one obtains In the process of image segmentation one obtains regionwise information regarding the individual segments. This information can then be subsequently used to classify the i di id individual segments. l t Detection of the boundaries between segments does not automatically yield regionwise information about the individual segments. So, further image analysis is necessary before any segment-based classification can be attempted.
WHY STATISTICAL METHODS? WHY STATISTICAL METHODS? They involve image features that are simple to interpret by using a model. They also involve features that are easy to compute from a given image compute from a given image They use merging methods that are firmly rooted in statistical/mathematical inference.
WHAT ARE BOUNDARIES? WHAT ARE BOUNDARIES?
THE MATHEMATICAL PROBLEM THE MATHEMATICAL PROBLEM If we define If we define Ω ={(m, , n): 1 ≤ m ≤ M and 1 ≤ n ≤ N } the domain where the image is defined. For any given point the segmentation g(m n) For any given point the segmentation g(m,n) at that point take a value from a set Γ . For example Γ ={ ξ : ξ = 0 or 1} for a binary l Γ { ξ ξ 0 1} f bi segmentation. Γ ={ ξ : ξ = 1,2,3,…k} for a multiclass segmentation.
BOUNDARY SEGMENTATION BOUNDARY SEGMENTATION Of course, g could also denote a boundary image: g ( m,n ) = 1 to denote the presence of a boundary. While g ( m n ) = 0 to denote the absence While g ( m,n ) = 0 to denote the absence.
GAUSSIAN STATISTICS GAUSSIAN STATISTICS M Measures the amount of variability in the pixels that th t f i bilit i th i l th t comprise f 1 along the (p,q) direction. •If T is very small then that implies that f 1 has little or no variability along the (0,1)th (horizontal) direction. •Computation of statistics is straightforward, as is merely a quadratic operation.
FOURIER STATISTICS FOURIER STATISTICS
COVARIANCE STATISTICS COVARIANCE STATISTICS
LABEL STATISTICS LABEL STATISTICS
FISHER COLOR DISTANCE FISHER COLOR DISTANCE
VEHICLE TRACKING USING MAP
FISHER COLOR SEGMENTATION FISHER COLOR SEGMENTATION A segmentation that yields all segments that do not contain g y g the color green.
TRACKING AN OBJECT OF INTEREST TRACKING AN OBJECT OF INTEREST Tracking of a human hearth from frame to frame using Tracking of a human hearth from frame to frame using elastic deformation model
ILLUSORY BOUNDARY TRACKING ILLUSORY BOUNDARY TRACKING Segmentation using texture phase in EdgeFlow Algorithm
SEGMENTATION USING TEXTURE ENERGY SEGMENTATION USING TEXTURE ENERGY Segmentation using color and texture energy Segmentation using color and texture energy
MOTION FIELD SEGMENTATION MOTION FIELD SEGMENTATION 2D dense motion field from the second frame to the first and resulting segmentation
AN EXAMPLE OF CAR TRACKING AN EXAMPLE OF CAR TRACKING • The vehicle is described by three parameters (V b ,V l ,V w ) corresponding to the bottom edges, left edges and width of the p g g , g square. • Vehicles seldom tend to be too big or small, and so depending on the distance of the vehicle from the camera, in is possible to expect the width of the vehicle to be within a certain range : W min and W max
PARAMETER ESTIMATION PARAMETER ESTIMATION Let (v b ,v l ,v w ) denote a specific hypothesis of the b l w unknown vehicle parameters ( V b ,V l ,V w ): the merit of this hypothesis is decided by the likelyhood. The merit of this hypothesis is decided by the color Th it f thi h th i i d id d b th l difference between pixels that are inside the square (i.e. pixels that are hypothesized to be the square (i.e. pixels that are hypothesized to be the vehicle) and pixels that are outside (pixels that are in the immediate background). The color difference evaluator is the Fisher distance:
FISHER DISTANCE FISHER DISTANCE 1 ad K 1 are the mean and covariance of the pixels that 1 ad K 1 are the mean and covariance of the pixels that are inside the hypothesized square while 2 ad K 2 are the mean and covariance of the pixels that are immediately surrounding the hypothesized square surrounding the hypothesized square. Hypotheses corresponding to a large color difference between pixels inside and immediately surrounding the between pixels inside and immediately surrounding the square have more merit (and hence a higher probability of occurrence) than those with smaller color difference. An optimal estimate of these parameters is the one that maximizes the product of the prior and likelihood probabilities: the so-called maximum a pos probabilities: the so called maximum a pos posteriori posteriori eriori (MAP) eriori (MAP) estimate.
AERIAL IMAGE SEGMENTATION AERIAL IMAGE SEGMENTATION Segmentation of an aerial image, a rural crop field area, using the texture-based maximum likelihood procedure
CHOOSING HOMOGENEOUS SEGMENTS FOR AERIAL IMAGES IMAGES The human operator examines the aerial image and chooses a collection of polygons corresponding to various homogeneous segments of the image. g g By use of the pixels with these polygons as a training sample, a statistical segmentation of the aerial image is effected; The segmentation procedure used for this map updating The segmentation procedure used for this map updating application is based on the Gaussian statistics. For each homogeneous polygonal region selected in the aerial image by the human operator the Gaussian statistics for that image by the human operator, the Gaussian statistics for that polygon are automatically computed. With these statistics, a model of probable variation in the ‘pixels' intensities within the polygon is subsequently created intensities within the polygon is subsequently created.
SEGMENTATION FOR OBJECT ORIENTED ENCODING SEGMENTATION FOR OBJECT-ORIENTED ENCODING Given an image is first divided into 8 × 8 blocks of Gi i i fi di id d i 8 8 bl k f pixels. FFT is applied to each block (Fourier statistics). If the pixels f 1 within a single block have little or no p g 1 variation then F f1 (0,0) will have a very large value. IF the blocks contain a vertical edge IF the blocks contain a vertical edge then will have a large value…
FOURIER DECOMPOSITION FOURIER DECOMPOSITION If g denotes the collection of unknown block labels, g , then an estimate of g from f would correspond to an object based segmentation of f .
RESULTS RESULTS
Recommend
More recommend
