Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW - PowerPoint PPT Presentation

Feature Extraction Manos Baltsavias, Haris Papasaika

OVERVIEW APPLICATIONS - Object detection and localisation (2D measurement) - Matching (3D information) - Image analysis and scene interpretation , e.g. Edgels -> edge segments -> closed edges (2D structures) -> 3D structures …. and much more TASKS - General -> e.g. extract image edges - Specific -> extract specific object (e.g. circles, crosses) FEATURE TYPES - Point corners, junctions, high curvature gradient, gravity center, line ends, edgels - Linear curved or straight edge or line segments, closed edges, polygonal approximations - Areal segmented regions, blobs, specific shapes (ellipses, etc.) - Structures combination of features and their relations

Examples From left to right: Different feature types, e.g. points, edges, regions

ATTRIBUTES OF FEATURES Position : 3nd points or center of line, gravity center, polygon nodes Geometric : orientation, length, curvature, area, perimeter, line width, quad min/max diameter, elongation, axes of symmetry, number/position of singular points, compactness Radiometric : edge sign and strength, grey level statistics on either side of edge, multiple spectral channels, autocorrelation Textural : co-occurence matrix, homogeneity, texture gradients, textural filters, grey level moments Topological : above, left, adjacency, common end points, intersect, parallel, vertical, touch, overlap, contains, contained Comparisons : ordering of attributes, seldomness of features or attributes Spectral : Motion : features on static or moving objects Time : change of feature attributes over time Symbolic : set of constituent features/attributes, their relationships descriptions Feature combination -> new features - > symbolic description CHOICE OF ATTRIBUTES Discrimination power : statistical properties of each attribute in all features, information-theoretic measures (e.g. entropy-based) Independence : correlations between attributes in all features

PROPERTIES OF GOOD FEATURES / ATTRIBUTES Detection - Density : appearance in all images, sufficient density Seldomness-Distinctness : unique or seldom in image, distinct in neighbourhood Invariance-Stability : robust with respect to geometric and radiometric transformations, insensitive to noise Localisation : permit precise localisation Interpretability : permit interpretation (for subsequent image analysis), e.g. junctions, edges, regions, YES; edgels, NO Speed : feature extraction speed

IMAGE FEATURES - Regions (have been treated in previous lecture) - Edges (will be treated here) - Points (will be treated here) EDGES - Discontinuities in intensity, color, texture, ... Here, focus on intensity - 2D edges, 3D (in surfaces), in multispectral images Here, focus on 2D edges - Similar terms: edges, lines, contours (special case: straight edges), ribbons = two parallel edges

The role of edges in object detection and reconstruction

Edge models (from Gonzalez and Woods) - both are too idealistic

Edges (the real life) 3D representation of grey values as surface. Below 3 zoomed parts showing the problems with the edge definition and the noise.

EDGE ATTRIBUTES AND RELATIONS Usual attributes - Position (center, end points, all points, polygonal approximation) - Strength (contrast, gradient magnitude) - Orientation More complicated attributes possible, e.g. - Properties of flanking regions - Relations between neighbouring edges, etc.

RICH FEATURE ATTRIBUTES AND RELATIONS (HENRICSSON, 1996) - Transformation of RGB to Lab color space - Detection of photometric and chromatic attributes using the color flanking regions The construction of a flanking region. The source edge is black, the interfering edges are gray and the region is encoded in gray-values indicating the distance from the source edge. (A) schematically shows the construction of a flanking region, (B-D) the effect of using different region widths and offsets for the same edge, (E) a “curved” region, (F) a uniform region with dark and light disturbances (outliers), (G) same as (F) after excluding the outliers with robust estimation.

EDGE ATTRIBUTES

EDGE RELATIONS

EDGE RELATIONS Grouping of edges based on similarity: - position - orientation - radiometry - color properties

Extraction of edge/linear features Other edge detection approaches possible, e.g. - Modelling of grey level surface (Haralick), or profiles vertical to edge - Template matching - Hough transform

Extraction of edge/linear features

Edge extraction (example with Sobel operator). From left: original image, magnitude of gradients, gradient orientation (0-180 o ) coded as grey values (0-255).

Ramp edge examples (from Gonzalez and Woods). From top to bottom, addition of Gaussian noise with mean 0 and sigma=0, 0.1,1 and 10. Left column shows grey values, second the first derivatives, third the second derivatives. The higher the derivatives, the higher the effect of noise.

Preprocessing • Can change results drastically • Typical operations: - reduce noise (but without edge smoothing) - sharpen edges - enhance contrast • Pay attention to saturation (can change edge position)

The edges are detected from the zero crossing of the LoG.

• Difference of Gaussians (DoG) - DoG looks like a Mexican hat - When s 1 / s 2 = 1:1.75, then DoG is approximately equal to the Laplacian of the Gaussian with s, i.e. first a low-pass filtering (Gaussian), then high-pass filtering (Laplace) - Result is an enhancement of the edges (zero crossings of the second derivatives) and edge detection

Examples of edge detection (all refer to gradient magnitude) Results with Canny. From top left to bottom right increasing value of σ .

Examples of edge detection (all refer to gradient magnitude) Results with Canny after deletion of the short edges.

Examples of edge detection (all refer to gradient magnitude) Top: left discrete differentiator, right Roberts. Bottom: left 3x3 Prewitt, right 3x3 Sobel.

Examples of edge detection (all refer to gradient magnitude) Top: left DoG, 7x7 mask size, excitatory σ =1, inhibitory s =1.6 ; right original – 5x5 local average Bottom: left 3x3 compass with 8 masks ; right Laplace

Examples of edge detection (all refer to gradient magnitude) 3x3 Kirsch with 8 masks.

Computation of threshold for binarising gradients • Interactively, e.g. by measuring gradients of low-contrast edges that should be extracted, or visual inspection of results • Empirically, e.g. if grey level difference between neighbours > 8, then edge • Automatically from the image, e.g. compute the histogram of the gradients. Choose the threshold such that N% (e.g. 10%) of the gradients is larger than the threshold, i.e. keep the N% higher gradients • Compute the noise variance (e.g. using homogeneous regions) and choose the threshold such that no edges due to noise are detected • Above approaches can be used globally or locally (in image parts)

Example: edge tracking in a 4-neighbourhood Efficient storage of edgels as chain code.

- Coding of the 8 neighbour directions - Necessary for storage (instead of x,y coordinates of all edgels): Start point, number of edgels, direction code for every edgel) - Example Original image

- Chain code with 4- and 8-neighbourhood. Start point is marked dark. 4-neighbourhood. Chain code is: 8-neighbourhood. Chain code is: Start point: x=3, y=5 Start point: x=3, y=5 Number of points: 44 Number of points: 28 Directions: 2,2,0,2,0,0,0,6,6,0,6, Directions: 2,1,1,0,7,6,7,1,2,1,0, ......4,2,2,4,2,2 .........4,3,4,3,3,3,2,2

Closing of gaps • Mathematical morphology (e.g. closing) - closes gaps < M ; M dimensions of Structural Element (SE ; like filter mask), SE symmetric - distance between different edges must be > M, otherwise wrong edge combination • Heuristic approaches (also for edge tracking), e.g. the gap should be small, the direction of the detected edge should not change much, i.e. search space is a circle segment Starting from an edge endpoint send 11 search rays of length 4 within an angle tolerance with respect to the orientation of the last edgel. Two other edge endpoints were detected and were connected to the first edge. The original edgels are dark, the new ones lighter.

• Ring operator (for edge tracking and gap closing) - define a circle arc (or more in different distances from the edge endpoint) - examine the grey values or gradients along the arc - multiple arcs useful to (a) bridge gaps, (b) recognise junctions - Simpler approach: instead of arcs, use lines vertical to the estimated edge direction possible gap other edge search space