Image Features: Detection, Description, and Matching and their Applications
Image Representation: Global Versus Local Features
Features/ keypoints/ interset points are interesting locations in the image. Features serve to give a new compact representation of an image. Features can describe objects. Types of image features: Edges , Corners, blobs, small image patches
Global Features Local Features Image is represented by one Imagemayhavehundredsof multidimensional localfeatures. feature vector, describing the information in the whole image. Measure various aspects of the Measure local structures that image such as color, texture or are more distinctive and stable shape. than other structures. Much faster and compact Local features have superior while easy to compute and performance. generally require small amounts of memory. Not invariant to significant Robust against occlusion and transformations and sensitive to background clutter . clutter and occlusion.
Features Corner Region Edge
Features Detection: Detection: identify the interest points Feature detection = how to find some interesting points (features) in the image.(Ex: find a corner, find a region and so on...)
Features Detection: Detectors Affine- Single-scale Multi-scale invariant Harris- Hessian- Hessian Harris- Harris FAST SUSAN LOG DOG Laplace Laplace -Affine Affine
Features description: Description: Extract feature vector surrounding each interest point. Feature description = how to represent the interesting points we found to compare them with other interesting points (features) in the image.
Features description Descriptors Binary-based Gradient-based (Floating – point) (Binary string) ORB BRISK FREAK BRIEF SURF GLOH SIFT
Features Matching: Matching: match/compare the extracted descriptor from the query image to those in the database. Feature Matching= find the distance between two or more descriptors/ finding the similarity.
Features Matching: Given two such patches, how can we determine their similarity? We can measure the pixel to pixel similarity by measuring their Euclidean distance, but that measure is very sensitive to noise, rotation, translation and illumination changes. In most applications we would like to be robust to such changes.
Applications 1.Object Recognition 2. Camera Calibration 3. Image Retrieval 4. Image Registration
Binary Descriptors Binary descriptors are composed of three parts: • A sampling pattern : where to sample points in the region around the feature/keypoint. • Orientation compensation : some mechanism to measure the orientation of the keypoint and rotate it to compensate for rotation changes. • Sampling pairs : which pairs to compare when building the final descriptor.
Why binary descriptors? • Pixel comparisons are much faster than gradients. • Matching binary representations is done using hamming distance which is much faster to compute than any distance metric. • Whereas a single gradient based descriptor commonly require 64 or 128 floating point values to store, a single binary descriptors requires only 512 bits – 4 to 8 times less.
Challenges • Change of scale • Change of orientation (rotation) • Change of viewpoint (affine, projective transformations) • Change of illumination • Noise, Clutter, and occlusion • Repetitive patterns (windows, street marks, etc.) Thus, we need robust similarity measures, detector, descriptors, and matchers
Ideas • Studying the effect of different sampling patterns on descriptors’ performance. • Investigating the effect of different similarity metrics on the matching performance. • Adaptation of binary descriptors for computational resources. • Enhancing existing descriptors’ robustness against geometric and photometric transformations. • Proposing our own descriptor and using it in applications.
Thank you
A schematic representation of the SIFT descriptor for a 16 × 16 pixel patch and a 4 × 4 descriptor array
schematic representation of the GLOH algorithm using log-polar bins
Recommend
More recommend
