Content Based Image Retrieval Techniques Ambrose Tuscano - - PowerPoint PPT Presentation

Content Based Image Retrieval Techniques

Ambrose Tuscano (atuscan1@umbc.edu) University of Maryland Baltimore County, CMSC 676 Information Retrieval

Introduction

Image retrieval systems aim to find similar images to a query image among an image dataset.

Represented as

• Pixels (Also called Rasters)
• Vectors

• By annotation (manual)
• Text retrieval
• Semantic level (good for picture with people, architectures)
• By the content (automatic)
• Color, texture, shape
• Vague description of picture (good for pictures of scenery and with

pattern and texture)

Features in an Image

• Color : Low level, Can't specify context.
• Texture : Produce a mathematical characterisation of a repeating

pattern in the image.

• Shape: Region based and Contour(outline) based.
• Local Image Features : small parts of a big image.

○ extracted from the images at salient points and dimensionality reduced using Principal Component Analysis (PCA) transformation ○ SIFT using Harris interest points

Structure

IMAGE RETRIEVAL METHODS

Text based Image Retrieval

First annotated the images by text and then used text-based database management systems to perform image retrieval.

Text based Image Retrieval

Three Ways to go

• Manually Assign Keywords to each image
• Use text associated with the images (captions, web pages)
• Analyse the image content to automatically assign keywords(Computer Vision?)

Content Based Image Recognition

• A technique which uses

visual contents to search images from large scale image databases according to users' interests.

• CBIR research

is mainly contributed by the computer vision community

Content based Image Recognition

To use local features for image retrieval, three different methods are available:

• Direct transfer: nearest neighbors for each of the local features of the query

searched and the database images containing most of these neighbors returned.

• Local feature image distortion model (LFIDM): Compares the distances

between local features from the query image to the local features of each image

• f the database . The images with the lowest total distances are returned.
• Histograms of local features: A reasonably large amount of local features from

the database is clustered and then each database image represented by a histogram of indices of these clusters.

• Color Histogram
• Color Correlogram
• Color AutoCorrelogram
• Color Coherence vector
• Dominant Color Descriptors

• A shape is the form of an object or its external boundary, outline,or external

surface, as opposed to other properties like color, texture or composition.

• Fourier Descriptors
• Canny Algorithm
• SIFT Descriptors
• Moment Invariants
• Eccentric and Axis Oriented

• ○
• ○

○ ○

• Smoothing: Blur image to remove

Noise

• FInd Gradients : Edges are

marked where gradients of image have large magnitudes.

• Non-Max Suppression: Only local

Maxima is marked for edges.

• Double Thresholding: Potential

Edges are determined

• Hysteresis : Finally Edges

which are not connected/near to many other potential edges are removed.

Texture Extraction- Motif Co-Occurrence Matric

• MCM is used to represent transveral
• f adjacent pixel color difference in

an image.

• Each Pixel corresponds to four

adjacent pixel colors

• Each image can be presented by four

images of motifs of scan pattern, which can be further constructed into four two dimensional matrices of the image size.

• The attribute of the image will be

computed with motifs of scan pattern and a color motif cooccurence matrix(CMCM) will be obtained

• Euclidean DIstance
• Mahalanobis Distance
• MInkowski Distance
• Histogram Intersection Distance
• Quadratic Form Distance

Techniques used by CBIR

• K-Means

K-means clustering algorithm is proposed as it improves the scalability.

• Wavelet Transform

Feature vectors of images are be constructed from wavelet transformations, which can also be utilized to distinguish images through measuring distances between feature vectors.

• Support Vector Machine:

SVM classifier can be trained using training data of images marked by users .

• Neural Network:

A CNN doesn’t need complex work like feature extraction to work. Having a proper labelled data, we can train the system to learn the data features using complex layer structure.

CBIR + TBIR

• CBIR can be costly in the fact that it needs a lot of complex

computations.

• TBIR can be comparatively fast but has low precision.
• A hybrid model is currently being implemented.
• a text-based image meta-search engine retrieves images from the Web

using the text information from the Query.

• Techniques like matching term frequency-inverse document

frequency (tf-idf) weightings and cosine similarity are used.

• use the CBIR approach to re-filter the search results.

• X.Y. Wang,Y.J. Hong and H.Y.Yang,”An effective image retrieval scheme using color,

texture and shape features”

• Nidhi Singh ,Kanchan Singh and Ashok Sinha “A Novel Approach for Content Based Image

Retrieval”

• Yogita Mistry,and D. T. Ingole “Survey on Content Based Image Retrieval Systems”
• John Canny, “A Computational Approach to Edge Detection”
• Mussarat Yasmin, Muhammad Sharif and Sajjad Mohsin, “Use of Low Level Features for

Content Based Image Retrieval: Survey”

• N. Jhanwar, Subhasis Chaudhuri, et.al. Content based image retrieval using motif

cooccurrence matrix