GNR607 Principles of Satellite Image Processing Instructor: Prof. - - PowerPoint PPT Presentation

GNR607 Principles of Satellite Image Processing

Instructor: Prof. B. Krishna Mohan CSRE, IIT Bombay bkmohan@csre.iitb.ac.in

Slot 2 Lecture 32-34 Principal Component Transform and Band Arithmetic October 14, 2014 10.35 AM – 11.30 AM October 16, 2014 11.35 AM – 12.30 PM, 3.30 – 5.00 PM

Example

IIT Bombay Slide 14 GNR607 Lecture 32-34 B. Krishna Mohan Original Image After increased saturation

IIT Bombay Slide 14a GNR607 Lecture 32-34 B. Krishna Mohan

Volcanic lava flows Source: web.pdx.edu/~jduh/courses/Archive/.../Welch_HSI-RGB-presentation.pdf

Multiband Operations

Multiband Image Operations

• Operations performed by combining gray levels

recorded in different bands for the same pixel

• Applications

– Data reduction through decorrelation – Highlighting specific features with significant difference in response in different bands – The transformed data may be viewed like enhanced versions compared to originals

IIT Bombay Slide 15 GNR607 Lecture 32-34 B. Krishna Mohan

Principal Component Transform

• Highlights the redundancy in the data sets

due to similar response in some of the wavelengths

• Original bands variables represented along

different coordinate axes, redundancy implies variables are correlated, not independent

• Gray level in a band at a pixel can be

predicted from the knowledge of the pixel gray level in other bands

IIT Bombay Slide 16 GNR607 Lecture 32-34 B. Krishna Mohan

Example of Redundancy in Data

• Example: Highly

correlated data

• Values along band b1

leads to knowledge along band b2 of the data element

• Linear variation (nearly)

between b1 and b2

• Often true in case of

visible bands

b1 b2

IIT Bombay Slide 17 GNR607 Lecture 32-34 B. Krishna Mohan

Example of Redundancy in Data

• Points projected
• nto the line a

small error in the position of the point.

• Points represented

by only one coordinate b1’  half data reduced

• For highly correlated

data, this error will be minimal

b1 b2 b1’ b2’

IIT Bombay Slide 18 GNR607 Lecture 32-34 B. Krishna Mohan

Decorrelating Multispectral Remotely Sensed Data

• How do we identify the optimum axes along

which the remotely sensed data should be projected so that the transformed data would be uncorrelated?

• What should be the way to rank the new axes

so that we can discard the least important dimensions of the transformed data?

• Invertibility of the transformation?

IIT Bombay Slide 19 GNR607 Lecture 32-34 B. Krishna Mohan

Useful band statistics

IIT Bombay Slide 20 GNR607 Lecture 32-34 B. Krishna Mohan

1 1

.

M N k ij i j

g M N

= =

∑ ∑

2 1 1

( ) .

M N k ij k i j

g M N

µ

= =

  −          

∑∑

1 1

( )( ) .

M N k l ij k ij l i j

g g M N µ µ

= =

− −

∑∑

Mean  Variance   Covariance

Covariance Matrix

• C = {Ckl | k = 1, …, K, l = 1, …, K}
• K is the number of bands in which the

multispectral dataset was generated

• C is a symmetric matrix
• Ckl = Clk
• Diagonal elements of C are the intra-band

variances

• Off-diagonal elements are the inter-band

covariances

IIT Bombay Slide 21 GNR607 Lecture 32-34 B. Krishna Mohan

Relation between correlation and covariance

• Correlation Rkl =
• It can be shown that Rkl = Ckl + µkµl
• For data with zero-mean, correlation and co-variance will

be equal IIT Bombay Slide 22 GNR607 Lecture 32-34 B. Krishna Mohan

1 1

.

M N k l ij ij i j

g g M N

= =

∑ ∑

Principal Component Transformation

Problem to solve:

• Find a transformation to be applied to the input

multispectral image such that the covariance matrix of the result is reduced to a diagonal matrix

• Further, we should find an axis v such that the

variance of the projected coordinates (zk = vk

t x)

is maximum.

IIT Bombay Slide 23 GNR607 Lecture 32-34 B. Krishna Mohan

Solution

Given the transformed vector zk = vk

t x

The variance σz

2 =

This simplifies to σz

2 = vtCv

C, the covariance matrix is a positive, semi-definite, real symmetric matrix.

IIT Bombay Slide 24 GNR607 Lecture 32-34 B. Krishna Mohan

1 1

( )( ) .

M N t t ij k ij l i j

v x x v M N µ µ

= =

− −

∑∑

Finding vector v

• To maximize the projected variance σz

2, find a v

such that vtCv is maximum, subject to the constraint vtv = 1. Combining the maximization function with the constraint, we can write

• vtCv – λ(vtv – 1) = maximum
• Differentiating w.r.t. v,

IIT Bombay Slide 25 GNR607 Lecture 32-34 B. Krishna Mohan

( 1)

t t

C v λ ∂   − − =   ∂ v v v v

Finding v

The derivative results in Cv = λv (Verify!) Therefore, v is an eigenvector of C vtCv = vt(λv) = λvtv = λ This implies that v is the eigenvector of C with the largest eigenvalue Therefore all the eigenvectors with decreasing eigenvalues lead to axes with decreasing variance along them.

IIT Bombay Slide 26 GNR607 Lecture 32-34 B. Krishna Mohan

Alternative Explanation

Let the transformed pixel vector y = Dtx Covariance matrix of y = Sy = DtSxD (Note that Sy = E{(y – my)(y-my)t} = E{(Dtx – Dtmx)(Dtx – Dtmx)t} This simplifies to Sy = DtE(x – mx)(x – mx)tD D is a set of vectors independent of x)

IIT Bombay Slide 27 GNR607 Lecture 32-34 B. Krishna Mohan

Alternative Explanation

Covariance matrix of y = Sy = DtSxD It is desired that Sy be diagonal, i.e., the data in the transformed domain is uncorrelated Let Sy =

IIT Bombay Slide 28 GNR607 Lecture 32-34 B. Krishna Mohan

1 2

0 ... 0 ... 0 ... 0 0 ...

n

λ λ λ            

Alternative Explanation

Let Sy =

IIT Bombay Slide 29 GNR607 Lecture 32-34 B. Krishna Mohan

1 2

0 ... 0 ... 0 ... 0 0 ...

n

λ λ λ             Then Sy = DtSxD is a similarity transformation with D containing eigenvectors of Sx We can order λi in such a way that they are in descending

• rder. Given that y = Dtx, y1 corresponds to direction given

by e1, that is the first row of Dt, … Each transformed pixel vector y is obtained from scalar products of eigenvectors of Sx and x

Sample Eigenvectors and Eigenvalues

IIT Bombay Slide 30 GNR607 Lecture 32-34 B. Krishna Mohan 34.89 55.62 52.87 22.71 55.62 105.95 99.58 43.33 52.87 99.58 104.02 45.80 22.71 43.33 45.80 21.35 Covariance Matrix

0.34 −0.61 0.71 −0.06

0.64 −0.40 −0.65 −0.06 0.63 0.57 0.22 0.48 0.28 0.38 0.11 −0.88 Eigenvalues 253.44 7.91 3.96 0.89 Eigenvectors

Sample Eigenvectors and Eigenvalues

IIT Bombay Slide 31 GNR607 Lecture 32-34 B. Krishna Mohan

Transformation

IIT Bombay Slide 32 GNR607 Lecture 32-34 B. Krishna Mohan New component value = dot product of eigenvector and pixel vector (i,j)  pixel position n eigenvectors for n principal components 1st principal component  dot product of pixel vector with eigenvector corresponding to largest eigenvalue

Principal Components

For n input bands, n principal components are computed The utility of the principal components gradually decreases from 1st towards the last e.g., For Landsat TM, last three PCs are generally of very little value

IIT Bombay Slide 33 GNR607 Lecture 32-34 B. Krishna Mohan

Visualization of PCT

IIT Bombay Slide 34 GNR607 Lecture 32-34 B. Krishna Mohan

From J.R. Jensen’s lecture notes at Univ. South Carolina; used with permission

Comments on PCT

IIT Bombay Slide 35 GNR607 Lecture 32-34 B. Krishna Mohan

• For IRS / IKONOS images, out of four bands, 2-3

principal components capture most of the useful

• information. The last 1-2 bands are redundant.
• Advantages

– Smaller data volume to handle – Principal components appear to be enhanced versions of the originals, having contributions from all the four input bands

• Application scientists use composites of PC 1-2-3 for

interpretation of various features such as geology

Band 1 (Blue)

IIT Bombay Slide 36 GNR607 Lecture 32-34 B. Krishna Mohan

Band 2 (Green)

IIT Bombay Slide 37 GNR607 Lecture 32-34 B. Krishna Mohan

Band 3 (Red)

IIT Bombay Slide 38 GNR607 Lecture 32-34 B. Krishna Mohan

Band 4 (NIR)

IIT Bombay Slide 39 GNR607 Lecture 32-34 B. Krishna Mohan

Band 5 (SWIR)

IIT Bombay Slide 40 GNR607 Lecture 32-34 B. Krishna Mohan

Band 7 (SWIR)

IIT Bombay Slide 41 GNR607 Lecture 32-34 B. Krishna Mohan

PC1

IIT Bombay Slide 42 GNR607 Lecture 32-34 B. Krishna Mohan

PC2

IIT Bombay Slide 43 GNR607 Lecture 32-34 B. Krishna Mohan

PC3

IIT Bombay Slide 44 GNR607 Lecture 32-34 B. Krishna Mohan

PC6

IIT Bombay Slide 45 GNR607 Lecture 32-34 B. Krishna Mohan

Input Image FCC

IIT Bombay Slide 46 GNR607 Lecture 32-34 B. Krishna Mohan

Decorrelation Stretch

IIT Bombay Slide 47 GNR607 Lecture 32-34 B. Krishna Mohan

Decorrelation Stretch

IIT Bombay Slide 47a GNR607 Lecture 32-34 B. Krishna Mohan

• Variance of lower order principal

components is low

• Apply enhancement to these lower order

PCs

• Apply Inverse PCT (discussed next)
• Form color composites (FCC, True color

composites)

• See improvement in visual quality

IIT Bombay Slide 47b GNR607 Lecture 32-34 B. Krishna Mohan ASTER Satellite Image Enhancement Source:

http://www.gisdevelopment.net/technology/rs/techrs0023a.htm

IIT Bombay Slide 47c GNR607 Lecture 32-34 B. Krishna Mohan Source: http://www.dstretch.com/AlgorithmDescription.html Burham Canyon (KER-273) Enhancement of Rock Art Paintings

Inverse PCT

IIT Bombay Slide 48 GNR607 Lecture 32-34 B. Krishna Mohan

• Inverse PCT is used to generate the

bands in the original domain

• If ALL PCTs are retained, inverse will give

back the original bands

• If any PCTs are dropped, inverse will give

new bands in the original domain that may be close to the original bands depending

• n how many PCTs are discarded

Inverse PCT

IIT Bombay Slide 49 GNR607 Lecture 32-34 B. Krishna Mohan

From the principle of PCT, we have y = Dtx Dt contains eigenvectors of Sx, covariance matrix from the original image. D has eigenvectors as columns, thus Dt has the eigenvectors as rows Since Dt is an orthonormal matrix, Dt .D = I (each row is orthogonal to other rows) (Dt)t = (Dt)-1 From each pixel vector in PC domain, x = (Dt)t y

Inverse PCT

IIT Bombay Slide 50 GNR607 Lecture 32-34 B. Krishna Mohan

For k band image, matrix D is square, of size k x k If m principal components are dropped, we are left with a matrix (D1) of size k x (k-m) The vector y is reduced to y1 of size k-m x 1 Therefore the modified vector x1 is given by x1 = D1y1 The difference between x and x1 is a measure of the loss of information due to removal of some

• f the PCs

Comments on PCT

IIT Bombay Slide 51 GNR607 Lecture 32-34 B. Krishna Mohan

• One of the other important applications of

PCT is data fusion

• Images from two sensors can be fused to

produce a new image that has the strong points of both the input images

• PCT based fusion is a well known

approach

Data Fusion

Data Fusion

• Combine datasets to prepare a superior

dataset

• Stack up all the datasets to create a large

higher dimensional dataset – e.g., multitemporal data from same sensor

• Fuse the datasets to create a higher

resolution dataset

• Fuse the datasets to create a new dataset

that has attributes of individual ones

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 52

Data Fusion

• Most commonly employed by endusers of

remotely sensed data

• Supported by most software packages

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 53

Introduction

• Merging multi-sensor data can help exploit

strengths of various data sets

– Radiometric resolution advantage – Spatial resolution advantage – Spectral resolution advantage – Temporal resolution advantage

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 54

Spatial Resolution Enhancement

• This is the most common application of

data fusion

– Low resolution images have fewer pixels per unit area due to larger pixel size – Improve spatial resolution – High resolution images provide more pixels per unit area by smaller sampling interval (pixel size)

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 55

Zooming is NOT resolution enhancement

• How is spatial resolution enhanced?
• Low resolution  absence of high spatial

frequency content

• High frequency information is to be

transferred from another data source (of higher resolution)

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 56

Resolution Sharpening

• Most often, data from the lower spatial

resolution multispectral sensors and the higher spatial resolution panchromatic sensors are merged

• Results in multispectral data at higher

spatial resolution

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 57

Multi-sensor Data Merging

Most common operation

• PAN images to sharpen multispectral data

e.g., IRS pan + IRS ms

• Sharpening low resolution multispectral

images with high resolution multispectral images For instance, SPOT ms + TM ms (20 metres) (30 metres)

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 58

Input Image Preparation

• Contrast Adjustment

– Zoom low resolution image to the same physical size of the high resolution image – Match histogram of the MS image with that of PAN image using histogram based techniques

• Image Registration

– Register the zoomed low resolution image to the high resolution image. This should be accurate to a fraction of a pixel

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 59

Image Sharpening

• MShr = f(MSlr, PANhr) , where
• MS = multispectral Image
• PAN = Panchromatic Image
• lr = low resolution
• hr = high resolution

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 60

Sharpening Techniques

• Principal Component Analysis method
• Intensity-Hue-Saturation method
• Ratio-based (Brovey Transform)
• Arithmetic algorithm
• Multiplicative
• Wavelet Transform method

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 61

PCA Merge

• The 1st PC is most influential
• It should be used while merging so that the

effect is felt on all bands

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 62

PCA Merge

• The 1st principal component is replaced by

the high resolution image

• Inverse PCT is applied

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 63

Results

• This technique is useful to transform all

bands at a time

• Often works well in producing good fusion

results

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 64

PCA Merge

• Very effective when the correlation

between PAN image and the multispectral image is good

• Does not work very well when fusion is

done with images from different types of sensors such as SAR and optical

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 65

Input High Resolution Image

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 66

Input Multispectral Image

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 67

PCT Resolution Merge

IIT Bombay Slide 68 GNR607 Lecture 32-34 B. Krishna Mohan

RGB-HSI Transform Method

• In color images, the spectral information is

contained in the hue and the saturation.

• Hue denotes the basic dominant wavelength of

the radiation

• Saturation denotes the purity of the color or is a

function of the amount of dilution of the color with white light

• Intensity is an indicator of the strength of the

color or the magnitude of the energy that reaches our eye

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 69

RGB-HSI Transform Method

• The philosophy in HSI based fusion is to replace

the intensity with the new data set first and then compute the inverse transform of the HSI data set to the RGB coordinate system

• The spatial resolution of the added component

and the spectral information in the hue and saturation together provide an enhanced data set compared to the original low resolution multispectral and high resolution panchromatic data sets.

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 70

RGB-HSI Transform Method

Algorithm:

– Choose any three bands of the multispectral input data set; denote them by the Red, Green and Blue coordinates respectively. – Transform the RGB data to IHS color space. – Replace the Intensity component with PAN image. – Perform an inverse IHS to RGB color space.

• Good results are obtained for visualization
• Limited to only three bands

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 71

IHS Resolution Merge

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 72

IHS Resolution Merge - FCC

GNR607 Lecture 32-34 B. Krishna Mohan IIT Bombay Slide 73