Classification of Remotely Sensed Images for Landuse Information - - PowerPoint PPT Presentation

• Prof. Krishna Mohan Buddhiraju

Centre of Studies in Resources Engineering IIT Bombay INDIA bkmohan@csre.iitb.ac.in

Classification of Remotely Sensed Images for Landuse Information

Today’s Presentation

(Very brief) Introduction to Remote Sensing – source of images Image Classification Principles Texture based segmentation High Resolution Image Classification Hyperspectral Image Classification

What is Remote Sensing?

Remote sensing is the art and science of making measurements about an object or the environment without being in physical contact with it

CSRE 0.6m x 0.6m

5.8m x 5.8m

23.25m x 23.25m

High Spectral Resolution

Large number of contiguous sensors Narrow bandwidth wavelength Response

Low Contrast Image

Contrast Enhanced Image

Input Image FCC

NDVI

Concept of Image Classification

Image classification - assigning pixels in the image to categories or classes of interest Examples: built-up areas, waterbody, green vegetation, bare soil, rocky areas, cloud, shadow, …

Why Classification?

• Quantitative information
• Acreage of each category
• Spatial location of each category
• Identifying any changes happening in one or

more categories since the last time the classification was done of an image of the same area for a past date

Types of Classification

• Supervised Classification
• Partially Supervised Classification
• Unsupervised Classification

Supervised Classification

• Familiarity with geographical area
• Small sets of pixels can be identified for each

class

• Statistics for the classes can be estimated from

the samples

• Separate sets can be identified for classifier

learning and post-classification validation

Unsupervised Classification

• Domain knowledge or the experience of an

analyst may be missing

• Data analyzed by numerical exploration
• Data are grouped into subsets or clusters

based on statistical similarity

• K-Means and its many variants, hierarchical

methods are often used

Partially Supervised Classification

When prior knowledge is available

– For some classes, and not for others, – For some dates and not for others in a multitemporal dataset,

Combination of supervised and unsupervised methods can be employed for partially supervised classification of images

Statistical Characterization of Classes Each class has a conditional probability density function (pdf) denoted by p(x | ck) The distribution of feature vectors in each class ck is indicated by p(x | ck) We estimate P(ck | x), the conditional probability of class ck given that the pixel’s feature vector is x

Supervised Classification Principles

• Typical characteristics of classes

– Mean vector – Covariance matrix – Minimum and maximum gray levels within each band – Conditional probability density function p(Ci|x) where Ci is the ith class and x is the feature vector

• Number of classes L into which the image is to

be classified should be specified by the user

Inputs to a Classifier

• How many and what classes to map input

data into?

• What are the attributes of each data

element? (In case of images, the data element is a pixel, attributes are measurements in various wavelengths made by imaging sensors)

• Samples to help classifier learn relationship

between input raw data and information classes

• Validation data to test the performance of

classifier

How are known sample locations marked?

Necessary conditions for successful classification

• Rich set of attributes (called features in

machine learning literature)

• Adequate number of samples for classifier

learning (called training data) and validation (called test data)

• Capability of learning algorithm – should be

able to exploit all information that can be exploited from the sample data

Support Vector Machines

Slides on SVM originally from Prof. Andrew Moore’s lectures on Machine Learning

Linear Classifiers

f

x

a yest

denotes +1 denotes -1 f(x,w,b) = sign(w. x - b)

Maximum Margin

denotes +1 denotes -1

The maximum margin linear classifier is the linear classifier allowing maximum margin for test samples to vary from training samples

Linear SVM

Maximize Margin

denotes +1 denotes -1

wx +b = 0

 

2 , 1

argmaxarg min subject to :

i

i d b D i i i i i

b w D y b

 

      



w x

x w x x w

Margin

Strategy:

: 1

i i

D b      x x w

 

2 1 ,

argmin subject to : 1

d i i b i i i

w D y b



    



w

x x w

Multilayer Perceptron Neural Networks

Mathematical Representation

Inputs Output w2 w1 wn . . … y

1

net b y f(net)

n i i i

wx



 



＋

x2 xn b x1

Mathematical Representation

• f the Activation Function

Mathematical Representation

• f the Activation Function

I N P U T N O D E S

H I D D E N L A Y E R S

O U T P U T N O D E S

Multilayer Perceptron Network

Selected Applications

• Landuse/Landcover classification
• Edge and line detection

Input Image

NN Supervised Classification

Texture Analysis

MUMBAI Data: IRS-1C, PAN Consists of 1024x1024 pixels.

LEGEND

Texture Classification by neural networks

WATER MARSHY LAND / SHALLOW WATER HIGHLY BUILT-UP AREA PARTIALLY BUILT-UP AREA OPEN AREAS/ GROUNDS

Identification of Informal Settlements based

• n Texture

39

Classification Strategies

Pre-processing Spectral Features Spatial Features Texture Features Context High Resolution Satellite Image Decompose image at different level Segment image at Different Resolutions Linking the regions of different resolutions Connected Component Labeling General Purpose Classification Object-Specific Classification Post-processing (Relaxation Labeling Process) Classified Image

Object based classification

Grass Vegetation Roof top Concrete Open ground

Buildings1 Open ground Road Shadow Buildings2 Vegetation

Object based classification

Object based classification

Buildup Open ground Vegetation

Examples

Road Extraction

Biplab Banerjee, Siddharth Buddhiraju and Krishna Mohan Buddhiraju, Proc. ICVGIP 2012

Examples

Building outline extraction by object based image analysis

Biplab Banerjee and Krishna Mohan Buddhiraju, UDMS 2013, Claire Ellul et al. (ed.), CRC Press, May 2013

Object Specific Classification Examples

Buildings Planes Trees Ashvitha Shetty and Krishna Mohan B., Building Extraction in High Spatial Resolution Images Using Deep Learning Techniques, LNCS10962, pp. 327–338, 2018

Hyperspectral Imagery

INTRODUCTION

Hyperspectral sensors

• Large number of contiguous

bands

• Narrow spectral BW

Advantages

• Better

discrimination among classes on ground is offered

• Highly correlated bands
• Huge

information from a contiguous and smooth spectra

6/14/2019 Centre of Studies in Resources Engineering, IIT BOMBAY 47

Hyperspectral data of a scene

(Source: remotesensing.spiedigitallibrary.org)

Tea Spectra for different conditions

AVIRIS-NG red-green-blue (visible) aerial image of the Refugio Incident oil spill, near Santa Barbara Channel beaches

Airborne Visible and InfraRed Imaging Spectrometer – Next Generation (AVIRIS- NG)

Source: https://aviris-ng.jpl.nasa.gov/

Atmospheric Correction High spectral resolution image Dimensionality Reduction Pure Pixel / Training Data Identification Supervised Classification Mixture Modeling Abundance Mapping General Purpose classification

High Spectral Resolution Image Analysis

Spectral libraries Spectral Matching Classification Sub-pixel Mapping & Super-Resolution

End Member Extraction

• Pixel Purity Index

(Source:https://www.researchgate.net/figure/T

• y-example-illustrating-the-performance-of-

the-PPI-endmember-extraction-algorithm-in- a_fig2_228856827)

51

Endmember Extraction Algorithm Demonstration : Samson Dataset

5,88 68, 62 4,8

• Fig. 1 Samson FCC
• Fig. 2 Auto-EME Endmembers
• Fig. 3 Endmember 1 (4,8)
• Fig. 4 Endmember 2 (5,88)
• Fig. 5 Endmember 3 (68,62)

Abundance Distribution of Endmembers with GDME algorithm on Samson dataset

Coordinates

• n Image

Ground Truth Entropy Output

(13, 3) 0.0767 0.1189 1 0.8042 (67, 53) 0.7974 0.7411 0.1966 0.1804 0.0061 0.0783 (88, 70) 0.7760 0.7556 0.2065 0.1653 0.0174 0.0789 (6, 3) 0.0737 0.1195 1 0.8066

Coordinates Ground Truth Entropy

• utput

(90, 95) 0.9531 0.9657 0.0198 0.0469 0.0144 (56, 3) 0.0777 0.1233 1 0.7989 (19, 17) 0.1563 0.0671 0.1387 0.8437 0.7940 (29, 10) 0.0272 0.0707 0.1292 0.9728 0.8000

Water Vegetation Rock Integrated Abundance Image

Hyperion Hyperspectral Data

• Number of rows = 1400
• Number of columns = 256
• Number of bands = 242

54

SVM Classification

55

Deep Learning by CNN

• Train networks with many layers (vs. shallow nets

with just a couple of layers)

• Multiple layers work to build an improved feature

space

• Biological Plausibility – e.g. Visual Cortex
• Proven - Problems which can be represented with a

polynomial number of nodes with k layers, may require an exponential number of nodes with k-1 layers (e.g. parity)

• Highly complex functions can be efficiently

represented with deep architectures

Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks

• Cascading Convolutional and Pooling layers
• Pooling
• Flattening for Feature Vector Generation
• Classification by Fully Connected Neural Net
• Trained by Backpropagation Algorithm

(Source: Chen, Yushi et al.“Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks.” IEEE Transactions on Geoscience & Remote Sensing (2016): 6232-6251.)

57

Indian Pines

• 145 145 pixels and 224 spectral bands
• wavelength range of 400 to 2500 nm

Salinas

• 512 217 pixels and 224 spectral

bands

• wavelength range of 400 to 2500

nm

Some public domain datasets with labels

Spatial Spectral Classification

a) Spatial Spectral classification result b) Result of SVM c) Ground truth

Pavia University (PU)

• 610 340 pixels and 103

spectral bands

• wavelength range of 430 to

860 nm AVIRIS-NG Dataset (AN)

• 777 449 pixels and 445 spectral bands
• wavelength range of 380 to 2500 nm

Result of Salinas Dataset

Ground truth Image Prediction with PCA+CNN

61

Result of UAV Dataset

Ground truth Image Prediction with PCA+CNN

62

Recent Application of CNN

• Sub-pixel Analysis (estimation of

classification at higher resolution when spatial resolution is coarse)

• Super-resolution (estimating the image

itself at higher resolution when imaging is done at coarse resolution)

a) Original HR image b) Simulated coarser version (Z=1/4) c) Sub-pixel level classified image (architecture-2) Urban built-up Bare soil Bitumen Water body Vegetation

Illustration of proposed sub-pixel mapping framework

• Reconstruction of finer scale images from coarser ones, thereby increasing

the probability of pure pixels

Simulated coarse resolution image Higher resolution image

Super-resolution of hyperspectral images

Results: Proposed SR approach on AVIRIS Dataset

a) Original HR image b) Simulated coarser version

Super-resolved image

Thank You!