of Hyper-Spectral Images via Diffusion Bases The 9th International - - PowerPoint PPT Presentation

▶
of hyper spectral
SMART_READER_LITE
LIVE PREVIEW

of Hyper-Spectral Images via Diffusion Bases The 9th International - - PowerPoint PPT Presentation

Nov 15, 2022 203 likes •440 views

~ 1 ~ Unsupervised Segmentation of Hyper-Spectral Images via Diffusion Bases The 9th International Joint Conference on Computational Intelligence NCTA 2017, Funchal, Madeira, Portugal Dr. Alon Schclar School of Computer Science, Academic

slide-1
SLIDE 1

~ 1 ~

Unsupervised Segmentation

  • f Hyper-Spectral

Images via Diffusion Bases

  • Dr. Alon Schclar

School of Computer Science, Academic College of Tel-Aviv Yafo

  • Prof. Amir Averbuch

School of Computer Science, Tel-Aviv University

The 9th International Joint Conference

  • n Computational Intelligence

NCTA 2017, Funchal, Madeira, Portugal

slide-2
SLIDE 2

Introduction

  • Segmentation

– Cluster similar pixels in an image

~ 2 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-3
SLIDE 3

Introduction – cont.

  • Hyper-spectral imagery provides much more

information than RGB images

  • Currently cheaper than before to acquire
  • However

– Large volume – Redundant – Noise

  • Efficiently utilize the

extra information

~ 3 ~

Washington DC Mall

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-4
SLIDE 4

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 4 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-5
SLIDE 5

~ 5 ~

Dimensionality reduction

  • Definition:

– Embedding of high-dimensional data into a low- dimensional space that preserves the “important” characteristics of the data. – Minimum distortion of the geometry

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-6
SLIDE 6

~ 6 ~

Dimensionality reduction

  • Formal definition

– A set of vectors

𝚫 = 𝒚𝒋 𝒋=𝟐

𝒐

, 𝒚𝒋 ∈ ℝ𝑬

is mapped (while satisfying a constraint for example all pair- wise distances preserved) into a low dimensional space

𝚫 = 𝒚 𝒋 𝒋=𝟐

𝒐

, 𝒚 𝒋 ∈ ℝ𝜽

where D >> 𝜽

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-7
SLIDE 7

Dimensionality reduction in Hyper-spectral images

~ 7 ~

  • Represent the information in each hyper-pixel

by a small number of values

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-8
SLIDE 8

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 8 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-9
SLIDE 9

~ 9 ~

Diffusion Bases Dimensionality Reduction

Schclar and Averbuch (IJCCI 2015)

  • Utilizes the similarity among the coordinates of the

dataset

  • When the dimensionality is much smaller than the

number of the points

– Useful for hyper-spectral images – Point-wise dimensionality reduction is these cases requires sampling and out-of-sample extension (e.g. Nyström)

  • Dual to Diffusion Maps (Coifman and Lafon 2006)

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-10
SLIDE 10

~ 10 ~

Diffusion Bases

1. Construct the similarity matrix 𝑿 ∈ ℝ𝑬×𝑬 between the coordinates of the dataset

– Each waveband image is a coordinate

2. Normalize each row to sum to 1 giving P (Markov matrix) 3. Find the right eigenvectors {𝜊𝑗} ∈ ℝ𝑬 of P and their eigenvalues {λi} 4. Embedding of every hyper-pixel u is given by

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

Φ: 𝑣 ⟼ 𝜊1, 𝑣 , 𝜊2, 𝑣 , … , 𝜊𝜃, 𝑣 Result:

slide-11
SLIDE 11

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 11 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-12
SLIDE 12

Normalization

~ 12 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

Each coordinate in the reduced representation All pixels in each reduced waveband will be in [0..1]

slide-13
SLIDE 13

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 13 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-14
SLIDE 14
  • Quantize each pixel coordinate to 𝑚 values

where Result:

Quantization

~ 14 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-15
SLIDE 15

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 15 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-16
SLIDE 16
  • Count the quantized pixels with the same value

in the dimension reduced representation

– Largest number belongs to the largest segment – In hyper-spectral images it is the most common material in the image – where for every 𝜆 ∈ 1, … , 𝑚 , 𝑔(𝜆) is the number of quantized color vectors that are equal to 𝜆.

Color frequency calculation

~ 16 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-17
SLIDE 17

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized color

~ 17 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-18
SLIDE 18

Finding most frequent colors (peaks)

~ 18 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

  • In

Input

  • 𝜄 – the number of peaks
  • 𝑔 – the frequency of each color
  • 𝜊 – neighborhood size of each peak
  • Ou

Outp tput

  • Ψ = 𝜍

𝑗 𝑗=1,…,𝜄

  • 𝜍

𝑗 = 𝜍i

1, … , 𝜍i 𝜃 ∈ ℕ𝜃

slide-19
SLIDE 19

The Wavelength-wise Global (WWG) Segmentation Algorithm

  • Diffusion Bases Dimensionality Reduction
  • Normalization
  • Quantization
  • Color frequency calculation
  • Finding most frequent colors (peaks)
  • Finding the nearest peak to each quantized

color

~ 19 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-20
SLIDE 20

Results 1: Hyper-spectral Microscopy (D=128)

~ 20 ~ Band 35 Band 50

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

Band 95 𝜽 = 𝟓, 𝜾 = 𝟒, 𝝄 = 𝟒, 𝒎 = 𝟒𝟑

slide-21
SLIDE 21

Results 2: Remote Sensing (D=100)

~ 21 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

𝜽 = 𝟓, 𝜾 = 𝟗, 𝝄 = 𝟖, 𝒎 = 𝟒𝟑

slide-22
SLIDE 22

Future Work

  • Finding the parameter values by non-

parametric optimization e.g. Nelder-Mead

  • Find sub-pixel segments (work in progress)

~ 22 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017

slide-23
SLIDE 23

Thank you 

~ 23 ~

Schclar & Averbuch, Unsupervised Segmentation of Hyper-Spectral Images via Diffusion, IJCCI 2017