HYPERSPECTRAL RESOLUTION ENHANCEMENT USING MULTISENSOR IMAGE DATA - - PowerPoint PPT Presentation

▶
hyperspectral resolution enhancement using multisensor
SMART_READER_LITE
LIVE PREVIEW

HYPERSPECTRAL RESOLUTION ENHANCEMENT USING MULTISENSOR IMAGE DATA - - PowerPoint PPT Presentation

Nov 25, 2023 222 likes •596 views

HYPERSPECTRAL RESOLUTION ENHANCEMENT USING MULTISENSOR IMAGE DATA J. Bieniarz, D. Cerra, X. X. Zhu, R. Mller, P. Reinartz German Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), Oberpfaffenhofen, 82234 Wessling, Germany.

slide-1
SLIDE 1

HYPERSPECTRAL RESOLUTION ENHANCEMENT USING MULTISENSOR IMAGE DATA

  • J. Bieniarz, D. Cerra, X. X. Zhu, R. Müller, P. Reinartz

German Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), Oberpfaffenhofen, 82234 Wessling, Germany. jakub.bieniarz@dlr.de +49 8153 28-2790

slide-2
SLIDE 2
slide-3
SLIDE 3
slide-4
SLIDE 4
slide-5
SLIDE 5
slide-6
SLIDE 6
slide-7
SLIDE 7

Unmixing Based Resolution Enhancement

www.DLR.de • Chart 7

slide-8
SLIDE 8

Unmixing Based Resolution Enhancement

www.DLR.de • Chart 8

slide-9
SLIDE 9

Highly Mixed Scenario

www.DLR.de • Chart 9

? ?

slide-10
SLIDE 10

Multi-Look Joint Sparsity Fusion (MLJSF)

  • Unmixing based image fusion
  • Use of external (pre-defined) spectral libraries

containing pure endmemebers

  • Jointly sparse unmixing concept – spatial

spectral imnixing

www.DLR.de • Chart 10

slide-11
SLIDE 11

significant endmembers

y = A x

  • Use of large libraries
  • Aster Spectral Library from

JPL, USGS, JHU with 2463 spectra

  • DLR Spectral Archive with 1609 spectra

Requirements:

  • Higly accurate Atmospheric Correction

Sparse Spectral Unmixing – The Concept

www.DLR.de • Chart 11

=

slide-12
SLIDE 12

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(window based)

Select Endmembers

(window based)

Resample endmembers to MSI SRF Joint Sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(window based)

HSI

www.DLR.de • Chart 12

slide-13
SLIDE 13

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(2f x 2f window)

Select Endmembers

(window based)

Resample endmembers to MSI SRF Jointly sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(af x bf Window)

HSI . . . . . . From a window select active endmembers and construct a new pruned dictionary for this region. . . . . . .

www.DLR.de • Chart 13

slide-14
SLIDE 14

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(window based)

Select Endmembers

(2 x 2 window)

Resample endmembers to MSI SRF Joint Sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(window based)

HSI Unmix the window of the MSI using the Joinr Sparsity Model (JSM)

www.DLR.de • Chart 14

slide-15
SLIDE 15

EnMAP + World_View2

Thanks to Karl Segel

www.DLR.de • Chart 15

+

slide-16
SLIDE 16
slide-17
SLIDE 17

WV2 EnMAP MLJSF Reference Standard method

slide-18
SLIDE 18

WV2 EnMAP MLJSF Reference Standard method

slide-19
SLIDE 19

Experiments – spatial resolution

HYPERION

  • 30m spatial resolution
  • 220 spectral bands

(152 after band reduction)

  • 10nm spectral resolution
  • 0.4-2.5 spectral range

www.DLR.de • Chart 19

slide-20
SLIDE 20

Experiments – spatial resolution

LANDSAT 7 (simulated from AISA)

  • 3m spatial resolution
  • 6 spectral bands
  • 0.4-2.4 spectral range

www.DLR.de • Chart 20

slide-21
SLIDE 21

Experiments – spectral resolution

Landsat 7 SRF Spectrum LANDSAT 7 (simulated from AISA)

  • 3m spatial resolution
  • 6 spectral bands
  • 0.4-2.4 spectral range

HYPERION

  • 30m spatial resolution
  • 220 spectral bands

(152 after band reduction)

  • 10nm spectral resolution
  • 0.4-2.5 spectral range

www.DLR.de • Chart 21

slide-22
SLIDE 22

Results

MLJSF

  • 3m spatial resolution
  • 152 spectral bands
  • 10nm spectral resolution
  • 0.4-2.5 spectral range

www.DLR.de • Chart 22

slide-23
SLIDE 23

Evaluation – 2 ways

  • Performance measure
  • Quantitative comparison
  • Application example
  • SAM Classification
slide-24
SLIDE 24

Quantitative analisys

www.DLR.de • Chart 24

slide-25
SLIDE 25

Quantitative analisys

www.DLR.de • Chart 25

slide-26
SLIDE 26

Application to classification

HYPERION

  • 30m resolution
  • OA: 42%
  • Kappa:0.36

Compared to classification results

  • f AISA original image

www.DLR.de • Chart 26

slide-27
SLIDE 27

Application to classification

MLJSF

  • 3m resolution
  • OA: 79%

increase of 37%

  • Kappa:0.76

increase of 0.40

www.DLR.de • Chart 27

slide-28
SLIDE 28

Application to classification

HYPERION

  • 30m resolution
  • OA: 42%
  • Kappa:0.36

Compared to classification results

  • f AISA original image

www.DLR.de • Chart 28

slide-29
SLIDE 29

Application to classification

CNMF

  • 3m resolution
  • OA: 64%

increase of 22%

  • Kappa:0.59

increase of 0.23

www.DLR.de • Chart 29

slide-30
SLIDE 30

Conclusions

www.DLR.de • Chart 30

  • MLJSF exploits the sparsity in hyperspectral mixtures

and the HR spatial information in the multispectral image.

  • MLJSF achieves
  • Comparable quantitative metrics to the state-of-the-art

methods, even when using external libraries containing pure spectra.

  • Significantly better performance when aiming at

applications like classification.

  • Future work: Robustness analysis against co-

registration error, different conditions of the data acquisition and missing materials in the library etc.

slide-31
SLIDE 31

HYPERSPECTRAL RESOLUTION ENHANCEMENT USING MULTISENSOR IMAGE DATA

  • J. Bieniarz, D. Cerra, X. X. Zhu, R. Müller, P. Reinartz

German Aerospace Center (DLR), Remote Sensing Technology Institute (IMF), Oberpfaffenhofen, 82234 Wessling, Germany. jakub.bieniarz@dlr.de +49 8153 28-2790

slide-32
SLIDE 32

Spectral Resolution

www.DLR.de • Chart 32

slide-33
SLIDE 33

Spectral Resolution

www.DLR.de • Chart 33

slide-34
SLIDE 34

Highly mixed scenario

www.DLR.de • Chart 34

slide-35
SLIDE 35

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(2f x 2f window)

Select Endmembers

(2 x 2 window)

Resample endmembers to MSI SRF Jointly sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(af x bf Window)

HSI . . . . . . 1) Unmix the whole HSI (pixel-wise) using the sparse spectral unmixing method and a priori given spectral dictionary.

www.DLR.de • Chart 35

slide-36
SLIDE 36

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(window based)

Select Endmembers

(2 x 2 window)

Resample endmembers to MSI SRF Joint Sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(window based)

HSI 3) Resample endmember spectra to the MSI sensor spectral resolution using the spectral response function (SRF)

www.DLR.de • Chart 36

slide-37
SLIDE 37

Hyperspectral Image Resolution Enhancement Based on Jointly Sparse Spectral Unmixing

Sparse unmixing

  • f HSI

MSI

(window based)

Select Endmembers

(2 x 2 window)

Resample endmembers to MSI SRF Joint sparse unmixing

  • f MSI

Spectral dictionary High resolution HSI

(window based)

HSI 5) reconstruct the high resolution HSI, with the resulting MSI abundances and the

  • riginal HSI spectral dictionary using the

LMM. . . . . . .

www.DLR.de • Chart 37