A Multivariate Statistical Model for Multiple Images Acquired by - - PowerPoint PPT Presentation

A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors

Jorge Prendes1,2, Marie Chabert1,3, Fr´ ed´ eric Pascal2, Alain Giros4, Jean-Yves Tourneret1,3

1 T´

eSA Laboratory, 2 Sup´ elec - SONDRA,

3 University of Toulouse, 4 CNES (French Space Agency)

ICASSP 2014

Introduction Image Model Similarity Measure Results Conclusions

Outline

1 Introduction 2 Image Model 3 Similarity Measure 4 Results 5 Conclusions

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 2 / 23

Introduction Image Model Similarity Measure Results Conclusions

Introduction

Motivation: Change detection on remote sensing images Monitor urban/rural area evolution

Detect new constructions Track changes in agricultural areas Track urban growth

Coordinate efforts after natural disasters

Volcano eruptions Floodings Earthquakes

Improve the analysis of remote sensing images

Find new objects

Different type of sensors: Optical, SAR, Hyperspectral, etc. Joint analysis of heterogeneous sensors!

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 3 / 23

Introduction Image Model Similarity Measure Results Conclusions

Introduction

Change Detection Framework Sliding window W Similarity measure on W Threshold Statistical Similarity Measures Dependency between pixel intensities

Correlation Coefficient

Linear dependency, Fails on homogeneous areas

Mutual Information

Requires pdf estimation, Fails on homogeneous areas

Optical SAR Images WOpt WSAR Sliding Window: W d = f(WOpt, WSAR) Similarity Measure H0 : Absence of change H1 : Presence of change d

H0

≷

H1

τ Decision . . . Using several windows Result

Objective: Similarity measure for homogeneous and heterogeneous sensors based on a statistical model

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 4 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – Optical image for Homogeneous Regions

Optical Sensor Affected by additive Gaussian noise IOpt = TOpt(P) + νN(0,σ2) IOpt|P ∼ N

• TOpt(P), σ2

where TOpt(P) is how an object with physical properties P would be ideally seen by an

• ptical sensor

σ2 is associated with the noise variance

1 5 10 IOpt

Histogram of the normalized image

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 5 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – SAR Image for Homogeneous Regions

Radar Sensor Affected by multiplicative speckle noise (with gamma distribution) ISAR = TSAR(P) × νΓ(L, 1

L)

ISAR|P ∼ Γ

• L, TSAR(P)

L

• where

TSAR(P) is how an object with physical properties P would be ideally seen by a SAR sensor L is the number of looks of the SAR sensor

1 2 4 ISAR

Histogram of the normalized image

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 6 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – Generic Image for Homogeneous Regions

Generic Model: Sensor S IS|P = fS[TS(P), νS] Optical Image IOpt = TOpt(P) + νN(0,σ2) TOpt(P) = µP SAR Image ISAR = TSAR(P) × νΓ(L, 1

L)

TSAR(P) = αP × θP

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 7 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – Joint Distribution for Homogeneous Regions

Independence assumption for the sensor noises p(IS1, IS2|P) = p(IS1|P) × p(IS2|P) Conclusion Statistical dependency (CC, MI) is not always an appropriate similarity measure

1 1 IOpt ISAR

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 8 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – Heterogeneous Regions

Sliding window W Usually includes a finite number of objects, K Different values of P for each object Pr(P = Pk|W ) = wk p(IS1, IS2|W ) =

K

• k=1

wkp(IS1, IS2|Pk) Mixture distribution!

1 1 IOpt ISAR

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 9 / 23

Introduction Image Model Similarity Measure Results Conclusions

Image Model – Mixture Distribution

Mixture Distribution p(IS1, IS2|W ) =

K

• k=1

wkp(IS1, IS2|Pk) Parameter Estimation Expectation Maximization Iteratively Algorithm

Estimate class prob. π(i)

n,k

Maximize parameters θ(i)

k

Repeat

Selection of the number of classes [1]

[1] M. A. T. Figueiredo and A. K. Jain, ”Unsupervised learning of finite mixture models,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 381–396, March 2002.

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 10 / 23

Introduction Image Model Similarity Measure Results Conclusions

Similarity Measure – Introduction

Mixture distribution Parameter Estimates

Related to P Can be used to derive [TS1(P), TS2(P), . . . ] for each object Example:

TOpt(Pk) = µk TSAR(Pk) = L × θk

1 1 IOpt ISAR 1 1 P1 P2 P3 P4

TOpt (P) TSAR (P)

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 11 / 23

Introduction Image Model Similarity Measure Results Conclusions

Similarity Measure – Manifold

Main assumption For each unchanged window,

v(P) = [TS1(P), TS2(P), . . . ] can be

considered as a point on a manifold Manifold Describes the joint behavior of the different images Belongs to a D-dimensional space

D: Number of combined channels Several unchanged windows

. . .

1 0.3

TOpt (P) TSAR (P)

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 12 / 23

Introduction Image Model Similarity Measure Results Conclusions

Similarity Measure – Manifold

Unchanged regions Pixels belong to the same

• bject

P is the same for both images

1 0.3 TOpt (P) TSAR (P)

Changed regions Pixels belong to different

• bjects

P changes from one image to another

1 0.3 TOpt (P) TSAR (P)

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 13 / 23

Introduction Image Model Similarity Measure Results Conclusions

Similarity Measure – Manifold

Distance measure between Optical and SAR images PDF of v(P) Good distance measure Learned using training data from unchanged images Learning strategies

Histogram Parzen windows Mixture models

H0 : Absence of change H1 : Presence of change

K

• k=1
• wk

pT( v W ,k)

H0

≷

H1

τ

where

• wk is the estimated wk
• vW ,k is the estimated vector v for the k-th

component of the window W

• pT is the estimated density of v(P)

τ is an application dependent threshold

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 14 / 23

Introduction Image Model Similarity Measure Results Conclusions

Similarity Measure – Summary

WOpt WSAR Sliding Window: W Mixture

• µ1,

σ2

1,

k1, α1

• θ1 :
• TS1(P1),

TS2(P1)

• vP1 :
• µ4,

σ2

4,

k4, α4

• θ4 :
• TS1(P4),

TS2(P4)

• vP4 :

. . . . . .

1 0.3 P1 P2 P3 P4

TS1 (P) TS2 (P)

Manifold Samples

. . .

1 0.3

TOpt (P) TSAR (P)

Using several windows 1 0.3

TOpt (P) TSAR (P)

Manifold Estimation

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 15 / 23

Introduction Image Model Similarity Measure Results Conclusions

Results – Synthetic Optical and SAR Images

Synthetic optical image Synthetic SAR image Change mask Mutual Information Correlation Coefficient Proposed Method

1 1 PFA PD

Proposed Correlation Mutual Inf.

Performance – ROC

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 16 / 23

Introduction Image Model Similarity Measure Results Conclusions

Results – Real Optical and SAR Images

Optical image before the flooding SAR image during the flooding Change mask

[1] G. Mercier, G. Moser, and S. B. Serpico, “Conditional copulas for change detection in heterogeneous remote sensing images,” IEEE Trans. Geosci. and Remote Sensing, vol. 46, no. 5, pp. 1428–1441, May 2008.

Mutual Information Conditional Copulas [1] Proposed Method

1 1 PFA PD

Proposed Copulas Correlation Mutual Inf.

1 1 TOpt (P) TSAR (P)

Performance – ROC Manifold Projection

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 17 / 23

Introduction Image Model Similarity Measure Results Conclusions

Results – Pl´ eiades Images

Pl´ eiades – May 2012 Pl´ eiades – Sept. 2013

1 1 TPleiades (P) TPleiades (P)

Change mask Manifold Projection

Special thanks to CNES for providing the Pl´ eiades images

Change Map

1 1 PFA PD

Proposed Correlation Mutual Inf.

Performance – ROC

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 18 / 23

Introduction Image Model Similarity Measure Results Conclusions

Results – Pl´ eiades and Google Earth Images

Pl´ eiades – May 2012 Google Earth – July 2013

1 1 TPleiades (P) TGoogle (P)

Change Mask Manifold Projection Change Map

1 1 PFA PD

Proposed Correlation Mutual Inf.

Performance – ROC

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 19 / 23

Introduction Image Model Similarity Measure Results Conclusions

Results

Homogeneous images

Pl´ eiades – Pl´ eiades 1 1 PFA PD

Proposed Correlation Mutual Inf.

CC and MI Similar performance Proposed method Improved performance Heterogeneous images

Pl´ eiades – Google Earth 1 1 PFA PD

Proposed Correlation Mutual Inf.

CC Reduced Performance Proposed method and MI Performance not affected

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 20 / 23

Introduction Image Model Similarity Measure Results Conclusions

Conclusions and Future Work

Conclusions New statistical model to describe multi-channel images

Analyze the joint behavior of the channels to detect changes, in contrast with channel by channel analysis

New similarity measure showing encouraging results for homogeneous and heterogeneous sensors Interesting for many applications

Change detection Registration Segmentation Classification

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 21 / 23

Introduction Image Model Similarity Measure Results Conclusions

Conclusions and Future Work

Future Work Model validation on larger datasets. Include priors on the sensor parameters: Bayesian methods Study the method performance for different image features

Texture coefficients: Haralick, Gabor, QMF Wavelet coefficients Gradients

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 22 / 23

Introduction Image Model Similarity Measure Results Conclusions

Thank you for your attention Jorge Prendes

jorge.prendes@tesa.prd.fr

• J. Prendes

T´ eSA – Sup´ elec-SONDRA – INP/ENSEEIHT – CNES A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors 23 / 23