Principles of object-oriented image analysis Image mining and - - PowerPoint PPT Presentation

Principles of object-oriented image analysis Image mining and knowledge-driven analysis in disaster risk management

• Dr. Norman Kerle

2

Lecture outline

Image Mining 2013 – Barcelonette - 28 August 2013

(1) Me @ (1) Me @ ITC ITC (3) Recent research (3) Recent research (4) Outlook & (4) Outlook & trends trends (2) Principles of (2) Principles of OOA & use OOA & use for DRM for DRM

3

ITC/University Twente

• ITC now faculty of Uni Twente
• Houses the United Nations

University- ITC Centre for Spatial Analysis and Disaster Risk Management

• Training, education and curriculum

development

• Knowledge development and

research collaboration

• Advisory services
• In collaboration with many partners

Image Mining 2013 – Barcelonette - 28 August 2013

www.unu-drm.nl

(1) Me @ (1) Me @ ITC ITC

4

Me @ ITC

• Geographer with study background in Hamburg (D), Ohio State (US) and

Cambridge (UK)

• Since early 1990s work in the hazards & disaster field, with focus on remote

sensing

• PhD in volcano remote sensing (lahars)
• Advanced image analysis, and focus on object-oriented analysis

Image Mining 2013 – Barcelonette - 28 August 2013

http://www.itc.nl/ooa-group http://www.itc.nl/about_itc/resumes/kerle.aspx

5

Object-oriented analysis for disaster risk management

DRM OOA

Image Mining 2013 – Barcelonette - 28 August 2013

Disaster risk

• Different concepts
• Expected losses (f[hazard, period])
• Risk = Hazard * VulnerabilityEaR * Amount (R=H*V)
• EaR (elements at risk): not only physical
• H: f(type, magnitude)
• V: physical, social, economic, environmental, etc.
• Amount: quantifiable?
• Note: all elements of risk are spatial

(2) Principles of (2) Principles of OOA & use OOA & use for DRM for DRM

6

Basics of object-oriented analysis

DRM OOA

Image Mining 2013 – Barcelonette - 28 August 2013

OOA

• OOA is a form of image classification
• Objects = segments (segmentation-based analysis, OBIA, GEOBIA)
• 1. step: segmentation: old concept (~1970s) – partition an image into

homogenous units

• 2. step: classification of those units

7

Basics of object-oriented analysis

DRM OOA

Image Mining 2013 – Barcelonette - 28 August 2013

OOA

• Segmentation also at multiple scales, and using auxiliary information
• Note: we do most OOA work in eCognition software

Pixel Classification level Sub-objects Super-objects

Any type of raster image, and thematic GIS layer

8

Basics of object-oriented analysis

DRM OOA

Image Mining 2013 – Barcelonette - 28 August 2013

OOA

• Main difference over pixel-based methods: objects have extra features

(spectral, geometric, contextual) = useful for classification

• Allows use of feature and process knowledge

9

Basics of object-oriented analysis

Image Mining 2013 – Barcelonette - 28 August 2013

• Pixel-based: landcover (spectal information); OOA - landuse
• Challenge: we need detailed feature and process knowledge

10

OOA for DRM

Image Mining 2013 – Barcelonette - 28 August 2013

• Our group addresses
• Use of OOA for different hazards and risk elements
• Different aspects of risk
• Methodological work (better segmentation, feature and threshold

selection)

(3) Recent research (3) Recent research

Vulnerability EaR Risk Damage (Recovery)

Remote sensing for DRM

Hazard

Domain focus Technical focus OOA (in eCognition)

Landslides/ erosion Social Urban/ infra- structure Refugee camps; metrics for recovery Pictometry-/UAV- based damage

11

OOA for DRM

Image Mining 2013 – Barcelonette - 28 August 2013

• Focus on landslide work – find solutions to this type of

mapping problem

12

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

OOA-based landslide mapping Martha et al., 2010 (Geomorphology)

• Work with several PhD students and postdocs
• work of Tapas Martha
• conceptualization of a landslide
• segmentation based on satellite data and elevation data
• removal of false positives
• classification of different landslide types

Full PhD thesis: www.itc.nl/library/papers_2011/phd/martha.pdf

Problem: knowledge incorporation

13

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

OOA-based landslide mapping Martha et al., 2010 (Geomorphology) Objective segmentation (POF) Martha et al., 2011 (IEEE TGRS)

• Problem: trial & error work
• What segmentation parameters?
• One-fits-all?
• Work on statistical optimization of

segmentation

• Balancing intra-segment

homogeneity and inter-segment heterogeneity

• Plateau objective function (POF) to

select appropriate scale factors

Scale factor 20 Scale factor 50

Problem: scale parameter

14

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

OOA-based landslide mapping Martha et al., 2010 (Geomorphology) Objective segmentation (POF) Martha et al., 2011 (IEEE TGRS) Change detection Lu et al., 2011 (IEEE GSRL)

• Ping Lu (Uni Florence): OOA-based

landslide change detection

• Also focused on multi-scale

segmentation optimization

Pre-event image Post-event image Landslide map

Problem: change detection

15

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

• Andre Stumpf: classification parameter and threshold selection
• How to chose from hundreds of object features and the best threshold?
• Random Forest method (data mining/active learning based on samples)

Problem: object feature selection

16

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

• Tested on air- and spaceborne data of

4 different sites

• Accuracies of 73-87%

OOA-based landslide mapping Martha et al., 2010 (Geomorphology) Objective segmentation (POF) Martha et al., 2011 (IEEE TGRS) Change detection Lu et al., 2011 (IEEE GSRL) Objective parameter selection Stumpf & Kerle, 2011 (RSE)

17

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

• Tapas Martha: OOA-based landslide detection

based only on pan-chromatic data

• Again use of POF
• Focus on texture measures, segment refinement
• Time-series analysis

Landslides

Problem: limits of pan-chromatic data

18

Hazard: Landslide work

Image Mining 2013 – Barcelonette - 28 August 2013

Objective segmentation (POF) Martha et al., 2011 (IEEE TGRS) Change detection Lu et al., 2011 (IEEE GSRL) Objective parameter selection Stumpf & Kerle, 2011 (RSE)

…

Use of pan- chromatic data Martha et al, 2012 (ISPRS)

19

Hazard: Landslide work with LiDAR data

Image Mining 2013 – Barcelonette - 28 August 2013

• So far all work focused on optical data
• Miet Van Den Eeckhaut: detection of forested landslides in single-pule LiDAR

data

• No use of additional optical data

→ focus on geomorphometry

(Elevation exaggeration x1; @Google Earth)

Rotational slide Complex slide

• Area in Flanders, Belgium; > 200 old

deep-seated and shallow slides

• Almost impossible to detect in
• ptical data

Problem: work with lidar data

Hazard: Landslide work with LiDAR data

Image Mining 2013 – Barcelonette - 28 August 2013

• Procedure:
• Creation of LiDAR derivatives
• Multiple segmentation based on

POF

• Detection of main scarp
• Downslope growing using

evidence from side and base scarp, as well as interior

• Good detection of deep slides

(71% of main scarps, >50% of associated landslide body

20

Hazard: Landslide work with LiDAR data

Image Mining 2013 – Barcelonette - 28 August 2013

• Promising results given the challenging terrain

OOA and LiDAR Van Den Eeckhaut et al., 2012 (Geomorphology)

21

Hazard: Erosion detection

Image Mining 2013 – Barcelonette - 28 August 2013

• Shruthi Rajesh: use of high-resolution satellite data to map gully erosion
• Similar approaches to what we developed for landslides (directional texture, etc.)
• Removal of false positives was challenging

22

Hazard: Erosion detection

Image Mining 2013 – Barcelonette - 28 August 2013

Gully erosion detection Shruthi et al., 2011 (Geomorphology)

23

Problem: linear element detection

Hazard: Erosion detection

Image Mining 2013 – Barcelonette - 28 August 2013

Gully system change detection Shruthi et al., in press (Catena)

• Change detection for gully systems (2001-2009)

24

Problem: change detection of lines

Gully detection with Random Forests Shruthi et al., in review (Geomorphology)

25

Other risk aspects – Elements at risk

Image Mining 2013 – Barcelonette - 28 August 2013

• Janak Joshi: Problem - building extraction from optical satellite data
• Chicken & egg: we’d like to have a DEM/DSM, but photogrammetry is imperfect
• Solution:

 Create an (imperfect) DEM/DSM  Use in OOA (distinguish buildings from similar looking low features)  Use the extracted buildings to correct the DEM/DSM

X, Y Z

Problem: building extraction

26

Other risk aspects – Elements at risk

Image Mining 2013 – Barcelonette - 28 August 2013

Initial DSM Evident errors Assignment of height Corrected DSM OOA-derived buildings Geoeye image

27

Other risk aspects – Elements at risk

Image Mining 2013 – Barcelonette - 28 August 2013

• Improved DSM, useful for example for flood modeling

28

[Other risk aspects – Social vulnerability]

Image Mining 2013 – Barcelonette - 28 August 2013

• Annemarie Ebert: Social vulnerability (SV): ‘‘people’s differential incapacity to

deal with hazards, based on the position of the groups and individuals within both the physical and social worlds” (Clark et al., 1998)

• Traditionally assessed using census data (that often don’t exist)
• Solution: use physical proxies
• We selected 47 variables
• In stepwise multiple regression against census-based SV index found 8

variables that explained 60% of the variance

29

[Other risk aspects – Social vulnerability]

Image Mining 2013 – Barcelonette - 28 August 2013

SV mapping with OOA Ebert et al., 2009 (Natural Hazards)

30

[Other risk aspects – Deprivation]

Image Mining 2013 – Barcelonette - 28 August 2013

• We use similar approaches to map deprivation (e.g. slums)
• Divyani Kohli: use of spatial metrics to describe urban units extracted with OOA
• Ontology used to formalize (local/specific) knowledge of slums
• Currently being used to parameterize OOA-based slum detection

Ontology-based slum detection Kohli et al., 2012 (CEUS) OOA-based slum detection Kohli et al., in press (Remote Sensing)

31

[Other risk aspects – Deprivation]

Image Mining 2013 – Barcelonette - 28 August 2013

32

Other risk aspects – Damage mapping

Image Mining 2013 – Barcelonette - 28 August 2013

• With Markus Gerke: use of oblique image data

from Pictometry

• 5 perspectives, in principle allowing comprehensive

damage evaluation

• Example from Haiti (2010)

33

Other risk aspects – Damage mapping

Image Mining 2013 – Barcelonette - 28 August 2013

• Many features were calculated

from the data (digital elevation model, texture, etc.)

34

Other risk aspects – Damage mapping

Image Mining 2013 – Barcelonette - 28 August 2013

• Images were segmented

(OOA) and classified based

• n training samples

Intact roof Broken roof/ rubble Intact facade Bare ground Vegetation

Damage mapping with Pictometry data Gerke & Kerle, 2011 (PE&RS)

We now continue this work with UAV data

35

Outlook & trends

Image Mining 2013 – Barcelonette - 28 August 2013

Status quo: OOA has proven a versatile and useful image analysis concept It allows effective use of process and feature knowledge Good progress in automating methods High dependence on eCognition (>50% of all papers; high cost) Faster development in the life sciences field than in Earth sciences

(4) Outlook & (4) Outlook & trends trends

36

Outlook & trends

Image Mining 2013 – Barcelonette - 28 August 2013

Research needs:

• Segmentation scale, feature and threshold selection remain difficult
• Actual multi-scale analysis is still rare
• Feature behavior across scales poses challenge
• Work on better image metrics

(4) Outlook & (4) Outlook & trends trends

37

Outlook & trends – 3D data processing

Image Mining 2013 – Barcelonette - 28 August 2013

• 3D data are becoming increasingly available (LiDAR point clouds,

geophyscial data)

• Many inspiring developments from biomedical field. Example: > 500 CT

slices of a mouse

• eCognition result:

(4) Outlook & (4) Outlook & trends trends

38

Outlook & trends – 3D data processing

Image Mining 2013 – Barcelonette - 28 August 2013

• Islam Fadel: OOA-processing of geophysical data (seismic/ tomographic data)

(4) Outlook & (4) Outlook & trends trends

OOA of 3D geophysical data II Fadel et al., in review (JAG) OOA of 3D geophysical data I Fadel et al., in review (Computers & Geosciences)

39

Questions?

Image Mining 2013 – Barcelonette - 28 August 2013

• Our OOA work continues – check www.itc.nl/ooa-group for updates
• Same for full references
• Papers also on

https://www.researchgate.net/profile/Norman_Kerle

• Or email: kerle@itc.nl

Thank you

(4) Outlook & (4) Outlook & trends trends