Machine Learning-aided Disaster Response Supporting faster - - PowerPoint PPT Presentation

▶

Feb 09, 2023 283 likes •607 views

Machine Learning-aided Disaster Response Supporting faster humanitarian relief efforts Ben Bischke Jakub Fil Ramona Pelich Tim G. J. Rudner Marc Ruwurm Dr Simon Jackman simon.jackman@mpls.ox.ac.uk The Team Marc Ruwurm Ramona Pelich

SLIDE 1

Machine Learning-aided Disaster Response

Supporting faster humanitarian relief efforts Ben Bischke Jakub Fil Ramona Pelich Tim G. J. Rudner Marc Rußwurm Dr Simon Jackman simon.jackman@mpls.ox.ac.uk

SLIDE 2

The Team

Marc Rußwurm Ramona Pelich Ben Bischke Jakub Fil Tim G. J. Rudner

SLIDE 3

Our Mentors

Veronika Kopaˇ ckov´ a Piotr Bili´ nski Earth Observation Artificial Intelligence

SLIDE 4

Case Studies

Ecuador Earthquake

Apr. 2016

Haiti Hurricane

Oct. 2016

Houston Floods

Aug. 2017

SLIDE 5

SLIDE 6

SLIDE 7

Our Approach

Key idea Fast, high-accuracy building and damage detection by fusion of multi-resolution and multi-temporal satellite imagery. Input data sources:

◮ Radar: Sentinel-1 (public) ◮ Optical: Sentinel-2 (public) ◮ Very high resolution (commercial)

SLIDE 8

Multi-temporal Data

time disaster radar radar radar radar radar

ptical
ptical
ptical
ptical

VHR optical

SLIDE 9

Multi-temporal Data

time disaster radar radar radar radar radar

ptical
ptical
ptical
ptical

VHR optical

pre during post

SLIDE 10

Multi-temporal Data

time disaster radar radar radar radar radar

ptical
ptical
ptical
ptical

VHR optical

coherence pre high correlation values coherence post correlation decreases

coherence RGB pre & post

SLIDE 11

Multi-temporal Data

time disaster radar radar radar radar radar

ptical
ptical
ptical
ptical

VHR optical

VHR post

SLIDE 12

Multi-resolution Data

0.5m post-disaster 10m pre-disaster 10m pre-disaster

very high resolution

ptical

radar

SLIDE 13

Multi-resolution Data

0.5m post-disaster 10m post-disaster 10m post-disaster

very high resolution

ptical

radar

SLIDE 14

Ground truth, towards two objectives

building footprints damaged sites

Open Street Map UNOSAT

SLIDE 15

Semantic Segmentation with PSP-Net

Input

CNN (ResNet)

classifier

Hengshuang Zhao et al. “Pyramid scene parsing network”. In:IEEE Conf. on Computer Vision and PatternRecognition (CVPR). 2017

SLIDE 16

Semantic Segmentation with PSP-Net

Input

CNN (ResNet)

Feature Maps

PSP Decoder

Output

Hengshuang Zhao et al. “Pyramid scene parsing network”. In:IEEE Conf. on Computer Vision and PatternRecognition (CVPR). 2017

SLIDE 17

Semantic Segmentation with PSP-Net

Input

CNN (ResNet)

Feature Maps

pool conv conv conv conv Pyramid Spatial Pooling (PSP) Module

Feature Maps Output

Hengshuang Zhao et al. “Pyramid scene parsing network”. In:IEEE Conf. on Computer Vision and PatternRecognition (CVPR). 2017

SLIDE 18

Extending PSP-Net to Multi-resolution Input

96px 192px

ResNet PSP Module

96px

ResNet PSP Module

96px

SLIDE 19

Extending PSP-Net to Multi-resolution Input

96px 192px 768px

ResNet PSP Module

96px

ResNet PSP Module

96px

ResNet PSP Module

96px

SLIDE 20

Houston—Building Prediction Results

Individual data sources:

Data mIoU Building IoU Accuracy S2 (10m & 20m bands) 73.1% 66.7% 85.4% S1 (Int, IntMT , Coh) 69.3% 63.7% 82.6% VHR 78.9% 74.3% 88.8%

Fused data sources:

Data mIoU Building IoU Accuracy S1 (IntMT , Coh) + S2 (10m & 20m bands) 76.1% 70.5% 87.3% S1 (IntMT , Coh) + S2 (10m & 20m bands) + VHR 79.9% 75.2% 89.5%

Training epochs = 20, Batch size = 4, learning rate = 0.001

SLIDE 21

Extending PSP-Net to Multi-temporal Input

ResNet PSP Module

pre post

ResNet PSP Module

pre post

ResNet PSP Module

SLIDE 22

Houston—Damage Prediction Results

Data mIoU Building IoU Accuracy S1 (IntMT, Coh) + S2 (10m & 20m bands) 59.7% 34.1% 86.4% VHR 74.2% 56.0% 93.1% S1 (IntMT, Coh) + S2 (10m & 20m bands) + VHR 75.3% 57.5% 93.7% Training epochs = 20, Batch size = 4, learning rate = 0.001

SLIDE 23

Houston—Building Prediction (1/2)

Sentinel based predictions: RGB input target (10m) prediction

SLIDE 24

Houston—Building Prediction (2/2)

VHR based predictions: RGB input target (2m) prediction

SLIDE 25

Houston—Damage Prediction (1/4)

RGB input target prediction

SLIDE 26

Houston—Damage Prediction (2/4)

VHR + Sentinel VHR only difference

verlap

added by fusion removed by fusion

SLIDE 27

Houston—Damage Prediction (3/4)

RGB input target prediction

SLIDE 28

Houston—Damage Prediction (4/4)

VHR + Sentinel VHR only difference

verlap

added by fusion removed by fusion

SLIDE 29

Use Cases & Users

Use cases

◮ Response prioritization: Reliable infrastructure mapping in developed and

developing countries

◮ Improvement of response time: Integration of our model into existing

disaster response pipelines to improve and speed up disaster response

◮ Better long-term planning: Rapid and accurate estimation of extent and

total economic cost of damage Users

◮ Humanitarian aid organizations ◮ National and local authorities in affected areas

SLIDE 30

Future Directions

Next steps

◮ Scale models to disaster-prone areas globally ◮ Create flexible model suite readily available for deployment ◮ Work with disaster relief organizations to optimize model usability ◮ Further extend models using multi-task and transfer learning

Interested in collaborating? Come talk to us—we’d love to work with you!

SLIDE 31