TRUEBRANCH METRIC LEARNING-BASED VERIFICATION OF FOREST - - PowerPoint PPT Presentation

Simona Santamaria* ssimona@ethz.ch David Dao* david.dao@inf.ethz.ch Björn Lütjens* lutjens@mit.edu

• Prof. Dr. Ce Zhang ce.zhang@inf.ethz.ch

TRUEBRANCH

METRIC LEARNING-BASED VERIFICATION OF FOREST CONSERVATION PROJECTS

Motivation

2

• Deforestation and forest degradation accounts for ~15% of all global

greenhouse gas emissions [1]

• International stakeholders are paying landowners for forest conservation,

if they can verify it [2]

[1] IPCC. 2019: Summary for policymakers, WWF [2] UN-REDD Programme, www.goldstandard.org www.reforestationworld.org

• On-ground inspection is expensive (300

USD/ha[3]), biased, hard to scale, and corruptible

Monitoring, reporting and verification process

3

• Carbon estimates by satellites can have

high uncertainties and long lead times

[3] Interviews with Ministry of Agriculture Peru

Reported High-Resolution Imagery Landowner International Stakeholder Payment for Ecosystem Services (PES)

Verification with Drones

• Low-cost monitoring via drones

4

Challenge

• Opens up possibility of untruthfully reported imagery
• Attack vectors

5

wrong time true location high true time wrong location high true time true location medium Reported Land-Use Detected Forest Cover modified image high

Reported High-Resolution Imagery Landowner TrueBranch Verification Automated Valuation

• f Forest Ecosystem

Services International Stakeholder Payment for Ecosystem Services (PES)

Approach - True Branch Verification System

Public Low-Resolution Imagery

6

Forest validation algorithm

How to distinguish truthful imagery from untruthful imagery?

• Image Registration: Matching Drone images with Satellite images

Model and Classifier

Satellite image Drone image Same location, same time?

Yes/No

7

Nominal Metrics not able to detect attacks

• Nominal distance metrics of MSE in pixels space

MSE

Satellite image Drone image

8

• MSE in pixel space

ground truth wrong time wrong location

Nominal Metrics

• Nominal distance metrics of MSE in feature space

g g

e1 e2

MSE

Pretrained Model Embeddings Drone image Satellite image

9

• MSE in feature space

ground truth wrong time wrong location

Learned Metrics

• MSE in pixel space and RESISC-45 feature space not sufficient

10

Satellite anchor rt-rl wt-rl rt-wl

MSE MSE

Learned Metrics

11

rt-rl

Learned distance

wt-rl

• adv. pert.

Satellite anchor

threshold

• MSE in pixel space and RESISC-45 feature space not sufficient
• Metric learning with triplet loss function

Learned distance

Metric Learning with Triplet loss

Training

12

Embeddings

Model

e1 e2

Triplet loss

anchor positive negative

Metric Learning with Triplet loss

same location, same time?

Yes/No Inference

13

Embeddings

Model

e1 e2

Classifier

First Results on Dataset in Indonesia

14

• Dataset:

10 different locations, 3 different years

First Results on Indonesian Dataset

15

Satellite anchor rt-rl wt/wl

• Difference between satellite image and drone images

Distance Location

Conclusion and Further Work

16

• Model with metric learning is able to distinguish truthfully reported imagery

from untruthfully reported imagery

• Model evaluation on more training and testing data to ensure high reliability
• Protecting model from Adversarial perturbation
• Metric learning with images from different sources with different resolutions

Simona Santamaria* ssimona@ethz.ch David Dao* david.dao@inf.ethz.ch Björn Lütjens* lutjens@mit.edu

• Prof. Dr. Ce Zhang ce.zhang@inf.ethz.ch

Thank you very much for your attention