Spotting Violence from Space The Detection of Housing Destruction in - - PowerPoint PPT Presentation
Spotting Violence from Space The Detection of Housing Destruction in Syria Andr Grger, Jonathan Hersh, Andrea Mantangra, Hannes Mueller, Joan Serrat Trinity College 22. February 2019 Hannes Mueller (Trinity College) Spotting Violence from
Spotting Violence from Space
The Detection of Housing Destruction in Syria André Gröger, Jonathan Hersh, Andrea Mantangra, Hannes Mueller, Joan Serrat
Trinity College
Hannes Mueller (Trinity College) Spotting Violence from Space
1 / 31
War Reporting
Hannes Mueller (Trinity College) Spotting Violence from Space
2 / 31
War Reporting
War reporting is at times highly controversial.
Hannes Mueller (Trinity College) Spotting Violence from Space
3 / 31
War Reporting
War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon.
Hannes Mueller (Trinity College) Spotting Violence from Space
3 / 31
War Reporting
War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area.
Hannes Mueller (Trinity College) Spotting Violence from Space
3 / 31
War Reporting
War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion.
Hannes Mueller (Trinity College) Spotting Violence from Space
3 / 31
War Reporting
War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion. It might also affect data gathering.
Hannes Mueller (Trinity College) Spotting Violence from Space
3 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities.
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
2) Use satellite imagery to measure destruction.
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
2) Use satellite imagery to measure destruction.
deep learning architecture trained using UNOSAT data
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
2) Use satellite imagery to measure destruction.
deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
2) Use satellite imagery to measure destruction.
deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
This Paper (once it’s done!)
Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases.
reporting function of source/military control
2) Use satellite imagery to measure destruction.
deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data
3) Compare destruction and reports
Hannes Mueller (Trinity College) Spotting Violence from Space
4 / 31
Related Literature (Econ.) - Media Bias
Measures: Groseclose and Milyo (2005, QJE), Gentzkow and Shapiro (2010, Econometrica) Political economy of bias: Leeson (2008, JEP), Prat and Stromberg (2013), Lacrinese, Puglisi and Snyder (2011, JPubE) Government reaction to media: Snyder and Strömberg (2010, JPE), Durante and Zhuravskaya (2018, JPE) Effect of mass media on population: DellaVigna and Kaplan (2007, QJE), DellaVigna et al (2014, AEJ:Applied), Yanagizawa-Drott (2014, QJE)
Hannes Mueller (Trinity College) Spotting Violence from Space
5 / 31
Related Literature - Violence Data
Thousands of papers based on violence data. Serious investments in data gathering: ACLED, UCDP (GED) In one way or the other all these measures are based on reports on
Potential problem: reporting costs are inversely proportional to intensity of conflict.
Hannes Mueller (Trinity College) Spotting Violence from Space
6 / 31
Related Literature - Social Science
Old debate in political science (For example, Sambanis (2004, JCR)) Davenport and Ball (2002, JCR), study Guatemalan state terror, compare newspapers and human rights violations reports, identify underreporting of violence by newspapers in rural areas Weidmann (2016, AJPS), compares media-based event and military sources, higher reporting rates of violence in cellphone-covered areas. Price, Gohdes, and Ball (2014, HRDAG), Updated Statistical Analysis
Hannes Mueller (Trinity College) Spotting Violence from Space
7 / 31
Plan for Alternative Measure
Use satellite imagery to scan for destruction. Based on this construct a panel of violence. Advantages:
Potentially available with frequency of satellite updates (in 2017+ quarterly) Availability is improving dramatically (monthly/weekly). Spatially very disaggregated.
Study reporting news bias in space and time. Caveats:
Destruction is due to a particular subset of violence Satellites are operated, i.e. they might follow news Measurement error is not trivial without human coders
Hannes Mueller (Trinity College) Spotting Violence from Space
8 / 31
Related Literature - Remote Sensing
Witmer (2014) Remote Sensing of Violent Conflict: Eyes from Above. Overview. Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis. 86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2, patch classifier for 11 different places, balanced test set. 80% TPR, 12% FPR. Kahraman, Imamoglu, and Ates (2016) Disaster Damage Assessment
Earthquake and 2013. balanced test sets, 75/82% TPR and 25/15% FPR. Fujita et al (2017) Damage Detection from Aerial Images via Convolutional Neural Networks: pairs of pre- and post-tsunami image patches, balanced test sets, 94%-96% accuracy, pre not necessary.
Hannes Mueller (Trinity College) Spotting Violence from Space
9 / 31
Related Literature - Remote Sensing
Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis.
86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2 patch classifier for 11 different places (!) balanced test set stats: 80% true positive rate, 12% false positive rate
Still: UN/Amnesty International use human coders in applications.
Hannes Mueller (Trinity College) Spotting Violence from Space
10 / 31
Remainder of Talk
Data Sources Some Evidence on Media Bias Presentation of Method Results
Hannes Mueller (Trinity College) Spotting Violence from Space
11 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times)
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Data Sources
GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery Unit of analysis is currently city but we are working on a match to control points.
Hannes Mueller (Trinity College) Spotting Violence from Space
12 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry)
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian)
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with:
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with:
UNOSAT labels - destruction score
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with:
UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with:
UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
Media Reporting Dataset
News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with:
UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory
Hypothesis: reporting is not consistent and function of control.
Hannes Mueller (Trinity College) Spotting Violence from Space
13 / 31
aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs
1000 2000 3000 4000 5000 UNOSAT destruction measure 500000 1000000 1500000 2000000 GDELT fighting measure
aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs aleppo aleppo aleppo aleppo ar-raqqa ar-raqqa ar-raqqa damascus damascus dar'a dar'a dar'a dar'a hamra hamra hamra homs homs
1000 2000 3000 4000 5000 UNOSAT destruction measure 2000 4000 6000 8000 10000 ICEWS fighting measure
Comparison of Fighting News Events and Housing Destruction (using UNOSAT labels)
.2 .4
1 2 3 4 Fighting News Events around Government Taking Control of City (at 0)
Media Reporting Dataset
Strong deviation between UNOSAT destruction and news reporting
Hannes Mueller (Trinity College) Spotting Violence from Space
15 / 31
Media Reporting Dataset
Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources.
Hannes Mueller (Trinity College) Spotting Violence from Space
15 / 31
Media Reporting Dataset
Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/UNOSAT and news reports on city i, in source j in month t through ln(newsijt) = αjt + θij + β1 ∗ ln(violenceit) +β2 ∗ sourcej ∗ ln(violenceit) + ǫijt
Hannes Mueller (Trinity College) Spotting Violence from Space
15 / 31
Media Reporting Dataset
Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/UNOSAT and news reports on city i, in source j in month t through ln(newsijt) = αjt + θij + β1 ∗ ln(violenceit) +β2 ∗ sourcej ∗ ln(violenceit) + ǫijt The coefficient β2 captures the fact that some sources report less.
Hannes Mueller (Trinity College) Spotting Violence from Space
15 / 31
Media Reporting Dataset
Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/UNOSAT and news reports on city i, in source j in month t through ln(newsijt) = αjt + θij + β1 ∗ ln(violenceit) +β2 ∗ sourcej ∗ ln(violenceit) + ǫijt The coefficient β2 captures the fact that some sources report less. We look at sources from Syria, Russia, US and UK.
Hannes Mueller (Trinity College) Spotting Violence from Space
15 / 31
News Reporting in Different Outlets: US and UK media vs. Russian and Syrian media (1) (2) (3) (4) VARIABLES fighting news events heavy fighting news events fighting news events fighting news events ACLED fighting events 0.178*** 0.0865*** (0.0230) (0.0156) ACLED fighting events * (Russian or Syrian news outlet)
(0.0172) (0.0114) ACLED state gains territory 0.609*** (0.0989) ACLED state gains territory * (Russian or Syrian news outlet)
(0.0797) ACLED opposition gains territory 0.419*** (0.0920) ACLED opposition gains territory * (Russian or Syrian news outlet)
(0.0495) Source/city Fixed Effects YES YES YES YES Source/time Fixed Effects YES YES NO YES Observations 89,680 89,680 89,680 89,680 R-squared 0.617 0.564 0.617 0.613 Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. All variables x are in given in ln(x+1). Hannes Mueller (Trinity College) Spotting Violence from Space
16 / 31
Our Method
Supervised learning
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Our Method
Supervised learning Supervision - show the network 0s and 1s and it learns
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Our Method
Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried:
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Our Method
Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried:
1) mark destruction in images (first alley taken)
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Our Method
Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried:
1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best)
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Our Method
Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried:
1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best)
2) Offers no pixel-level labels but a LOT of labels (several thousand)
Hannes Mueller (Trinity College) Spotting Violence from Space
17 / 31
Neural Network Architecture
We use what is called a convolutional neural network (CNN)
Hannes Mueller (Trinity College) Spotting Violence from Space
19 / 31
Neural Network Architecture
We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use.
Hannes Mueller (Trinity College) Spotting Violence from Space
19 / 31
Neural Network Architecture
We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16.
Hannes Mueller (Trinity College) Spotting Violence from Space
19 / 31
Neural Network Architecture
We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16. 16 because it has 16 layers
Hannes Mueller (Trinity College) Spotting Violence from Space
19 / 31
Neural Network Architecture
We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16. 16 because it has 16 layers The first layers are based on many convolutional filters interrupted by max pooling layers.
Hannes Mueller (Trinity College) Spotting Violence from Space
19 / 31
Idea of Convolutional Filter
Use small filter (3X3), apply it to the different parts of the image.
Hannes Mueller (Trinity College) Spotting Violence from Space
20 / 31
Idea of Convolutional Filter
Use small filter (3X3), apply it to the different parts of the image. This leads to a scoring on the right.
Hannes Mueller (Trinity College) Spotting Violence from Space
20 / 31
Idea of Max Pooling
Make local summaries (example: 2X2, stride 2)
Hannes Mueller (Trinity College) Spotting Violence from Space
21 / 31
Idea of Max Pooling
Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1)
Hannes Mueller (Trinity College) Spotting Violence from Space
21 / 31
Idea of Max Pooling
Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1) For a fantastic explanation see Brandon Rohrer’s blog.
Hannes Mueller (Trinity College) Spotting Violence from Space
21 / 31
(Modified) Very Deep Convolutional Networks for Large‐Scale Image Recognition, K. Simonyan, A. Zisserman
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images.
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Take a satellite image from the same place years before.
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Take a satellite image from the same place years before.
Sample "negatives" randomly, far away from positives.
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Take a satellite image from the same place years before.
Sample "negatives" randomly, far away from positives.
Need to restrict sampling to urban area
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Take a satellite image from the same place years before.
Sample "negatives" randomly, far away from positives.
Need to restrict sampling to urban area
This gives us 6 layers to feed into Neural Network.
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Method Based on UNOSAT/UNITAR Tags
We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives"
Take a satellite image from the same place years before.
Sample "negatives" randomly, far away from positives.
Need to restrict sampling to urban area
This gives us 6 layers to feed into Neural Network. We train and test with 5 folds and sample 20 negatives for one positive.
Hannes Mueller (Trinity College) Spotting Violence from Space
23 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities"
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times.
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change.
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change.
→ Domain transfer is very hard (our holy grail)
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change.
→ Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding.
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Project is Now in Second Gear
Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change.
→ Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding.
Second problem: imbalance explodes when scanning.
Hannes Mueller (Trinity College) Spotting Violence from Space
25 / 31
Problem in Applications
Literature is focusing on 1:1 evaluation (TPR = 0.8, FPR = 0.12) We deviate from this on purpose. Reason: reality on the ground is far from 1:1 A LOT more patches contain no destruction. Statistics of 1:1 test are misleading
Hannes Mueller (Trinity College) Spotting Violence from Space
26 / 31
Why is this a problem?
An example: True positive rate (share of 1’s predicted correctly - recall) TPR = TP TP + FN = 80% False positive rate (share of 0’s not predicted correctly) FPR = FP FP + TN = 12% Imagine you have 1 million patches Imagine of these 100 are destroyed
Hannes Mueller (Trinity College) Spotting Violence from Space
27 / 31
Why is this a problem?
12% FPR means your model produces 1Mio ∗ 0.12 FP = 120, 000 FP 80% TPR means your model produces 100 ∗ 0.8 TP = 80 TP The probability that you are right if you find destruction is... 80/120, 000 = 0.06 %!!! This means we need to get false positives (FP) down!
Hannes Mueller (Trinity College) Spotting Violence from Space
28 / 31
20000 40000 60000 80000 destroyed cells 2013m7 2014m7 2015m7 2016m7 time
Scan Results: 7 X 4 million scanned patches
Domain Transfer Across Cities
Currently: train on one city then use for monitoring. Goal: train on some cities and apply to others. Working on: train on Aleppo, then scan Homs
Hannes Mueller (Trinity College) Spotting Violence from Space
30 / 31
Summary
Clear patterns in reporting. Propose a method based on image. Scanning within city is clearly possible and generates new data. Next step: train on one city test on others If we manage this we can scan more cities.
Hannes Mueller (Trinity College) Spotting Violence from Space
31 / 31