Spotting Violence from Space - PDF Free Download

Spotting Violence from Space The Detection of Housing Destruction in Syria André Gröger, Jonathan Hersh, Andrea Mantangra, Hannes Mueller, Joan Serrat Trinity College 22. February 2019 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 1 / 31

War Reporting Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 2 / 31

War Reporting War reporting is at times highly controversial. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 3 / 31

War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 3 / 31

War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 3 / 31

War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 3 / 31

War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion. It might also affect data gathering. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 3 / 31

This Paper (once it s done!) Study of violence in Syrian cities. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data 3) Compare destruction and reports Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 4 / 31

Related Literature (Econ.) - Media Bias Measures: Groseclose and Milyo (2005, QJE), Gentzkow and Shapiro (2010, Econometrica) Political economy of bias: Leeson (2008, JEP), Prat and Stromberg (2013), Lacrinese, Puglisi and Snyder (2011, JPubE) Government reaction to media: Snyder and Strömberg (2010, JPE), Durante and Zhuravskaya (2018, JPE) Effect of mass media on population: DellaVigna and Kaplan (2007, QJE), DellaVigna et al (2014, AEJ:Applied), Yanagizawa-Drott (2014, QJE) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 5 / 31

Related Literature - Violence Data Thousands of papers based on violence data. Serious investments in data gathering: ACLED, UCDP (GED) In one way or the other all these measures are based on reports on violence. Counts often controversial. Potential problem: reporting costs are inversely proportional to intensity of conflict. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 6 / 31

Related Literature - Social Science Old debate in political science (For example, Sambanis (2004, JCR)) Davenport and Ball (2002, JCR), study Guatemalan state terror, compare newspapers and human rights violations reports, identify underreporting of violence by newspapers in rural areas Weidmann (2016, AJPS), compares media-based event and military sources, higher reporting rates of violence in cellphone-covered areas. Price, Gohdes, and Ball (2014, HRDAG), Updated Statistical Analysis of Documentation of Killings in the Syrian Arab Republic. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 7 / 31

Plan for Alternative Measure Use satellite imagery to scan for destruction. Based on this construct a panel of violence. Advantages: Potentially available with frequency of satellite updates (in 2017+ quarterly) Availability is improving dramatically (monthly/weekly). Spatially very disaggregated. Study reporting news bias in space and time. Caveats: Destruction is due to a particular subset of violence Satellites are operated, i.e. they might follow news Measurement error is not trivial without human coders Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 8 / 31

Related Literature - Remote Sensing Witmer (2014) Remote Sensing of Violent Conflict: Eyes from Above. Overview. Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis. 86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2, patch classifier for 11 different places, balanced test set. 80% TPR, 12% FPR. Kahraman, Imamoglu, and Ates (2016) Disaster Damage Assessment of Buildings Using Adaptive Self-Similarity Descriptor. 2010 Haiti Earthquake and 2013. balanced test sets, 75/82% TPR and 25/15% FPR. Fujita et al (2017) Damage Detection from Aerial Images via Convolutional Neural Networks: pairs of pre- and post-tsunami image patches, balanced test sets, 94%-96% accuracy, pre not necessary. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 9 / 31

Related Literature - Remote Sensing Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis. 86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2 patch classifier for 11 different places (!) balanced test set stats: 80% true positive rate, 12% false positive rate Still: UN/Amnesty International use human coders in applications. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 10 / 31

Remainder of Talk Data Sources Some Evidence on Media Bias Presentation of Method Results Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 11 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from 2014-2017 UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery Unit of analysis is currently city but we are working on a match to control points. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 12 / 31

Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 13 / 31

Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 13 / 31

Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 13 / 31

Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 13 / 31

Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities 2011-2018 Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory Hypothesis: reporting is not consistent and function of control. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 13 / 31

Comparison of Fighting News Events and Housing Destruction (using UNOSAT labels) UNOSAT destruction measure 0 1000 2000 3000 4000 5000 hamra hamra homs aleppo aleppo aleppo ar-raqqa homs ar-raqqa aleppo ar-raqqa damascus hamra dar'a damascus UNOSAT destruction measure 0 1000 2000 3000 4000 5000 hamra hamra homs aleppo aleppo ar-raqqa homs ar-raqqa aleppo hamra dar'a ar-raqqa damascus damascus aleppo 0 2000 4000 6000 8000 10000 ICEWS fighting measure 0 500000 1000000 1500000 2000000 GDELT fighting measure

Fighting News Events around Government Taking Control of City (at 0) -.2 0.2.4-4 -3-2 -1 0 1 2 3 4

Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 15 / 31

Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 15 / 31

Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt The coeffi cient β 2 captures the fact that some sources report less. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 15 / 31

Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt The coeffi cient β 2 captures the fact that some sources report less. We look at sources from Syria, Russia, US and UK. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 15 / 31

News Reporting in Different Outlets: US and UK media vs. Russian and Syrian media (1) (2) (3) (4) VARIABLES fighting news events heavy fighting news events fighting news events fighting news events ACLED fighting events 0.178*** 0.0865*** (0.0230) (0.0156) ACLED fighting events * (Russian or Syrian news outlet) 0.0848*** 0.0527*** (0.0172) (0.0114) ACLED state gains territory 0.609*** (0.0989) ACLED state gains territory * (Russian or Syrian news outlet) 0.316*** (0.0797) ACLED opposition gains territory 0.419*** (0.0920) ACLED opposition gains territory * (Russian or Syrian news outlet) 0.0896* (0.0495) Source/city Fixed Effects YES YES YES YES Source/time Fixed Effects YES YES NO YES Observations 89,680 89,680 89,680 89,680 R squared 0.617 0.564 0.617 0.613 Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. All variables x are in given in ln(x+1). Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 16 / 31

Our Method Supervised learning Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best) 2) Offers no pixel-level labels but a LOT of labels (several thousand) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 17 / 31

destroyed, severe damage, moderate

Neural Network Architecture We use what is called a convolutional neural network (CNN) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 19 / 31

Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 19 / 31

Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16. 16 because it has 16 layers Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 19 / 31

Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16. 16 because it has 16 layers The first layers are based on many convolutional filters interrupted by max pooling layers. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 19 / 31

Idea of Convolutional Filter Use small filter (3X3), apply it to the different parts of the image. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 20 / 31

Idea of Convolutional Filter Use small filter (3X3), apply it to the different parts of the image. This leads to a scoring on the right. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 20 / 31

Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 21 / 31

Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 21 / 31

Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1) For a fantastic explanation see Brandon Rohrer s blog. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 21 / 31

(Modified) Very Deep Convolutional Networks for Large Scale Image Recognition, K. Simonyan, A. Zisserman

Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 23 / 31

Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 23 / 31

Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 23 / 31

Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area This gives us 6 layers to feed into Neural Network. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 23 / 31

Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area This gives us 6 layers to feed into Neural Network. We train and test with 5 folds and sample 20 negatives for one positive. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 23 / 31

All positives and negatives (20 negs/pos)

Fold 1

Fold 2

Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 25 / 31

Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 25 / 31

Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 25 / 31

Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 25 / 31

Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding. Second problem: imbalance explodes when scanning. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 25 / 31

Problem in Applications Literature is focusing on 1:1 evaluation (TPR = 0.8, FPR = 0.12) We deviate from this on purpose. Reason: reality on the ground is far from 1:1 A LOT more patches contain no destruction. Statistics of 1:1 test are misleading Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 26 / 31

Why is this a problem? An example: True positive rate (share of 1 s predicted correctly - recall) TPR = TP TP + FN = 80% False positive rate (share of 0 s not predicted correctly) FPR = Imagine you have 1 million patches Imagine of these 100 are destroyed FP FP + TN = 12% Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 27 / 31

Why is this a problem? 12% FPR means your model produces 1Mio 0.12 FP = 120, 000 FP 80% TPR means your model produces 100 0.8 TP = 80 TP The probability that you are right if you find destruction is... 80/120, 000 = 0.06 %!!! This means we need to get false positives (FP) down! Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 28 / 31

Scan Results: 7 X 4 million scanned patches 0 20000 40000 60000 80000 destroyed cells 2013m7 2014m7 2015m7 2016m7 time

Domain Transfer Across Cities Currently: train on one city then use for monitoring. Goal: train on some cities and apply to others. Working on: train on Aleppo, then scan Homs Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 30 / 31

Summary Clear patterns in reporting. Propose a method based on image. Scanning within city is clearly possible and generates new data. Next step: train on one city test on others If we manage this we can scan more cities. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February 2019 31 / 31