Spotting Violence from Space

Transcription

1 Spotting Violence from Space The Detection of Housing Destruction in Syria André Gröger, Jonathan Hersh, Andrea Mantangra, Hannes Mueller, Joan Serrat Trinity College 22. February 2019 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

2 War Reporting Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

3 War Reporting War reporting is at times highly controversial. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

4 War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

5 War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

6 War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

7 War Reporting War reporting is at times highly controversial. Mother of all fake news (Wag the Dog etc.) Important aspects are not agreed upon. Reporting could be driven by politics/spin but also access to an area. It might affect public opinion. It might also affect data gathering. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

8 This Paper (once it s done!) Study of violence in Syrian cities. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

9 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

10 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

11 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

12 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

13 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

14 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

15 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

16 This Paper (once it s done!) Study of violence in Syrian cities. 1) Analyze reports on violence using massive news event databases. reporting function of source/military control obvious relative holes in reporting 2) Use satellite imagery to measure destruction. deep learning architecture trained using UNOSAT data trained network then used to "scan" cities repeatedly new, "objective" way of building violence panel data 3) Compare destruction and reports Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

17 Related Literature (Econ.) - Media Bias Measures: Groseclose and Milyo (2005, QJE), Gentzkow and Shapiro (2010, Econometrica) Political economy of bias: Leeson (2008, JEP), Prat and Stromberg (2013), Lacrinese, Puglisi and Snyder (2011, JPubE) Government reaction to media: Snyder and Strömberg (2010, JPE), Durante and Zhuravskaya (2018, JPE) Effect of mass media on population: DellaVigna and Kaplan (2007, QJE), DellaVigna et al (2014, AEJ:Applied), Yanagizawa-Drott (2014, QJE) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

18 Related Literature - Violence Data Thousands of papers based on violence data. Serious investments in data gathering: ACLED, UCDP (GED) In one way or the other all these measures are based on reports on violence. Counts often controversial. Potential problem: reporting costs are inversely proportional to intensity of conflict. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

19 Related Literature - Social Science Old debate in political science (For example, Sambanis (2004, JCR)) Davenport and Ball (2002, JCR), study Guatemalan state terror, compare newspapers and human rights violations reports, identify underreporting of violence by newspapers in rural areas Weidmann (2016, AJPS), compares media-based event and military sources, higher reporting rates of violence in cellphone-covered areas. Price, Gohdes, and Ball (2014, HRDAG), Updated Statistical Analysis of Documentation of Killings in the Syrian Arab Republic. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

20 Plan for Alternative Measure Use satellite imagery to scan for destruction. Based on this construct a panel of violence. Advantages: Potentially available with frequency of satellite updates (in quarterly) Availability is improving dramatically (monthly/weekly). Spatially very disaggregated. Study reporting news bias in space and time. Caveats: Destruction is due to a particular subset of violence Satellites are operated, i.e. they might follow news Measurement error is not trivial without human coders Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

21 Related Literature - Remote Sensing Witmer (2014) Remote Sensing of Violent Conflict: Eyes from Above. Overview. Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis. 86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2, patch classifier for 11 different places, balanced test set. 80% TPR, 12% FPR. Kahraman, Imamoglu, and Ates (2016) Disaster Damage Assessment of Buildings Using Adaptive Self-Similarity Descriptor Haiti Earthquake and balanced test sets, 75/82% TPR and 25/15% FPR. Fujita et al (2017) Damage Detection from Aerial Images via Convolutional Neural Networks: pairs of pre- and post-tsunami image patches, balanced test sets, 94%-96% accuracy, pre not necessary. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

22 Related Literature - Remote Sensing Gueguen and Hamid (2016) Toward a Generalizable Image Representation for Large-Scale Change Detection: Application to Generic Damage Analysis. 86 pairs of pre- and postevent VHR optical satellite imagery covering 4665 km2 patch classifier for 11 different places (!) balanced test set stats: 80% true positive rate, 12% false positive rate Still: UN/Amnesty International use human coders in applications. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

23 Remainder of Talk Data Sources Some Evidence on Media Bias Presentation of Method Results Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

24 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

25 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

26 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

27 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

28 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

29 Data Sources GDELT and ICEWS: scrape internet/news sources, give CAMEO scale events, give source information ACLED: coded violence events starting January 2017 Carter Centre: control data for thousands of locations from UNOSAT/UNITAR labels for six Syrian cities (up to 4 times) Google Earth archive imagery Unit of analysis is currently city but we are working on a match to control points. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

30 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

31 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

32 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

33 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

34 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

35 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

36 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

37 Media Reporting Dataset News reports from GDELT and ICEWS of fighting (and heavy weaponry) Monthly panel of more than 1100 cities Many sources: we coded their country origin (e.g. UK for the Guardian) We match this with: UNOSAT labels - destruction score Carter centre control: government, isis, opposition, kurds ACLED fighting events, change of territory Hypothesis: reporting is not consistent and function of control. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

38 Comparison of Fighting News Events and Housing Destruction (using UNOSAT labels) UNOSAT destruction measure hamra hamra homs aleppo aleppo aleppo ar-raqqa homs ar-raqqa aleppo ar-raqqa damascus hamra dar'a damascus UNOSAT destruction measure hamra hamra homs aleppo aleppo ar-raqqa homs ar-raqqa aleppo hamra dar'a ar-raqqa damascus damascus aleppo ICEWS fighting measure GDELT fighting measure

39 Fighting News Events around Government Taking Control of City (at 0)

40 Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

41 Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

42 Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

43 Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt The coeffi cient β 2 captures the fact that some sources report less. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

44 Media Reporting Dataset Strong deviation between UNOSAT destruction and news reporting ACLED tries to explicitly tackle reporting bias by cross-verification and through additional sources. We look at relationship between ACLED reports/unosat and news reports on city i, in source j in month t through ln(news ijt ) = α jt + θ ij + β 1 ln(violence it ) +β 2 source j ln(violence it ) + ɛ ijt The coeffi cient β 2 captures the fact that some sources report less. We look at sources from Syria, Russia, US and UK. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

45 News Reporting in Different Outlets: US and UK media vs. Russian and Syrian media (1) (2) (3) (4) VARIABLES fighting news events heavy fighting news events fighting news events fighting news events ACLED fighting events 0.178*** *** (0.0230) (0.0156) ACLED fighting events * (Russian or Syrian news outlet) *** *** (0.0172) (0.0114) ACLED state gains territory 0.609*** (0.0989) ACLED state gains territory * (Russian or Syrian news outlet) 0.316*** (0.0797) ACLED opposition gains territory 0.419*** (0.0920) ACLED opposition gains territory * (Russian or Syrian news outlet) * (0.0495) Source/city Fixed Effects YES YES YES YES Source/time Fixed Effects YES YES NO YES Observations 89,680 89,680 89,680 89,680 R squared Robust standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1. All variables x are in given in ln(x+1). Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

46

47

48

49

50

51

52

53

54

55 Our Method Supervised learning Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

56 Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

57 Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

58 Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

59 Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

60 Our Method Supervised learning Supervision - show the network 0s and 1s and it learns Need a set of 0s and 1s. Two ways we tried: 1) mark destruction in images (first alley taken) 2) UNOSAT/UNITAR labels (currently best) 2) Offers no pixel-level labels but a LOT of labels (several thousand) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

61 destroyed, severe damage, moderate

62 Neural Network Architecture We use what is called a convolutional neural network (CNN) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

63 Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

64 Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG16. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

65 Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG because it has 16 layers Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

66 Neural Network Architecture We use what is called a convolutional neural network (CNN) Tensorflow gives a lot of options for networks to use. We use a standard network called VGG because it has 16 layers The first layers are based on many convolutional filters interrupted by max pooling layers. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

67 Idea of Convolutional Filter Use small filter (3X3), apply it to the different parts of the image. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

68 Idea of Convolutional Filter Use small filter (3X3), apply it to the different parts of the image. This leads to a scoring on the right. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

69 Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

70 Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

71 Idea of Max Pooling Make local summaries (example: 2X2, stride 2) Network ends with fully connected layers (voting on 0 or 1) For a fantastic explanation see Brandon Rohrer s blog. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

72 (Modified) Very Deep Convolutional Networks for Large Scale Image Recognition, K. Simonyan, A. Zisserman

73 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

74 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

75 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

76 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

77 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

78 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area This gives us 6 layers to feed into Neural Network. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

79 Method Based on UNOSAT/UNITAR Tags We do change detection: use before/after images. Make a patch around tag (64 X 64) pixels: "positives" Take a satellite image from the same place years before. Sample "negatives" randomly, far away from positives. Need to restrict sampling to urban area This gives us 6 layers to feed into Neural Network. We train and test with 5 folds and sample 20 negatives for one positive. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

80

81

82

83 All positives and negatives (20 negs/pos)

84 All positives and negatives (20 negs/pos)

85 Fold 1

86 Fold 2

87

88 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

89 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

90 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

91 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

92 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

93 Project is Now in Second Gear Instead of using it on pre-defined patches use it to "scan cities" Goal: use trained classifier to scan unseen places or at least unseen times. First problem: image quality, angle and lighting change. Domain transfer is very hard (our holy grail) Might be the reason why UNOSAT/UNITAR still use hand coding. Second problem: imbalance explodes when scanning. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

94 Problem in Applications Literature is focusing on 1:1 evaluation (TPR = 0.8, FPR = 0.12) We deviate from this on purpose. Reason: reality on the ground is far from 1:1 A LOT more patches contain no destruction. Statistics of 1:1 test are misleading Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

95 Why is this a problem? An example: True positive rate (share of 1 s predicted correctly - recall) TPR = TP TP + FN = 80% False positive rate (share of 0 s not predicted correctly) FPR = Imagine you have 1 million patches Imagine of these 100 are destroyed FP FP + TN = 12% Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

96 Why is this a problem? 12% FPR means your model produces 1Mio 0.12 FP = 120, 000 FP 80% TPR means your model produces TP = 80 TP The probability that you are right if you find destruction is... 80/120, 000 = 0.06 %!!! This means we need to get false positives (FP) down! Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

97

98

99

100

101 Scan Results: 7 X 4 million scanned patches destroyed cells 2013m7 2014m7 2015m7 2016m7 time

102 Domain Transfer Across Cities Currently: train on one city then use for monitoring. Goal: train on some cities and apply to others. Working on: train on Aleppo, then scan Homs Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31

103 Summary Clear patterns in reporting. Propose a method based on image. Scanning within city is clearly possible and generates new data. Next step: train on one city test on others If we manage this we can scan more cities. Hannes Mueller (Trinity College) Spotting Violence from Space 22. February / 31