Deliverable 2

NaturalDisasterProject/Datasets/export_dataframe.csv

@@ -0,0 +1,112 @@
+BEGIN_LOCATION,STATE_ABBR,BEGIN_DATE
+ENOLA,MS,04/24/2010
+ZEIGLERVILLE,MS,04/24/2010
+RODENTOWN,AL,04/24/2010
+NORMAN,OK,05/10/2010
+(TIK)TINKER AFB,OK,05/10/2010
+BOWDLE,SD,05/22/2010
+MOLINE,OH,06/05/2010
+URBANK,MN,06/17/2010
+PORTLAND JCT,ND,06/17/2010
+HOLMES,ND,06/17/2010
+BLUFFTON,MN,06/17/2010
+WADENA,MN,06/17/2010
+MANSFIELD,MN,06/17/2010
+SIBLEY,IA,06/25/2010
+TYLER,ND,08/07/2010
+DORAN,MN,08/07/2010
+ATLANTA,LA,11/29/2010
+VARINA,IA,04/09/2011
+VIGUS,MO,04/22/2011
+LONGINO,MS,04/27/2011
+CRANE HILL,AL,04/27/2011
+PRINCE CHAPEL,MS,04/27/2011
+FEARNS SPGS,MS,04/27/2011
+BARNESVILLE,AL,04/27/2011
+GUM POND,AL,04/27/2011
+RUTH,AL,04/27/2011
+NIX,AL,04/27/2011
+SMITHVILLE,MS,04/27/2011
+MT HOPE,AL,04/27/2011
+NEW SALEM,MS,04/27/2011
+HODGE,AL,04/27/2011
+LAWNGATE,AL,04/27/2011
+IDER,AL,04/27/2011
+MARIETTA,AL,04/27/2011
+CLUTTSVILLE,AL,04/27/2011
+RALPH,AL,04/27/2011
+ARKADELPHIA,AL,04/27/2011
+FACKLER,AL,04/27/2011
+WELLER,AL,04/27/2011
+HAMLET,MS,04/27/2011
+TEN BROECK,AL,04/27/2011
+WAUTUBBEE,MS,04/27/2011
+SANIE,AL,04/27/2011
+MIDDLETON,AL,04/27/2011
+CITICO BEACH,TN,04/27/2011
+CHILHOWEE,TN,04/27/2011
+DAUS,TN,04/27/2011
+HENDON,TN,04/27/2011
+BLUE SPRING,GA,04/27/2011
+APISON,TN,04/27/2011
+PINE HILL,TN,04/27/2011
+BARNESVILLE,AL,04/27/2011
+MORGAN SPGS,TN,04/27/2011
+CENTRAL CITY,MO,05/22/2011
+CALUMET,OK,05/24/2011
+NORGE,OK,05/24/2011
+CRINER,OK,05/24/2011
+BLANCHARD,OK,05/24/2011
+PETER PENDER,AR,05/24/2011
+TIPTON ARPT,OK,11/07/2011
+CARRIERS MILLS,IL,02/29/2012
+EQUALITY,IL,02/29/2012
+FREDDRICKSBURG,IN,03/02/2012
+BLUE LICK,IN,03/02/2012
+NABB,IN,03/02/2012
+CHELSEA,IN,03/02/2012
+BRACHT,KY,03/02/2012
+KANAPOLIS RES,KS,04/14/2012
+OAK GROVE,MS,02/10/2013
+MAMBRINO,TX,05/15/2013
+BURDETT RUCKER ARPT,KS,05/18/2013
+PINK,OK,05/19/2013
+NEWCASTLE,OK,05/20/2013
+MOORE,OK,05/20/2013
+ALTONA,NE,10/04/2013
+CLIMBING HILL,IA,10/04/2013
+MARQUETTE HGTS,IL,11/17/2013
+ADDIEVILLE,IL,11/17/2013
+PALARM,AR,04/27/2014
+ZAMA,MS,04/28/2014
+STANTON,NE,06/16/2014
+STANTON,NE,06/16/2014
+WISNER,NE,06/16/2014
+ALTONA,NE,06/16/2014
+ALTONA,NE,06/16/2014
+LANE,SD,06/18/2014
+CARTHAGE,IL,04/09/2015
+TYRO,MS,12/23/2015
+CHULAHOMA,MS,12/23/2015
+ASHLAND,MS,12/23/2015
+TIPLERSVILLE,MS,12/23/2015
+POOR,TN,12/23/2015
+CHEWALLA,TN,12/23/2015
+NEW HOPE,TX,12/26/2015
+HENNEPIN,OK,05/09/2016
+SOLOMON,KS,05/25/2016
+SILVER LAKE,MO,02/28/2017
+BIG ROCK,TX,04/29/2017
+BUPREE,AL,03/03/2019
+PYRMONT,OH,05/27/2019
+LINWOOD,KS,05/28/2019
+ENSOR,TN,03/03/2020
+ENON,MS,04/12/2020
+BASSFIELD,MS,04/12/2020
+WILLIAMSBURG,MS,04/12/2020
+HEBRON,MS,04/12/2020
+MOSS,MS,04/12/2020
+ESTILL MUNI ARPT,SC,04/13/2020
+ST PAUL,MS,04/19/2020
+ASHBY,MN,07/08/2020
+DALTON,MN,07/08/2020

NaturalDisasterProject/Deliverable_1/Deliverable_Checklist.md

@@ -0,0 +1,14 @@
+# CS 506 - Final Project Repository
+
+## Project Deliverable 1
+
+- [x] Collect and pre-process a preliminary batch of data
+  - Tornado data collected from NOAA Storm Events Database
+  - Tornado data cleaned
+- [x] Perform a preliminary analysis of the data
+  - created heat-map & clhoropleth-map from tornado data for preliminary analysis
+- [x] Answer one key question
+  - Where are Stage 4 & higher tornados likely to appear? Answered through the visual maps. [heatmap](heatmap_weather.html), [clhoropleth-map](choropleth_weather.html)
+- [x] Refine project scope and list of limitations with data and potential risks of achieving project goal
+- [x] Submit a PR with the above report and modifications to original proposal
+  - [Proposal](Project_Proposal_CS506.md)

NaturalDisasterProject/Deliverable_1/Project_Proposal_CS506.md

@@ -0,0 +1,17 @@
+# Project Proposal CS506
+
+## Correlating Natural disasters to Climate change and other environmental factors
+
+
+| Syntax      | Description |
+| ----------- | ----------- |
+| Contact      | Name: Simon Angel, Abdullaah Robins, Frazier Horn, Alvaro Carrascosa|
+| Email | [email protected], [email protected], [email protected], [email protected] |
+|Cell Phone| (786) 863-0176, (470) 269-5940, (617) 840-4970, (305) 393-9302 |
+| Organization   |         |
+| Project Type   |  Data Science  |
+| Project Description  |  This project invloves collecting data about the prevalence of occurrences of certain natural disasters, specifically tornadoes and earthquakes, with the purpose of identifying climate and weather trends leading up to the occurrence. <br><br> The environmental patterns we would like to investigate are surface weather, rainfall, and general weather data to determine patterns and correlation between the various natural disasters. <br><br> Today we can find historical weather data and data on the significant natural disasters, but there isn’t much indication to the general public when these natural disasters will happen. <br><br> Our goal is to bring knowledge of trends and visualized quantification of aspects within the environment that lead up to the occurrence and to help predict when these disasters will happen. |
+| DataSets |  Climate Change: Earth Surface Temperature Data <br>https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data <br><br> Significant Earthquakes <br> https://www.kaggle.com/usgs/earthquake-database <br><br> Fires <br> https://www.kaggle.com/rtatman/188-million-us-wildfires<br><br> Glaciers <br> http://www.glims.org/download/ <br><br> Floods <br> https://www.eea.europa.eu/data-and-maps/data/european-past-floods/flood-phenomena/flood-phenomena-csv-files <br><br> Storm Events Database <br> https://www.ncdc.noaa.gov/stormevents/choosedates.jsp?statefips=-999%2CALL <br><br> Weather API <br> https://oikolab.com/api-details#api=weather&operation=weather-data         |
+| Suggested Steps | <ul><li>Clean temperature data put into Pandas dataframes.<ul><li>This could be the “n” most volatile days or the days on which the temperature changes, for example a 3% change.</li></ul></li><li>Clean the data from the non temperature data from various datasets into respective data frames.</li><li>We will make time series analytics for each distinct type of phenomenon.</li><li>Each form of data will be correlated with temperature change over time.<ul><li>We understand different phenomena can have a different relationship with temperature change, therefore they must be analyzed separately.</li></ul></li><li>Analyze the phenomena (disasters or disastrous trends) to see if there is acorrelation between them and the rise of temperatures.</li></ul>                           |
+| Questions to be Answered in Analysis       | <ul><li>When have each of these phenomena peaked in happening?</li><li>What are some consistent weather patterns leading up to the phenomena?</li><li>What weather data is unique to the phenomena?</li><li>When have the highest continuous periods of increase been?<ul><li>Are these periods related?<ul><li>Could this prove correlation?</li></ul></li></ul></li><li>Are there outlier years or periods?</li><li>Where do Tornados occur more often geographically?</li></ul>|
+|               |              |

NaturalDisasterProject/Deliverable_1/heatmap_weather.html

@@ -0,0 +1,76 @@
+<!DOCTYPE html>
+<head>    
+    <meta http-equiv="content-type" content="text/html; charset=UTF-8" />
+
+        <script>
+            L_NO_TOUCH = false;
+            L_DISABLE_3D = false;
+        </script>
+
+    <style>html, body {width: 100%;height: 100%;margin: 0;padding: 0;}</style>
+    <style>#map {position:absolute;top:0;bottom:0;right:0;left:0;}</style>
+    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/leaflet.js"></script>
+    <script src="https://code.jquery.com/jquery-1.12.4.min.js"></script>
+    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
+    <script src="https://cdnjs.cloudflare.com/ajax/libs/Leaflet.awesome-markers/2.0.2/leaflet.awesome-markers.js"></script>
+    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/[email protected]/dist/leaflet.css"/>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css"/>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap-theme.min.css"/>
+    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.6.3/css/font-awesome.min.css"/>
+    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/Leaflet.awesome-markers/2.0.2/leaflet.awesome-markers.css"/>
+    <link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/python-visualization/folium/folium/templates/leaflet.awesome.rotate.min.css"/>
+
+            <meta name="viewport" content="width=device-width,
+                initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
+            <style>
+                #map_caa5ac6661f14f718f208d08c1c906fb {
+                    position: relative;
+                    width: 100.0%;
+                    height: 100.0%;
+                    left: 0.0%;
+                    top: 0.0%;
+                }
+            </style>
+
+    <script src="https://cdn.jsdelivr.net/gh/python-visualization/folium@master/folium/templates/leaflet_heat.min.js"></script>
+</head>
+<body>    
+
+            <div class="folium-map" id="map_caa5ac6661f14f718f208d08c1c906fb" ></div>
+
+</body>
+<script>    
+
+            var map_caa5ac6661f14f718f208d08c1c906fb = L.map(
+                "map_caa5ac6661f14f718f208d08c1c906fb",
+                {
+                    center: [32.7003, -90.6783],
+                    crs: L.CRS.EPSG3857,
+                    zoom: 10,
+                    zoomControl: true,
+                    preferCanvas: false,
+                }
+            );
+
+
+
+
+
+            var tile_layer_631210ae6ca6470abb864414d414b0af = L.tileLayer(
+                "https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png",
+                {"attribution": "Data by \u0026copy; \u003ca href=\"http://openstreetmap.org\"\u003eOpenStreetMap\u003c/a\u003e, under \u003ca href=\"http://www.openstreetmap.org/copyright\"\u003eODbL\u003c/a\u003e.", "detectRetina": false, "maxNativeZoom": 18, "maxZoom": 18, "minZoom": 0, "noWrap": false, "opacity": 1, "subdomains": "abc", "tms": false}
+            ).addTo(map_caa5ac6661f14f718f208d08c1c906fb);
+
+
+            var heat_map_3be44ae715354bd3b151cf3c61afd564 = L.heatLayer(
+                [[32.7003, -90.6783], [32.9456, -90.1539], [34.2203, -85.9602], [35.1808, -97.4273], [35.377, -97.305], [45.45, -99.71], [41.5512, -83.538], [46.11, -95.425], [47.54, -97.3], [47.68, -97.32], [46.41, -95.18], [46.43, -95.15], [43.5755, -93.6404], [43.404, -95.8605], [46.11, -96.63], [46.13, -96.563], [31.7947, -92.808], [42.7293, -94.8557], [38.7364, -90.4863], [32.7992, -89.1088], [34.0757, -87.0097], [32.9017, -88.9145], [32.9262, -88.8719], [34.1043, -88.1479], [34.3054, -86.6549], [34.3528, -86.5813], [34.3109, -87.7858], [34.0455, -88.445], [34.4368, -87.5307], [34.0891, -88.36], [34.6208, -85.9814], [34.662, -87.0705], [34.82, -85.6314], [33.6448, -87.4238], [34.8017, -86.7855], [33.0606, -87.8315], [33.8668, -86.995], [34.7948, -85.909], [33.3512, -87.1975], [32.0712, -89.3201], [34.3907, -85.9784], [32.1439, -88.9154], [33.6839, -86.5234], [33.8291, -86.0526], [35.5324, -84.0504], [35.5459, -84.0274], [35.34, -85.42], [35.3969, -85.3253], [34.8744, -85.1785], [34.9868, -85.0485], [35.0371, -84.9875], [32.7545, -85.8826], [35.5535, -85.1077], [37.056, -94.5701], [35.444, -98.287], [35.008, -97.961], [34.939, -97.67], [35.189, -97.67], [35.3182, -93.9044], [34.357, -99.194], [37.6945, -88.65], [37.771, -88.3751], [38.4316, -86.1893], [38.5156, -85.8678], [38.6061, -85.5808], [38.6096, -85.57], [38.796, -84.6092], [38.523, -98.008], [31.2802, -89.4665], [32.412, -97.754], [38.1429, -99.4706], [35.308, -97.142], [35.284, -97.628], [35.303, -97.605], [42.128, -97.077], [42.3319, -96.1086], [40.622, -89.5678], [38.42, -89.45], [34.913, -92.4572], [32.932, -89.3067], [41.893, -97.33], [41.9528, -97.1369], [41.9797, -97.0198], [42.0908, -96.931], [42.127, -96.8686], [44.0372, -98.4032], [41.8886, -89.2381], [34.5955, -89.7055], [34.6131, -89.6691], [34.8117, -89.2983], [34.9428, -89.0174], [34.9958, -88.8952], [35.0471, -88.7866], [32.796, -96.5894], [34.559, -97.357], [38.9917, -97.3713], [37.7238, -90.0055], [32.357, -95.9549], [32.4571, -85.4362], [39.8243, -84.4632], [38.9689, -95.108], [36.1715, -85.6628], [31.2763, -90.1732], [31.4641, -89.7579], [31.5475, -89.6194], [31.6661, -89.4004], [31.81, -89.1863], [32.7045, -81.2899], [31.075, -89.9209], [46.09, -95.94], [46.11, -95.9]],
+                {"blur": 15, "maxZoom": 18, "minOpacity": 0.5, "radius": 25}
+            ).addTo(map_caa5ac6661f14f718f208d08c1c906fb);
+
+
+            var heat_map_b66a17aeb5714612b5e5eb5d33e771a5 = L.heatLayer(
+                [[32.7003, -90.6783], [32.9456, -90.1539], [34.2203, -85.9602], [35.1808, -97.4273], [35.377, -97.305], [45.45, -99.71], [41.5512, -83.538], [46.11, -95.425], [47.54, -97.3], [47.68, -97.32], [46.41, -95.18], [46.43, -95.15], [43.5755, -93.6404], [43.404, -95.8605], [46.11, -96.63], [46.13, -96.563], [31.7947, -92.808], [42.7293, -94.8557], [38.7364, -90.4863], [32.7992, -89.1088], [34.0757, -87.0097], [32.9017, -88.9145], [32.9262, -88.8719], [34.1043, -88.1479], [34.3054, -86.6549], [34.3528, -86.5813], [34.3109, -87.7858], [34.0455, -88.445], [34.4368, -87.5307], [34.0891, -88.36], [34.6208, -85.9814], [34.662, -87.0705], [34.82, -85.6314], [33.6448, -87.4238], [34.8017, -86.7855], [33.0606, -87.8315], [33.8668, -86.995], [34.7948, -85.909], [33.3512, -87.1975], [32.0712, -89.3201], [34.3907, -85.9784], [32.1439, -88.9154], [33.6839, -86.5234], [33.8291, -86.0526], [35.5324, -84.0504], [35.5459, -84.0274], [35.34, -85.42], [35.3969, -85.3253], [34.8744, -85.1785], [34.9868, -85.0485], [35.0371, -84.9875], [32.7545, -85.8826], [35.5535, -85.1077], [37.056, -94.5701], [35.444, -98.287], [35.008, -97.961], [34.939, -97.67], [35.189, -97.67], [35.3182, -93.9044], [34.357, -99.194], [37.6945, -88.65], [37.771, -88.3751], [38.4316, -86.1893], [38.5156, -85.8678], [38.6061, -85.5808], [38.6096, -85.57], [38.796, -84.6092], [38.523, -98.008], [31.2802, -89.4665], [32.412, -97.754], [38.1429, -99.4706], [35.308, -97.142], [35.284, -97.628], [35.303, -97.605], [42.128, -97.077], [42.3319, -96.1086], [40.622, -89.5678], [38.42, -89.45], [34.913, -92.4572], [32.932, -89.3067], [41.893, -97.33], [41.9528, -97.1369], [41.9797, -97.0198], [42.0908, -96.931], [42.127, -96.8686], [44.0372, -98.4032], [41.8886, -89.2381], [34.5955, -89.7055], [34.6131, -89.6691], [34.8117, -89.2983], [34.9428, -89.0174], [34.9958, -88.8952], [35.0471, -88.7866], [32.796, -96.5894], [34.559, -97.357], [38.9917, -97.3713], [37.7238, -90.0055], [32.357, -95.9549], [32.4571, -85.4362], [39.8243, -84.4632], [38.9689, -95.108], [36.1715, -85.6628], [31.2763, -90.1732], [31.4641, -89.7579], [31.5475, -89.6194], [31.6661, -89.4004], [31.81, -89.1863], [32.7045, -81.2899], [31.075, -89.9209], [46.09, -95.94], [46.11, -95.9]],
+                {"blur": 15, "maxZoom": 18, "minOpacity": 0.5, "radius": 25}
+            ).addTo(map_caa5ac6661f14f718f208d08c1c906fb);
+
+</script>