INFN ISS Dashboard Scraper.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "98014532",
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "import json\n",
    "import requests\n",
    "import pandas as pd\n",
    "from os.path import join"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10ce9e54",
   "metadata": {},
   "source": [
    "# INFN ISS Scraper\n",
    "\n",
    "There are no open sources of complete data for new cases, hospitalizations, intensive care occupancy and deaths for the COVID-19 related crisis in Italy. However, in an agreement with the ISS (Istituto Superiore di Sanità), the INFN (Istituto Nazionale di Fisica Nucleare) is publishing an updated dashboard with these information for each and every province of Italy.\n",
    "\n",
    "This is why I have made this little script that scrapes informations from the dashboard and saves them into a csv file.\n",
    "\n",
    "## List of units\n",
    "\n",
    "I have manually written a csv file with the list of available provinces."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7f337530",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(join('data', 'province.csv'), newline='') as f:\n",
    "    reader = csv.reader(f)\n",
    "    province = [provincia[0] for provincia in reader]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744d48e3",
   "metadata": {},
   "source": [
    "## List of datasets\n",
    "\n",
    "There are four available datasets, all already averaged out on a weekly basis:\n",
    "- `positivi`, which refers to new positive cases;\n",
    "- `ricoveri`, which refers to hospitalizations;\n",
    "- `terapia_intensiva`, which refers to the occupancy of ICUs;\n",
    "- `deceduti`, which refers to COVID-19 related deaths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "d99a265e",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_names = ['positivi', 'ricoveri', 'terapia_intensiva', 'deceduti']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c0e87c4",
   "metadata": {},
   "source": [
    "We can cycle through the datasets and scrape the data from the requests that the dashboard sends to the server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "977340ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "datasets=[]\n",
    "for dataset_name in dataset_names:\n",
    "    url=f'https://covid19.infn.it/iss/plots/iss_bydate_agrigento_{dataset_name}.div'\n",
    "    s=requests.get(url).text\n",
    "    datasets.append(json.loads(s.split('\\n')[11].strip()[:-1])[0]['y'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "7fc3c6ed",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[None,\n",
       "  None,\n",
       "  None,\n",
       "  3.2857142857142856,\n",
       "  3.142857142857143,\n",
       "  1.8571428571428572,\n",
       "  2.0,\n",
       "  2.0,\n",
       "  2.142857142857143,\n",
       "  2.5714285714285716,\n",
       "  3.4285714285714284,\n",
       "  4.428571428571429,\n",
       "  4.571428571428571,\n",
       "  4.714285714285714,\n",
       "  5.285714285714286,\n",
       "  5.428571428571429,\n",
       "  4.857142857142857,\n",
       "  5.285714285714286,\n",
       "  5.142857142857143,\n",
       "  5.0,\n",
       "  4.714285714285714,\n",
       "  4.142857142857143,\n",
       "  3.4285714285714284,\n",
       "  3.4285714285714284,\n",
       "  2.5714285714285716,\n",
       "  2.2857142857142856,\n",
       "  2.7142857142857144,\n",
       "  3.2857142857142856,\n",
       "  3.857142857142857,\n",
       "  4.571428571428571,\n",
       "  4.857142857142857,\n",
       "  4.285714285714286,\n",
       "  4.0,\n",
       "  3.7142857142857144,\n",
       "  3.2857142857142856,\n",
       "  2.2857142857142856,\n",
       "  1.7142857142857142,\n",
       "  1.2857142857142858,\n",
       "  1.5714285714285714,\n",
       "  1.4285714285714286,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.7142857142857142,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.5714285714285714,\n",
       "  1.2857142857142858,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  4.571428571428571,\n",
       "  4.857142857142857,\n",
       "  4.857142857142857,\n",
       "  4.714285714285714,\n",
       "  4.428571428571429,\n",
       "  4.428571428571429,\n",
       "  4.285714285714286,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  4.285714285714286,\n",
       "  4.142857142857143,\n",
       "  5.0,\n",
       "  5.0,\n",
       "  4.857142857142857,\n",
       "  4.571428571428571,\n",
       "  4.428571428571429,\n",
       "  1.2857142857142858,\n",
       "  1.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  8.142857142857142,\n",
       "  8.0,\n",
       "  8.428571428571429,\n",
       "  9.428571428571429,\n",
       "  9.571428571428571,\n",
       "  9.571428571428571,\n",
       "  9.571428571428571,\n",
       "  2.4285714285714284,\n",
       "  4.857142857142857,\n",
       "  6.142857142857143,\n",
       "  5.142857142857143,\n",
       "  6.857142857142857,\n",
       "  7.714285714285714,\n",
       "  8.571428571428571,\n",
       "  8.857142857142858,\n",
       "  7.0,\n",
       "  6.571428571428571,\n",
       "  6.714285714285714,\n",
       "  5.428571428571429,\n",
       "  5.0,\n",
       "  4.571428571428571,\n",
       "  5.0,\n",
       "  6.0,\n",
       "  7.142857142857143,\n",
       "  7.0,\n",
       "  7.0,\n",
       "  7.428571428571429,\n",
       "  8.0,\n",
       "  7.571428571428571,\n",
       "  6.428571428571429,\n",
       "  5.714285714285714,\n",
       "  7.857142857142857,\n",
       "  8.285714285714286,\n",
       "  8.857142857142858,\n",
       "  8.428571428571429,\n",
       "  8.142857142857142,\n",
       "  8.142857142857142,\n",
       "  7.142857142857143,\n",
       "  6.142857142857143,\n",
       "  6.428571428571429,\n",
       "  5.857142857142857,\n",
       "  5.571428571428571,\n",
       "  5.714285714285714,\n",
       "  6.285714285714286,\n",
       "  6.714285714285714,\n",
       "  6.571428571428571,\n",
       "  6.714285714285714,\n",
       "  7.428571428571429,\n",
       "  8.428571428571429,\n",
       "  10.428571428571429,\n",
       "  11.571428571428571,\n",
       "  21.142857142857142,\n",
       "  23.142857142857142,\n",
       "  25.857142857142858,\n",
       "  25.857142857142858,\n",
       "  27.428571428571427,\n",
       "  27.285714285714285,\n",
       "  28.0,\n",
       "  21.285714285714285,\n",
       "  27.285714285714285,\n",
       "  28.285714285714285,\n",
       "  31.428571428571427,\n",
       "  32.142857142857146,\n",
       "  33.285714285714285,\n",
       "  36.57142857142857,\n",
       "  42.42857142857143,\n",
       "  42.285714285714285,\n",
       "  46.42857142857143,\n",
       "  51.142857142857146,\n",
       "  55.857142857142854,\n",
       "  54.42857142857143,\n",
       "  59.42857142857143,\n",
       "  61.714285714285715,\n",
       "  64.14285714285714,\n",
       "  64.57142857142857,\n",
       "  62.42857142857143,\n",
       "  66.85714285714286,\n",
       "  74.85714285714286,\n",
       "  72.71428571428571,\n",
       "  78.42857142857143,\n",
       "  77.85714285714286,\n",
       "  85.14285714285714,\n",
       "  88.28571428571429,\n",
       "  80.71428571428571,\n",
       "  77.28571428571429,\n",
       "  82.0,\n",
       "  77.14285714285714,\n",
       "  77.85714285714286,\n",
       "  83.0,\n",
       "  86.14285714285714,\n",
       "  85.71428571428571,\n",
       "  84.42857142857143,\n",
       "  81.85714285714286,\n",
       "  79.42857142857143,\n",
       "  74.57142857142857,\n",
       "  61.42857142857143,\n",
       "  52.0,\n",
       "  50.714285714285715,\n",
       "  48.0,\n",
       "  44.0,\n",
       "  39.285714285714285,\n",
       "  38.857142857142854,\n",
       "  36.0,\n",
       "  37.42857142857143,\n",
       "  36.285714285714285,\n",
       "  36.857142857142854,\n",
       "  34.714285714285715,\n",
       "  31.428571428571427,\n",
       "  30.142857142857142,\n",
       "  28.571428571428573,\n",
       "  29.285714285714285,\n",
       "  30.0,\n",
       "  31.428571428571427,\n",
       "  32.57142857142857,\n",
       "  33.714285714285715,\n",
       "  32.142857142857146,\n",
       "  30.714285714285715,\n",
       "  26.714285714285715,\n",
       "  25.857142857142858,\n",
       "  22.285714285714285,\n",
       "  20.571428571428573,\n",
       "  23.285714285714285,\n",
       "  25.428571428571427,\n",
       "  28.0,\n",
       "  26.0,\n",
       "  25.571428571428573,\n",
       "  25.857142857142858,\n",
       "  26.285714285714285,\n",
       "  27.285714285714285,\n",
       "  32.0,\n",
       "  32.714285714285715,\n",
       "  35.285714285714285,\n",
       "  39.142857142857146,\n",
       "  41.142857142857146,\n",
       "  45.714285714285715,\n",
       "  50.0,\n",
       "  45.42857142857143,\n",
       "  52.42857142857143,\n",
       "  59.57142857142857,\n",
       "  60.0,\n",
       "  61.714285714285715,\n",
       "  65.28571428571429,\n",
       "  65.28571428571429,\n",
       "  65.85714285714286,\n",
       "  59.57142857142857,\n",
       "  54.857142857142854,\n",
       "  53.714285714285715,\n",
       "  52.714285714285715,\n",
       "  47.0,\n",
       "  39.714285714285715,\n",
       "  39.285714285714285,\n",
       "  37.714285714285715,\n",
       "  39.857142857142854,\n",
       "  38.57142857142857,\n",
       "  36.42857142857143,\n",
       "  37.0,\n",
       "  41.714285714285715,\n",
       "  41.285714285714285,\n",
       "  40.42857142857143,\n",
       "  38.714285714285715,\n",
       "  39.0,\n",
       "  41.285714285714285,\n",
       "  38.57142857142857,\n",
       "  35.142857142857146,\n",
       "  33.57142857142857,\n",
       "  33.714285714285715,\n",
       "  33.285714285714285,\n",
       "  33.42857142857143,\n",
       "  32.142857142857146,\n",
       "  35.42857142857143,\n",
       "  34.42857142857143,\n",
       "  34.285714285714285,\n",
       "  33.142857142857146,\n",
       "  35.0,\n",
       "  33.285714285714285,\n",
       "  32.42857142857143,\n",
       "  29.714285714285715,\n",
       "  27.285714285714285,\n",
       "  27.142857142857142,\n",
       "  27.285714285714285,\n",
       "  22.714285714285715,\n",
       "  23.0,\n",
       "  24.571428571428573,\n",
       "  23.428571428571427,\n",
       "  30.142857142857142,\n",
       "  29.0,\n",
       "  28.714285714285715,\n",
       "  31.142857142857142,\n",
       "  31.571428571428573,\n",
       "  30.0,\n",
       "  31.714285714285715,\n",
       "  29.571428571428573,\n",
       "  32.142857142857146,\n",
       "  39.142857142857146,\n",
       "  44.285714285714285,\n",
       "  46.285714285714285,\n",
       "  48.714285714285715,\n",
       "  52.714285714285715,\n",
       "  54.857142857142854,\n",
       "  56.42857142857143,\n",
       "  55.857142857142854,\n",
       "  53.714285714285715,\n",
       "  53.57142857142857,\n",
       "  51.285714285714285,\n",
       "  49.714285714285715,\n",
       "  52.857142857142854,\n",
       "  57.42857142857143,\n",
       "  65.57142857142857,\n",
       "  67.57142857142857,\n",
       "  68.42857142857143,\n",
       "  76.0,\n",
       "  81.85714285714286,\n",
       "  91.42857142857143,\n",
       "  96.28571428571429,\n",
       "  94.71428571428571,\n",
       "  102.71428571428571,\n",
       "  106.28571428571429,\n",
       "  106.0,\n",
       "  103.14285714285714,\n",
       "  98.85714285714286,\n",
       "  101.57142857142857,\n",
       "  102.71428571428571,\n",
       "  101.57142857142857,\n",
       "  103.71428571428571,\n",
       "  97.71428571428571,\n",
       "  94.42857142857143,\n",
       "  94.0,\n",
       "  99.28571428571429,\n",
       "  94.71428571428571,\n",
       "  94.57142857142857,\n",
       "  96.85714285714286,\n",
       "  100.14285714285714,\n",
       "  106.28571428571429,\n",
       "  106.0,\n",
       "  100.28571428571429,\n",
       "  98.28571428571429,\n",
       "  94.14285714285714,\n",
       "  86.71428571428571,\n",
       "  86.57142857142857,\n",
       "  81.42857142857143,\n",
       "  78.0,\n",
       "  69.14285714285714,\n",
       "  68.85714285714286,\n",
       "  66.57142857142857,\n",
       "  67.85714285714286,\n",
       "  66.0,\n",
       "  67.85714285714286,\n",
       "  64.14285714285714,\n",
       "  67.42857142857143,\n",
       "  65.42857142857143,\n",
       "  62.285714285714285,\n",
       "  59.57142857142857,\n",
       "  60.857142857142854,\n",
       "  55.42857142857143,\n",
       "  55.285714285714285,\n",
       "  49.285714285714285,\n",
       "  46.57142857142857,\n",
       "  47.285714285714285,\n",
       "  47.857142857142854,\n",
       "  45.285714285714285,\n",
       "  47.142857142857146,\n",
       "  46.857142857142854,\n",
       "  45.285714285714285,\n",
       "  44.285714285714285,\n",
       "  41.142857142857146,\n",
       "  38.57142857142857,\n",
       "  37.0,\n",
       "  35.857142857142854,\n",
       "  32.42857142857143,\n",
       "  34.0,\n",
       "  30.714285714285715,\n",
       "  33.142857142857146,\n",
       "  31.857142857142858,\n",
       "  33.57142857142857,\n",
       "  32.714285714285715,\n",
       "  32.714285714285715,\n",
       "  30.857142857142858,\n",
       "  34.857142857142854,\n",
       "  33.0,\n",
       "  35.42857142857143,\n",
       "  34.714285714285715,\n",
       "  33.285714285714285,\n",
       "  31.857142857142858,\n",
       "  27.428571428571427,\n",
       "  25.571428571428573,\n",
       "  24.285714285714285,\n",
       "  23.0,\n",
       "  21.285714285714285,\n",
       "  20.857142857142858,\n",
       "  19.571428571428573,\n",
       "  22.285714285714285,\n",
       "  24.285714285714285,\n",
       "  24.142857142857142,\n",
       "  22.142857142857142,\n",
       "  27.0,\n",
       "  26.428571428571427,\n",
       "  25.285714285714285,\n",
       "  26.857142857142858,\n",
       "  23.571428571428573,\n",
       "  22.285714285714285,\n",
       "  22.714285714285715,\n",
       "  18.571428571428573,\n",
       "  17.428571428571427,\n",
       "  20.0,\n",
       "  15.714285714285714,\n",
       "  14.714285714285714,\n",
       "  15.571428571428571],\n",
       " [None,\n",
       "  None,\n",
       "  None,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  1.0,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.7142857142857142,\n",
       "  1.8571428571428572,\n",
       "  1.8571428571428572,\n",
       "  2.0,\n",
       "  2.0,\n",
       "  1.8571428571428572,\n",
       "  1.7142857142857142,\n",
       "  1.4285714285714286,\n",
       "  1.1428571428571428,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  1.0,\n",
       "  1.5714285714285714,\n",
       "  1.5714285714285714,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  3.142857142857143,\n",
       "  3.857142857142857,\n",
       "  3.5714285714285716,\n",
       "  3.2857142857142856,\n",
       "  3.2857142857142856,\n",
       "  3.857142857142857,\n",
       "  3.7142857142857144,\n",
       "  2.7142857142857144,\n",
       "  2.142857142857143,\n",
       "  3.142857142857143,\n",
       "  3.2857142857142856,\n",
       "  4.0,\n",
       "  3.4285714285714284,\n",
       "  4.285714285714286,\n",
       "  4.571428571428571,\n",
       "  4.857142857142857,\n",
       "  4.142857142857143,\n",
       "  4.428571428571429,\n",
       "  4.285714285714286,\n",
       "  4.285714285714286,\n",
       "  3.5714285714285716,\n",
       "  4.142857142857143,\n",
       "  4.0,\n",
       "  4.714285714285714,\n",
       "  4.714285714285714,\n",
       "  6.0,\n",
       "  6.285714285714286,\n",
       "  7.285714285714286,\n",
       "  6.571428571428571,\n",
       "  7.0,\n",
       "  7.0,\n",
       "  7.0,\n",
       "  5.714285714285714,\n",
       "  5.857142857142857,\n",
       "  5.0,\n",
       "  5.857142857142857,\n",
       "  5.428571428571429,\n",
       "  4.714285714285714,\n",
       "  5.857142857142857,\n",
       "  6.285714285714286,\n",
       "  5.714285714285714,\n",
       "  6.142857142857143,\n",
       "  5.714285714285714,\n",
       "  5.714285714285714,\n",
       "  6.0,\n",
       "  4.714285714285714,\n",
       "  4.0,\n",
       "  4.571428571428571,\n",
       "  4.285714285714286,\n",
       "  3.5714285714285716,\n",
       "  3.5714285714285716,\n",
       "  3.142857142857143,\n",
       "  2.7142857142857144,\n",
       "  2.4285714285714284,\n",
       "  2.7142857142857144,\n",
       "  2.2857142857142856,\n",
       "  2.0,\n",
       "  1.8571428571428572,\n",
       "  2.142857142857143,\n",
       "  2.0,\n",
       "  2.2857142857142856,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  1.0,\n",
       "  2.0,\n",
       "  1.8571428571428572,\n",
       "  2.0,\n",
       "  1.8571428571428572,\n",
       "  1.7142857142857142,\n",
       "  2.0,\n",
       "  2.857142857142857,\n",
       "  2.0,\n",
       "  2.142857142857143,\n",
       "  2.142857142857143,\n",
       "  2.7142857142857144,\n",
       "  3.2857142857142856,\n",
       "  3.4285714285714284,\n",
       "  3.142857142857143,\n",
       "  3.5714285714285716,\n",
       "  3.5714285714285716,\n",
       "  4.142857142857143,\n",
       "  4.142857142857143,\n",
       "  3.857142857142857,\n",
       "  4.0,\n",
       "  4.571428571428571,\n",
       "  4.714285714285714,\n",
       "  5.0,\n",
       "  4.571428571428571,\n",
       "  4.142857142857143,\n",
       "  4.571428571428571,\n",
       "  4.0,\n",
       "  3.2857142857142856,\n",
       "  3.5714285714285716,\n",
       "  3.2857142857142856,\n",
       "  3.4285714285714284,\n",
       "  3.5714285714285716,\n",
       "  3.142857142857143,\n",
       "  3.2857142857142856,\n",
       "  3.2857142857142856,\n",
       "  2.4285714285714284,\n",
       "  2.7142857142857144,\n",
       "  2.5714285714285716,\n",
       "  2.2857142857142856,\n",
       "  2.4285714285714284,\n",
       "  2.4285714285714284,\n",
       "  2.142857142857143,\n",
       "  2.4285714285714284,\n",
       "  2.0,\n",
       "  2.142857142857143,\n",
       "  2.4285714285714284,\n",
       "  2.142857142857143,\n",
       "  2.4285714285714284,\n",
       "  2.5714285714285716,\n",
       "  2.4285714285714284,\n",
       "  2.4285714285714284,\n",
       "  2.2857142857142856,\n",
       "  3.142857142857143,\n",
       "  3.0,\n",
       "  3.0,\n",
       "  3.142857142857143,\n",
       "  3.2857142857142856,\n",
       "  3.142857142857143,\n",
       "  2.857142857142857,\n",
       "  1.8571428571428572,\n",
       "  2.0,\n",
       "  1.5714285714285714,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  1.5714285714285714,\n",
       "  2.142857142857143,\n",
       "  2.0,\n",
       "  2.5714285714285716,\n",
       "  2.857142857142857,\n",
       "  3.0,\n",
       "  3.0,\n",
       "  2.2857142857142856,\n",
       "  1.8571428571428572,\n",
       "  2.2857142857142856,\n",
       "  2.142857142857143,\n",
       "  2.0,\n",
       "  2.857142857142857,\n",
       "  2.857142857142857,\n",
       "  3.5714285714285716,\n",
       "  3.7142857142857144,\n",
       "  3.7142857142857144,\n",
       "  3.2857142857142856,\n",
       "  3.4285714285714284,\n",
       "  3.2857142857142856,\n",
       "  3.857142857142857,\n",
       "  4.428571428571429,\n",
       "  4.714285714285714,\n",
       "  5.285714285714286,\n",
       "  6.142857142857143,\n",
       "  6.285714285714286,\n",
       "  6.285714285714286,\n",
       "  5.857142857142857,\n",
       "  4.571428571428571,\n",
       "  4.571428571428571,\n",
       "  4.714285714285714,\n",
       "  4.714285714285714,\n",
       "  5.142857142857143,\n",
       "  5.0,\n",
       "  5.285714285714286,\n",
       "  5.857142857142857,\n",
       "  5.571428571428571,\n",
       "  4.857142857142857,\n",
       "  5.0,\n",
       "  4.857142857142857,\n",
       "  4.714285714285714,\n",
       "  4.714285714285714,\n",
       "  5.714285714285714,\n",
       "  6.142857142857143,\n",
       "  6.571428571428571,\n",
       "  6.428571428571429,\n",
       "  6.857142857142857,\n",
       "  7.142857142857143,\n",
       "  7.571428571428571,\n",
       "  6.571428571428571,\n",
       "  6.714285714285714,\n",
       "  7.0,\n",
       "  7.0,\n",
       "  6.714285714285714,\n",
       "  7.0,\n",
       "  6.285714285714286,\n",
       "  6.428571428571429,\n",
       "  7.0,\n",
       "  6.428571428571429,\n",
       "  5.571428571428571,\n",
       "  4.857142857142857,\n",
       "  4.0,\n",
       "  4.714285714285714,\n",
       "  4.857142857142857,\n",
       "  3.5714285714285716,\n",
       "  3.7142857142857144,\n",
       "  4.142857142857143,\n",
       "  4.428571428571429,\n",
       "  4.714285714285714,\n",
       "  4.428571428571429,\n",
       "  3.857142857142857,\n",
       "  3.857142857142857,\n",
       "  3.2857142857142856,\n",
       "  3.4285714285714284,\n",
       "  3.2857142857142856,\n",
       "  3.2857142857142856,\n",
       "  2.857142857142857,\n",
       "  3.0,\n",
       "  3.0,\n",
       "  4.0,\n",
       "  3.5714285714285716,\n",
       "  3.5714285714285716,\n",
       "  3.142857142857143,\n",
       "  3.0,\n",
       "  2.5714285714285716,\n",
       "  3.142857142857143,\n",
       "  3.0,\n",
       "  3.142857142857143,\n",
       "  2.857142857142857,\n",
       "  2.857142857142857,\n",
       "  2.857142857142857,\n",
       "  3.142857142857143,\n",
       "  2.4285714285714284,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.8571428571428572,\n",
       "  1.8571428571428572,\n",
       "  1.7142857142857142,\n",
       "  1.2857142857142858,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.4285714285714286,\n",
       "  1.7142857142857142,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143],\n",
       " [None,\n",
       "  None,\n",
       "  None,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  1.1428571428571428,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.4285714285714286,\n",
       "  1.4285714285714286,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.5714285714285714,\n",
       "  1.5714285714285714,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.2857142857142857],\n",
       " [None,\n",
       "  None,\n",
       "  None,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.14285714285714285,\n",
       "  0.0,\n",
       "  0.14285714285714285,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.2857142857142858,\n",
       "  1.5714285714285714,\n",
       "  2.2857142857142856,\n",
       "  2.4285714285714284,\n",
       "  2.5714285714285716,\n",
       "  2.7142857142857144,\n",
       "  2.7142857142857144,\n",
       "  2.5714285714285716,\n",
       "  2.2857142857142856,\n",
       "  1.7142857142857142,\n",
       "  1.8571428571428572,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  1.4285714285714286,\n",
       "  1.8571428571428572,\n",
       "  2.2857142857142856,\n",
       "  2.5714285714285716,\n",
       "  2.5714285714285716,\n",
       "  2.4285714285714284,\n",
       "  2.7142857142857144,\n",
       "  3.142857142857143,\n",
       "  2.857142857142857,\n",
       "  2.5714285714285716,\n",
       "  2.4285714285714284,\n",
       "  2.0,\n",
       "  2.0,\n",
       "  1.8571428571428572,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  1.5714285714285714,\n",
       "  1.4285714285714286,\n",
       "  1.5714285714285714,\n",
       "  1.5714285714285714,\n",
       "  1.4285714285714286,\n",
       "  1.8571428571428572,\n",
       "  1.7142857142857142,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.0,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  1.0,\n",
       "  1.4285714285714286,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  1.7142857142857142,\n",
       "  1.4285714285714286,\n",
       "  1.4285714285714286,\n",
       "  1.0,\n",
       "  0.8571428571428571,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.1428571428571428,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.4285714285714286,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  1.4285714285714286,\n",
       "  1.8571428571428572,\n",
       "  2.0,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  2.0,\n",
       "  2.0,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  1.7142857142857142,\n",
       "  1.5714285714285714,\n",
       "  1.5714285714285714,\n",
       "  1.5714285714285714,\n",
       "  1.1428571428571428,\n",
       "  1.2857142857142858,\n",
       "  1.4285714285714286,\n",
       "  1.2857142857142858,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.42857142857142855,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.42857142857142855,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  0.8571428571428571,\n",
       "  1.0,\n",
       "  1.2857142857142858,\n",
       "  1.2857142857142858,\n",
       "  0.8571428571428571,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.7142857142857143,\n",
       "  0.5714285714285714,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.2857142857142857,\n",
       "  0.14285714285714285,\n",
       "  0.42857142857142855,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.5714285714285714,\n",
       "  0.14285714285714285]]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "0fdbcd7f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['2020-03-20T00:00:00',\n",
       " '2020-03-21T00:00:00',\n",
       " '2020-03-22T00:00:00',\n",
       " '2020-03-23T00:00:00',\n",
       " '2020-03-24T00:00:00',\n",
       " '2020-03-25T00:00:00',\n",
       " '2020-03-26T00:00:00',\n",
       " '2020-03-27T00:00:00',\n",
       " '2020-03-28T00:00:00',\n",
       " '2020-03-29T00:00:00',\n",
       " '2020-03-30T00:00:00',\n",
       " '2020-03-31T00:00:00',\n",
       " '2020-04-01T00:00:00',\n",
       " '2020-04-02T00:00:00',\n",
       " '2020-04-03T00:00:00',\n",
       " '2020-04-04T00:00:00',\n",
       " '2020-04-05T00:00:00',\n",
       " '2020-04-06T00:00:00',\n",
       " '2020-04-07T00:00:00',\n",
       " '2020-04-08T00:00:00',\n",
       " '2020-04-09T00:00:00',\n",
       " '2020-04-10T00:00:00',\n",
       " '2020-04-11T00:00:00',\n",
       " '2020-04-12T00:00:00',\n",
       " '2020-04-13T00:00:00',\n",
       " '2020-04-14T00:00:00',\n",
       " '2020-04-15T00:00:00',\n",
       " '2020-04-16T00:00:00',\n",
       " '2020-04-17T00:00:00',\n",
       " '2020-04-18T00:00:00',\n",
       " '2020-04-19T00:00:00',\n",
       " '2020-04-20T00:00:00',\n",
       " '2020-04-21T00:00:00',\n",
       " '2020-04-22T00:00:00',\n",
       " '2020-04-23T00:00:00',\n",
       " '2020-04-24T00:00:00',\n",
       " '2020-04-25T00:00:00',\n",
       " '2020-04-26T00:00:00',\n",
       " '2020-04-27T00:00:00',\n",
       " '2020-04-28T00:00:00',\n",
       " '2020-04-29T00:00:00',\n",
       " '2020-04-30T00:00:00',\n",
       " '2020-05-01T00:00:00',\n",
       " '2020-05-02T00:00:00',\n",
       " '2020-05-03T00:00:00',\n",
       " '2020-05-04T00:00:00',\n",
       " '2020-05-05T00:00:00',\n",
       " '2020-05-06T00:00:00',\n",
       " '2020-05-07T00:00:00',\n",
       " '2020-05-08T00:00:00',\n",
       " '2020-05-09T00:00:00',\n",
       " '2020-05-10T00:00:00',\n",
       " '2020-05-11T00:00:00',\n",
       " '2020-05-12T00:00:00',\n",
       " '2020-05-13T00:00:00',\n",
       " '2020-05-14T00:00:00',\n",
       " '2020-05-15T00:00:00',\n",
       " '2020-05-16T00:00:00',\n",
       " '2020-05-17T00:00:00',\n",
       " '2020-05-18T00:00:00',\n",
       " '2020-05-19T00:00:00',\n",
       " '2020-05-20T00:00:00',\n",
       " '2020-05-21T00:00:00',\n",
       " '2020-05-22T00:00:00',\n",
       " '2020-05-23T00:00:00',\n",
       " '2020-05-24T00:00:00',\n",
       " '2020-05-25T00:00:00',\n",
       " '2020-05-26T00:00:00',\n",
       " '2020-05-27T00:00:00',\n",
       " '2020-05-28T00:00:00',\n",
       " '2020-05-29T00:00:00',\n",
       " '2020-05-30T00:00:00',\n",
       " '2020-05-31T00:00:00',\n",
       " '2020-06-01T00:00:00',\n",
       " '2020-06-02T00:00:00',\n",
       " '2020-06-03T00:00:00',\n",
       " '2020-06-04T00:00:00',\n",
       " '2020-06-05T00:00:00',\n",
       " '2020-06-06T00:00:00',\n",
       " '2020-06-07T00:00:00',\n",
       " '2020-06-08T00:00:00',\n",
       " '2020-06-09T00:00:00',\n",
       " '2020-06-10T00:00:00',\n",
       " '2020-06-11T00:00:00',\n",
       " '2020-06-12T00:00:00',\n",
       " '2020-06-13T00:00:00',\n",
       " '2020-06-14T00:00:00',\n",
       " '2020-06-15T00:00:00',\n",
       " '2020-06-16T00:00:00',\n",
       " '2020-06-17T00:00:00',\n",
       " '2020-06-18T00:00:00',\n",
       " '2020-06-19T00:00:00',\n",
       " '2020-06-20T00:00:00',\n",
       " '2020-06-21T00:00:00',\n",
       " '2020-06-22T00:00:00',\n",
       " '2020-06-23T00:00:00',\n",
       " '2020-06-24T00:00:00',\n",
       " '2020-06-25T00:00:00',\n",
       " '2020-06-26T00:00:00',\n",
       " '2020-06-27T00:00:00',\n",
       " '2020-06-28T00:00:00',\n",
       " '2020-06-29T00:00:00',\n",
       " '2020-06-30T00:00:00',\n",
       " '2020-07-01T00:00:00',\n",
       " '2020-07-02T00:00:00',\n",
       " '2020-07-03T00:00:00',\n",
       " '2020-07-04T00:00:00',\n",
       " '2020-07-05T00:00:00',\n",
       " '2020-07-06T00:00:00',\n",
       " '2020-07-07T00:00:00',\n",
       " '2020-07-08T00:00:00',\n",
       " '2020-07-09T00:00:00',\n",
       " '2020-07-10T00:00:00',\n",
       " '2020-07-11T00:00:00',\n",
       " '2020-07-12T00:00:00',\n",
       " '2020-07-13T00:00:00',\n",
       " '2020-07-14T00:00:00',\n",
       " '2020-07-15T00:00:00',\n",
       " '2020-07-16T00:00:00',\n",
       " '2020-07-17T00:00:00',\n",
       " '2020-07-18T00:00:00',\n",
       " '2020-07-19T00:00:00',\n",
       " '2020-07-20T00:00:00',\n",
       " '2020-07-21T00:00:00',\n",
       " '2020-07-22T00:00:00',\n",
       " '2020-07-23T00:00:00',\n",
       " '2020-07-24T00:00:00',\n",
       " '2020-07-25T00:00:00',\n",
       " '2020-07-26T00:00:00',\n",
       " '2020-07-27T00:00:00',\n",
       " '2020-07-28T00:00:00',\n",
       " '2020-07-29T00:00:00',\n",
       " '2020-07-30T00:00:00',\n",
       " '2020-07-31T00:00:00',\n",
       " '2020-08-01T00:00:00',\n",
       " '2020-08-02T00:00:00',\n",
       " '2020-08-03T00:00:00',\n",
       " '2020-08-04T00:00:00',\n",
       " '2020-08-05T00:00:00',\n",
       " '2020-08-06T00:00:00',\n",
       " '2020-08-07T00:00:00',\n",
       " '2020-08-08T00:00:00',\n",
       " '2020-08-09T00:00:00',\n",
       " '2020-08-10T00:00:00',\n",
       " '2020-08-11T00:00:00',\n",
       " '2020-08-12T00:00:00',\n",
       " '2020-08-13T00:00:00',\n",
       " '2020-08-14T00:00:00',\n",
       " '2020-08-15T00:00:00',\n",
       " '2020-08-16T00:00:00',\n",
       " '2020-08-17T00:00:00',\n",
       " '2020-08-18T00:00:00',\n",
       " '2020-08-19T00:00:00',\n",
       " '2020-08-20T00:00:00',\n",
       " '2020-08-21T00:00:00',\n",
       " '2020-08-22T00:00:00',\n",
       " '2020-08-23T00:00:00',\n",
       " '2020-08-24T00:00:00',\n",
       " '2020-08-25T00:00:00',\n",
       " '2020-08-26T00:00:00',\n",
       " '2020-08-27T00:00:00',\n",
       " '2020-08-28T00:00:00',\n",
       " '2020-08-29T00:00:00',\n",
       " '2020-08-30T00:00:00',\n",
       " '2020-08-31T00:00:00',\n",
       " '2020-09-01T00:00:00',\n",
       " '2020-09-02T00:00:00',\n",
       " '2020-09-03T00:00:00',\n",
       " '2020-09-04T00:00:00',\n",
       " '2020-09-05T00:00:00',\n",
       " '2020-09-06T00:00:00',\n",
       " '2020-09-07T00:00:00',\n",
       " '2020-09-08T00:00:00',\n",
       " '2020-09-09T00:00:00',\n",
       " '2020-09-10T00:00:00',\n",
       " '2020-09-11T00:00:00',\n",
       " '2020-09-12T00:00:00',\n",
       " '2020-09-13T00:00:00',\n",
       " '2020-09-14T00:00:00',\n",
       " '2020-09-15T00:00:00',\n",
       " '2020-09-16T00:00:00',\n",
       " '2020-09-17T00:00:00',\n",
       " '2020-09-18T00:00:00',\n",
       " '2020-09-19T00:00:00',\n",
       " '2020-09-20T00:00:00',\n",
       " '2020-09-21T00:00:00',\n",
       " '2020-09-22T00:00:00',\n",
       " '2020-09-23T00:00:00',\n",
       " '2020-09-24T00:00:00',\n",
       " '2020-09-25T00:00:00',\n",
       " '2020-09-26T00:00:00',\n",
       " '2020-09-27T00:00:00',\n",
       " '2020-09-28T00:00:00',\n",
       " '2020-09-29T00:00:00',\n",
       " '2020-09-30T00:00:00',\n",
       " '2020-10-01T00:00:00',\n",
       " '2020-10-02T00:00:00',\n",
       " '2020-10-03T00:00:00',\n",
       " '2020-10-04T00:00:00',\n",
       " '2020-10-05T00:00:00',\n",
       " '2020-10-06T00:00:00',\n",
       " '2020-10-07T00:00:00',\n",
       " '2020-10-08T00:00:00',\n",
       " '2020-10-09T00:00:00',\n",
       " '2020-10-10T00:00:00',\n",
       " '2020-10-11T00:00:00',\n",
       " '2020-10-12T00:00:00',\n",
       " '2020-10-13T00:00:00',\n",
       " '2020-10-14T00:00:00',\n",
       " '2020-10-15T00:00:00',\n",
       " '2020-10-16T00:00:00',\n",
       " '2020-10-17T00:00:00',\n",
       " '2020-10-18T00:00:00',\n",
       " '2020-10-19T00:00:00',\n",
       " '2020-10-20T00:00:00',\n",
       " '2020-10-21T00:00:00',\n",
       " '2020-10-22T00:00:00',\n",
       " '2020-10-23T00:00:00',\n",
       " '2020-10-24T00:00:00',\n",
       " '2020-10-25T00:00:00',\n",
       " '2020-10-26T00:00:00',\n",
       " '2020-10-27T00:00:00',\n",
       " '2020-10-28T00:00:00',\n",
       " '2020-10-29T00:00:00',\n",
       " '2020-10-30T00:00:00',\n",
       " '2020-10-31T00:00:00',\n",
       " '2020-11-01T00:00:00',\n",
       " '2020-11-02T00:00:00',\n",
       " '2020-11-03T00:00:00',\n",
       " '2020-11-04T00:00:00',\n",
       " '2020-11-05T00:00:00',\n",
       " '2020-11-06T00:00:00',\n",
       " '2020-11-07T00:00:00',\n",
       " '2020-11-08T00:00:00',\n",
       " '2020-11-09T00:00:00',\n",
       " '2020-11-10T00:00:00',\n",
       " '2020-11-11T00:00:00',\n",
       " '2020-11-12T00:00:00',\n",
       " '2020-11-13T00:00:00',\n",
       " '2020-11-14T00:00:00',\n",
       " '2020-11-15T00:00:00',\n",
       " '2020-11-16T00:00:00',\n",
       " '2020-11-17T00:00:00',\n",
       " '2020-11-18T00:00:00',\n",
       " '2020-11-19T00:00:00',\n",
       " '2020-11-20T00:00:00',\n",
       " '2020-11-21T00:00:00',\n",
       " '2020-11-22T00:00:00',\n",
       " '2020-11-23T00:00:00',\n",
       " '2020-11-24T00:00:00',\n",
       " '2020-11-25T00:00:00',\n",
       " '2020-11-26T00:00:00',\n",
       " '2020-11-27T00:00:00',\n",
       " '2020-11-28T00:00:00',\n",
       " '2020-11-29T00:00:00',\n",
       " '2020-11-30T00:00:00',\n",
       " '2020-12-01T00:00:00',\n",
       " '2020-12-02T00:00:00',\n",
       " '2020-12-03T00:00:00',\n",
       " '2020-12-04T00:00:00',\n",
       " '2020-12-05T00:00:00',\n",
       " '2020-12-06T00:00:00',\n",
       " '2020-12-07T00:00:00',\n",
       " '2020-12-08T00:00:00',\n",
       " '2020-12-09T00:00:00',\n",
       " '2020-12-10T00:00:00',\n",
       " '2020-12-11T00:00:00',\n",
       " '2020-12-12T00:00:00',\n",
       " '2020-12-13T00:00:00',\n",
       " '2020-12-14T00:00:00',\n",
       " '2020-12-15T00:00:00',\n",
       " '2020-12-16T00:00:00',\n",
       " '2020-12-17T00:00:00',\n",
       " '2020-12-18T00:00:00',\n",
       " '2020-12-19T00:00:00',\n",
       " '2020-12-20T00:00:00',\n",
       " '2020-12-21T00:00:00',\n",
       " '2020-12-22T00:00:00',\n",
       " '2020-12-23T00:00:00',\n",
       " '2020-12-24T00:00:00',\n",
       " '2020-12-25T00:00:00',\n",
       " '2020-12-26T00:00:00',\n",
       " '2020-12-27T00:00:00',\n",
       " '2020-12-28T00:00:00',\n",
       " '2020-12-29T00:00:00',\n",
       " '2020-12-30T00:00:00',\n",
       " '2020-12-31T00:00:00',\n",
       " '2021-01-01T00:00:00',\n",
       " '2021-01-02T00:00:00',\n",
       " '2021-01-03T00:00:00',\n",
       " '2021-01-04T00:00:00',\n",
       " '2021-01-05T00:00:00',\n",
       " '2021-01-06T00:00:00',\n",
       " '2021-01-07T00:00:00',\n",
       " '2021-01-08T00:00:00',\n",
       " '2021-01-09T00:00:00',\n",
       " '2021-01-10T00:00:00',\n",
       " '2021-01-11T00:00:00',\n",
       " '2021-01-12T00:00:00',\n",
       " '2021-01-13T00:00:00',\n",
       " '2021-01-14T00:00:00',\n",
       " '2021-01-15T00:00:00',\n",
       " '2021-01-16T00:00:00',\n",
       " '2021-01-17T00:00:00',\n",
       " '2021-01-18T00:00:00',\n",
       " '2021-01-19T00:00:00',\n",
       " '2021-01-20T00:00:00',\n",
       " '2021-01-21T00:00:00',\n",
       " '2021-01-22T00:00:00',\n",
       " '2021-01-23T00:00:00',\n",
       " '2021-01-24T00:00:00',\n",
       " '2021-01-25T00:00:00',\n",
       " '2021-01-26T00:00:00',\n",
       " '2021-01-27T00:00:00',\n",
       " '2021-01-28T00:00:00',\n",
       " '2021-01-29T00:00:00',\n",
       " '2021-01-30T00:00:00',\n",
       " '2021-01-31T00:00:00',\n",
       " '2021-02-01T00:00:00',\n",
       " '2021-02-02T00:00:00',\n",
       " '2021-02-03T00:00:00',\n",
       " '2021-02-04T00:00:00',\n",
       " '2021-02-05T00:00:00',\n",
       " '2021-02-06T00:00:00',\n",
       " '2021-02-07T00:00:00',\n",
       " '2021-02-08T00:00:00',\n",
       " '2021-02-09T00:00:00',\n",
       " '2021-02-10T00:00:00',\n",
       " '2021-02-11T00:00:00',\n",
       " '2021-02-12T00:00:00',\n",
       " '2021-02-13T00:00:00',\n",
       " '2021-02-14T00:00:00',\n",
       " '2021-02-15T00:00:00',\n",
       " '2021-02-16T00:00:00',\n",
       " '2021-02-17T00:00:00',\n",
       " '2021-02-18T00:00:00',\n",
       " '2021-02-19T00:00:00',\n",
       " '2021-02-20T00:00:00',\n",
       " '2021-02-21T00:00:00',\n",
       " '2021-02-22T00:00:00',\n",
       " '2021-02-23T00:00:00',\n",
       " '2021-02-24T00:00:00',\n",
       " '2021-02-25T00:00:00',\n",
       " '2021-02-26T00:00:00',\n",
       " '2021-02-27T00:00:00',\n",
       " '2021-02-28T00:00:00',\n",
       " '2021-03-01T00:00:00',\n",
       " '2021-03-02T00:00:00',\n",
       " '2021-03-03T00:00:00',\n",
       " '2021-03-04T00:00:00',\n",
       " '2021-03-05T00:00:00',\n",
       " '2021-03-06T00:00:00',\n",
       " '2021-03-07T00:00:00',\n",
       " '2021-03-08T00:00:00',\n",
       " '2021-03-09T00:00:00',\n",
       " '2021-03-10T00:00:00',\n",
       " '2021-03-11T00:00:00',\n",
       " '2021-03-12T00:00:00',\n",
       " '2021-03-13T00:00:00',\n",
       " '2021-03-14T00:00:00',\n",
       " '2021-03-15T00:00:00',\n",
       " '2021-03-16T00:00:00',\n",
       " '2021-03-17T00:00:00',\n",
       " '2021-03-18T00:00:00',\n",
       " '2021-03-19T00:00:00',\n",
       " '2021-03-20T00:00:00',\n",
       " '2021-03-21T00:00:00',\n",
       " '2021-03-22T00:00:00',\n",
       " '2021-03-23T00:00:00',\n",
       " '2021-03-24T00:00:00',\n",
       " '2021-03-25T00:00:00',\n",
       " '2021-03-26T00:00:00',\n",
       " '2021-03-27T00:00:00',\n",
       " '2021-03-28T00:00:00',\n",
       " '2021-03-29T00:00:00',\n",
       " '2021-03-30T00:00:00',\n",
       " '2021-03-31T00:00:00',\n",
       " '2021-04-01T00:00:00',\n",
       " '2021-04-02T00:00:00',\n",
       " '2021-04-03T00:00:00',\n",
       " '2021-04-04T00:00:00',\n",
       " '2021-04-05T00:00:00',\n",
       " '2021-04-06T00:00:00',\n",
       " '2021-04-07T00:00:00',\n",
       " '2021-04-08T00:00:00',\n",
       " '2021-04-09T00:00:00',\n",
       " '2021-04-10T00:00:00',\n",
       " '2021-04-11T00:00:00',\n",
       " '2021-04-12T00:00:00',\n",
       " '2021-04-13T00:00:00',\n",
       " '2021-04-14T00:00:00',\n",
       " '2021-04-15T00:00:00',\n",
       " '2021-04-16T00:00:00',\n",
       " '2021-04-17T00:00:00',\n",
       " '2021-04-18T00:00:00',\n",
       " '2021-04-19T00:00:00',\n",
       " '2021-04-20T00:00:00',\n",
       " '2021-04-21T00:00:00',\n",
       " '2021-04-22T00:00:00',\n",
       " '2021-04-23T00:00:00',\n",
       " '2021-04-24T00:00:00',\n",
       " '2021-04-25T00:00:00',\n",
       " '2021-04-26T00:00:00',\n",
       " '2021-04-27T00:00:00',\n",
       " '2021-04-28T00:00:00',\n",
       " '2021-04-29T00:00:00',\n",
       " '2021-04-30T00:00:00',\n",
       " '2021-05-01T00:00:00',\n",
       " '2021-05-02T00:00:00',\n",
       " '2021-05-03T00:00:00',\n",
       " '2021-05-04T00:00:00',\n",
       " '2021-05-05T00:00:00',\n",
       " '2021-05-06T00:00:00',\n",
       " '2021-05-07T00:00:00',\n",
       " '2021-05-08T00:00:00',\n",
       " '2021-05-09T00:00:00',\n",
       " '2021-05-10T00:00:00',\n",
       " '2021-05-11T00:00:00',\n",
       " '2021-05-12T00:00:00',\n",
       " '2021-05-13T00:00:00',\n",
       " '2021-05-14T00:00:00',\n",
       " '2021-05-15T00:00:00',\n",
       " '2021-05-16T00:00:00',\n",
       " '2021-05-17T00:00:00',\n",
       " '2021-05-18T00:00:00',\n",
       " '2021-05-19T00:00:00',\n",
       " '2021-05-20T00:00:00',\n",
       " '2021-05-21T00:00:00',\n",
       " '2021-05-22T00:00:00',\n",
       " '2021-05-23T00:00:00',\n",
       " '2021-05-24T00:00:00',\n",
       " '2021-05-25T00:00:00',\n",
       " '2021-05-26T00:00:00',\n",
       " '2021-05-27T00:00:00',\n",
       " '2021-05-28T00:00:00',\n",
       " '2021-05-29T00:00:00',\n",
       " '2021-05-30T00:00:00',\n",
       " '2021-05-31T00:00:00',\n",
       " '2021-06-01T00:00:00',\n",
       " '2021-06-02T00:00:00',\n",
       " '2021-06-03T00:00:00',\n",
       " '2021-06-04T00:00:00',\n",
       " '2021-06-05T00:00:00',\n",
       " '2021-06-06T00:00:00',\n",
       " '2021-06-07T00:00:00',\n",
       " '2021-06-08T00:00:00',\n",
       " '2021-06-09T00:00:00',\n",
       " '2021-06-10T00:00:00',\n",
       " '2021-06-11T00:00:00',\n",
       " '2021-06-12T00:00:00',\n",
       " '2021-06-13T00:00:00',\n",
       " '2021-06-14T00:00:00',\n",
       " '2021-06-15T00:00:00',\n",
       " '2021-06-16T00:00:00',\n",
       " '2021-06-17T00:00:00',\n",
       " '2021-06-18T00:00:00',\n",
       " '2021-06-19T00:00:00']"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.loads(s.split('\\n')[11].strip()[:-1])[0]['x']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "5c5cbb1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "datasets.insert(0, json.loads(s.split('\\n')[11].strip()[:-1])[0]['x'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e1bbbb2",
   "metadata": {},
   "source": [
    "We obtain the four datasets as columnar data, then zip the lists together and store them in a Pandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "f8b4e083",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame.from_records(zip(*datasets), columns=['datetime', 'new_cases', 'hospitalizations', 'intensive', 'deaths']).dropna()\n",
    "df['province'] = 'agrigento'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "7ad665e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>datetime</th>\n",
       "      <th>new_cases</th>\n",
       "      <th>hospitalizations</th>\n",
       "      <th>intensive</th>\n",
       "      <th>deaths</th>\n",
       "      <th>province</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2020-03-23T00:00:00</td>\n",
       "      <td>3.285714</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2020-03-24T00:00:00</td>\n",
       "      <td>3.142857</td>\n",
       "      <td>0.142857</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2020-03-25T00:00:00</td>\n",
       "      <td>1.857143</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2020-03-26T00:00:00</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2020-03-27T00:00:00</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>0.428571</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>448</th>\n",
       "      <td>2021-06-11T00:00:00</td>\n",
       "      <td>33.285714</td>\n",
       "      <td>2.857143</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>449</th>\n",
       "      <td>2021-06-12T00:00:00</td>\n",
       "      <td>31.857143</td>\n",
       "      <td>2.857143</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>450</th>\n",
       "      <td>2021-06-13T00:00:00</td>\n",
       "      <td>27.428571</td>\n",
       "      <td>2.857143</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>451</th>\n",
       "      <td>2021-06-14T00:00:00</td>\n",
       "      <td>25.571429</td>\n",
       "      <td>3.142857</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>0.571429</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>452</th>\n",
       "      <td>2021-06-15T00:00:00</td>\n",
       "      <td>24.285714</td>\n",
       "      <td>2.428571</td>\n",
       "      <td>0.285714</td>\n",
       "      <td>0.142857</td>\n",
       "      <td>agrigento</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>450 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                datetime  new_cases  hospitalizations  intensive    deaths  \\\n",
       "3    2020-03-23T00:00:00   3.285714          0.285714   0.428571  0.428571   \n",
       "4    2020-03-24T00:00:00   3.142857          0.142857   0.428571  0.428571   \n",
       "5    2020-03-25T00:00:00   1.857143          0.285714   0.428571  0.285714   \n",
       "6    2020-03-26T00:00:00   2.000000          0.571429   0.428571  0.285714   \n",
       "7    2020-03-27T00:00:00   2.000000          0.571429   0.428571  0.285714   \n",
       "..                   ...        ...               ...        ...       ...   \n",
       "448  2021-06-11T00:00:00  33.285714          2.857143   0.000000  0.571429   \n",
       "449  2021-06-12T00:00:00  31.857143          2.857143   0.000000  0.571429   \n",
       "450  2021-06-13T00:00:00  27.428571          2.857143   0.285714  0.571429   \n",
       "451  2021-06-14T00:00:00  25.571429          3.142857   0.285714  0.571429   \n",
       "452  2021-06-15T00:00:00  24.285714          2.428571   0.285714  0.142857   \n",
       "\n",
       "      province  \n",
       "3    agrigento  \n",
       "4    agrigento  \n",
       "5    agrigento  \n",
       "6    agrigento  \n",
       "7    agrigento  \n",
       "..         ...  \n",
       "448  agrigento  \n",
       "449  agrigento  \n",
       "450  agrigento  \n",
       "451  agrigento  \n",
       "452  agrigento  \n",
       "\n",
       "[450 rows x 6 columns]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "c0174139",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 450 entries, 3 to 452\n",
      "Data columns (total 6 columns):\n",
      " #   Column            Non-Null Count  Dtype  \n",
      "---  ------            --------------  -----  \n",
      " 0   datetime          450 non-null    object \n",
      " 1   new_cases         450 non-null    float64\n",
      " 2   hospitalizations  450 non-null    float64\n",
      " 3   intensive         450 non-null    float64\n",
      " 4   deaths            450 non-null    float64\n",
      " 5   province          450 non-null    object \n",
      "dtypes: float64(4), object(2)\n",
      "memory usage: 24.6+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "d3b8730c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.6123046875"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(len(province)*25)/1024"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c0d071b",
   "metadata": {},
   "source": [
    "In total the upper bound of the entire dataset should be less than 3MB, which is a reasonable size."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80757517",
   "metadata": {},
   "source": [
    "## Writing the script that automates all of this\n",
    "\n",
    "We can now cycle through all the provinces and gather the data for all of them. We can then store the data in a csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "3f289887",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading files and preparing dataframe... 106/107\n",
      "Made the df dataframe!\n"
     ]
    }
   ],
   "source": [
    "prov_dfs=[]\n",
    "i=0\n",
    "tot=len(province)\n",
    "for provincia in province:\n",
    "    datasets=[]\n",
    "    for dataset_name in dataset_names:\n",
    "        url=f'https://covid19.infn.it/iss/plots/iss_bydate_{provincia}_{dataset_name}.div'\n",
    "        s=requests.get(url).text\n",
    "        datasets.append(json.loads(s.split('\\n')[11].strip()[:-1])[0]['y'])\n",
    "    datasets.insert(0, json.loads(s.split('\\n')[11].strip()[:-1])[0]['x'])\n",
    "    df = pd.DataFrame.from_records(zip(*datasets), columns=['datetime', 'new_cases', 'hospitalizations', 'intensive', 'deaths']).dropna()\n",
    "    df['province'] = provincia\n",
    "    prov_dfs.append(df)\n",
    "    print(f'Downloading files and preparing dataframe... {i}/{tot}', end='\\r')\n",
    "    i+=1\n",
    "df = pd.concat(prov_dfs)\n",
    "print('\\nMade the df dataframe!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "id": "c1b04950",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(join('data', 'covid_iss.csv'), index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}