notebooks/intro_to_generative_replay.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Generative_Replay.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "9Ur_3TCd35dG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# A simple Example of Continual Learning with Generative Replay\n",
        "\n",
        "Generative Replay is a continual learning method that use a generative models to generate samples from past experiences. The idea is : by learning from actual data and generated data from the past we can learn without forgetting.\n",
        "\n",
        "In this notebook, we take on of the simpliest setting : Mnist disjoint - 10 tasks\n",
        "\n",
        "In the setting, we have 10 task with each time one class per task. There is no overlapping between task, and we can not use data from past tasks. \n",
        "\n",
        "The goal is to train a generative model to generate all the 10 classes of mnist even if a receive data from each classes one by one.\n",
        "\n",
        "To do so we use Generative Replay !\n",
        "\n",
        "- **At task 0 (T0) :**  We train a generator on real data (for Mnist only samples of 0 ) only and then we save it as G0. G0 will be used as a memory of T0.\n",
        "- **At task 1 (T1) :**  We clone G0 as G1 and we train it with both real data (samples of 1) and generated data of T0 with G0. At the end of the task G1 learned to generate both 0 and 1. Then we delete G0, G1 will be use as a memory for **both** T0 and T1\n",
        "- **At task 2 (T2) :** We clone G1 as G2 and ...\n",
        "\n",
        "After 10 task if the generative models is efficient we are able to generate all the 10 classes by this method. (Here by default we will only train on 3 tasks but you can easily modify the notebook to do all the 10 tasks :) )\n",
        "\n",
        "In this notebook we only train a generative model to experiment how it works, but this method could also be use to realize an efficient memory for classification algorithms or reinforcement learning algorithms !!\n",
        "\n",
        "\n",
        "Let's start !"
      ]
    },
    {
      "metadata": {
        "id": "-UmlWNS-8Ubp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "1. We Create the environment to execute the code, we install pytorch with cuda 8.0 for GPU acceleration."
      ]
    },
    {
      "metadata": {
        "id": "Fx6pWPAGY_S_",
        "colab_type": "code",
        "outputId": "8a70f1b7-3a9f-4b04-876c-314e0dc2f3bf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        }
      },
      "cell_type": "code",
      "source": [
        "# http://pytorch.org/\n",
        "from os import path\n",
        "from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
        "platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
        "\n",
        "accelerator = 'cu80' #'cu80' if path.exists('/opt/bin/nvidia-smi') else 'cpu'\n",
        "print('Platform:', platform, 'Accelerator:', accelerator)\n",
        "\n",
        "!pip install --upgrade --force-reinstall -q http://download.pytorch.org/whl/{accelerator}/torch-0.4.0-{platform}-linux_x86_64.whl torchvision\n",
        "\n",
        "import torch\n",
        "print('Torch', torch.__version__, 'CUDA', torch.version.cuda)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Platform: cp36-cp36m Accelerator: cu80\n",
            "tcmalloc: large alloc 1073750016 bytes == 0x5ba4c000 @  0x7ff44315c2a4 0x591a07 0x5b5d56 0x502e9a 0x506859 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x502209 0x502f3d 0x506859 0x504c28 0x502540 0x502f3d 0x507641 0x504c28 0x502540 0x502f3d 0x507641\n",
            "\u001b[31mpymc3 3.6 has requirement joblib<0.13.0, but you'll have joblib 0.13.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mjupyter-console 6.0.0 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.15 which is incompatible.\u001b[0m\n",
            "\u001b[31mgoogle-colab 0.0.1a1 has requirement six~=1.11.0, but you'll have six 1.12.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mfeaturetools 0.4.1 has requirement pandas>=0.23.0, but you'll have pandas 0.22.0 which is incompatible.\u001b[0m\n",
            "\u001b[31mcufflinks 0.14.6 has requirement plotly>=3.0.0, but you'll have plotly 1.12.12 which is incompatible.\u001b[0m\n",
            "Torch 0.4.0 CUDA 8.0.61\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Gzh8v-jf8ihv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "2. We import all library needed for training and visualisation"
      ]
    },
    {
      "metadata": {
        "id": "bkcYXgb9aRLV",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torchvision.datasets as datasets\n",
        "import torchvision.transforms as transforms\n",
        "import torch.optim as optim\n",
        "import torch.nn.functional as F\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from torch.autograd import Variable\n",
        "from copy import deepcopy"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "lePKqo_e8obk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "3. We clone the continualAI repo ta get an easy to use data loader for mnist"
      ]
    },
    {
      "metadata": {
        "id": "H9JYmfPzaTCi",
        "colab_type": "code",
        "outputId": "d5925c22-35e4-41ff-bc99-ccc98376fa0f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/ContinualAI/colab.git continualai/colab"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Cloning into 'continualai/colab'...\n",
            "remote: Enumerating objects: 84, done.\u001b[K\n",
            "Unpacking objects:   1% (1/84)   \rUnpacking objects:   2% (2/84)   \rUnpacking objects:   3% (3/84)   \rUnpacking objects:   4% (4/84)   \rUnpacking objects:   5% (5/84)   \rUnpacking objects:   7% (6/84)   \rUnpacking objects:   8% (7/84)   \rUnpacking objects:   9% (8/84)   \rUnpacking objects:  10% (9/84)   \rUnpacking objects:  11% (10/84)   \rUnpacking objects:  13% (11/84)   \rUnpacking objects:  14% (12/84)   \rUnpacking objects:  15% (13/84)   \rUnpacking objects:  16% (14/84)   \rUnpacking objects:  17% (15/84)   \rUnpacking objects:  19% (16/84)   \rUnpacking objects:  20% (17/84)   \rUnpacking objects:  21% (18/84)   \rUnpacking objects:  22% (19/84)   \rUnpacking objects:  23% (20/84)   \rUnpacking objects:  25% (21/84)   \rUnpacking objects:  26% (22/84)   \rUnpacking objects:  27% (23/84)   \rUnpacking objects:  28% (24/84)   \rUnpacking objects:  29% (25/84)   \rUnpacking objects:  30% (26/84)   \rUnpacking objects:  32% (27/84)   \rUnpacking objects:  33% (28/84)   \rUnpacking objects:  34% (29/84)   \rUnpacking objects:  35% (30/84)   \rUnpacking objects:  36% (31/84)   \rUnpacking objects:  38% (32/84)   \rUnpacking objects:  39% (33/84)   \rUnpacking objects:  40% (34/84)   \rremote: Total 84 (delta 0), reused 0 (delta 0), pack-reused 84\u001b[K\n",
            "Unpacking objects:  41% (35/84)   \rUnpacking objects:  42% (36/84)   \rUnpacking objects:  44% (37/84)   \rUnpacking objects:  45% (38/84)   \rUnpacking objects:  46% (39/84)   \rUnpacking objects:  47% (40/84)   \rUnpacking objects:  48% (41/84)   \rUnpacking objects:  50% (42/84)   \rUnpacking objects:  51% (43/84)   \rUnpacking objects:  52% (44/84)   \rUnpacking objects:  53% (45/84)   \rUnpacking objects:  54% (46/84)   \rUnpacking objects:  55% (47/84)   \rUnpacking objects:  57% (48/84)   \rUnpacking objects:  58% (49/84)   \rUnpacking objects:  59% (50/84)   \rUnpacking objects:  60% (51/84)   \rUnpacking objects:  61% (52/84)   \rUnpacking objects:  63% (53/84)   \rUnpacking objects:  64% (54/84)   \rUnpacking objects:  65% (55/84)   \rUnpacking objects:  66% (56/84)   \rUnpacking objects:  67% (57/84)   \rUnpacking objects:  69% (58/84)   \rUnpacking objects:  70% (59/84)   \rUnpacking objects:  71% (60/84)   \rUnpacking objects:  72% (61/84)   \rUnpacking objects:  73% (62/84)   \rUnpacking objects:  75% (63/84)   \rUnpacking objects:  76% (64/84)   \rUnpacking objects:  77% (65/84)   \rUnpacking objects:  78% (66/84)   \rUnpacking objects:  79% (67/84)   \rUnpacking objects:  80% (68/84)   \rUnpacking objects:  82% (69/84)   \rUnpacking objects:  83% (70/84)   \rUnpacking objects:  84% (71/84)   \rUnpacking objects:  85% (72/84)   \rUnpacking objects:  86% (73/84)   \rUnpacking objects:  88% (74/84)   \rUnpacking objects:  89% (75/84)   \rUnpacking objects:  90% (76/84)   \rUnpacking objects:  91% (77/84)   \rUnpacking objects:  92% (78/84)   \rUnpacking objects:  94% (79/84)   \rUnpacking objects:  95% (80/84)   \rUnpacking objects:  96% (81/84)   \rUnpacking objects:  97% (82/84)   \rUnpacking objects:  98% (83/84)   \rUnpacking objects: 100% (84/84)   \rUnpacking objects: 100% (84/84), done.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "aIzKP9GU81TT",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "4. We import mnist and inialize it"
      ]
    },
    {
      "metadata": {
        "id": "hEC8SMuwACqn",
        "colab_type": "code",
        "outputId": "d23ebcef-a7df-4a3c-e74b-f29e5548f531",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "!ls colab"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "extras\tLICENSE  notebooks  README.md  scripts\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "192J3FHLa0nE",
        "colab_type": "code",
        "outputId": "6a78c9ea-c7b9-45b5-c332-8c5a1e549fbe",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 121
        }
      },
      "cell_type": "code",
      "source": [
        "from continualai.colab.scripts import mnist\n",
        "mnist.init()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading train-images-idx3-ubyte.gz...\n",
            "Downloading t10k-images-idx3-ubyte.gz...\n",
            "Downloading train-labels-idx1-ubyte.gz...\n",
            "Downloading t10k-labels-idx1-ubyte.gz...\n",
            "Download complete.\n",
            "Save complete.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "JHdbAdK-8_xs",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "5. We can now load the data set and get training and testing data and labels"
      ]
    },
    {
      "metadata": {
        "id": "ec_SejbFa39O",
        "colab_type": "code",
        "outputId": "b91e8ae5-eb2b-4d4a-9229-8947c8d2850c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 86
        }
      },
      "cell_type": "code",
      "source": [
        "x_train, t_train, x_test, t_test = mnist.load()\n",
        "\n",
        "print(\"x_train dim and type: \", x_train.shape, x_train.dtype)\n",
        "print(\"t_train dim and type: \", t_train.shape, t_train.dtype)\n",
        "print(\"x_test dim and type: \", x_test.shape, x_test.dtype)\n",
        "print(\"t_test dim and type: \", t_test.shape, t_test.dtype)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x_train dim and type:  (60000, 1, 28, 28) float32\n",
            "t_train dim and type:  (60000,) uint8\n",
            "x_test dim and type:  (10000, 1, 28, 28) float32\n",
            "t_test dim and type:  (10000,) uint8\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "V3bh6Gns9J4M",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "6. We set a flag to use cuda for GPU acceleration and we fix the seed for reproducibility purpose"
      ]
    },
    {
      "metadata": {
        "id": "FbJnUx1Ma87D",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# switch to False to use CPU\n",
        "use_cuda = True\n",
        "\n",
        "use_cuda = use_cuda and torch.cuda.is_available()\n",
        "device = torch.device(\"cuda\" if use_cuda else \"cpu\");\n",
        "torch.manual_seed(1);"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "RU9L7kJs9UX-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "7. Creation of a generator. We will use a GAN to achieve Generative Replay."
      ]
    },
    {
      "metadata": {
        "id": "-ZyVDR0TbM1a",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Generator(nn.Module):\n",
        "    # Network Architecture is exactly same as in infoGAN (https://arxiv.org/abs/1606.03657)\n",
        "    # Architecture : FC1024_BR-FC7x7x128_BR-(64)4dc2s_BR-(1)4dc2s_S\n",
        "    def __init__(self):\n",
        "        super(Generator, self).__init__()\n",
        "\n",
        "        self.latent_dim = 1024\n",
        "        self.input_height = 28\n",
        "        self.input_width = 28\n",
        "        self.input_dim = 62\n",
        "        self.output_dim = 1\n",
        "\n",
        "        self.fc = nn.Sequential(\n",
        "            nn.Linear(self.input_dim, self.latent_dim),\n",
        "            nn.BatchNorm1d(self.latent_dim),\n",
        "            nn.ReLU(),\n",
        "            nn.Linear(self.latent_dim, 128 * (self.input_height // 4) * (self.input_width // 4)),\n",
        "            nn.BatchNorm1d(128 * (self.input_height // 4) * (self.input_width // 4)),\n",
        "            nn.ReLU(),\n",
        "        )\n",
        "        self.deconv = nn.Sequential(\n",
        "            nn.ConvTranspose2d(128, 64, 4, 2, 1),\n",
        "            nn.BatchNorm2d(64),\n",
        "            nn.ReLU(),\n",
        "            nn.ConvTranspose2d(64, self.output_dim, 4, 2, 1),\n",
        "            nn.Sigmoid(),\n",
        "        )\n",
        "\n",
        "        self.maxPool = nn.MaxPool2d((2, 2), stride=(2, 2))\n",
        "        self.Sigmoid = nn.Sigmoid()\n",
        "        self.apply(self.weights_init)\n",
        "    \n",
        "    def reinit(self):\n",
        "      self.apply(self.weights_init)\n",
        "\n",
        "    def forward(self, input):\n",
        "        input = input.view(-1, self.input_dim)\n",
        "        x = self.fc(input)\n",
        "        x = x.view(-1, 128, (self.input_height // 4), (self.input_width // 4))\n",
        "        x = self.deconv(x)\n",
        "        return x\n",
        "\n",
        "    def weights_init(self, m):\n",
        "        classname = m.__class__.__name__\n",
        "        if classname.find('Conv') != -1:\n",
        "            m.weight.data.normal_(0.0, 0.02)\n",
        "        elif classname.find('BatchNorm') != -1:\n",
        "            m.weight.data.normal_(1.0, 0.02)\n",
        "            m.bias.data.fill_(0)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "sBQ3Emmz9pW_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "7. Here is the Discriminator of our GAN"
      ]
    },
    {
      "metadata": {
        "id": "g6EMu6RjcNMU",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Discriminator(nn.Module):\n",
        "    # Network Architecture is exactly same as in infoGAN (https://arxiv.org/abs/1606.03657)\n",
        "    # Architecture : (64)4c2s-(128)4c2s_BL-FC1024_BL-FC1_S\n",
        "    def __init__(self):\n",
        "        super(Discriminator, self).__init__()\n",
        "        self.input_height = 28\n",
        "        self.input_width = 28\n",
        "        self.input_dim = 1\n",
        "        self.output_dim = 1\n",
        "\n",
        "        self.latent_dim = 1024\n",
        "\n",
        "        shape = 128 * (self.input_height // 4) * (self.input_width // 4)\n",
        "\n",
        "        self.fc1_1 = nn.Linear(784, self.latent_dim)\n",
        "        self.fc1_2 = nn.Linear(10, self.latent_dim)\n",
        "        self.fc2 = nn.Linear(self.latent_dim * 2, self.latent_dim // 2)\n",
        "        self.fc2_bn = nn.BatchNorm1d(self.latent_dim // 2)\n",
        "        self.fc3 = nn.Linear(self.latent_dim // 2, 256)\n",
        "        self.fc3_bn = nn.BatchNorm1d(256)\n",
        "        self.fc4 = nn.Linear(256, 1)\n",
        "\n",
        "        self.conv = nn.Sequential(\n",
        "            nn.Conv2d(self.input_dim, 64, 4, 2, 1),\n",
        "            nn.LeakyReLU(0.2),\n",
        "            nn.Conv2d(64, 128, 4, 2, 1),\n",
        "            nn.BatchNorm2d(128),\n",
        "            nn.LeakyReLU(0.2),\n",
        "        )\n",
        "        self.fc = nn.Sequential(\n",
        "            nn.Linear(shape, self.latent_dim),\n",
        "            nn.BatchNorm1d(self.latent_dim),\n",
        "            nn.LeakyReLU(0.2),\n",
        "            nn.Linear(self.latent_dim, self.output_dim),\n",
        "            nn.Sigmoid(),\n",
        "        )\n",
        "        self.aux_linear = nn.Linear(shape, 10)\n",
        "        self.softmax = nn.Softmax()\n",
        "        self.apply(self.weights_init)\n",
        "    \n",
        "    def reinit(self):\n",
        "      self.apply(self.weights_init)\n",
        "\n",
        "    def weights_init(self, m):\n",
        "        classname = m.__class__.__name__\n",
        "        if classname.find('Conv') != -1:\n",
        "            m.weight.data.normal_(0.0, 0.02)\n",
        "        elif classname.find('BatchNorm') != -1:\n",
        "            m.weight.data.normal_(1.0, 0.02)\n",
        "            m.bias.data.fill_(0)\n",
        "\n",
        "    def forward(self, input):\n",
        "        x = self.conv(input)\n",
        "        x = x.view(x.data.shape[0], 128 * (self.input_height // 4) * (self.input_width // 4))\n",
        "        return self.fc(x)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "wT1xKrgw9vQS",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "8. The function get_iter_dataset makes possible to load data from only one particular class. It will help to load data class by class."
      ]
    },
    {
      "metadata": {
        "id": "LmzvXxzHcrGp",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# function to get digits from only one class\n",
        "def get_iter_dataset(x_train, t_train, classe=None):\n",
        "    if classe is not None:\n",
        "        return x_train[np.where(t_train==classe)[0]]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FkFgz_Vg9-Qz",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "9. The function \"run_batch\" train the Generator and Discriminator one a particular batch of data."
      ]
    },
    {
      "metadata": {
        "id": "MeBNcBSqvqUM",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def run_batch(G, D, G_optimizer, D_optimizer, x_):\n",
        "      x_ = x_.view((-1, 1, 28, 28))\n",
        "    \n",
        "      # y_real and y_fake are the label for fake and true data\n",
        "      y_real_ = Variable(torch.ones(x_.size(0), 1))\n",
        "      y_fake_ = Variable(torch.zeros(x_.size(0), 1))\n",
        "      if use_cuda:\n",
        "        y_real_, y_fake_ = y_real_.cuda(0), y_fake_.cuda(0)\n",
        "\n",
        "      z_ = torch.rand((x_.size(0), z_dim))\n",
        "\n",
        "      x_, z_ = Variable(x_), Variable(z_)\n",
        "\n",
        "      if use_cuda:\n",
        "        x_, z_ = x_.cuda(0), z_.cuda(0)\n",
        "\n",
        "      # update D network\n",
        "      D_optimizer.zero_grad()\n",
        "\n",
        "      D_real = D(x_)\n",
        "      D_real_loss = BCELoss(D_real, y_real_[:x_.size(0)])\n",
        "\n",
        "      G_ = G(z_)\n",
        "      D_fake = D(G_)\n",
        "      D_fake_loss = BCELoss(D_fake, y_fake_[:x_.size(0)])\n",
        "\n",
        "      D_loss = D_real_loss + D_fake_loss\n",
        "\n",
        "      D_loss.backward()\n",
        "      D_optimizer.step()\n",
        "\n",
        "      # update G network\n",
        "      G_optimizer.zero_grad()\n",
        "\n",
        "      G_ = G(z_)\n",
        "      D_fake = D(G_)\n",
        "      G_loss = BCELoss(D_fake, y_real_[:x_.size(0)])\n",
        "\n",
        "      G_loss.backward()\n",
        "      G_optimizer.step()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "I_3so87yxYX6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "#Declarations and Hyper-parameters\n",
        "\n",
        "# Training hyper-parameter\n",
        "nb_task = 3  # change this to 10 if you want the experiment of the full 10 classes setting\n",
        "batch_size = 64\n",
        "lr = 0.0002\n",
        "z_dim = 62\n",
        "epoch_number = 10\n",
        "nb_samples = 10\n",
        "\n",
        "G = Generator()\n",
        "D = Discriminator()\n",
        "\n",
        "# G and D are put in train mode\n",
        "G.train()\n",
        "D.train()\n",
        "\n",
        "if use_cuda:\n",
        "  G.cuda(0)\n",
        "  D.cuda(0)\n",
        "\n",
        "# Use of Adam optimizer \n",
        "G_optimizer = optim.Adam(G.parameters(), lr=lr)\n",
        "D_optimizer = optim.Adam(D.parameters(), lr=lr)\n",
        "BCELoss = nn.BCELoss()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "aY4lGuxSyZ4q",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Training Generator without Replay\n",
        "In the following cell we train the generator without using Generative Replay. It show what happen when the catastrophics forgetting appear. Each line of samples is ramdomly sampled from the generator at the end of each task. We can see that at the end of each task the generator is only able to generate data from the present and not anymore from the past task. "
      ]
    },
    {
      "metadata": {
        "id": "tkQyFHFQeGBZ",
        "colab_type": "code",
        "outputId": "793bf281-fb3b-487e-da41-c32662610ac5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 233
        }
      },
      "cell_type": "code",
      "source": [
        "for task in range(nb_task):\n",
        "  \n",
        "  data = get_iter_dataset(x_train, t_train, classe=task)\n",
        "  nb_batch = int(len(data)/batch_size)\n",
        "  \n",
        "  for epoch in range(epoch_number):\n",
        "    for index in range(nb_batch):\n",
        "      x_=torch.FloatTensor(data[index*batch_size:(index+1)*batch_size])\n",
        "      run_batch(G, D, G_optimizer, D_optimizer, x_)\n",
        "\n",
        "  \n",
        "  z_ = Variable(torch.rand((nb_samples, z_dim)))\n",
        "  \n",
        "  if use_cuda:\n",
        "    z_ = z_.cuda(0)\n",
        "  samples = G(z_).data\n",
        "  samples = samples.cpu().numpy()\n",
        "  #plt.plot(samples[0])\n",
        "  \n",
        "  f, axarr = plt.subplots(1,nb_samples)\n",
        "  for j in range(nb_samples):\n",
        "    axarr[j].imshow(samples[j, 0], cmap=\"gray\")\n",
        "  np.vectorize(lambda ax:ax.axis('off'))(axarr);"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl4U1X+/9/33iRNk65AW6DsS4tQ\nBgQVERQEURQEEQcHRMRRQYRBRAdFRHABBUR90IFxGBWnCg7INi5siqjAiLKMIJsoCoW2FAo0LWnT\nLOf3R+Z8uEmT3JulTef7O6/n6fO0zc25n3vuWT/bkRhjDAKBQCAQCOocOd4CCAQCgUDw/ytiEhYI\nBAKBIE6ISVggEAgEgjghJmGBQCAQCOKEmIQFAoFAIIgTYhIWCAQCgSBOiElYIBAIBII4ISZhgUAg\nEAjihJiEBQKBQCCIE4a6uInFYkFVVRUAQJIkGI1GGAzeW1dVVUGdtMs/gVeohF6SJMFgMEBRFPpb\nfb3T6YTH4wlYjslkgsPhCCl3586dUVBQAACorq5GYmIiKioqAAAulwuMMUiSRPfmchiNRmRkZKC8\nvBwAYLPZwBiD2+2msmVZhix710CKooAxBqfTGfC51b+bzWZUVlYGlbl169Yks8fj8fkul1WN+nNZ\nlukaWZbh8XjomTweD9xut+b7UJfL/05LS8P58+eDfg8AUlNTqb78ZVPLxP/P2w//3OVyAYBPHauf\ni5flX2ao50hJScGFCxeCXmc2m4O2IUmSfNoEY8ynLdZWojqtdt2oUSOfdxGobtSo64vXP1CzbUWD\nLMsB35ua5s2bo6SkhO4tSRLVZ7A+Hgz1e1EUBU6ns8Z3eRvw74eyLFPb69atG/79738HvU92djbO\nnDnjI6O6/Fi1Az5+8Hv4vytZlpGYmAgA6NOnD/71r3+FLC85ORl2ux0AarQNXm5SUhIA7xjL+x6H\n/x3o2dT9jo8xavzHK369xWKpMT6o0eqLwe7BP/cfi/l3XC5XDRkDlRnof2azGZcuXQoqM32vLtJW\nhhrwYoF6cA7ncbSuNZlMPhOjFvw51QMvEHhiUL94/tK1BiL+vUCNIlKZQ92HDzpAzUEk3LK0vq8o\niuY1gQZG/4VXLNGSu7bbdaSEqg/1pFuf0JLJYDD49I9YvfdoytFaEMeqL8YC9cLy4sWLIa/V00b8\nF9yRyqT3u4qi1Jjsg8lTn9DzfEIdLRAIBAJBnKgTdXRtUps7oVArr1B4PB7NXZ1afcT/jgWRysxR\nq3/cbjcSEhIAAOXl5RHvhvU8W6TX1MddXX3mf7W+QqktoyGacrR2udFoj6LFX4PFn5ObBaPlf7Ud\naRFI21bb1MkkHOuJUm0jNBqNIe2B0d4nEvV2sO+o7cdcdsArt8Vigc1mo+/XRiMIZIPxJzMzEwAw\nfvx4/O53v6OB5rPPPsPKlSs17eiREmkb4XajYDZfrlqL54BYn4hlX1SriOty0IrlQMltvLzMcNq3\nnoV2PMjIyEBpaWnAz/Q8X22adxRFoQUC96vR+g5QuwsaWZbRsWNHAF47vsFgwNatWwHUXLToMa2F\nS51Mwlr6/HDp0qULAOCaa67Bt99+S45IZWVlMW1AsW6MvEEZjUa43W6a4DIyMtChQwd88803AEIv\nJtQOF+Gix+baqlUrAMDtt9+O5s2b06Q8YsQI5OXlYdasWQAu7wJitfCJtK7VThTA5UF1zJgxAICh\nQ4di6dKl2Lx5M10Tq91ArLFYLACAdu3a4dixYyHtjZESjf0uJyeH+t7gwYPRvXt3pKSkAAC2bt2K\nxx57jBaSsezv/P5ai9xw4P0oLy8Pr776Krp27QoAOHv2LO655x4cPXoUAHQ51sQbs9mMrl27oqio\nCIC3727cuBEA8PPPP9e5PLw/ms1mMMYwaNAgAEBBQQEKCwtx7tw5AOFpv7RsvtEs4ocNG4a33noL\ngHdsvnTpEjngbtq0CStWrCBnRqfTiRMnTsS0fQubsEAgEAgEcaJOvKNj6ZHZqVMnLF++HADQoEED\nLF++nMpetGgRioqKdN8rXI/McOGhDIqioFOnTmjQoAEAoHv37ti2bRuuuuoqAMCQIUPQqlUr/P73\nvwcAnD9/HsXFxQBQIyxISx0SaV1LkoTU1FSsW7cOAHDhwgX069ePdjoAsH79ekyYMAEAYLfbUVlZ\nierqal3la8mkxztaDyaTCXfeeSfeeecdAEBCQgKKi4spLOPkyZPIz8/HqVOnNMvSCp2JVbtOSkrC\nnj170LJlSwDeusjPz8ezzz4LwLs7q66ujkm7jlTmnJwcfPrpp2jTpg3dg4dzAN4d46hRo1BYWAgA\n2LNnT1jl13Zf9Kd58+YAgDVr1qBTp04wm80AvDudbdu24fHHHwcAHDt2jLQ+gcKMaqMv6oX7a3z4\n4Ye4/fbbcfbsWQBAeno6aSz4jl5NOJ7P4dKoUSP8+c9/BgDcdtttyM7OJg3PxYsXMWvWLOzevRsA\n8MMPP+jeUdZWpEK3bt2wc+dOGqvVYZmAt10fPnwYGzZsAACsW7cO+/fv1zVW6VVd14k6OtqGyCs4\nOTkZb775Jlq0aEHl9u3blwavTp06YcyYMeSCrzemtTbkTktLw7JlywB4O3xOTg69EIPBgEmTJiEr\nK4tk8Xg8NAkXFhYiPz8fAEgtUluoHbHGjBlDk+7q1avRqFEj5OXlkcw7duzwsefoDb+IpK4jVS+l\np6fjhRdeoLhIwKvuHz16NADvIFtZWYlFixYBiJ/zjCRJGD9+PABgwYIFFHcJeOuia9eumD9/PgCg\ntLQUL774IsWcapUbinDrlA9It9xyC5o3b07v3+12w+1202T7008/4corr6R637t3b701DaWkpOCT\nTz4BAOTm5vq0NUmS0LJlS3Tv3h0AUFJSQir2cM0YtTkBy7KMVatWAQD69u0LRVHQuHFj+nzfvn0A\nLps46oKsrCwUFhYGNZllZmZi7ty5pI4eMmQICgoKKCa5LuFzyMaNG5GQkOATcy7Lss/Cy2w248Yb\nbwQA7Nq1Cz/88IOue+g1Hdb7OGGj0Yjs7GwAQH5+Pnr27EnOBU6nE06nE8nJyQC8FVZcXEyrwFDB\n3UDsV9/c0Wrp0qUYNWoU/R3oXuo64d7Ue/fuBQC8+OKL+PjjjyOSO5wYv4yMDFxxxRUAgAceeAB5\neXnUGN98800UFRVRJ7Zardi/fz9OnDhB5VRUVOhKlKBnRcjt5JxIJ4uFCxdi0qRJPqtZnmiEy/LV\nV1/hjjvuoGeIVO5I40BNJhPuvvtuvPvuuz6y82f+9ddfUVpaSp04LS0Nb7/9Nl599VUAoZ1rYrlj\nkGWZFmGbN29GWloatWmbzYY5c+bg7bffBuAd7IcMGYKpU6cCAK699tqgzkGBqCtNyY033ogNGzb4\nJGdQJ2SoqKiAzWajiWH79u2YN28eAODEiRM15KyrmGxu9wWAqVOnYuDAgT6798TExBpjCn8+f2Jd\n19xvpLi4OGT74mXyvlhYWIiPP/4Yjz76qM/nwYh2U8Vp1KgRjWM8CoTvyCsrK+F0OnH69GkAXo0g\nTwoFAMuXL0d+fj75Cmi9fz3zh7AJCwQCgUAQJ+p9nHDbtm3x3nvvAfDq79WhPYcOHcLjjz+OBx54\nAAAwatQotGzZMm7ZU7j6cMyYMTVkCJQ6ja+iPB4P7HY7Zs+eDQBkM6ktMjIyAAA9evRA3759AQAd\nO3ZESkoKqbjWrFnjk/aTp9ls2LAhAGDAgAHYvHkz2a71hBuEIpo0iGazmdrIXXfd5aMGYozBbrfT\njiAhIQE9e/bEjBkzAAAzZ84MapfSkiccLYkkSWR+mDNnDsaOHeuTMc3lcpG6eeLEiSguLkavXr0A\nAM899xyeeOIJXHfddQCAYcOGBZU50M4nXLhcCQkJ5C3P081y7VLLli1RXl5OdVRRUYGzZ8+SnXLN\nmjXo27dvvYjnBYCBAwcCANauXQuTyUT/dzqdPh7QaWlpsFqt9JxXXXUV+W6otUB1SVJSEtatW4f+\n/fvT/xhjZANetmwZ2rdvj549ewIAGjduTO+wd+/e2L59e1j3C6euc3NzcfjwYQCBd6NqLRRvm3xX\n2aJFC0ycOBHHjh0D4NW+Rart0GO+4vLl5+eTBuvChQtwu92Unnby5MkwGo0UspSXlweXy4UmTZoA\n8PbFdu3a4ZlnngGAkBEMetXR9XoSliQJ119/PanEXC4XqqqqMGnSJADAqlWrYLfb8eOPPwIAevXq\nhTZt2vjkFY4GPQ2CV3SzZs1w6623ktzA5cb8yy+/ICUlhf7vdDqxdetW3HLLLQC8g53D4aAXesMN\nN2D9+vX0zOGg1RhlWcadd94JAHj44Yexdu1aAF675Pr160nV6R8aYDab0bp1azz55JMAvJ3bZrPh\n008/1ZRTT2OMdJC1WCzYu3cvcnNzA37+22+/YeXKlTQBXnPNNbBYLBg5ciQAr2qYO5KEOwDo6fTc\nGa9Xr16YOHEiAKB///4+dXLgwAHk5+fjs88+A+BtL06nkwanjRs3YtWqVTQh9u/fn0Ku/GXQE4am\nJTcvw+FwkP2uQYMGOHfuHIYMGQIAZCfluN1u7Nu3j+ySLpdLtzquNpEkCatWrSK5y8rKUFFRgRUr\nVgAATp8+jdTUVPzpT3+i681mM8ndvn17GqBrK3afE8jxC/CaitQTMABs2LCB/BzKy8uRmpqK9PR0\nAKB2A3jbXbiTsJ42MnfuXADA9OnTg16zcuVKzJw5k+pv6NCh6N27N4YPHw7A23dlWcacOXMAeEOY\n+LjnXxexGNOHDh0KwLuBUC9gvvjiCzJL7d69G4wx/Oc//wEA9OvXD6dOncLMmTMBeMfqwYMHk114\nxYoVQU1SescToY4WCAQCgSBesDoAQEQ/bdu2Zb/88gu7cOECu3DhAsvPz2ddu3ZlsiwzWZbpOkmS\nmCRJbM+ePcztdrORI0eykSNHhixbkqSYyG0wGJjBYGBWq5WtW7eOrVu3jnk8HuZyudiGDRvYhg0b\nWLt27VheXh5LTk5mycnJTFEUZrVa2c6dO9nOnTuZzWZju3fvZpMmTWKTJk1iq1evZj169GA9evSo\ncT9ZlkPKLElSSHmbN2/ObDYbs9lsrKysjD300EPsoYceYiaTSbO+WrRowXbs2MF27NjBysrK2Nat\nW1lOTg7LyclhRqMx6He1ZI6kjWRlZbGsrCxWWVlZoyyXy8V2797Ndu/ezcxmM5MkidqMyWRieXl5\n7LvvvmPfffcdKykpYVarlVmt1oD3iUZmRVHYXXfdxe666y5WUFDAnE4nczqdjDHGPB4PKygoYAUF\nBSwlJYWZTCZqS4HK6tSpEztw4AA7cOAAmz9/ftBrtdq1Vvvw/8nMzGSZmZnM4XCwffv2UV8LdG12\ndjZzuVzM5XIxt9ut2abC6Yvhys1/+vfvzy5dusTKy8tZeXk569q1q8+7liSJNW3alJWVlbGysjLm\ndrsZY4xVV1ez6upqVlZWxlq1asVatWoV8/ahrl/1eKbuN7Iss2PHjvmUe/z48YD1wa8vLS2laydP\nnhyWzHrqOj09nXk8HubxeGp81+PxUDsNJmOHDh1Yhw4d2KZNm5jL5aK6ttlsNB75f9dqtUYlsyzL\n7PTp0+z06dPM6XSycePGsXHjxtH4EKwuLRYLa9euHVu5ciVbuXIls9vt7NKlS2zx4sVs8eLFLDk5\nOeQ99VAv1dHc227//v0oKyvDm2++CQB46aWXArqzs/+qTs6fPw9ZlvHaa68BAKmcagv1yUeSJFEI\ng9vtxpEjR/Dcc88B8NqS1LYRwKuSvvLKKwF4bSWKouCJJ54AADRp0oRU8D169AgrE1goVZKiKHjl\nlVfIq9LlclH8m1a8ryRJSElJocwxZ86cQUZGBkaMGAHAa88JdjpLrMOAEhMTSR3Ln4XDGMPKlSvx\n8MMPA7gcVsLrpLq6GgcPHiT7VIMGDZCamgog/OxIWmo7xhg6d+4MwJsOT20zs9ls9Jm/ajcQR48e\npfq99957SR3oX+d624leeP0ajUZ8/fXXIcu/cOGCj527WbNmOH78uOY99MhsMBjC8kTn5odFixZh\n165d1Le4mlF97zNnzlD7d7vdPn4n5eXl5Ckba5YsWUK2x9GjR1PmP86AAQMAgKJDuKkoJycnZB51\nde76SPqeVrt+5ZVXfP5WH6HocDhw7733+sjjfy2PXZ48eTK2bt1KJgyr1Urx2Zs3b0ZBQYFPv41G\n5vHjx6NRo0YAvPPE6tWrAQQPO1ObZGRZJh+Bs2fPokGDBhRnHsrcorfu4zIJh6owSZLIxmuxWFBd\nXU0pxbTiybjjyk8//RRDaYPD2OWj/lJTU6nSq6qq0KxZM4qZDOS0ZDabaVBJSEhAdnY2fvvtNwBe\nxyneYNq2bUuhS0B0aStTU1PRtGlTGmAcDofuAcbj8eDHH38ke/Lrr7+OMWPGUKzriRMn8MEHH9C1\ntcnw4cNpAuPwzjB27FisWLEiZOdgjOGNN94AALz66qvkdMETTcQKxhjFfqsn4LKyMrRs2RJlZWW6\ny3K73dTxk5KS0KxZMyornIk33Ema24RdLhd69uwZMm+z3W6nvpebm4sPPviAnIVCEU2bDkR6ejqF\nFf3888948sknQ6Zv9Hg8lOaR+wrw59u4cWPMU3ACXse28ePHU31+8sknFJvscrlgNpsxatQoAF7n\nH6PRSA5QwfoXL6uqqorq9Nlnn6VNTKzIzMyk+nE6nVS3TZo0gc1m0zw7nH/3+PHj2Lx5M6W1TE9P\nR/v27QF443HbtGlDfjJ60laGwmaz0cL7/PnzIc8JV+N2u3Hq1CkKxevfvz/cbjc2bdoEILxc48EQ\nNmGBQCAQCOJEvVNHP/LII0hLSwPgXTE99NBDmrs1vsJxOBywWCy47bbbNO+jJ4xJS8XBT0MCvGEB\nnTp1AuDd5RYVFVFC9UC43W4fGUpLS0l9zQ9PALw7EbUMkewyuYxGoxGtWrWiHfiBAwfC3hnx7z71\n1FMYMmQIZXq64447sGXLFgBeVbW6XD11reeEp3bt2gEA/vKXv9Qok3u3rlq1SpdHLn+O5ORk0qD4\np1qMNvuU2WxGTk5OjeuHDRsW1i6Yl8WzmVmtVmprBw8eDKuccFGr6zZu3Kh5gpF6h9G6dWtd99DT\nBhVFCamOVhSFNGg5OTn44osvAHgTWxw9elTTbMDHGK6O5olGPv/885iq+Hn9PfjggzXME2q1rqIo\npDp/4YUXsHDhQixYsIDkDQQPPeTqawC0gwuHUM8ryzK6dOlC7Xfbtm1Ys2YNAG+aynnz5unWrrlc\nLrz22ms4efIkAK96Wt3GnU4nyaKljdDqqy6Xi3at/rtXfhIbENhD3WKx4JprrgHg7Q8ej4fq2GKx\nBE0KpTdUtt5MwlzgadOm0aT6448/YvXq1ZqdgKu8UlJSUFpaGrNUj3omYa4WXLBgAWWfYoxh/fr1\npKYJVIbT6SQbUE5ODvbu3Ytvv/0WAHDkyBFSGftP5JFMDLxhWa1WpKamklp/ypQpIcsKRXl5OT78\n8EP88Y9/BOAN/eFhQnrSK/qj9Y6NRiPZgf1TPK5duxZLly4FoD9296abbgLgHVTuueceAF4bXSxV\njw0aNPCJ2+WqtXBDRgCvTUydgS2ck2iiQW0bu/rqqzUXSmqVvl4brp5n0LIJvv/+++jQoQMAbxu4\n++67AXht5lrlK4qCbt260e8Oh4Mmln/84x+askUCt1Fz2d555x2fSbiyshKvv/46XT9y5MiQx0ZK\nkkS5xoHL742H/4RDqPpKTk6G0WiksW3JkiX4/vvvAYBOtNPbBxljOHToEL3bgQMH4uqrrwbg9c8I\nJ4OeVrvcsmULybxr1y6fmGWTyRQ03pffl5sWFUWBJEnUXnJzc4PmdahXk7CiKJoVygPos7OzqUIH\nDRqkWflt2rSh+MqKigpKPqFFrAYvnv5Qfd8dO3Zg+vTpIRtjamoq2X0Bb+PmE4A6KYC/nJE6WgBA\nnz59YLFYaBVbUlISdllq3G43pQy1WCy0K/OXOdq6NhqNGDFiBC141LvmdevWYfTo0WFPnjzuUpZl\nWK1WANGlVw0EP5qSwyffSFJdGo1Gn1hJbsf271u1RWJiIjmzhGLYsGH0u7+jUTSEakNTpkzBiBEj\naHE5a9Ys3TY/SZKQnZ1NOx1JkmC328nxLZq6DfVu+IKKOwPu27fPR8vg/7x2uz1o+zSbzZg7dy4d\n38kYw/PPPw8g9rnnk5KSYLFYSP6kpCTK3W8wGGhXqxfGGNq2bQvAO6Hx516xYoXPWKc1hmh9LkkS\nJZLhu3nAO39s2LAh5Ht2OByUsMVsNvvsjAcPHhx0EtY77gmbsEAgEAgEcaLenKLEMzEpikLedVre\nqhaLBXPnziW71a+//oojR45EKe1l9KyuFi5cSL/z6+fNm4eqqqqQ32/bti2tJmVZRk5ODq3kGWP0\nGT+tJhr4Duq6666DLMuUWee9995Dnz59Ii536NChZG8uLy8PGqKkh1B11bBhQ8yePdtHHctTlb73\n3nth77TV3sWAb5pG9S5Vz/sPdc3Bgwd97KIffvhhWHKqadasGantDAYDrr/+egA1w/BqK2WrzWbD\njBkz6IjIYBoZXn8mk4nC7GJBKO3ZI488Ao/HQ2k+/cOQAsH9TsaMGYOpU6dSmBrgNaeEu6MLRKB3\nwdsa/4yrOZ988knyhg4GD2dyOBxwOBxUxvPPP48BAwbg888/J/n5KWGxpnv37pAkiXwdlixZQhqe\n6upqfPnll3QwiR7tVEZGBmbNmgXg8tGMACgNrV60+qraY/zmm28mc05paammZurSpUt0qt2gQYNg\nMpmo/XMTYiDqVdpKPYMkV6UAoDCYQJOPJElke9yyZQtsNhvZIAcMGKBbLRkLdfQNN9zg09G4XYGf\nN+nvxMIHjZtvvhmPPfYYqXIZY/j+++9JdsaYT/xxOHKH+tzpdPpMHLm5uSST3nOBOQaDwUd9dPTo\n0aCLhUgnBt6IrVYrnWMLeJ+Rd4pw32Pjxo1RUFBAMrlcLrIJ+8cJ6wmLCHV/f9s4D+EKF0mSKC85\n4K0Xvtisq7SQycnJYIyRGjyY841aZf7RRx/Vqkz8xKbGjRvD7XbrPmJOURQKUbv77rt96tBms6FP\nnz662lUk7Vp9xKa6jG7dulFe9oqKCrhcLrq2srISHTt2pBSxFosFJ0+epDbw1ltvYf78+RTCBEQ3\nvvk7SXJHMf672+2mCTM5OZkW8kajEVlZWXQmOU8NqcZgMJDzVZs2bTB//nzypTGZTHRWvJ6FVDhc\nunQJixcvBuCNs2/atCkA74JcluWQDocej4cWSHwjwMNGv/zyy6D31PsO6sUkLEkSJehwuVw+cbH+\n5ObmYufOnQC8dlW32012WT1JD9T3jJY+ffr4lMOdqBo3boyioiKfibRhw4bkqdi9e3dkZGRQw/Z4\nPDh58mTIAPxo4JP7oUOHwBgjmRs0aEAOBrt27QrrXvfdd59PgpI5c+bUmpPQwIEDfeo5HOcPwDuo\n3HDDDQC8Czd14oeFCxfS4B1r+T/66CPasQPh5wHnZGZm+pzla7fb8f7770dUlp68wIGQZRlpaWkU\necAd4dQYDAYf56JIPHPDgfuCTJ8+PaxjA6+99lpy3DIajZBlmfwkMjIyIrLZByKQtoBru+x2O5KS\nkqhdN23alGyrHTt2hMvlot15VVUVXnvtNZ/d+sGDB2mi+uqrr2Ladv3LUicl2r59O44dO0bxvFar\nlZ4hISEBaWlp5H3MJ3M+zrVu3Rrz5s0jfwy32w1FUSghTEFBAe67776YPYc/fOGem5tLTnyJiYlo\n1aoVLWgC1aPBYCCHMZ6QZPDgwQBCxwkLm7BAIBAIBPWcOtkJ+6++/f9WFIXUoYqikJrWP/6qd+/e\n2LBhg0+Iyu7du3WrodToWaVonQIzduxYn6P+1AeDm81m+nvy5Ml45plnfLzz1DL8+uuveOGFF8J+\nBr3w+2zdupVOtwG8Kzye+eX555/HG2+8Qbu1QKt4SZLIfsNPT+H2IJ7+MpZwOTMzM312wocOHaLP\nAskpyzIdgN6yZUssWrSI1E/8e/zIxqeffjpoW4h2d/GHP/yBfnc6nT6ZyvTAdwhXXHEF8vLyqD9M\nnz49oKoPiP1untc732mqjwFUoygKFi9eTPVbVlYW0yM5A+3guYdqcnIyJEkitej27dvpWsYYFEUh\nG+aUKVMwduxYH/+C3bt30wlo4eyCIzEN8fZ65swZJCUl0TVmsxnbtm0DcPmgef6ZwWDwsS86HA6M\nGzeO0j/G+p2bTCYajxljPn2srKwMW7dupdPBeIgP4H1HhYWFPmN5ixYtKI3w0KFDffoxP76Ue7Ln\n5ubWarY97mv09ddf09GWjRo1wtSpU+lo00Dhlfn5+aSlYIzhrbfe0uX/ovu9sDrAP0G2f7J/WZbZ\nxYsX2cWLF5nL5WLLly9ny5cvZykpKSw9PZ0NHTqUDR06lBUVFfkkDT937hxLTEz0KUvvj57k2oqi\nhCxj//79Ptdv2rSJbdq0iY0cOZKtXbuWHTlyhB05coS5XK4aZbvdblZVVcWqqqrYtGnTQh6A4P8T\nTl37P/PSpUspYTpjjA4VOHfuHNu5cyd78MEH2YMPPsisViuTZZkOCujatSvbtWsXc7vdzO12M4/H\nw0pLS1lCQgJLSEgIKa+iKJp1Heh7JpOJmUwmdv/997PKykpKGv/VV1+xtLQ0lpaWxhITE1njxo1Z\n9+7dWffu3dkbb7zBTp8+HfJeR44cCXkQQSzqGgC79dZb6Vqn08maN2/OmjdvrusdS5LE2rRpw9q0\nacOOHj3KHA4HmzZtGps2bVrIgxG0DkOIpK8AoPZy+PBhdvjwYWqv/B3de++9rKioiNqHzWZjmZmZ\nMWnTwep62bJlbNmyZTUOEnA6nay4uJgVFxez0tJSOpSBMVbj2p9++knXWCHLckAZIm0f6enprLy8\n3OcgBF53/vDxg/fbzz//XHe9+vdNPYdlBDtEhH9/+PDh7Pjx4+z48ePMZrOxkydPspMnTzK73c4K\nCgrYhAkT2IQJE1jv3r3Znj17gh72wBhjJSUl1IaiaSPhtOWOHTvSAS52u52VlZWxWbNmsVmzZrHE\nxEQmSRIzm83MbDazp556ilUamZadAAAH+0lEQVRVVdG7KSkpYe3atYtZu2aMMem/D1CrGAwGH/uo\nJEk1DP/8QPtnnnmGVqMVFRWwWCw+zkOKolDQ9VVXXRXyUGUttB5dy860bds28lKVZZm8uUtKStC2\nbVuKP/Uv6+LFi7jhhhtoJRuuDSqUTIqihFxNWiwW8ugePXo07c4NBgMYY+QVmpycDKvVSnUvyzKc\nTie9x5UrV+Kxxx7TzBMLeO1uehKwB/tfQkIC3n77bXLY++2338iJqkmTJsjKyqrhdaqGMUa7yHfe\neQezZ8/WlbHKv536o9U+evToga+//hqAd9XPHWsmTJhQYycr/fccW8B7hq3b7caSJUsAeA8/b9iw\nIdkEtWQKpb2J1BeiqKgIWVlZFHc6YcIE9OvXj/L+ZmVl4cyZM9Qfe/XqFVYubq2+GEhunpd7xowZ\nFPfJUWuo1PBoA+59O2HChJD1ZTQaqQ/Y7fawYuC12seIESNojLjpppswbdo0AMDy5cvx17/+lfwt\nPvvsMzRu3Jj6pp4hW90f/J8vkrpWM3HiRMpQl5GRQYd0dO7cGdXV1Th16hQA7zh4/fXXB/US/uab\nb3DzzTcHPUTBX6Zo+qIak8lEToM8FwUfH4qKisAYo6iGxMREVFZWUt999NFHUVxcrGvHriUzya5L\naoFAIBAIBDGnTnbCWrszAGRjOHz4MMXw+WeVGTFiBL788ktdKyc9RLsi7Nu3L62QzGYzyWU2m31W\nf/w+PBPWyy+/TCdDhYvW6kqtdQgGX9mnpaVh5syZALwhYupTqlJSUmCxWHyeYdSoUeSRWl5ernvl\nGelOWP1ZSkoKnQbTq1cv2jVmZWXV+K66jgoLC3H11Vf7nAakF6261mofycnJePrppwF4j1Lj9WUy\nmbBmzRra2WzatAnDhg2jHNapqalYsGABeXAWFhZi+fLlQe3AamprJ/zUU09hxowZFDbjcDh8wm1c\nLhemTZuGv//97wDCz9QUiVZKHTbz8ssvY9y4cQC8Hru8DpxOJ+x2O/72t78B8HrxV1ZW6m67eo6r\nDEfmeKNnd6bVRlJTUynOvkWLFrSbX7p0KRITE338CNQha4wx2Gw20pD069cPxcXFuuSOdbvm4WCn\nT58mD/lAuFwuZGdnU98L530qiqJrvKmTSTghIUF3HGrLli3Jyad9+/YoLi6mOE6u2osV0U7CZrMZ\nkydPBgDMnDmTJi3/F2q327Fnzx7ccsstABCVCl2rExmNxrAmGt5J0tLSUFFRQU5vXbp0wdSpUyn9\n3ZkzZ+ioxVCyATXrVU/H11o8GAwGilG95557SA3apUsXlJSU0IKNq8z5Wc1ax19qEc0gy2MQAW9Q\nPz+Awmg0+oSK8fNfuWp/9erVmD17NoXchbPoDHew0huyZDQaMXz4cHLIc7lc6Ny5M/75z38C8Dq4\nRZOmMtq+CFzudxaLhcpzuVxwuVxhx1OrwwcjnYT1bD7iQbR1rSgKlSHLMrXrLVu2IC0tjUxciqL4\nOKyuXr0a8+fPp9CqcMYprXYd6YJn0KBBuOOOOyg0iptLX3rpJQDehUWoxC3+daVOVap16AjJHrbU\nAoFAIBAIYkKd7ITD3Z3xFS13la+t1WQsVt88+8v9999Pq6kOHTpAURTayQwYMAAHDhyISXYjrRWh\nyWSKWbKBcAmVdUarrvVoS/hul6nCJvjKU72rjFWTjlYdrb4mISGBVGBr1qxBhw4daMfw6aef4uOP\nP6aQnqNHj4b9Dvl9tFbf/jLrOUJS/d1gR75FS7ROkrVFNOro/6s7YX+4eWvKlCm4/fbbaaz/9ttv\n0b59ezpW8t13343YlJiYmBhSqxVt++AaQf6+eFl66oo/v9FohMPhoOfXuxOuNzbheBDrjs8bb3Jy\nMqqqqmiyjGVqwViro+sKrXqM5+IhGLGYhOsaLTtUpOro2kSPuaI+2lcBMQn7o86aFstnV2e5C0Q8\n24e6vtQyaG2YOPUibWU8qI0BlD9nOOkzw0VLbr1Jw+sb9XGwqo+TbCDUcqrPL9ZDIPt9XffX2m6z\nkiT5JHepq+erj+NebVNbG4D6OD5wgr1nve//f3PEFggEAoHg/wB1shO2Wq1BQxb4yjvYrkO9mgiW\niEEPansWgKDp99QkJCRQisHaWNVyeQwGg0+yBo/HQ7YTtf0TgE/YUCDS0tJoJ+52u+HxeGKiDuf1\n528r4c+g3m2oU+4Bl9MvhiI5OZnkZozV2B0xxnTZaULZpbW+49++tOparUKv612POgxE7a3KfRSC\noTZXsP+mdFT7YDDGyDbvv2vUY75R4/9ddf2q+6OWzIC3XfNUgcHkCKQRYIz5hAzy5wv13vzbQbD7\naY0hPNGDHpnV9SPLss970tpd6pUXgE+6zmCYzWafcS+acTdc1O9N7a+gNYakpKT4pDlWmwLC0X7w\nduk/JqhtxYG0R/wz9RipZ44B6sgmLBAIBAKBoCZCHS0QCAQCQZwQk7BAIBAIBHFCTMICgUAgEMQJ\nMQkLBAKBQBAnxCQsEAgEAkGcEJOwQCAQCARxQkzCAoFAIBDECTEJCwQCgUAQJ8QkLBAIBAJBnBCT\nsEAgEAgEcUJMwgKBQCAQxAkxCQsEAoFAECfEJCwQCAQCQZwQk7BAIBAIBHFCTMICgUAgEMQJMQkL\nBAKBQBAnxCQsEAgEAkGcEJOwQCAQCARxQkzCAoFAIBDECTEJCwQCgUAQJ8QkLBAIBAJBnBCTsEAg\nEAgEcUJMwgKBQCAQxIn/B87Tj9J0k40KAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d97e37e80>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAFTRJREFUeJzt3X1sFEUfB/Dvvty11x7QQhtaCoIR\noxLxJQJBFKlEMahRwJSYiEDUYOJbQEUTKb4gEgmIEnkAjcEIhqDRBJFGXkR8hQiIprS8FIi8VMqL\n0PYOete725vnj8tM70rbe9nZ3Sv+PkmTazlmZ2dnZ3bnVWGMMRBCCCHEdqrTESCEEEL+q6gSJoQQ\nQhxClTAhhBDiEKqECSGEEIdQJUwIIYQ4hCphQgghxCFUCRNCCCEOoUqYEEIIcQhVwoQQQohDdDsO\nUlBQgEuXLgEAotEoGGNov1CXoigAkPB3RVGgKIr4m8fjAWMMqhp7dmhpaYHb7UZOTg4AwO12w+fz\nIRKJiGO1x49TUFCACxcudBnvoqIiNDY2dhgWD4djjHV4Dp2dWzxd1xGNRsX3eBp1xO12o7W1tdM4\n9+3bF+fPnwcAGIbR6fdSlSzu8d/r7Lw9Ho+4/p3p27evSGvDMBLySLI4pHMtOqLrbbdBfJq5XK4u\n07pHjx5oaWkB0HFeAyDyany+5d9tf37x8Y//O0/b+PzRGU3TRP7viNfrFXHu6Hp1dg07+n4q4s+/\ns3CSxRkAevfujebmZgCd39e8HAiFQtA0DUDneYnHyzAMqKraZfw6OhYA5OTkIBAIdPq9+PzBrz+P\ne/u0TjfPZipZ+QEAPXv2xMWLFzuNi6IocLlcAIBwOJzw9/hz5P+X31/RaBQ5OTkJ1zocDifksfZ5\nnIfhcrkQCoU6jXOyfM3jBrTliY7oug7GmMg/4XA4IX8oigLDMC67Zzs6rqqqKZXBih3LVrbP5LJl\nknmTXVQglmHjM1m26Oo8rU7rTCWLU3eMd0c3Yjag/JFZmZCJKzGtNU3r8kHPKVdiWgPUHE0IIYQ4\nxpbmaCefRjvDmxu6ko1Pg8nY0eJgxXGy8Sn2StRd0zndeMc3gzt1H3fXtKZyz162VMLZKFkfFJB6\nm/5/haZpyMvLAwD4fD6HY0My0VG/75WI37e8Mo5nVxr8V9I6mdzcXACxvvpMK/iOrmM60rkWiqKI\nfmzDMCx/KOl2zdFVVVWio92MVC6IXRVwdXU1fv/9dxQXF6O4uNiWY3YmPz8f+fn50DRNDMBwuVwY\nOHAgDhw4gJqaGtTU1CA/P9/ReKaL5xmZ/bgyw9J1HSNGjMCIESPEoJfugOePbJOXl4e8vDxMmTIF\nR48exbx58zBv3jzs2rULhw4dEv9OrLVjxw74/X74/X5s27YNTz75JDweDzwej9TjJLsX03kYKi8v\nR3V1Naqrq/Hll19i/Pjx8Hq98Hq9ZqPZoW5XCRNCCCFXDGYDAFJ+CgsL2YULF6SEpShK0ngriiIt\n7h39aJrGNE0Tx6uqqmJVVVVJ/58dceZxc7vdzO12s+nTp7OTJ0+yDRs2sA0bNqR9HLvySEc/uq4z\nv9/P/H4/q6urY6qqSom3pmnS4vjtt9+KcI8cOWIqrK6kc+7JfiZOnMjC4TALh8PsyJEjbOTIkZbE\nOZP8oaoqU1WVDRgwgPl8PnH9g8Ege/LJJ6WlgV35I/4nJyeHrVy5kn388cfs448/Tvs4ycgs906d\nOiXCbWpqYrNmzcoonGTltcw4v/POO8wwDGYYBguHw+z3338X5aHMOHPdqk84FArB6/WK9vpU+nXN\nYBb352zdujXhWA8//LDpMM3GOb5Zx+Px4JprrgEALFq0CIWFhfjzzz9Nhe+EjRs3iqaka6+9Frm5\nuWJOoRky+4qGDh0qPu/cuVNauO3JjHNlZaW4FwcMGJB03r2d+HmGw2FEo1HR9BwIBDBkyBC43W4A\nSDpNUUYcZCsvL8f06dNFHn733Xfx999/J3zHTDkgs9yLL0/8fj++/fZbaWHHkxnne++9N2FOcV1d\nXUbhpNpd1a0q4ZKSEmiaJiXBra5gU1FeXi4+NzY2SrlpMx0MUlxcDJ/PJ+ZFG4aBgoICrF27FgDQ\np08fGIaBq6++GgBQWlqKU6dOmY6v1RRFwR133CF+Z4whGAxKCVtmHmpoaEBZWRkA4JVXXpEWbnsy\nBwuVlpaKz6qqor6+Xkq4MpWUlIgFOYBYpTxu3DjxALxp0yYno5cWPqPjww8/BGNMPBCfPn06YbEP\ns2TmkV69eonPeXl5CQvjpMPO8vr777/HsGHDAMTKwUAgkFAppyrV8pz6hAkhhBCHdKs34crKSqiq\nipdffhkAsHDhQkuPZ+UUg1GjRonP4XAYZWVlKT05pTK/OVWKoqB///4AgEOHDuGTTz7BnDlzAMSa\n7R555BEMGjRIfF/XdQwePBhA27SDbNe7d++EkZhWd2FkQlEUFBYWit///fdfy44lMz/Hj+SPRqMZ\nTyOxeuWx+Pu4sLAQBQUFmDdvHgBg8+bNWdEqloqJEycCAK666ioYhiHKv86Wa3SaqqoJb75VVVWX\nNZtno2AwKNKyubkZgUAgYRlL2WyphGVVZidPngQAfP7556bDclL//v2xbds2kSYVFRVJ13PlkqVj\nKuvwxn/3jTfeABCrVJ999lnRL/3ll19iwoQJYj3e9mvuNjU1pXSMVMl+4OEV7/79+wG0Vb67d++W\ntiyfrDjruo6ysjI0NDQAsOZGl83j8SQUsKdPn5bSz96Z+DWJ0/HXX3+hT58+oqm8X79+iEaj2Ldv\nH4D07pd0ye5b/fTTTwHE1qyuq6vD9u3bpYUfT1a8XS5XQlhDhgzpFusujBo1SpR3xcXFuPbaay2N\nNzVHE0IIIQ7pVs3Rq1atQmVlpeg0/+effyw9nuzmHf50tWvXLiiKgm3btgGINdPIOlY6T2yRSASv\nvvoqAOC+++5D3759xb8NHToUO3bsQElJCYDYIh6KomDz5s0AIHY8kkVmWufm5mLNmjUAYgNDDMPA\nli1bAAAzZ87MujdNxhhcLpdodTDDrk0lZsyYkfD7M888Y+nKQmbCZozhgQceAADs2bMHFy9eFKN0\ns60JtzNTp04VC+REo1Fs3rw5K7tW4g0ePDjhbfitt97K+jjn5uZi9OjR4j7y+XxYuHChpW/C3aoS\n9vl8UBTF8srXKidOnAAQG1Xa3NyMxx57DEB6/ZTJCqN0CxW+7eGYMWOwc+dOVFZWAgAOHz6MgoIC\nTJo0SYTr8/nw/PPPZ3QcO61fvx733nsvgFil1NjYiPnz5wNAVo7g5cvk1dTUOB2VpHh3xGuvvQag\nLR/88ccfGYdpR16qra0FELvXNE2T8sBjF0VRsGDBAlExRCIRfPbZZ7btFJUp3qzLy6yffvrJ4Rgl\nF781LgC0traipqbG0jTuVpUwL1hXrVoFALjpppucjE5aysrK0K9fPwCxm+jxxx/HuXPnHI5Vm/Pn\nz6OhoQE//vgjgNib7/Dhw8XT96VLlxAKhcSertmqqKgIw4cPFzfN3r178c0334hCONvegrlwOOz4\nkqWp4H3t7Zd8tDpdzfa/X3fddQBi/alutxtLliwBAPz6668ZP9Tb1erQp0+fhFaqFStWoLq6Omsr\nX54uc+fOBWMMhw8fBhCbJ5zt+HRGnrbvv/8+GhsbLU1r6hMmhBBCHGLLm7Cs3Yj4mwIf0dudfP31\n16IpT1VV8caZLUpLS7F3715MnToVQKxPeOTIkeKp1uPxIBgMWjoC1gw+heCbb75BYWGhaAJrbGzE\n8uXLcfHiRenH1HVdyhug2+0GY0zKIh3JntgzHWXcPvz4rQKB2L3JuzasYHYU80svvSQ+K4oiypLS\n0lKcPXs2o+todVpzc+bMgaZpYpGZ1atXIxKJZO2bMG/O9Xq9MAwDS5cuBWBts7msmQqTJ0+Gqqoi\nrPvuuw+LFy+2dLxDt9pPeNWqVVi6dCn27t0rJTy7KIqC2267Tfy+ZcuWjJtmrGwCCwQCOHv2LABg\n0KBB6NOnj/i3aDSKkydPZuXAClVVMXv2bADAyJEjAbQtRzhx4kRcunTJkuPKGqzhcrkQDAZtWRLU\n7L3IK6tQKCQezADg+PHjpuPWFbOFYPvz5g9ty5Ytw6OPPopjx46ZCr8jZuPM7/Wnn34aAHDgwAEA\nwJ9//mlrBZzuNoB8MKeqqvD7/d0iX/O0vv/++wG0jdNZsmSJ5WWebfOEZWhpaUEkEsHtt98OwNyN\nb1d/DhB7O4t/a3r88cczDsuqm6+xsRGPPfaYiFtzczP8fr9Yc/nChQuYNGlSVm74XVRUhDfffBNA\n2xvanj17AMQeLKwi81rIGh2djKxKuL6+Hr169RIPk1amM2A+3nxGRftFRQ4dOmTpG7wZfMyLx+MB\nY0ws1pGN92A8Hm9N06BpmljfIZvx8Tr8xYOPffn5558tPzb1CRNCCCEO6Vajo4HYGyxfWnHdunWm\nwrEa3+z8wQcfBGNM9Etl06hobvjw4WhpacGiRYsAxEYJFhcXi6alYcOGoaGhIeumRWiahnXr1iVs\nLH/8+HExJSnb3xqAWNO/y+USS1fyqWzZiL9FDhgwAACyauekzqiqih9++AEAcMMNN0BVVXEeeXl5\ntraKpWPjxo3i8+nTpx3bMCWdsQ9utxszZ84EEJth4ff7UVRUBMDadR3M9gnz8Rh8/NL3338PAJZ1\nZSVIacNDk9xut7S9Hk+cOMGCwSALBoOmw01G13WxL2m6+7B6vd6EsFpbW6WlQVfSDaukpISVlJSw\nUCjEotGo2B82EAiw5uZm1tTUxJqamlh+fr6laZ3pfqBvv/32ZWENHjzYlrSWtYfp+PHjWTAYZMOG\nDWPDhg3LqvzR/ofvLx0OhxljjO3evZvt3r3b0jjLiDffD/bYsWMJ4Z44cYLdfPPNlsTbzN7Nzz33\nXEJYd999ty152my8i4qK2JkzZ9iZM2cYY4wdPHiQ9e7dm/Xu3dvSeJuJc35+PmttbWWtra2MMcYa\nGhqYx+NhHo8npf+vKEqnZUEqbGmOlvk2wleXYoxlvC1WqiKRCKLRqPhJlaIol73tDh8+XHb0pNi0\naRM2bdokniSbmprQ1NQEVVVFXxRjTKxtrSiKJW8OLM2nWF3Xoev6ZSOKN27ciCNHjsiMWqfSjXNn\ncnJyYBgG6uvrs3IxkXj8LZK/SQYCAcv7g2UwDAOGYWD16tUJ1+348eOWve2YKfdeeOEFce+dO3fO\n1oUuzMS7Z8+eIq3D4TC2bt2KlpYWy2dVmInz9ddfL/qvo9EolixZkla+5tcpU9QnTAghhDjElj7h\n/Px8aSstnTlzRvT/9evXz7a3nnTU1NQkbPXX2NiI6upqB2PUMZ/Phx49eiT8jfffBAIBuFwuMZUj\nLy8Pfr9fjHysra11dLrSe++9ByDWBwXE+syAtukc3cmAAQOg67qUNzKr+zf5aHnuxRdfNB2mnX2y\ny5cvx9y5c8Xvo0aNwoIFCzB58mTb4pBMTk4OysrKxNz2+fPn2zq2wUz/6rRp08RqaoFAAPX19Y6U\nE6meg6ZpWLZsmSjnWltbsWzZsrSOw/NvptfIlko4HA5LG9Bzyy23iASbM2cOnnjiiawZJNSzZ08A\nseYNoG06R7YtR8ivRUfN+Twjud1uRCIR/PrrrwBiTfMFBQVYsGABAGDChAk2xfZyeXl5qKioABDL\nT2fOnMHAgQMBtM0PtkP7Gz3TwquiogJut1sMduLbL2bC6nuBb4TAFzTgDz/dRa9evRJ+V1UVEydO\nFA9zMvNPpvkhJycH+/fvF5XBF198IS1OVlJVFU899ZRY6pYxJrparJZpWo8dOxYjRowQv9fW1qbd\nvWL2IdK2SlhW4bB582Y89NBDAGKVnux9aOOlO0n9q6++AtA2gpTPY86mPTRLSkowbtw4ALGRf3yF\nrNmzZ6Ourg433HADAGDfvn3Yvn071q9fDyD2hNjS0iIKYdnSSevXX39dfG5sbMSQIUNsrXy59nHO\nNB8OGjQI0Wg0q0dFc2PHjhWfDcOQMmI3lXSTdZ/zvB9P13WxZrBVe/SmghfmM2bMwIYNG/Ddd98B\nQMp7jcuMRyZpzRhDfX29mHMLAHfddZctMyoyvRfXrl2bMG88k/LNbPlOfcKEEEKIU1IaQ20SJA2t\nB8DuvPNOEa7f72c33nijZUP105mCous6C4VCLBQKMcYYMwwjo6lNZuOdSvq5XC7mcrk6HGovO65W\npHVFRQUrLy9n5eXl7KOPPmKapjkSb1nH+OWXX1gwGGRer5d5vd6sjvOUKVPYlClTWDQaZaFQSFoe\nT0ZWWg8cOLDD8Hft2sV27dolNd7p3k+FhYWssLCQ1dbWsqlTp4ppVXbmabNpPWbMGBaJRFgkEmGG\nYbDVq1fbEu90w+L5tqWlhTHGxFTMTMIxk9aMMdbtFuv47bffxAbt27dvR11dnWXHYmk0n6iqig0b\nNgCILXQxevTorFwoYvfu3Z1OvE/nfGVL1gSmKIoYNHb06FGx1OD+/fulbRDilMrKSnzwwQdZuzlG\nPD6OgC//yMdBNDU1ORmtlB0/fhyjR48WmwosXrwY06dPlzLArL107idd13HPPfcAAA4ePIg1a9Y4\ndj+aafr/7bff4PP5AMTyyMqVK2VGTRpeNtfW1uLWW2/FrFmzTIVjBjVHE0IIIQ5RmA2PW7KnIPCO\ndEVRoOt6xgMXkp16/JZW2STZG2O2SeVN1eVyie90dH65ublig4NwOCxGs5aUlODo0aOWbSrfVVrL\nyh89e/bEE088gRUrVgBIbSAOnyHQUbpamT/4yNcdO3bA4/GIRWjMTkFMlo5W5msz00zM5g9elo0Z\nMwbTpk0DEGut+t///pdWPNJhdbkXf61klp9W5Gtd19G/f38xiFZ2eZ9KeLZUwnwlkmzj5I1vxpVY\nCeu6npVNynZUwrLZEWdN0+B2u8VWhmbDpAdi+1C5Z59U8qwtfcLZWAET+/C3tq50xzySjZVCMrLi\nbBgGgsGgbWnQHdOakFRQnzAhhBDiEFsq4XQ3WuCLafP+p2Qy2VQglbezVL7TFb7QfW5urujDNCvZ\necZv6ec0vskC36Iv2XezTbJrFj/J3wrxGyWkKlmelR1nGRt6pHKfpXsMHi+v12vZdZIRrlUbonQm\nlfssWXzSOW9elmuahtzcXKiqmtE5Z5JGmR5HVn5JtUyzpU+YEEIIIZej5mhCCCHEIVQJE0IIIQ6h\nSpgQQghxCFXChBBCiEOoEiaEEEIcQpUwIYQQ4hCqhAkhhBCHUCVMCCGEOIQqYUIIIcQhVAkTQggh\nDqFKmBBCCHEIVcKEEEKIQ6gSJoQQQhxClTAhhBDiEKqECSGEEIdQJUwIIYQ4hCphQgghxCFUCRNC\nCCEOoUqYEEIIcQhVwoQQQohDqBImhBBCHEKVMCGEEOIQqoQJIYQQh/wfQcb7pEkLcvEAAAAASUVO\nRK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d94b35080>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl0VPUVx7/vzZqZbCSBBALIEiAQ\ndhG3IqCHFhcMVqUuKFGhpMheBOVQxeopCkWxFFSUqgfRKgc5ZRFBC6INQtu4IQoaICQSE5IQwmSZ\nzGTm9o/pu85knUlmCXo/57xzMpN57933e/e33Xt/96cQEUEQBEEQhLCjRloAQRAEQfi5Ip2wIAiC\nIEQI6YQFQRAEIUJIJywIgiAIEUI6YUEQBEGIENIJC4IgCEKEkE5YEARBECKEdMKCIAiCECGkExYE\nQRCECKEPx01UVUVHTMzVmkwXo9wXo8wAoChKmCQJjHCWtVYGiqLA7Xa3+ToXm37487wXo37o9Xq4\nXK4wSuMfrb3/i1Hui1E/NMLSCXe0Su8vF6PcF6PMFyvBLuuGDYn2OZj36Yj60RFlCgYdsSPzh4tV\n7osVMUcLgiAIQoQIy0y4Paiq2i7TnOA/BoMB9fX1AH66s5OOitVqxW233QYAqKmpQW5uLkaMGAEA\nOH78OAoKCnDhwoVIiigIQgjocJ2wqqpYuXIlAODmm29GVFQU+vbtCwBwOByRFC0gFEXp8B2Zoii4\n4447AAB33XUXkpOTce211wLw+IUuXLjAJtGOaqJSVRVJSUno2rUrAODBBx/E7bffjoceeggA8N57\n76GkpKTDDy6mTJmC559/HgBQV1eHqKgo6HQ6AJ739MEHH2DSpEn8/0ihqh7jmV6vR0pKCn744QcA\nHv1QVRUWiwWAZyChlbkQOEajEXv37gUAXHnllaioqMDll18OACgoKOiwetwUJpMJgKf91ul03JZc\nTM8QUigMAGj1GDp0KA0dOpSKiop8znW5XHT99dfT9ddfT6qq+nUtf49gyN3ckZ6eTjExMRQTE0MG\ngyGosrdXZlVVSVVVmjlzJjkcDj4++eQT6tmzJ/Xs2ZOio6PJaDR2+LJeuXIlud3uVq9vs9nIZrNR\nenp6WMva38NisVBtbS3V1tbShQsXqK6ujlwuF7lcLrLb7fT44493CJmtVitZrVZKS0uj7Oxsmjp1\nKk2dOpXS0tIoPT2dFixYQAsWLCCdTtch9KO1IzU1lfbv30+VlZVUWVlJhYWF9M033/BztvYcLWEy\nmdok0+WXX96kTtfX11N9fT3V1tZScXEx9erVi3r16hXw9UNV1qqq0vLly8lut5Pdbqfa2loqKyuj\nY8eO0bFjx8hms9GSJUvIZDK1qWzCqR86nS5gHW5LWRMRiU9YEARBECJExM3RqqoiMzMTb7/9NgD4\nmCiKi4sRHx+PAwcOAADcbjd0Oh369esHABg4cCA++OADNnecO3cORBRxM8cVV1yBrVu3IiYmBgBQ\nWlqKnJwcPPLIIwCAM2fOREy2Ll26IDc3FwDQvXt3/r6qqgqFhYX4/vvvAXjeg6IobBLtaOZorWwX\nLlzo1/IEzUx64MABdO3atcPFGcTExOD8+fMAgJdffhn79u3DkCFDAAD79u3DV199FUnxAHjq6k03\n3QQAuO222zBmzBjU1NQA8JjTY2JiUFhYCMD3eToSqqri008/xdChQwE0jkiPjY0FABw7dgwAcP31\n17e57Ntqjj98+DAcDge3awBQWVkJq9UKwGPejY2NRe/evQEA+fn5bbpPMOjSpQt+//vfAwDmzZvn\nIzPgKYPOnTsD8LQxBw4c4DalI6LX67F9+3b0798fgEevP/3009De1K/5cjtBgym6oig+JoyVK1fS\n0aNH6ejRo7R48WLq0aMH9ejRg5KSkig6Otrn3OXLl5Pb7ebD6XTysWnTpoDMM4HK3doxevRoGj16\nNDmdzibNSZrZNzU1NWQmjpbOMxqNtGnTJjZzelNQUEBdu3b1eUdTpkyhjRs30saNG33eWbBlbktZ\nL1++nJYvX97ktbz1w2azNXoXmZmZIS/rQA5VVam2tpZlXrBgAcXFxVFaWhqlpaU1qgORkFlRFLru\nuuvYtJifn08Oh4Oqq6upurqaxo8fT9OmTaNDhw7RoUOH6L777gtIZ4KtH03JrygKlZSUtHovIvJx\nBWgumnDqh16vp5EjR9LIkSOpb9++ZDKZKCUlhVJSUmjr1q0+Op6QkBD2uhgVFUVRUVG0ZMkSqqmp\noZqaGiLy1D3NHP3ZZ5/RpEmTKDY2lmJjY2ncuHH08ccf0+bNm2nz5s20fPnygFxeodSPSZMm0aRJ\nk8hms/lcd9u2bRQTE0NGo7FN7jl/6BA+Yb1eT2azmcxmc6sV6fDhw83ex+12044dO8hisZDFYgmL\nMmq+gyNHjnClaA2Xy9XqswZLGRVFoS1bttCWLVuotLSUnE4n+5bq6+t5APP0009Tly5duLHq1asX\nnT9/nq+7a9euFv3arSloMMra+8jLy6O8vLxG1ykpKaHc3FzavXs37d69m5KSkqiwsNDn3Zw7d84v\nfWtPxfenA4qPj6f4+PhGFb+iooJuuukmSk9Pp/T0dMrOzqaUlJSQ6Edrh1ZOzz//PNXV1bHf2uFw\n+AyAH3jgAdq8eTMdOXKEjhw5QrW1tbRgwYKgyNwWuRseH374IX344Yc+nZfb7abi4mKfwYTD4WhU\nhysqKqiioqJJfQmlzM0dDdvAQOIFglXWms98/fr17E9/6623KD4+vsXzpk2bxmVdWlpKcXFxIdFr\nVVUpNTWVUlNTW/Q/x8XF0RNPPNHsdd1uN+Xk5NCoUaNo1KhRZDAYglrWROITFgRBEITI4VdX3U4Q\nxFHg4sWLeSRls9nou+++49F3UVERPfHEE9SnTx/q06cPRUVFhXREqCgKzZ49m2bPnk0Oh4PP82c2\n/NBDD7W5DAKRWVEUyszMpMzMTHr88cdp5syZbDa/5ZZb2MSl1+tJVVWeCWdlZfk8R21tbYsyRUdH\n87mhKOuGR7du3ahbt270xRdfkMvlotLSUiotLaXk5GTS6/U+skycOJFnbESeKFN/ox8Dldnf6xoM\nBh8zXkPOnTtHJ06coBMnTpDD4SC73c7R9sGWuSX9Xrt2La1du5ZqamrIbrfTyZMn6eTJk/Tcc89R\nXl4elZeXU3l5OT3xxBONLA6VlZVBmzG0t93Q3oterye9Xk9xcXEUFxdHZrOZdDodrxjo1q0b5eTk\n+Nxba2/69u0bsrIO5Ni2bZvPfcaOHRvQ+cEs60BXfaxYsYJXKrz77rutzpzbWtaqqrI5PzEx0UdO\nVVXZCrVq1Sqqra1t8dp1dXV09uxZOnv2LM2ePduvOuhvWRMRKf9/gJASzLyeFosF7777LgAgKysL\nZ86cgdlsBgAsXrwYGRkZsNvtAICXXnoJH374IQA0GazV2qO3JrfRaMTEiRMBAL/4xS9QW1sLAHj/\n/fcxZMgQfPHFFwA86+OeeeYZjBkzhs8tLi7mta2B0pLcTclsMBgAeALbWguw0s6fMmUK/v73v/P3\n58+fR6dOndoiLoD2l3VDBg8eDAC45557kJ2djUWLFgHwBDU1vJeiKCgoKADgCUZzOBwcqNVaeQRa\n1v5y6NAhXHbZZQB+XHurcf78eUybNg3nzp0DAOzfvx92ux1paWkAgJKSkrDIbDQaOUBlxIgRSEpK\nwrp16wAATqcTMTExXI4lJSXQ6/X44x//CABYsmQJxo0bh48++sivewVbP9qKqqooLy9HfHw8f6cF\nMo4ePbpRQF+o9KMpuf7xj38AAAfHaZw4cQIDBw6E0+n061qRKmu9Xo/y8nIOfquvr8ejjz6KFStW\n+HV+IGXdVJInLSAsKioKM2bMAAA8+eSTrMPN3dPpdHIwrcPhQEZGht+Bqv50r2KOFgRBEIQIEfEl\nSoFiNpvxzjvvAACKiorgdDp5BPjcc89h7ty5PEvKysri5U2hmPA7nU7s2LEDALB9+3af//3rX//y\n+ZydnY2jR4/y58TERM4EduLEiaDL1lBOf9HKKS8vz+f78vLyoMrUXo4fPw4A2LBhAz7++GO2jjRn\n8UhJSeHPiqIgNTUVAHiGHE50Oh0++ugjznw1ZswYn5H8tGnTcODAAdx+++0APKPvgwcPhv0dOBwO\nXp7z7bffwul0+pRvVVWVTypNp9PJM0in04mDBw+GVd5g4Ha7cerUKQwfPhyAR1dmzZrF/wsHl1xy\nCaxWK86ePcv33bx5M1vdGmKz2QKq4+FG0+19+/bxLBjwzIy7desWkns29a602WtVVRVeffVVAMDc\nuXNxySWXNDv7P336NGw2G3r16gUAeOONN4KuBxddJ2y327Fv3z4AjTsXq9WKqVOn8otdvXp1SCtO\nIB37gAEDfD47nU6cOnUq2CIFjb/97W8+nz/77LMISdI02ns9ceJEq4OYhhWsvr4+oqkfXS4Xli5d\nyutIb731VmzcuJHNZdu3b4fBYOB1oWazGQUFBRFNA9lUytim6pZWJ3Q6HSwWS4fId+1tmlQUBXq9\nvtk0poqioE+fPqwzbrc7pOtENVcEESE5ORkAMHbsWNxwww28zvqaa67BwIEDm73GrbfeGjL5msPf\nHb6MRiP++c9/AvC47LwhIuzcuTM0ArZCRUUFAOCyyy7DjBkzuOyzs7Oh1+v5+RYuXIjHH38cRUVF\nAICrr74aSUlJKC0tDZosHaYT1h7aYDA0myNar9dj0KBBXLETExMbzQ5UVeV8tocPHw6hxP6jqipe\ne+01n+9eeOGFiCeMaCq/tdbwDxo0yOf7lvwmkSCQ5CGdO3eGXv+jqtvt9lb9qqHGu0PdunUrcnNz\nebauKAqsVivmz5/PnzvyTEcjOTkZv/3tbwF4dL5Pnz74/PPPw3Jv732SG+q0wWBgfRkzZgzmzJmD\nuXPnAvAkzvH+PRHhvffew1VXXQUA2Lt3b0gHP94yFxcXAwA2bdqELVu2oGfPngCAN998E6tWrcKo\nUaMAePTB5XKxf/7kyZMhk68pVFXlRDI//PADt8FutxuKovBs12w2Y/369WxVqKurQ1VVFbfPW7Zs\nwZEjR8Iqe0PKysp8fNLz58+HTqdjK2VZWRny8/Nx5ZVXAvD0QQMGDAhqJyw+YUEQBEGIEB1mJqxF\n3jocDh71ExFUVWWTTffu3VFaWsq/ra+vh81mQ1RUFABg4sSJ6N69O5satdRzkUKTa82aNZxmEfCY\norVdfiKJ9wyBiBAXF4fHHnsMgGe2aDKZOL1fpMvSm5ZmPU2xbNky/p2iKHjmmWdCKl9byM/P53eh\n1+sxduxYjvo/f/48lixZEknxmkSv12P16tUAPPqyaNEirquabzVctGRV0ul0MBqNAIBRo0YhLS0N\nzz33HADg7rvv5tUUgKdOmEwm3H///QCAjz/+OIRSNx/DUFtby3EP3333HWJjY31M5Bs2bGBLSbjx\nLuv6+npu29LS0jBy5Ej84Q9/AOBpM8aPH8+//c1vfoNdu3Zx+xwdHY3q6uowSu4fRIQ+ffoAAIYN\nG4Z+/fr5mN+DHcPTITrhcePGsc/j9ddf5wfW6XTo3r075zO22+0YP348+xF0Oh3cbjcHYhUXF+Or\nr77C008/DQBsxw83mulz2LBhANAooGLOnDkRM0UrisJLlsxmMzeasbGxKCkpwVNPPQXAE4AQFRXV\noTpfTS+WLVuGu+++G4AnsGrSpEk+DamiKGzaveOOOzB79mwfv/CZM2f89mmFA0VRsGLFCvzyl78E\nAOzYsQNz587lxm3ixImw2Ww+vw+H3Kqq8iBs6dKlWLBgAdephIQE5OfnIzo6uslzFUVptPQqEphM\nJuj1em7se/bsiUGDBvFA33u7SMDj5pg+fTr7DLXOOxT4+x5VVUVCQgJ/PnXqFBYtWhRR3dXMyKqq\n8lLLu+66CxMmTOD2b/To0VBVlc3527dv93GrVFVVhVlq/0hISMCTTz4JwNNWOJ1O1o+cnBw2pwPB\nqYth6YSbElRT/h49emD27NlISkoC4Gn8tQdWFAXV1dU+s7V33nmHz+3Tpw/uvPNO3vxcURTk5eXx\n2sRwv2RN7qioKNjtdlx33XUAPEnOgR9HkC+//HJY5fLGZDJx0FW/fv3w9ddfAwBmzZoFl8vFwSAu\nl4sboubQnhMIT2emBbctX76cy7p///74/vvvsWzZMgDAjTfeiAkTJnDj2VTU4/3334+33noLAHgD\ngnDTtWtX9gHOnz8fRqOR/WyaD+3RRx8FAHzwwQc+54ayrBVFQUZGBgBg3bp1GDlyJACP9Wb8+PG8\n0crKlSvZ0tPcdV555RXcc889AID4+HifzUFCjdZGGAwGn3bA5XKhpKSEo7itVitqamq44zCbzais\nrOS66j24ay9RUVEc70J+bDSjlW9OTg5vggAAe/bs4ZwEkUIrHyJCYmIiACAzMxNJSUk+Vj9FUXhV\nSMO4hlDpQXs6RlVVcerUKR5cDh8+HCUlJZxv4tlnn/VpU1RVbffmNpEfqgqCIAjCz5SwzISbGpVo\npqr4+HhUV1ezuaVTp048YoqLi8PYsWOxe/duAD9uVaiNWhcsWIDJkyfzVl5LlixBRUVFxLYK1J7T\nZrNBURSWQ5NXM8t07doVZWVlSE9PB+DxB1ZWVvLvLBYLjyYzMjLw+eef85rB9oweTSYTFi1axOtP\nVVXFf//7XwCeGUNNTQ1vRZaYmIjKyspG19BkvPrqq7Fjxw5s3boVgGe9nbfJNBRomYsazm4TEhKw\nfv36Jv/XFD179gzLDNg7UrS2tpbff1xcHGbNmsWf58yZg7i4OJ8t3ubPn4+//OUvIZexIVarFXv2\n7AEApKSkcF3Mzc3Ftm3beNS/atUqXHHFFRxF73a7G5mfb7zxRnz77bcAPFGmWsRxqPXEO2bAYrFA\nURTExcUBAG644QYkJCSw3lssFhARv4va2tqQuIo015m3X7c5FEXB5MmT8corrwAAy65RUFDQIdwo\nGlqbceHCBfTu3dsnXqOurg5XX311WOVpz+w0MzPTx8WiqioOHTrEFqC8vDyf6wdFV/xKbtlO0ERO\nTW0rrGHDhtH+/fvp3nvvpXvvvZdzGKuqSkajkebOncuf9Xo96XQ6+uabb+ibb76huro6qq6u5hy7\ngW631xa5mzvMZjOtWbOG1qxZQ6WlpfT11183up62A01BQQEVFxdTWVkZlZWV8W5G2laHNpuNt1Jz\nuVz06quv8n0URQlY5ujoaIqOjqbHHnusUZ5ULedy//79afz48fTwww/Tww8/TOXl5bRjxw7KyMig\njIwMmjdvHuXn5ze6n5YruKamhp/P5XLRsWPHfPIot7esDQaDT/5n7/u3RHV1NVVWVlJRUREVFRWR\ny+WiBx54ICT5ar0PRVGoR48elJiYSImJiY1085JLLuFc0FOnTqWxY8fy+87Lyws4J28wZAZAX375\npc/v169fT+vXr6fOnTuTyWTibel27drl8y7Kysqourqadbop3n//fXr//febrKet4c9za+1E//79\neQed1157jY4cOcL5iomI9bWmpoaSkpJIp9PxblGBblfXEg2f0ztHdUv1tKioyEevtTqm7QzWVr0I\nRVmnpaVRYWEhFRYWNpL3hRdeCDivdaj0urVD2z63YftCRLRu3TqaMGECTZgwgQYMGEDJyclkMBjI\nYDC02uf4Q8QCs7TgoClTpmDw4MGcR9fbJzx8+HCcPn3aZ2Q1cOBAn9yj586dw5YtW/j/keKll17i\nYCFFUdjH7Y0W7dqjRw8AP8qrKArq6+t5JuHta6upqcHChQv5c6DPGBMTw4v5G266TUQs06pVq7B3\n716W22Aw4JprrmH/emxsrM9aW+187xGn9k61/2mz+WBs7m40GtkPFh0d7ZN8wdsH1DDgatmyZbBY\nLJg9ezYAj49v+PDh7BOsrKwMit4oiuITxHPttddiw4YNXPYN17NXVFTw87z99tuYO3cuCgsLAXhm\nkEQU9uCxbt26sV9a47bbbgPgKf/S0lJeqzp+/HifmfuqVauwbt06tu7s2rULnTt39rFMaLnUQ/E8\niqLwO33wwQc5utlqtTayjngHJ9rtdrjdbp4JB3NNcMPndLvdPt95y2EymfDCCy8AACeO0PIhnDx5\nEqmpqZgzZ07QZGsvWt1+6623WF4NzfqRn5+PrKwsbkO0Z9faNy0YNJKoqoprrrmGY0SaauOKi4s5\nj7vL5YLD4eDfkZcVpc0ytOtsQRAEQRDaTMRmwloGphEjRsBkMvFoirx8vrGxsXj++eexa9cuAMCM\nGTOQnp7Oy09UVYXJZOI0lpFAmw1cd911Ae0+QkTs57Vardi3bx/71/76179yVOqMGTPalS0pMTGR\nZ+gul8tnXd6ePXu43N9++21Mnz6dI5BjYmJ8Ru7esx7A49fbtGkTZyU7fvw45s2bB8CzfKGgoCAo\nM2ANRVE4UlXz4wE/xhY0LHvtty+++CJGjRrF67K1WbE2M66qqvKJ5mwrRITevXtzTtqMjAxYLBbc\nd999AIBPP/2UR8yKomDw4MGcTSoxMRFvvvkmpkyZAgBYsWIF+vXrx1H02prWYKMtI9JmY01lmNMi\nX7OyshrpAPDjbE1bFqj5WpOTk6EoCmcWOnHiBJYuXRr8h/g/sbGxmD59OgDPTLgpWTVcLhfnxNYi\np8OREtTbl6iqKqxWKzZs2ADAYxH09qnX1dWxfqxduxZPPfUUunfvDsCTS+Hs2bO45ZZbAHgyrjWX\nZTBUaP7q4cOH+8hdX1+PL7/8EgCwc+dOHD9+nK1vqqoiPj4ea9euBeCxCB48eJDz+3/33Xe8WqOh\nrzXYOztpMi9evBiLFy9udoc4IsKwYcPw5ptv8vMlJyfj9OnT/Lm9RKQTVhSFl78sXboUd999N+cX\n9a482dnZSEhI4HXAn3zyCW699VafF6LT6bjj2Lt3b7gegdEq1c0338ybCJjNZlit1mbXSTqdTpjN\n5rCsFT59+jSbRKurq32WRjQ0je3evRuZmZkAPI1uZWUlD4AWLVqEnJwc7hgOHTrUyBx91113hew5\n3G43B6i01MBqaEkkAE9H0VzazejoaH7Gm266qV2mUrvdziZmLf+stmXaoEGDOMAwKysLffv25U0y\nZsyYgfPnz6Nfv34AwK6ZNWvWAAC7XIK5XAZo7E74z3/+ww29RnM6XF9fj9WrV+Phhx9u8fpNuWVC\nQVVVFQ4dOgTAN0hMS/GoDQi//fZbzJkzJ6T5oJtCC9LzDkrr1KkTv3PvciYilJeXs15Onz4d3bt3\nx/79+wE0HQS3cOFCPPvss6F+DGbjxo0APMlPtA7MZrMhNzeX02i+/vrr6NKlCwdUbtq0CQcPHsTr\nr78OABg5ciR0Oh1f68yZMzxo/eyzz3x0M9guDK09uOWWW6DT6bgzbWiOdrlc2Lt3L783m82GmJiY\noC4R6xD7CXv/31scnU6HrVu3cmaYIUOGYPDgwfjTn/7kc772EmfMmBHQy2rtt4GOvrQZRdeuXfHI\nI4+wcj755JMwGo28CUJ7i7yl89s7YtT87Z06dUJJSQnPwtu7Fq69ZX3nnXfijTfe8OteTz31FK8b\n1uTetm0bAGDy5MnNnudwOJCSksIDREVRWo1i9SYqKorz/Q4ZMgTz5s3jGcqsWbPYF/bnP/8ZV111\nFWcTys3NhcvlYt+ZluheK7PNmzfzelt/aKt+6PV67Ny5E7/61a8AAF9++SUnb5k8eTIyMzN517Ap\nU6YEtWFsr37odDpcccUVAIADBw5wo6rT6XD06FH2EX/++edBHfy2JLd3lDaARrELMTExrGsNO1W3\n280Dh4aR882hzZy1vAltkVmTrTVycnL4b21As3r1avTq1Yt1fvDgwaipqeF4iD179viUfVZWFqZN\nm8br0jdv3owXX3wRgGdW3LDNCVa7Z7FYePY9cuTIRv2Pw+HwSTAyffp0HysW/X+PYX/wp46IT1gQ\nBEEQIkSHmAk3h16vx6WXXsoZVwYMGICFCxfizjvvBOAZLep0OvbrBOrbC/ZMOFyEciYcKtpb1sOH\nD+cRd1O/1fxMjz76KGw2W6NRtGal8M6O1BC3242JEyfyjLS+vr5Fn09rMhuNRj7fewaQmZmJsrIy\nn9kE8KOJ7PDhwz6pFaOjowPyPbVHP6KionjW5Z1pKtSpMturHxaLhcto5syZ7IecOXMmxo0bx6kG\ng+0Cam9d1PT2d7/7HctWUlKCH374gf2uS5cuRVJSEkffOxwOXtXgjeZfzs7OblGuYFgdfv3rXwPw\npFj1dpPodDqua6mpqbDb7c3mbTAajRgxYgTneaiurvYx84ZqJty3b1/e771z585wOp3497//DcDj\nArrqqqvYtbFjxw44nU6uE263OyAd8qfOdOhOWFEUmM1mXjxtMplQUVGBSy+9FIAnV/Q999zD6f0C\nfRTphIOL996oDWUMRllr5txu3bqhf//+AICvv/4a33//vd/m8qFDh2Lnzp28TMxbtt69e6OiosJn\nD9xIlLWiKJg+fTqb3wNNct9R9aMl2qIfzT2LTqdjt0rv3r3xxRdfhCzwKlhlbTAYuJ27cOECiIgb\ne6PRCIfDwR2By+XCsmXLMG3aNACeAKdBgwbxhhmtlWV766KiKCxLUAKTvJIUaYOLsrKyRp1dsMq6\nS5cumDVrFgBPHMjOnTs5sFDzDwdryZqYowVBEAShA9MhZsLR0dEcAZ2Xl8ejh6FDh+LMmTMcbRfs\n0azMhIOHtlwM+HFBuzdS1uHjpyYzcHHKfTHKDEROboPBwC4Zp9PZKLXsT7GsgQ7SCXv/xjtTkLZL\nT6iW8nRUZWyNn6IyXoxyi8zBQ/QjfEhZh4+LyhzdcP2q5hOJ1L67Pzc6qhL/VJHyFgQB6ECdsCAI\ngiD83OiQnbCWlD8cqeQED5Hc/OLniMyEA0PKS/ipEhafsCAIgiAIjemQM2FBEARB+DkgnbAgCIIg\nRAjphAVBEAQhQkgnLAiCIAgRQjphQRAEQYgQ0gkLgiAIQoSQTlgQBEEQIoR0woIgCIIQIaQTFgRB\nEIQIIZ2wIAiCIEQI6YQFQRAEIUJIJywIgiAIEUI6YUEQBEGIENIJC4IgCEKEkE5YEARBECKEdMKC\nIAiCECGkExYEQRCECCGdsCC3mH2OAAAANUlEQVQIgiBECOmEBUEQBCFCSCcsCIIgCBFCOmFBEARB\niBDSCQuCIAhChJBOWBAEQRAixP8A0WS3jTvGnQcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d94997b00>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "PmJPlExp--5A",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "10. This function help to generate replayed data from the GAN use as memory (Gt-1)"
      ]
    },
    {
      "metadata": {
        "id": "4PyScms7yAxH",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# New function to generate samples for replay\n",
        "def get_replay(generator, batchsize):\n",
        "  z_ = Variable(torch.rand((batchsize, z_dim)))\n",
        "  if use_cuda:\n",
        "    z_ = z_.cuda(0)\n",
        "  return generator(z_).cpu()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ErM-LpqUyjgs",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###Training Generator with Replay\n",
        "\n",
        "After seeing what happend when training without Generative Replay, we can now see how efficient generative replay is in this setting."
      ]
    },
    {
      "metadata": {
        "id": "p5IEH98if_Ju",
        "colab_type": "code",
        "outputId": "ad6df33a-1a1b-4784-fecc-6b6f15d65a1c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 233
        }
      },
      "cell_type": "code",
      "source": [
        "# We reinit D and G to not cheat\n",
        "G.reinit()\n",
        "D.reinit()\n",
        "\n",
        "for task in range(nb_task):\n",
        "  \n",
        "  data = get_iter_dataset(x_train, t_train, classe=task)\n",
        "  nb_batch = int(len(data)/batch_size)\n",
        "  \n",
        "  for epoch in range(epoch_number):\n",
        "    for index in range(nb_batch):\n",
        "      x_=torch.FloatTensor(data[index*batch_size:(index+1)*batch_size])\n",
        "       \n",
        "      if task > 0 :\n",
        "        # We concat a batch of previously learned data\n",
        "        # the more there is past task more data need to be regenerate\n",
        "        replay = get_replay(G_saved, batch_size*task)\n",
        "        x_=torch.cat((x_,replay),0)\n",
        "      \n",
        "      \n",
        "      run_batch(G, D, G_optimizer, D_optimizer, x_)\n",
        "      \n",
        "  G_saved = deepcopy(G)\n",
        "  \n",
        "  z_ = Variable(torch.rand((nb_samples, z_dim)))\n",
        "  \n",
        "  if use_cuda:\n",
        "    z_ = z_.cuda(0)\n",
        "  samples = G(z_).data\n",
        "  samples = samples.cpu().numpy()\n",
        "  #plt.plot(samples[0])\n",
        "  \n",
        "  f, axarr = plt.subplots(1,nb_samples)\n",
        "  for j in range(nb_samples):\n",
        "    axarr[j].imshow(samples[j, 0], cmap=\"gray\")\n",
        "  np.vectorize(lambda ax:ax.axis('off'))(axarr);"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl8U1X6/z83a5OmaWlrW3ZZCgWl\nLLKvVVAWGdyQcRcUFREGlEUFHR1wRMAXKCriKI4i4IgijMDAgIyCIGvZEaWAgLKUQuneJm2S7x/3\ndR7uTW/ukgTi7/U7739oyM2955x7luc82xECgUAAHA6Hw+FwrjmmWBeAw+FwOJz/X+GLMIfD4XA4\nMYIvwhwOh8PhxAi+CHM4HA6HEyP4IszhcDgcTozgizCHw+FwODGCL8IcDofD4cQIvghzOBwOhxMj\n+CLM4XA4HE6MsFyLh6SlpeHSpUuy/2OJugKBAARBgCAItX7HvjObzQAAv98Pv9+P4CRf7LeCICAQ\nCMi+l35nsVjoXgkJCcjPz1ctd1JSEkpKSmTlMZlMsvv6/X5ZfaTfBf+f1Wqlv30+n6wNQpU7GIvF\nAq/XG7LMbrcbZWVltZ4vhbUBay9pGaR1NYq0zIIgUFvVqVMHFy5cUP3tddddh8uXL8v+T608odq2\nuro6ZP9gv5P2N0EQar1D9l18fDxKS0tDljk1NRVFRUWKv2V/+3w++ltv+5pMJlkZ2T2k9w3uh+x+\nLperVjtKSUhIoP4hvZ8UaV2C6yX9LB2bgiAotr0erFarap8GxDmksLCQyqDWltK2Y+3EYO9a2n7S\n+tfU1NS6v/S+7N0AYluyMinRuHFjnDlzRvbcUGVm91erU6jfS/tyqHux6+Pi4lBRURGyzADQoEED\nnD9/nsodraSKoeY0JYKfabfbUVVVFfL61NRUFBcX0+eamppaz2Z9VXr/QCBQ692otbWe7xlOpxPl\n5eUhy0z3uxZpKy0WS62JJNZYLBZUV1erXmM2m2u9oD8Caq/sj9jWeibZSMotHWDBgy8SpAu0Ejab\nTbMPhYvJZJJNFGpIF0uz2azaBlerTwcLGUbR+u3VHIvhll2rrePj4zUXvFigVVer1RrVcRQt1Mr9\nRyyz1vzB4OpoDofD4XBixDVZhPVIuaFU0nowm810Dy3YcyKVqpl62+hvIqknu4caUpVLrGF11bPL\nCLdNpG0ajfY1Uqar2dZmsxlutxtut9vQ766WYkuq8lRq46utUNN6F+G+f6ZiZnOI3t9Eq4+Fw9Xo\n68H3v5pcjXJfy3kv2mW/JjZhtQHKBoHeRVE6qdvtdjRs2BAdO3YEAKxatQqVlZWq9zIyWWiV28i9\ngu3R5eXlZOOQ2prZZ3Zvm80Gn8+nu32MqmSU7KXs/yMVVFgd9KiZjTxLahO12Wzwer0y26v02Uqw\nSRcQ1VihVIZagy2aqmhpf3I4HGjcuDG9S6mtS+33VxNmvomLiwMAeL1emb3QbDYr2tf0oKfsWveV\nCsWBQED13Ujtus2aNUOHDh3Itrtx40ZVO6iRdo5W/5AKCgDQrVs3mEwmHD9+HABw+vTpqDyHEem4\nZ+UM9nWJj49HcnIy2UlLSkpkcz/z+VHiWgo90jKxfiWdw+x2O31fXV0dcXtdk0VYCTaJBgIBmZOS\n1m8yMjLot3PmzEGXLl2Qnp4OAJg1axbeeecdFBQURK2MwY410kle7yLsdDqxfft2NGnSBIA48e/Z\nswfPPfccAGDXrl0yJx72bABo1KgRzpw5A4/HQ+VQw+iORFofs9kMh8MBAKiqqjJUx1hRU1Ojusiz\n+jmdTgCiQLBixQq0aNECgLiIt2/fHgBQWloKj8dDba+l6VBrG5vNhrS0NHLMUbo+JSWF6uB2u2mx\nveeee1BVVYVt27apPj8cjE5m7HqLxQK/3y/bcUjHR0pKCoqKiqiO0baVS+2vSgIwm0ekZVaqR1pa\nGtavX0/vuHnz5vD7/SQQ9+3bFydPniSHu2Ckc0Cw05fStZEgnQMmTJhAZRozZgw2b96MBQsWAKi9\nCEdqn2cCFaBvnpP2EZ/PR4Iac9Rjvx8xYgRatmxJ2p0JEybA5XKR0yab49SeEQqHwyHzO9Fbf1Zm\ndn+r1YrU1FRaWOvXr4/8/HwSHIqLi2X9Xm0DqXescZswh8PhcDgxImY7Yan0oCW1MIkwMzMTs2fP\nRnZ2NgBRqpWqeQcOHIi9e/di5cqVmvfVIykpldGIhMnK9eqrryIrK0u2i2jRogXuueceAEBeXh6K\ni4sV733x4sWwQz+MYDabYTKZYLPZAFzx/K2srAQgSprV1dWy0KyMjAycO3cOgHJoECPaZZfuerRU\n3SaTCS6XC5MnT6Zy9unTh3a5ZrMZLVu2BAD8+OOPAK68d617a3lOd+nSBWvXrqXnsrarW7cusrOz\n8dZbbwEAKioqYLFY8N577wEA2rVrhw0bNmiGdTGuZt9g966qqlIMuWJjMzk5Ge3bt6fd/M6dO2tp\nd0JhVB2tVF9pn1Cid+/eAESTVUJCgkzd6Pf7qQwvv/wyli1bhuXLlwNQNu+w52upISONUmDamilT\npuC2224jDV9JSQk++eQTbNq0SfF3kfYHNbWwEmwsCYIAm80Gu90OAEhMTERycjKaNm0KAHjllVdQ\nVVVFoaGNGjVCUVER/b66ujpkm2n1Eenv9NQ/ISGBynD58mUySXXq1AmPPPIIrr/+egBiW8+cORNn\nz56lz1VVVYqhqcHo3o3ruipCIlGPCIKA5ORkAMCCBQvQs2dPioMsLy9HIBBAYmIiADHWdN68edi9\nezcA4MyZM1c1xEirXqwztmjRAiaTCb/99hv9zmq1YsCAAQCA1atXY8eOHbIBz+5bXFwMq9VK91JT\n2egpUzBsEk1JSYHX68WNN94IAOjTpw8yMzPx5ZdfAhA7ZyAQoAH10UcfYcCAAVi/fj0A4PDhwzRJ\nhPOujfgF6IEN2vr162PQoEEYOnQoAGDu3Lnwer2kMhMEAdOnTwcADBs2DBcvXpTFpWs9I1RdU1JS\nMGrUKHz//fcAxPjMhg0bAhAXKKkjEFNxPv/883RtXl4emjdvDgA4cOBAWG2ghFSVZgSz2Qyfz0f9\nLxAIyGLSk5KSMHjwYNx///0AxMlNbVKVotcUFa692Ww2k+qWmVsuXrxI9RIEgVSkHTp0QOvWrUl4\nKi8vjyh0Ltx4++TkZGzYsAGAGANrt9spZr2iogJVVVXIzMwEcKV/KMV3XwvYs5xOp8w8lJaWhvnz\n51M/r1OnDiorK5GamgpAbOsDBw7QOykuLsbvv/+u+oxQ2Gw21ThrKWazGQMHDgQAvPbaa3jooYdw\n8uRJAOK4vfXWW8mcYrVa0atXL+p7b731FjweT1TbOuaOWUDtikidgxwOB01OHTt2hNfrJSl19erV\nAECLdIMGDTBx4kS89NJLAIDZs2fj2LFj0a2MhFD1MpvNsFqt6NOnDwCgSZMm2L17N95//30AQFlZ\nGZ544gl06NABAPDwww+joKAAP/30U617MemSTX5aE5GRDmG1WhEfHw9A1DKcPXsWf//73wEAbdu2\nhcPhQP/+/QGIndPn89Gke+bMGQwYMAA9e/ak7++77z4AwJEjR2Q2wWg43uhFEARkZGSgcePGAIDN\nmzfD5/NR8oHt27fj119/JW2KIAho164dgNqOZJE4gxQWFsLr9ZLEPWrUKNx7770AQNoGaZkDgQBd\na7fbcffdd5ONq6CgAGazmaTxSNoq3N/6fD6YTCbUrVsXAMh/gE1eDocDAwYMoDr06NED3333Xdjl\nDMaoYMmErKFDh+KRRx6hBUsQBFy+fBlz5swBAKxbtw69e/fGs88+C0Bs+3r16uGmm24CgFp1MDLp\nhtvWb7/9NgYPHoz69evT/wUCAarT5cuX0a9fPxKQk5KS0LJlS5w6dQoAkJ+fH9HCoEf4ZJhMJkqU\n4/P5YLPZ8NBDDwEQhdrOnTvLnPfi4+Pp89ChQ2G1WknwP3bsGObNm0f3kqJVHy2HXCmTJk0izVhp\naSnKy8upDunp6XA4HFRHh8OBnJwc2q0vX74cXq+X7PDRcALmNmEOh8PhcGJEzNTRUu9o6fcmk4mk\njuzsbPTq1Qt//vOfAYi7i/Pnz+PVV18FABQVFcnCCQRBQNOmTfGnP/0JAHDnnXeSNBls29HybAy3\nnoAY9tC1a1eMGTMGgKg+WrRoEb766isAQMOGDdGqVSvyjh0xYgTuuusuUtNI28psNqNfv34kkQen\nHDRSruB7OxwO2hG+//77SE5OJk9z9pvrrruOPvv9frJTvv/++8jOzsYtt9wCQNxBMDPA1q1b0bdv\nX0M7gWipo2+66SZs2LCBTBRM9V+nTh0A4g7+6NGjJH1LQz+Yyt9ImUOpKv1+P9LT00mL89hjj5G0\nrXStNOzKYrGgRYsWGDlyJACgadOmaNWqFX0+deqUqhYm3DIrwdqEjVPWp/1+P/Ly8ihM5uzZs9i/\nfz/1F7V0joBxLYNSRjXWXqz92D2zsrLw5JNPAhA9iaXer6dOncKSJUtoJywIAgoKCkiL07FjR9hs\nNnTq1AkA8P3338va2mq1yjxjownTyIwdO7bWd1KVeVpaGlJTU2mey8vLQ05ODk6cOAFAtCGvWrVK\n8Rl6cymEgu1mpWYJRmJiIjp37ozZs2cDUB5P0veUk5ODLl26kMbn0qVL+OijjwCIc6YRO6+ePu1y\nuQCItmk21zDTJotc6dy5M+x2O3lDe71eNG7cmObF77//Hk6nEzfccAMAyKIfwiVmjlnSRpW6/cfH\nx5MKdMiQISgsLMRnn30GAPjss89w7Ngx1VjY0aNHk+01MTEx6rYRNskpvXSmZly4cCGysrLoBY8b\nNw7r16+nF3/8+HFMnz4dc+fOBSDaAJ1OZ8gY1++++45yWIdTn1C5T51OJ4YPHw5AVOWbTCZqW5PJ\nJEud6PV68eGHH+Kvf/0rAFEAuuuuu7B//34AQMuWLaltDh06VOv9GimjUTuLIAg0SF577TW43W7Z\nRFJaWkoqssrKSly4cIFCUuLj4/H5558DAKmspffVW+ZgKioq8MUXX5BgVV5eTpOAxWIhNRgg9qWi\noiKyqVutVrjdbupjPXv2RPPmzclUMGXKFFI9KrWFGkbiyAVBoAmSOWEdOXIEgNiHT506RWaHo0eP\nYsqUKSSIHThwQPVZSuYnNbTivqVq0W7dutH7Zg6HLHf95MmTsWrVKmrbQCCA8+fP44033gAALFu2\nDABkMfxSfD6fbnOFkX5ss9lCOloBoqAhjYOOi4ujcrRv317mc5CWlqbrmeHg9/tRWVmp+G7T09Px\nxhtv6BZmrVYrLl26RPOG0+lE27ZtAYhhm0aERa20uAAwePBgeu7Ro0cBiCrx48eP03vat28fXn/9\ndQoHa9iwIcaOHUuOfUlJSXA4HCSMvvTSSxE74HF1NIfD4XA4MeKaqaODkaqQWSIAAOjSpQs5NHXt\n2hVPPfUUDh06BAC1TmJSorq6mhycevXqRY4kwV53ej0ypVKOVro1pk7q3r07qqur8fPPPwMQHZWk\n96qpqcG6deswdepUAFe8H5kEycKCWH2kJ3FEIn0Hl79evXqUbczlcuHw4cO0G7RYLOjUqRNyc3MB\niKEdCxYskO0QPB4PXnzxRQDA119/TfdmHtRGkCbo10qMH/w7u91OO/RevXrBZDLRrjI/Px/PPvss\nOeh5PB788ssv5B3rdDoxceJEAMYdabRU6O3atSP1rN/vx549ewCIO4bBgwfLvEq7deuGf//733Tt\njTfeiJtvvhmAeDJWWloaunbtCgBYvHgxmQHYqT9S8060CL6X3+/H//73PwBiHw3Wdpw8eZL6jxFV\nbbhlDlYTs/EzaNAgJCUl0XeVlZUYPXo0AODbb7+lLGvSezCNhcfjQVxcHBo1agSg9niShjNpYaQf\njx07ljQlrFxsvjh//jwSEhLIsfP222+nEEFA9CpOS0ujOjGv+nBRq59SciX29+uvv06hPVJYGwQn\nvxEEAQkJCaTattls1H/CSSqjpS1hXth+v592wg0bNpQ5xHq9Xhw+fJg+nz17Fj///DOZEtPS0tC6\ndWsyZzVp0iRi59+YeUezQWo2m5GQkEAdcObMmbRw7t27l7xbjbBx40YA4oQcahCE67GrNunOnDkT\ngFi36upqvPbaawDEjDbBcY6///472WP/9re/YezYsWTjYSoxpWdGoiJl37HJ6osvviAv4urqarz9\n9tu0EDRq1AitW7fGt99+C0AUgJSyIP3yyy+17i914deLVJ2oh86dOwMAnnzyScydO5fUjz6fD2fP\nniVBgE1MrN0SExMxY8YMUoF9/fXXYdnZAe20plKV8pEjR/D6668DAH744QdUVlbK3uXevXtlas5z\n586RENe8eXMMGTIEDRo0ACAekcdCrpYvX04LcbRREjKYICyNlWTU1NRQmfWoB42gFq/L/BXYnNKz\nZ0+Zbfzw4cP45ptv6D7BAkIgEMDSpUsBiAL0yJEj0a9fP9kzgp+pB4vForoIm0wm6qdvvPGGrFw+\nn482DpMmTcLRo0fJ5nvixAmkpqbSYltYWIiHH34YO3bsAADs2LFDtuBF+3Sh4PfO7Kk5OTm1xrDX\n66U5pWvXrsjIyJBdwzy+AbFvSesUHAqkhtZ7adCgAUaNGgUAOHnyJMaPH09/a923qKiIzHYPPvgg\npk+fjpycHADAe++9hzvuuAMAVI9aVCMmO2FBEEjSdLvdePXVV2kibNasGSUtWLRokWZKQqXGZ1Kw\nyWRCq1atANS294VLqM7gdrvRo0cP+nzy5ElKOxhq4WZ1XrZsGcaPH4+3334bwJUc2Ox50sMptBxv\ntHbCbrebHCeaNWtG15aWlmLFihVUpsOHDyMvL4924aHuyWw47P4AyCnKCC6Xi56tNeDS0tLIjlu3\nbl307duX0lL+8ssvmDBhQq14ajbw582bB5fLRTb2p59+2nBZGWrlbNCgAfr3708Ds7y8nJ7J7IpS\nQTTYCUUaVtWkSRN4vV7Z9e+88w4A0aYtHSfRzLEr9QlgC53UKUmakhAQ2/iZZ54BIMZC6wlNBCLf\nvTOfErYLYw6P7Lv58+fLtCw2m63WwsSEhh9//BEjR44kgSf43UjLrSU0SnNUK9WxQYMGFJITvEss\nKSnBli1bAIi795KSErrHN998Q+GaALBp0yZ4vV6KI+7bty9peqqqqpCbmxtW7gAlgvtXeno6xVQH\n24Lz8vLIBs+unTt3LgmQTAvKFmKbzUYOfcnJybJzsY2GuUoxm83Ytm0bObYVFhYazrfNHFI/++wz\nTJo0iea4vn37Yty4cQCAd999V9f5wcFwmzCHw+FwODEiJupos9lMIQEDBw5EdnY2qbkOHjyIadOm\nAQidHUpL2me7D4/HQyocrTLpuUYtNV5ycjKlPispKUGHDh00k9hLPfJqamqwd+9eWfnZNUqZtEKh\n1TaJiYl48MEH6VpWnwcffBCXL1+W3V8rOxcAPP744/Q3220wydAIHo9HMexBCpO0V65cSUlGjh8/\njo8//ph2Lq+88oqiepnZg4YMGQJAlFqBK5mTlNCyEat9z2y2bLeyYcMG7Ny5E8CVww2kp8cowa4r\nLS1FcXEx6tWrB0BsH3bfQ4cOyTx2tTDisSsN7cnIyEB5eTmFZmzbtq3WfeLj4xUTzoRCr6pRem0o\nampqKDpBqmUoLi7GypUrqS0FQVBMsco+b968GYFAgNTXau2qVSY1s4zNZsPs2bPRq1cvALVNTvv3\n7yeNVfBhEpcuXcLmzZuxefNmAKIn/qFDh8iuPX/+fAqNW7NmDfbs2WOorfX4RrDx9+GHH8rswH6/\nn9IG33fffbI58Ny5c9i5cyel62XmDjbu8/Pz6V6hDtAIhdq7SE9PR2pqKj2nW7duhv0/2Hg7e/Ys\n2rZtS1rOOnXqkN/HH1odrWRbuvvuuwEAbdq0gc1mo06zdOlSzdM0tDo/m3AtFktI9YCeicjIhJWT\nk0MvYe3atYZOkfF4PCgoKCBBRJohi5VVmrNZDaV4SobJZMKiRYtkdhgW57lr166w7Ljdu3enzyxv\ncCjBRw2tk7Ti4+PRrFkzAKIaj6mtVq5ciXfeeadWtqtgFi5cCODKaUr//Oc/DZcxGLUJbffu3RAE\ngcwKW7duNXyyEDM9FBYW4uuvv0bfvn0BiOn+mKqaqbiN9A8j9lrmJLlu3Tr06NGDQsGA2uM6LS0N\nkyZNAiBmfQp1RCQrrxHVuVZ8s9VqpTzmlZWVVMfz588bUhEWFBSgpqaG1I9Kc5deNbpa/VJSUtC3\nb18SsKuqqqhPp6en46WXXpL5W0jx+XyorKyUhUtKQ5j69etHdS4vL6fYWz1l1ip3IBCA1WrFrbfe\nCkB0EpNeP2bMGEoRqiTo5Obm0vWs3GzBTU5Opjp5vV5D86/0WvY3e07Lli1htVppk8NyV4fLmTNn\nKFvZ2LFjKReFkulCD9dkEQ7uxMz7ExB3NwUFBdRRIk3LKAgC2YHz8/NldoVw0JuPdPjw4WTkZw4A\nRtizZw/lGVYSQtizteqvdahAs2bNZHViweZ6zqwNZv/+/STwAKB4bql3N6BP+laSItnvBgwYgAkT\nJlC60osXL1Kc3u7du1UdTwRBQHZ2Njm9sVjocAQFI/Tv3x8mk4mk5Mcffxz/+Mc/AOibCC0WC8Um\nZmVloXnz5jR5HD16lJwPXS6X6vm3wUjtlFplMZvNWLx4MQCx7zRs2FAWrx4c61tWVkbvUc8xkNIE\nPVroSdbAtCGVlZWkDXnooYdk/UPJli2lRYsWEAQBWVlZIZ8rPUtWq0yhGDBgAFwuF40VdtAIIM5b\nu3btUnUqlWpDGCyFrM1mo76yY8cOmUZD671olRsQFzW2S5f2JY/HgwULFqj6pLDc4uyzz+ejVKcV\nFRWkmSooKDC0KVDKO8Hq+thjjyEQCFB0QqRxvcCVnbogCFTmcP0xuE2Yw+FwOJwYERN1NFNpsL9/\n/vlnXTp6PWqgmTNnUvL1EydORJwKUcuWwv6/V69epAKKi4uTqeJYSsJQx7EJgoDMzEySCCM5KFpt\nV9G5c2fUq1dPdg/mZawV08ikb2aXHD9+PGkzAHF38MILL9Sqm9JnvbBd5LJlyxAfH0824R07duDg\nwYP0XDXGjRuHxx9/nOyFfr8fly5dikqKTJvNFnLX37VrV9nRkG3atKE+r6UOtlqteP755zFhwgQA\nYqq8S5cu0e+bNGlC6sC6devi0UcfNaS2k6Z7VKNp06YULggA119/PdnCggkEArj33nvpnUk90PWU\nKRykv2vatCllRLLb7aTh6dixI/bt20d1jouLQyAQqOX1z2yckydPhsViIe9Xu90etq1PSZ3Kytym\nTRsKzwSuaGgA0XShFnbmcDiQkJBAc0xpaSm6dOlCccRms5niYJcuXaoY0xsJzz//vGIugBdffFH1\n/k6nE507d6Y2qKmpgdfrpfHg8/nIw9tms4XlaSyFqbZbt24NQRB09Ue9KnCWl8BiseDhhx8GIMZJ\nM5OIEWLimCUIAsXApaSk4Iknngj5W2mjsKQErHGtViu8Xi/lBc7JycHw4cNJ9z9s2DBV1YgWwc9W\ngsWrCoJAMZLBEzMARZUhmxiys7NRUlKCTz/9lK4NhR4HlVAwdRe7R01NDQ0mpfaQTtgZGRnIysoi\npws2abGydu/ePezJSnofKcy5iZ28whalvXv31nJqYuUXBAF169alxeOFF15AnTp1qM7r1q3DXXfd\nFXY5tcrMFsqmTZvK3pXdbqdwqHfffVemEmMJR9iC/dRTT+Gpp54ioWPevHnIz8+n5B19+vShRaJr\n164yO6+etJV6zkIFRHsqM43ExcVhxYoVIScZi8WCnj170uRvZDLSIxAplVUqIGdmZtJ7PXPmDIU5\nfvnllwgEAvSMysrKWv4HgiCQQDlw4EAEAgE6sUpP3L2R79n78Xg88Pl8NJcJgkDj5+DBg7KxxxzN\n2LWDBw+mWHJAtPt+8cUX5O8QCAQwYsQIAFfsn1o+E3rrZbVacdttt8n6GZtzWIhlqDqzdLLSsZuX\nl0fOfFu2bKH+YzR2X2nxZHW9ePGibudFPQvwDTfcIFPrM/twuGGw12QRDjZY+/1+sg08/fTTFL+l\nhLRRTCYTsrOz0aVLFwDA/fffj927d9PkdvToUXg8HpqsI5Wk9MBsolLp2ul0yp4dajCyZB2A6CjE\npEAl9O4W1BxY2rVrJ/OILi0txX/+8x8AtSV+FlPMYq4XLlyI9u3b00LOBtIjjzwCAJQzOFyUdv+D\nBg2iv30+H2VOkwoaLNsaO86yZcuWWLt2LUm9derUQVFREe0apJmqtNBqc4vFUst+z+yE//rXv2Qx\n1GazmeIjt27ditzcXBrIo0ePxssvv0zZyTp16gSv10uOetu3b0cgECBhMykpiWxSDocDiYmJZNPX\n45Ef7MASipKSEnJQatSoERITE2lxAsQ+w+rvdrvx22+/yRYCvRgViJV4+eWX6e/8/HyazFm/UrOD\nWywWciZiY4AJxA6HI6SjqFYdlcYi6+Nbt27F008/TQttfHw8XTt+/Hh8/vnnlBHr9OnTGD16NPnN\njBgxAm63mxJb5Obmkj0cEMdKODsyhppWzG63w+VyyfoQy9UfyomN9fO0tDQkJydTnUtKSnDo0CF8\n+OGHAETBhPXjSLLXsb7C6rBo0SLk5OTQ/GrE4UuJffv2yT6zXXG49+Q2YQ6Hw+FwYkTM0lYyFcSJ\nEycwZswY/OUvfwEgqjuYBMNO7GC7zZ49e+Lzzz+nEBur1YrevXvT/dPT09G2bVvyVotGJh4tpFlz\n2K5RK5OOy+XCrFmzyGN31qxZWLJkia5TZ7R2DWoql6KiIlLpA6L0zY6JLCkpwZEjR2iny8wELB/2\nLbfcIpMgT58+jT59+ujKPBNuHGi3bt3o75qaGlKBS1MP9uvXD0OHDqW4c4vFIrOvLVmyBNOmTaNT\nh6JhE2OoSesbN27EuXPnqE+4XC4K59qyZQtGjhxJYXk33HADBEEg79aioiJ8/PHH5E1dWVmJ1NRU\nism22Ww0BhwOB+rVq6doAlHCaP1ZGaZPn47u3bvT7v7UqVO45ZZbKFVr//79kZubS7nEjaCnTFo7\no//+97+UutRut9OOPNRpSAxRcsxZAAALs0lEQVRBEHDPPfeQN3RRURG++uorsn2rqUUjyU62c+dO\n7Nu3j8LuHA4H9RWbzYadO3eSvbiqqgolJSV0be/evWvZw4ErEQ6jR48OWd9I+39CQgJKS0tlURGh\n5ru4uDh07NiRTCczZsxAeno6lT0uLg6rV6/GgQMHAFwxFQDybG3ssxpKdm/275YtW1BYWEhzWYsW\nLSjfs97IAlbmHj16yFTRly9fjkjrAFyjRdhut9dSk7HOfezYMUycOJFycSYmJtJLYUelDRs2DID4\n0oJd7H0+H024gwYNwsWLFyOOhWPouQ9zEAJEBwBATNdWt25d2XFpgiBQZzx58qQsWXthYaHqJGpE\nfSI9CCEYp9Mp68xWq5ViT7dv3w6TyYQZM2YAEN9DVlaWTEVdVVVF6shWrVrpjjcNV93IzBSCIMDh\ncGDXrl0ARPUbs5/m5OTUqpff78dzzz0HQFyEo53HmGGz2WpN8qyup06dwpdffknJQZgAAYjv6KOP\nPqqV5o+p9ZYvX44333yT7m21WrFs2TKK2TWZTDTwrVYriouLqT2inSd469at9PegQYOo337yySd4\n4IEHKC+wy+VCbm6urgQv0cZkMiElJYXCETMzM3H77bdTuVasWFHrN+w9vfDCC5gyZQoJNeXl5Viw\nYEFUTFlqpqHS0lIsXbqUkp/cfffdtAjb7XYkJyfLwoocDgctIna7XTbGFy9ejAceeIBU8uydRFLu\nUHg8Hvz+++9k/rFarWSrrl+/Pk6fPk3OeTNmzMCwYcPofg6HQzYXsPzYaoKSNFVruFy4cAGHDx8m\nQWvBggV0LGheXh5+++03ek7w+DGbzahTpw45+rFxxlA6+9koQiCaW4MQ2O32kBNhu3btMGvWLGog\nt9tN9ry4uDgkJSXVkrRYkUtLS7FmzRrK0MROztGD1EFA7Rot2Mtbs2YN5Y52OByYM2cOZWUSBAET\nJ04km2B6erosN3DTpk11T55aSQvUknXcdtttWLVqFbVnTU0NJQhgh1Uze7vVakV8fLxMK/HSSy9h\n/vz5ALRtj1L0tLVUA8Jgu/SlS5fSoRiA6GjB8gMHD4rCwkL06dOH7MdGCBYEtMrtdDpVJxC73Y4B\nAwYAEG2MbKJnO3Vp2aurqykR/N69e2WenJmZmdi5cyct2oFAAI8++igAUQg8fvw4lcNkMqkKHWpC\nWqjrAeCDDz7A3LlzSZA8ffo0qquracxJfQ3CIZLEF4IgIDU1lQT5adOmyZyWRowYIfO3aNWqFbXf\niBEjYLfbqe+1b99e96k4WhEFamORlZu1b1ZWFmbNmgVAdLwrLCwkwe3gwYNYu3YtCTjjxo3DgQMH\nKCfzJ598IvNPiDTXglq5XS4Xpk2bRrkQ4uLiqA02btyIvXv30mEHaWlpiosnK9+oUaOwcOFCxfIG\nj0XmhBsKtX5ttVpx++2344EHHgAgHoDCFtXy8nLs2rVLlm1Nmvhl5syZdIawFHbm/fr160OWCdB5\nWp/mFRwOh8PhcK4K12QnHBcXp6qmcrvdFOozbtw4Uus2btxYlo6NSTosHm7u3Lk4deqUoby5Uozs\nhLVUwm63Gx988AEAMWdqWVkZSdQWiwWtW7cmVW5FRQXWr19PGbbmzJmjeyegteNQ2lEykpKS8MMP\nPyAtLQ2AuBNmavCSkhIcPHiQsjTZbDakpqbSEW+bNm3C4sWLDak7pWfcarW1w+GoFeLEfv/BBx9g\n+PDh9L3f76ddgtlsRnFxMdnDunXrhvz8/KhkxdHaCbvdbjKrhPKAZ+8tPT2dVFd33HEHZs2aRaaA\n3NxcNGvWjLz6GUz9XFRUhDVr1tBOuLS0lOyDwfGkkfQPNZo3b47jx4/TzsbhcCA5OZls7ZGix9NY\nKwaVfT9nzhyyk/bs2ROffvoptV1SUhIGDhwoO/dYekzgM888o9l32Ds1m82qGiEjbS0IAqmjb7rp\nJmzbto20Dk6nEykpKRQCOWTIEGzatIm8542ozvVopdR2wnFxcViwYAEdgdq7d2+ZHbesrEwW+wzI\nTTUej4e0VF27dtVsn2i0tSAIyMjIoHWlc+fOZCZq164dtm/fjpYtWwIQ7eoZGRl0r5SUFHg8Htop\nT506FevWravlIR0KXabRa7EIG1GBWa1WSkl47733wuFw0AA6f/48Jk+eTOnHjLqxB+cX1fq91sAP\nhsUa7tq1C2VlZTI7XUVFBdq0aQNAfNFqMcjS74Ld/rXU0WptbTKZ0KlTJ+qAN998M6k9L1y4gPLy\ncjILxMXF4ccff8Qrr7wCQO40YRQ9ba1WbpfLhalTp9Jh23feeSc5bZ0+fRrDhg0j9VI0knDoLbdU\njannuUwdXa9ePZw7d07mM2CxWEjIsNlsMJlMpF799ddf0bNnTzpYvKysLOSEE4m5Qg9sYpXm/40G\nkS7C0jJdd9115PQ4depUFBQUoEWLFgDEtq2uriYTFovxZJsELfWxVDA3m82qKlKj84fac6WwYz/V\nnM3Yd+zYRqkQr2VKUhuLLA6/Y8eOAMTQStav7XY7ioqKyCmO2YuZyeKnn37Co48+SjkijJpF1BZs\nrTVGqvpPTk4mJ8f7778fS5YsoUQvSUlJSE1NpbzdKSkpWL16NfWTgwcPGjLFcXU0h8PhcDh/YP5w\nO2HmCQtcOT6NqTfKy8vplBOjKKlGtKR4o5Ise0bHjh1x9uxZ2lUeOXIEJSUlqsk4WJkAUdXHJGy7\n3Y7y8nKZx2IkOx2Hw0GS6sSJEyng32QyycIgDh8+jE8//ZSO34tW+s9QaPURs9lM97Db7TJvxqvl\nkavHMUv6bD1qPkAMkThz5oxME1JVVUUSNut3TJPi9XoRHx9PpgMtTYja+IiPj1c93UgLafpHu91O\nHslqqVaDUTLtRGMnzDCbzVTOzMxMVFRUkNPkr7/+SuF4wJXjGtnYU6tDQkICPB6P7Bq1XZG0T2uZ\ns4J3uyyiQvo9+31CQgLKysp0pbZ1uVyoqKigXaDf79fcyelxKGPPuP766ymF6mOPPYY333wT/fr1\nAyA6aqWlpZGZzu/3G9acsOdYrVbVca6nzNJr2TzXvn17rFy5Eg0bNgQgjrUmTZpQhI7T6cSFCxdo\nPjaahEZXOuZrsQhHEk93tbga6ujg+0vTzoWLUY9dI2o7p9NJE315ebnMk9Hr9UZN1QhEfg5yrFAr\nd7iq3Ugz9qih58i/aIQxsfelN/uWFtFchJVQy5ilF6baZeNPq62jpY6OhOBshVplZr8xInSztlUS\nxKJVf4vFolvg+SOhp/4xO0/4/wUimVgCgUDYUh/7vdY1ob7Xm9O2srJSFmJzteJp9XC1+0jwomHk\nN9EmlhNztJ5tdCerRjTSVmoRjXqzlJ9/VIFRieA56Gr0PXbPaArtoZ4Rij/qIqwHbhPmcDgcDidG\nXJOdsMPhMOxGHwojak2p1MrsGExaCk7yoERcXBzZ4aTeyux+wUcTMnUu+1t6Wo2W557aLliq1mL2\n3FA4nc6QqfakaqNglCR8Na9tabuya0O9m+DMUEoEH/8YTaT1Ze9Fz46AeX2qfa9WZqX2k/4drMoN\nNjuESlWq1j+MlDmSXRF7ZnC/1Wv3lNZPq8yA/IARIxnx9GqU1Pq11J/EarXSjk+rXysdiqKG9Dla\nmgbpO1fqZ8EZ5Bh62trhcOjuI2oaCr12cDZvSnfS0nYH5BnnlHC73XSoidJcG2oshmMm06qT1HdF\nD9fEJszhcDgcDqc2XB3N4XA4HE6M4Iswh8PhcDgxgi/CHA6Hw+HECL4IczgcDocTI/gizOFwOBxO\njOCLMIfD4XA4MYIvwhwOh8PhxAi+CHM4HA6HEyP4IszhcDgcTozgizCHw+FwODGCL8IcDofD4cQI\nvghzOBwOhxMj+CLM4XA4HE6M4Iswh8PhcDgxgi/CHA6Hw+HECL4IczgcDocTI/gizOFwOBxOjOCL\nMIfD4XA4MYIvwhwOh8PhxAi+CHM4HA6HEyP4IszhcDgcTozgizCHw+FwODGCL8IcDofD4cSI/wON\nTHcJkF9L/gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d97e3b9b0>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHnNJREFUeJztnXl4VNX5x7/3zpqZTBYSEllCIAQI\nypIQi2wiArIookiLFp+21FoBLRV5QGgREGurj0CrRSxdQUBAC5SKLSiCaARUxFKJgiyCEkiQBLKQ\nZdbz+2M8h5lhljvbveHX9/M8eeQ6M/ece+455z3n3Y7EGGMgCIIgCEJ1ZK0rQBAEQRD/q5AQJgiC\nIAiNICFMEARBEBpBQpggCIIgNIKEMEEQBEFoBAlhgiAIgtAIEsIEQRAEoREkhAmCIAhCI0gIEwRB\nEIRG6NUoxGg0wul0Bv1MkiTEmrQr2G+V3k+v14esk++9WiPhni8vLw+VlZUAAI/Ho7htJUkSf9H+\nliPLsl/9+H9TUlLQ1NQU9rcZGRmoq6u7qj68LpHqzglWZ9/PDQYDAIh3H+4ZZVmG2+0O+bnBYIDL\n5Qpbt1gI1u90Op2isoxGI+x2e1T31hpJkiK+48zMTNTW1iatfE40fd5kMqGlpSXk5zqdLuJz+daB\njx+9Xg+32y36Xqg6xVLvSH068L6tiUhjVevkj/x983oo6dcAIKmRtvJafKnAtVnvaAa+mrSWtuaD\nVWm3D/e9a7F/XIt1Bq7NekdbZ/59pZN3rFBbq4eSeYbU0QRBEAShESSE/5+RzBWhJEnQ6/XQ66Oz\nYmi9SpUkCTqdDjqdDkB0KsdwcPVhoomnvbRu62SRrLZuLcSqEieufVSxCUciHrsw4U+8aiw+2cmy\njJSUFDQ3NwMAioqKkJGRgeuvvx4AsHr1arhcLkXlJfvdyrKMNm3aAADMZjM6duyITz/9FIDXRpqa\nmopOnToBAD7++OOQ9Y62HwbaoRL1nMHuYzabRZ0lSQpp91V7HHHhkexyE6me9X3PfFHJ7+9wOBT9\njl8nikjtJ8tyqzQzhYMv2iP53vyv0yqEcCIHsE6ng8ViAeAdUOGcVLTEarUKRxuHw5GwNoh3QTN6\n9GgAQO/evTF9+nT86U9/AgDMmDEDNTU14nsbN26Ew+EI65jiW6dkYbFYoNfrMW3aNADAiBEjMGDA\nAJw+fRqA1ymspaVFTASDBg0CEHyyjbbtAp0wEokkScjNzQUApKWlYdeuXbBarQCACxcuoKioCMDV\nY0fNnXCHDh3w3HPPAQB+9atf4ejRo0krK95+7bu4vOeee3DhwgUAwKJFi/DZZ5/hwIEDAICXX34Z\nQPA5iTHmVw+12tpkMqFbt2744osvAHgXDna7vdUKZd4uBw4cQK9evURf5WPyWiJaH5KYykjanQmC\nIAiCCIsqO2E11c1ut1uoULOysnD+/HlVyo2GnJwcnDx5EiaTCQCwbNky/OIXv0jIveNZHZtMJqxb\ntw6AdwdpMpnw2GOPAfCG9BgMBpw5cwYAcP311+Pw4cOK7puMHQMPNZo6dSr69u2LO++8U9Rbr9ej\nsLAQgHfHW19fj3PnzgEA2rdvj4qKiqAhStG2ne/3k9G/uYq9Q4cOyMjIEM+cnp4esjw11dErVqwQ\n7T58+HB06NAhabuzYM/Fd7ehTAuAV41vsVjQp08fAMD3v/99jBs3DqmpqeK3ubm5Yp6ItPNJhvkh\nEmlpaVi9ejV2794NAHjnnXewc+fOVrsT5nTq1AkGgwFZWVkArp2dcEpKCv7yl78AACZMmIB77rkH\nO3bsSFp5qghhpTGOiaJt27YAgAcffBBLly4FACGYW4PjyqZNm8QkAADjxo1LmBCORKgFkdVqxZo1\na5CRkQHgiuqNC6+0tDSkpKRgyJAhALwLnMzMTDQ2NkYsU4kjVzQLNUmS8OSTTwIA5s2bh5qaGiGg\n6uvrkZmZiYqKClG22+0WKrGSkhKMGzcOv//97wEALperVfoj2Gw2zJo1C4B3cWQ2m4VjWWZmpmq2\n2FDIsowhQ4YIQZibm4tBgwbh/fffT0p5weyx4Z6dL3CfeOIJnDhxQphNTCYTUlNT0dDQAMA7N7W0\ntOCBBx4AALz00kvIzs5Oqmo9WhwOB7Kzs8Uz5OXlwWaziWdQc26NBpvNBlmWMXPmTADAD37wA41r\nFBzfhcLMmTPx2GOPwWg0is9jbV+lskYVIRwpOFwpgSvfUaNG4f333xed0+PxQJIkcZ2fn48ePXoA\nAMrLy/0mXC2FcUlJid/1008/rUq5wRwleGc7dOgQCgoK/BJueDwece12u4UQALzCr7a2FpMmTQJw\nZZETDCVCOFySjcDPunTpgrlz5wLw9om0tDSsWrUKgHeXcPbsWSF0O3fujO985ztiVzlv3jzodDq8\n+uqrAIDKysqE9c9E0bdvX5SUlKB///5+/5/3+yNHjmhRLT8YY1dNTp07d06aEA4G74/BJkm+aywt\nLcUnn3yCRx55BABw+PBhrFq1Cj179gQAnD17Fv369cMtt9wCAPj5z3+O0aNHC9+IcAlC1FoAtW3b\nFmazGb169QIA7NixA06nU3G/VdvxlZd15MgRlJSUoKCgQLWyY6Fr165iPigoKPATwA0NDXj77bdj\nuq/iBCox3Z0gCIIgiLi5JtTRgwcPBgAsWLAAPXr0EKoDm80Gl8uFjz/+GAAwcOBAdOrUCd27dwcA\n/PCHPxRq36eeegqff/65X53UJi8vDwD8vLcB4LXXXlOlfJ1Oh4KCAhw7dgyAVw1z9913A4Cwofoi\ny7Kw+w4dOhQmk0mofYuLi2EymYSt7cMPP7zq93wHHG1cMSfUTnjfvn1+72/t2rVC5eV2u8EYw3/+\n8x8AQLt27VBQUCDulZaWhnbt2mHLli0AgFmzZqm6ewuFzWZD165dAQB79+5FdXW1MKs0NTVBkiQx\nhiwWi+YqdL1eD7PZLK7dbjfeeeedpJUXbDcXqg0KCwtRXFwsvjN79mx88skn4j4AsH//fvH9Y8eO\niX6cnp6O4uJisfsZPnw46uvrQ9ZJDR599FG0bdsWN9xwAwCvpuT48eNC4xdpbtUqBDQzMxMAkJ2d\nrXrZSpEkCW3atEHHjh0BeO3BwBWt0/Tp05NeB1WEcKwOBJIkISUlRaiSrFYr0tPTRUMB3gnqu9/9\nrrg+e/Yshg8fDsA7MXDhcPz48YTUKVYkScL9998P4ErM30cffQQguWot3xzMjDE0NDQIFbPFYhHq\nZE5gXfhAmj59Om688UZMnDgRANCzZ09IkoR7770XAPDRRx9dZbPj6rJwqupwhAq/8R3Udrsds2fP\nvioUjV9XVFTgwIEDsNlsAIBevXrB5XKJhVxtba3m9lWDwYDFixcLFbrdbsfFixfFIi0jIwNOpxMn\nTpwAAIwdO1aTevpitVpht9tFm7W0tMT8npUQOF4ZYyHVsRs2bBDvdO/evX4Cl9eXjwHuuPfLX/4S\nADB79mzIsowuXboA8DpRai2Ed+/ejZ/+9KeivLKyMtTW1ipWR2vVr2tra+FyucRYbK35IGw2m5Ap\nvI35O9+0aVPM91XaP0gdTRAEQRAaocpO2GAwRJU0g68gxowZg7q6OqHG5eEG3HMxNzcXPXv2FF6C\ngFc1s2/fPgAI6+Gm5opMkiQMHz5cePRy1FKD+iaG56pNwBsCM3DgQPG9YDtPX8ero0ePCi9St9sN\no9Eodve7d+/Gv/71L3Ef3zSDsbZ14O+4x6vvirqlpSXoToXvEjweD8rLy3HHHXcA8KrNx48fL1S9\nEyZMwGeffRa0/Egr2XizGPH+uWDBAjz88MN48803AXidh7Zv3y5UYc899xxuvfVWEeIRTr2v1u6s\ntLQU6enpojxZlmGz2XDx4sWklBfNLmrfvn1iJ/vuu+8GfUd8PvB4PKipqRHfmTdvHiZOnIj09HQA\nwJYtW9CvXz+/3/jWSQ3Kysqg1+uFKSg1NdXvoJZIO2K1o1M4qamp8Hg8OHv2LIDWuRO2WCwoKioS\nY8pXQwIkzqk4HKoI4XCp4AIpLS0VNtKMjAz07dtXxGwBQP/+/fGzn/1MXAd7qVzVWFFRgT/84Q8h\nv6cWPXr0wLRp04QQYYzhzJkzWLRoUcLLCszo43siiyRJfiFFo0eP9guVAq6ojgcPHoxDhw75fXbi\nxAmh+t+1axeMRqMQZkuXLhV2twsXLiQ8VZ3BYMBDDz0EwP84tjlz5oR9t4wx7N69Gx06dAAATJky\nBbIsi6MVP/zww4QtEqKFC90bb7wRsixj2bJlALze319//bXwKt22bRs6duyINWvWAAC++uqrkPdU\nK8fy5MmT/a5bWlrEZJsMlLQ1F1IPPfSQWBCGijzw7Z92ux1VVVUAvAujiooKkZ61U6dOQiD7ZoxT\nk6amJrjdbpExraqqCg6HQ/ECUKt44urqauTk5IhFTGs84a2pqQkbNmzA7NmzAXhj8iVJEn478dRX\n6fzQKtJWAt6JCABef/11fP311wCAgwcP4vz58yKQvmfPnrjpppsiPhwXwuXl5SEXAFwgqsGcOXMw\nZswYcd3Y2Ijly5dHtThRSuB5oTzeF7jiTMMnq6effloMbMArgPluPVAAc3hOZj6J8Xt3794dDz74\nIABg5cqV+Oabb6Kqd+DiwbfejDF069YN8+fPv+p3vva+UHg8Hpw6dUrUW6/XIy0tDQAwadIkvPXW\nW1HVlaPkbNZgDB06FB6PB0OHDgVwpQ3z8/MBALfffjtee+015OTkAPAKhvz8fJFqMVz/55NHshk3\nbhyMRqOoy+uvv57U3VakXZQsy8KuazQahcCMdhJ1OBzYsmWLCAdKT08XCUlefvllTRbzGRkZ0Ov1\nwhmxS5cuUSW+0Ov1SZlrIpGZmQmLxSLGnk6na3V5pBljqK6uxh//+EcA3rhyk8kk6qnGooFswgRB\nEAShFUwFJEliAEL+de3alZ0/f56dP3+e2e12VlxczIqLi5ksywwA0+v1TK/XM1mWI95Lp9OxJUuW\nsCVLlrA9e/aE/J7ZbI5Y73DlRPO3Y8cO5nA4mNPpZE6nk+3fv5/p9fqY7xdNW/teS5LEdDodGzly\nJBs5ciSrra1ldrud2e125na72cGDB5kkSRHbGAA7ePCgX7kul4tt3bqVbd26NeizxdPWkiSxRYsW\nMbfbzdxuN2OMsZ07d7KdO3cqbjOTycRMJhN7/vnnWUNDA7t48SK7ePEiKy0tjbmtef9U+ldUVMSK\niopYdXU1+/LLL5nH42Eej4cdP36cjRgxglmtVma1Wln79u3ZXXfdxZYtW8aWLVvGvve977FJkyZF\nbCNJklhqampS+3R+fj7Lz89nLS0tzOPxsIaGBtbQ0MCysrLium+8/aN79+6submZNTc3M7vdztLS\n0lhaWlpMdUlNTRVj1W63s7y8PJaXlxd1veNta/5XXFzMPB4Pq6qqYlVVVaxdu3ZR/T6wn8bT1krL\nk2WZHTt2jLlcLnb48GF2+PBhZjabk9ZH4q2zxWJhFouFNTQ0MMYYq6mpYTU1NVGP8cB+qQRV1NFG\nozGkY1bbtm1x4MABYYv8xz/+IVSeXBXAvlUBGQwGWK3WsM4f69atw1133QXAq4YJFX6ihqMCVyf2\n69cPkiQJh7GFCxcmrfxwp7wwxmA2m69yx+f/njhxYkR1G//NqlWr0KtXL6Ha1ul0OHjwoCgnnnoH\nwhhDjx49RJsZjUZx6o1SuNqdh3vwes+ZMwf33Xdf1PUFvKpfbluOpLa68847RWyyXq/3s82Xl5ej\nuLgY7733HgCvjXLAgAHCBn7kyBER9xwK7uSVbHUft1vz9uOxwZcuXUpquZH6h81mw86dOwF444Qv\nX74cc1ncBgt421VJatZkMmbMGBGuGQtqZwfk5fH6bty4EYA6Tk6xEhhex8068ajyW1XaymAPwQP9\nKysrIcsy5syZAwBYv359yAmNfRvnyuFpGPv27QvAmwxj0qRJ4mV/+umnmia654PXYDDA6XTigw8+\nAOD12EwWvs8V7BkdDoewd9lsNj8vYyV2Jv793r17X3UQOV88xTLYIr2PS5cu+XkFr1+/PuoyAGD8\n+PHQ6XRiUfjvf/87pvsAUHScHHeUeuWVV/wSjFRVVYnnmTJlCi5fvizuZbVaMWrUKPHdo0eP4r//\n/W/IMiRJEmNMrQmXfXvIAV8MJdt2Fq5/SJKEdu3aoXfv3gC8C4V468P9UHzzTGsFj833je+PBrWF\nHy/vwoULyMnJESljW7MQ5v2rqqoKhYWFoq3btGkjnPaiRWlCKLIJEwRBEIRGaOYdzVXGOp0OjDFs\n3boVQPDsStyL9NChQzCZTGJlMm3aNGzfvl2oeWVZhizLIul6ODVjslfukiTh4YcfBuDd9VdXV+Ol\nl15KapkArjqgIRCbzYZx48aJOvKdk9JVKt+91dXVQafT+e28eBq9WOIBlXi/+nLdddcB8KpylbBw\n4UIAXg9kvV4vnsM39WK0RHpGk8mE7du3A4DI2MWZP3++OCaysbFRnPYEAH/+85/RoUMHUbdIyfp9\n2y6ZGp5gHuxqHRUarn9IkoR+/foJj/ef/OQnePHFF2Muq0+fPsjNzRXXWnn08ufhKTW5erdPnz44\nefKkJnWKBqvVKk6AAlpniFIgzc3NIvUtEF8UjdJnVUUIBxs8zz77rPj33r17g9p5ZVmG0WjEU089\nBQD44IMPcMcdd4iQgWeeeQa/+93vhH3K4XCgsbER7du3B4CwCUKSrY7W6/ViktXr9UhPTxfH6yWT\nSMLUbrejU6dO4pq3A18ERYLnWL3hhhv8TllijAmVaSxtG40AZoyJfOFKKCoqEio9m82G5uZmcS5y\nPOroSG39zDPPiKMfgSuDsqWlBZWVlSL+3WAwoE+fPiL+Nzs7GxcuXBDx8oEpQX2JN2FINEiShNLS\nUvFv4Eo4YLIJ1z9SUlKQmZkpfAYmTJgQV1m33XabWFBOnTo1rnvFA1fjOhwOP7+aWE/1UQveN775\n5hvk5+eL+aa1hScFY+7cuVi7dq1Q+ffu3RsVFRVX9T8lY65VCWG9Xu/niOSbjJ7nFvV9SD7pLl68\nGBs2bBDJ2G+66SYYDAax+zGbzXA4HOK3gwYNQnl5uaLsXMm2nW3btk3s1iRJwuXLl1uFTSQlJUUk\nfeBxlQBw//3348c//nFYhzGLxSIER1ZWlp/NQ5IkEceaaBhj6Nevn6ibwWAQiSx4gpBQGAwGvPHG\nG2Jh1tjYCKPRKN7NzTffjA0bNiS8zgaDAZMnTw7az8xmMxhjuP322wF4x8DGjRvF7q1Nmzb40Y9+\nJHJFq+G/oIQ+ffqIRRjfmf7zn/+M+77xjsWmpiZkZWXhyy+/BOB1ZFq5cmVM97JarXj22WfFBBqY\nzEZNuNBqaWlBamqqiGdPZo7uRMDn79LSUsiyLOabZJKobFxutxt1dXUiQcuKFStQW1srkr8sXrxY\nsaOW0vqQTZggCIIgNEITdTRjDKtXrwYAjBw5Em+++abYVaWnpwsV0NSpUzFs2DC/MBhfj0yTyYRR\no0aJlWE0ruTJ3F3IsiwOCedl8RNykk0kFUhNTY1QizOfrFQAcObMGbGKPXXqFM6dO4cZM2YAAN56\n6y20b99e2Dbdbrff6rOmpiapO/3y8nI/Vejy5csBAMOGDQuq5uIhSceOHcN1110nnstsNuPSpUtY\nsWIFgPiOkQzXhzIzM5GSkuLXvrx93G43HnnkEZF3+95774XBYBD2p5kzZ6KsrExRH1XTxjZ9+nQx\nFgHv8yciV3QitFIrV67ECy+8AMCbceyNN94A4D1VLVI7ZmZmipPaZsyYAVmWsW3bNgDAX//617jr\nFivcQ76xsRHZ2dmi/0STmlSLfM28flxzFY/fhdqUl5cjKytL+Izk5eWhY8eOwj6/YsUKnDt3LqFl\nqnaecOAEzW3CHo8HCxcuFB1l7ty5It705MmTePLJJ4U6w+l0wmq1CketeEimOnrgwIEwmUx+x2KF\nOg5NbRhjeOWVVwAAzz//vN8hF7m5uX7HBXo8HnGGKT+jmcMHGhcCS5YsCVlmvG3NGMN7770n8lan\npaUJgXzy5EkUFhb65eXu1auXUDFzNTSnubkZ69evF/0vWQsHvmDxfXbf85XHjx/vd3apLMsil/Se\nPXtajQral9tuu83vmjGWEDtfvM/Kvs3FzvvkyJEjxfnSgwYNwuDBg68qgy/6x44di7Vr14pJ1uVy\noamp6arc2FrAncMMBgMYY8JOeffddytePGohhHnb8kUoDx27FmCM+ZnZPB4PnE6nOBK3srIy4e2p\nmXc0n/xSU1PhcrlE3tz58+cLoetyueDxeMRnWpwEEgvr16/3OzihW7duGtfIHx73+Oijj4pc0W3b\ntoXT6fTzBpQkCYWFhUHv4Xa70djYKAZYOKezeDutJEnYvHmzsNP89re/FcItNzcXb7/9tpiwMjMz\nYbPZ/BYXvL6A1yv/3XffVVSneBYP/BQfX29iLrBkWUZOTo5fbOKWLVuwY8cOv9+0NqZPny4c2bjA\n4znf+QIiFhLxvJWVlXj88ccBeBfy/JzrU6dOYcyYMWISBbx9gdvfJ02a5JfTeNiwYXEd6pFI+Dw4\nd+5cOJ1OMW55NIgStPBG5gsaDh8LySRR72vAgAGwWq3ifvX19bh8+bI4bCcZ7Uk2YYIgCILQCFV2\nwrm5uSIkJBCDwQCz2YyysjIA3rSVgSquRK0+kq2a4XZf7kHKM2bx1IZqoOQZ+edr1qwR3p9PPPEE\nKisr0bVrVwBX0hL63svj8Yjd4eTJk7Fnzx5UV1crqlMkwoXaMMbQ3Nwsdi+33HKLMElkZ2ejpKRE\n2J14rLhvytPLly8Lz1mlu2Ag/Lm9/PNQ2pnGxkYwxkQ9ZFkWYS8WiwWbN28WbT1x4kRcvHixVey+\nwlFWVoa6ujoAXo2DXq/HsGHDAHh9BpLtZxFuHnC5XNizZw8Ar+bp17/+NQCv93tJSYkYi3V1dSgs\nLBTvjTGG06dPix0+z2qnhGRHWPCxKcsyDAaDSMupVmx2rPDQqubmZqSkpEQcR62JtLQ0PzNLTU0N\nnE6nOLL1iy++SHiZqrROuHM4V69eje3btwshnEyVc7AwqERy6623+pXD1Ubx5LGNlsCJMDC1pC9N\nTU1YunQpAODFF19EVlaWcGjp3r079Hq9OCP22LFj2Lhxo3DM2r17t2LBoWSyilRvX9vtfffdh9Gj\nRwMAli9fjs8//xwjRowAcOWowldffRUA8Pe//x27du0SznvR1DceIVxWVoYpU6aIc4yrqqpw5MgR\nAN4UnF999ZWiUDpOML8KtWlqahJHXN58880AIByzkr2ACDd2Aw+3379/v6jnoEGDsGXLFowfPx6A\nd5L1eDziCMxFixahrKwspoW+0rSEscL7Dl/QrVu3DkDrj7flgmrs2LHo3bs3Nm3apHGNlFNQUCDC\nZgFv/oTNmzdHnas+GkgdTRAEQRAaITEVdGBGozHk6k1N7z2TySTCmGRZjrjrjlbdxFWif/vb3zBk\nyBD0798fAGJOAB6KSMnsA6+5s1VgUpRg8NV9+/btYbfb/bQYjDFxr5aWFsUnLik5iSQwpZ0kSUId\nFyyBPt+lZmRkoL6+Xjhm1dXVweVyRbXzDVVvi8USVothNpvD7mYT2bejyYwVTf+Ilnbt2gEAXnjh\nBZw9exazZ88GEL+XuZJ+6fsdnU4n+mLgKUc6nQ4PPPAAAO8JZo8//rjI2lRZWYk2bdqI7GQul0vx\nOwp8nzqdLuwcEm9bd+7cGYA3QURNTQ1mzZoV1/04SsdtayOZ/ZozcuRI/OY3vxGnli1YsAD19fXC\nlBQtijRvaghhWZZbnb3L13s53HfiuX+ynjnazhjqOMdkE40QDrZ44GpHLdWw4dqstebCVWOySjSR\n+mbgHGIwGMSCMdgEyT8zGAwxT6Ac3mbB5oxktnWyxi0J4fD45sSP1zyq5N2poo5uLYb5ZNiBQ9Ga\nFh38oIZEdNRo7qPT6aDT6RS1ezAhrDWR6hAYBkUkD5PJJJzuTCaTiMEO9Y74UYut3X4aDv4MhLq4\n3W64XC7VQmLJJkwQBEEQGqHKFjUjI0N4UXo8nphXd74n9vgSqLYJtjo2GAxwuVziMzV3xWpiNpv9\n1G++al1JkuB2u4OqUEOpz31VyjxUArhihwv2G98dC3AlhWQ4TCaTsK9KkgS9Xu/3XkOpfbmaMloV\nmyI1UYQ+kpOTI+z9Tqcz5n4d2PaBfTSaMZMIj13fd95adpIFBQU4ffo0gCv+HLx/ORyOq0wWvB24\nLT1aPwrff/N7Wa1WEaLF7x0OJSYp37bmfTyeOfJ/lUSa/8JFlESDUhmjik2YIAiCIIir+f+5HSQI\ngiCIawASwgRBEAShESSECYIgCEIjSAgTBEEQhEaQECYIgiAIjSAhTBAEQRAaQUKYIAiCIDSChDBB\nEARBaAQJYYIgCILQCBLCBEEQBKERJIQJgiAIQiNICBMEQRCERpAQJgiCIAiNICFMEARBEBpBQpgg\nCIIgNIKEMEEQBEFoBAlhgiAIgtAIEsIEQRAEoREkhAmCIAhCI0gIEwRBEIRGkBAmCIIgCI0gIUwQ\nBEEQGkFCmCAIgiA04v8AfM4ud2A/J4cAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d97e0f6a0>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAABICAYAAADBLk2cAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl0lNX5x7/vvLNlskN2liBhCUQO\nEJFFkSUgBhRUyiIWRIpKxQOiFiplEykHaAscqoWqRaEItAqVNdIoUHYjyE7YIRBJCBBCMiaTZWbu\n74859zqTZeadJTPJz+dzzhwS8s59n/e+z92e57nPlRhjDARBEARB+B1VoAUgCIIgiF8qNAgTBEEQ\nRICgQZggCIIgAgQNwgRBEAQRIGgQJgiCIIgAQYMwQRAEQQQIGoQJgiAIIkDQIEwQBEEQAYIGYYIg\nCIIIEGp/3ESSJH/cxiX2cmg0GlRUVDi9XqvVoqqqqr7FchtnSc70er3L56oLlUoFq9XqqVh1otVq\nXcrUUHSkOs7quiHKrFarnepsWFgYjEajw//ZP0ddzytJElQqlbiWMQaNRoOIiAjx+/3792GxWADA\nLT1SqVTie3Uhy3K96KY3SJLkVKaGqB+uZObXNDRc6XVkZCSKi4vF7/bP4Ine6PV6AECrVq1w584d\nFBUVAXDeH9SGkuslf6StbIgvtbEqI9D4BgbAtTI2Rrkbo8wqlcrtjoQjSZLH33UF6Yf/+P9Y197o\ndV3lAUBQUBBMJpPHE0AlMpE5miAIgiACxC9qEJYkSXzUateW+IY6IySIunCls7Ise1y2s1m9LMsO\n7cvXeCO3t9TXMxENl9DQUISGhiItLQ1RUVFQqVRidexr/OIT9hbeABMTE5GTk+MT35CSMrw1b/CX\n1tB8Wb8EFixYAAD49ttvceXKFdy6dSvAEvkHVzrryveqFN4m7cvTarVo0qQJAOD27ds+NQ8G8rA3\nez+4v+43fvx47N69GwAwfPhwbN68GQMHDgQArFmzxi9yNCZ8+W50Oh1iY2MBAElJSbh48aKIfcjL\ny4NWq/XYR1wbjcInPGPGDADAnDlzcOLECfTt2xeAdxVQnz5hjUaDzp07Y/ny5QCA9PR0lJeX+6wD\nJD9U3Wg0GhQWFiI0NNTh3teuXQMAtGnTxq3y/FnX3DrDZbx48aJLGWqTyZle+8p3FhISgkWLFiEz\nMxMA8PTTT+Pu3bvo0KEDAODll19GZWUlKisrXZalJDDLV3UtSRIMBoPoZHv37o0hQ4Zg/fr1AIBd\nu3bBYrGIOnJVV77UD96vrV27FomJiSKAzmAwoLi4GGFhYQCAhQsXYs2aNcjJyXGrfMC/dV1X2Z7q\nnz/aoiRJaNeuHaZNmwYA+PWvf43c3Fw0a9YMADBixAiMGzcOEydOBAAHXamtLCULsF+UOZogCIIg\nGhKNwhz9j3/8AwAwZcoUxMXFQaPRAICiWXZd1IcBgK9kUlNT0b17d2GOjoqKQmVlJfLz831+z186\nKpUKw4cPx9tvvw0A6NGjRw3fjSRJSEpKAgDs3bsX/fv397ucStDpdACADz74ABaLBUOHDgUAt7bJ\nebNyUwLX8VdffRWDBg1Cv379xN/OnDmDyMhIADYXjFLLT320Ra4DsixDrVbjD3/4AwDbCj0mJkbI\nptFooFKpMHLkSABAVlYWDAYDBgwYAAAoLCz0uWx18corrwCAWHVptVoAgMlkwr179xAcHAwA6N+/\nP5KTk/Hb3/4WAPDgwQPF9wika0yWZfTp0wfHjx8HAJSVlTW4LaCMMeTn52PJkiUAgLZt2zpYz9q3\nb49Vq1bBbDYrKksJjWIQbt26NQCgefPmYIzhxRdfBBB434h9wIYsyxg2bBgA4MMPP8TKlSuRkJAA\nAJg0aRLS09Px2GOPAbBNHuqrMXi715c/j0ajQVVVlYM/LCkpCVeuXHG4nncUnTp1wg8//ODxfd1l\n69atACDqXCn9+vXDuHHjsG7duvoQyyv27dsHAHjkkUcA2Ey8ALBlyxaf3cMb/ZAkCV26dAEATJ48\nGbGxsfjuu+8A2CY3ffv2FRPjZ555BtnZ2Th79qzLcn0VdJWamgoAmDt3Lp544gkAtkEnNDRUTNz5\n4Mw7SIvF4tCOe/ToAcaYcAUMGDBAPIOv3El1UVZWBgAoLy9HSEgIfvzxR/G38+fPi/5Ep9Nh2LBh\nSEtLA2AbGLiPsiHC63zTpk0YPHgwPvzwQwDAkiVLcPfuXZ/cw5fb56xWK3766ScAQIsWLdCyZUuh\nH0OHDkXz5s1x5swZALZ35u19G4VPODk5GQCQnZ0NxhhOnToF4OdG5yne+ClbtmyJnTt3YurUqQCA\nJ554Qqxk0tLSsGvXLkRFRQGwBamEhobiv//9LwCgpKQE+fn5olEXFhYqmlkpkdtbnx9/hl69eiE6\nOhopKSkAgKlTp8JoNKJTp04AbM8A/LypfdGiRfj9738PoHYLhS99whqNxqkVpLy8XMh36dIl9OrV\ny6GjP3nypNAdf/r8XMETmmi1WjDGYDAYANiexx3qQz+aNWuGPn36YPbs2QBsHX9VVZXoRFetWoWB\nAwciMTERgE2P5syZo2iiLMuyS/13VdeSJGHDhg0AgJEjR4rrrVarwyArSRKqqqqEfvzrX/9CWFiY\nmOh37NgRkZGR4nqz2YxevXoBAI4dO1bjvr7UD35906ZNsXv3bvTp0weArS4LCwsRFBQEAPjpp58w\natQo8S70ej3atWun+D7+3ifcrVs3AEBmZiZCQkLERC47O9utcvzlEw4JCRGWs3//+98OdVtUVISJ\nEyfiwIEDAGx9tzO5aJ8wQRAEQTRgGsVKeNy4cQCAf/7znwAgTKJt27b1qlxvZoTjxo3DypUr8dln\nnwGwRc1xP8L58+exb98+8f3mzZvj1VdfxcqVKwHYsrCMHDlSrDIXLVqEGzduKPbtuNqv6Y05+tNP\nPwUAvPjii2JVbH/f999/HwAwf/58MMZExOaRI0fQs2dPAKiRFtGVzIAyHeHXlJWViRU456GHHgJg\n20JQWVkpVr4hISG4cuWKsEowxpCamoqTJ0+6vJ8ruX25YtDpdA4r3pKSEoSHh7tdjquITE/1486d\nOygtLUWLFi0A2FYEjDEsXboUAIRflet0ZWUl7ty5I1YUzvDFSjgxMREnTpwAYGtfXAdv3ryJpUuX\n4ptvvgFgszI8ePBAmH453Net1Wqxb98+sXoDgI0bNwKwRcpW14f60g+tVuvU2mMwGIRJNCYmBmFh\nYYotHP5eCXO/ryzLKCwsRExMjCI57HFl/fJV1L8kSVi8eDFatmwJABg9erRDfTx48AAbNmwQroLl\ny5c7tVQpkalR+IT5A3PTEu+A6zONniu2b9+OyZMnY/r06QCAd955p84ggytXrmDWrFmioVdVVeGr\nr74SHcGTTz6Jjz76yCdyeVsf0dHRAH729dpTWloqzI+84+TP8MUXX3h1XyVwmZ588kn85z//AWDL\n7WoymWo8Nzf1FxcXY9myZfjjH/8IwOYa4NuVGhKdO3d2eIZ3333Xo3JcvX93B2A+GEVHRyMyMhLn\nz58HABQUFKCgoEAEEe3evRvHjh0Tv4eFhSE0NFTRHlslMrlq64899piQ7caNG5gzZw4AIDc3V1EA\nJ58EmM1m9OzZ02FScPXqVZfP4GtcycwYE3UtSZKi/OyBoEmTJg7bvd59912P6tFVAJev3o1arUZq\naqqYtMXExODSpUui/IsXL+LIkSMiT7UvAsvIHE0QBEEQAaJRrIR5thIeYMFNi65MFPVJ+/btcf78\nebEqr80Ey5EkCeHh4SLizmKxoLy8HM899xwAmymtXbt2ePPNNwHYtiR4ipLN+HWRkpKCIUOGCJmB\nn2eYJSUlsFgseOGFFwDYItPNZrMwWffs2bPetz/wd33w4EFh0lJCXFycWLFHREQETGfqQq/XY+PG\njQ51zgPgAs28efPEz4WFhRg7diwAYOzYsUhPTxcJJkwmE27cuCG2+lRWViIqKkpsYdq7d2+d91Bi\n/tTr9XW2i7i4ODz//PPCLLhkyRKRIc2Td22xWIQuq1QqYZWYP3++WwGU9cnNmzcd3EWRkZG4fft2\nACWqiUqlwqZNm0RkOgA899xzWL16tdtlKQnM88VqeODAgejbty8mTJgAwOaeq6ioEG4Y3n/wYOHL\nly97fc9GMQjv2bMHgC16VKfTCeVr2bJljS0z/uLKlSvYvHmz6NydDUCMsRr7Da9duybS0k2aNAnx\n8fHiOSdMmCBebvUB1ZUyeqqIsbGxeOutt2rsseV7tHfs2IGIiAgxCEdFRaGqqgq9e/cGAOTn59f7\nIOzpsx0/fhxjxowBYDMfuRttXN80bdoUISEh4veSkhJMnjw5gBLZiI2NxeDBgwHY6p4xhtOnTwMA\npk+fLlwxnJMnT+LSpUsAfjZj861kTZo0qXMAUzIIOzP7lZaWQpIk5OXlAbB1jN6aCXl75e4ZpXL6\nA1mWRXpQjj/3MyslISHBIXNdcXExRowY4VFZ9b3/nce23Lx5Ex07dnTYHgZAuDrat2+Prl27CneY\nL/q8RjEI85UwD7bhDaxLly5+H4T5IPXCCy9gxIgRYoYfGxtb48U5w2q14ssvvwQAvP3223j66adF\nI8/OzhYriP3797sln7uBN7xuDxw4IIKbOGazGa+//rr4vVu3bqJznTt3LnJycsRWj9LS0gbpk9Jq\ntXj99dfF9g6eAKMh8dJLLzl0qvarT3fxxUDBJ7mXL18W5ZWWliIpKcllZ7do0SIAtmAmvV4vJhez\nZ8/Ge++9V+t3lCTGd3Zfk8mECxcu4Ny5cwBsfj1vrEnAz1vwoqKiRHsKCQlpEPtxR40a5VBnZrO5\nwSW9AGx1FxMTI+osLS2t3vqI6ithpSvj9u3bo6KiAh988AEA2wKotgkNH3t69uyJadOmiWA9X0A+\nYYIgCIIIEI1iJXz//n0AthmfLMu4ceMGAFuEsr/hs+LVq1dj8+bNYmbnySk9fOY+a9YsLFu2TERP\nM8bQtGnTWr/jS7OMwWAQ5qFWrVo5zB5feuklrF+/3qG8s2fPoqCgAIDNCvHMM88I8+Pdu3dFcomy\nsjKH1XggzHh8a8zSpUvRqVMnkSovKyvL77LUBZ9dP/vss1Cr1cIFsWLFCo/L9NYsFxERgZ07dwKw\nbfXhhwR07NhR0SqGZ/caPny4Q8T8V199Ved3lPhZnVl3LBYLsrKyRFyGki04rq7hFrZmzZrhr3/9\nKwAEfBUcHx8PwGaFsufjjz8OhDgu6dOnD6KiokQKUKXbAn2B0nZw+PBh5OTkiH7h3r17tV7Xvn17\nALZ+kSdJ8RWNYhDmgxMPzOKNLZBh+RaLBY8//rjoXDzp/PjgZDKZUFxcLEySPEUkv8adsiMjI0X4\nfPWgFEmSMGjQIHECSEpKisgGI8syCgoKRF0fO3bMIchLlmW8//776NixoyhPpVIJn5nVakVcXBwA\nm1/FPuWlfWCGP5BlWaR+7NGjBywWi0hj2JDgZufu3bsDsJ2e4y2uzsl2pk8qlQrDhg1zmDTt2LFD\nlOtOW8vMzEROTo7IRJWRkSG21HiCsyDMiIgIfP7552Kf/qVLl4TfX5ZlhxgAnU6HFStWCF/1N998\nA7PZ7BAUFxQUJFLMyrLsVm5mXzFp0iQxodm7dy82b94ssvPxYFAe6OmOadQfE2LeJyxcuBBBQUEi\nw2F9otFoxGROiTuOu9maNGmCsLAwPPzwwwCAadOm1dCzqKgosSfbbDZj3759ol34gkYxCPMZ6PHj\nx9GrVy/RmKtvuPcnVVVVyMzMFJu6PTlWjHeG+/fvR15envBb5uXliZV19SQOrigtLa2zs4qOjsYn\nn3wijnFTq9WiUfJJBfeH8IGcEx8fj7Fjx9Z6OAJgU04+ccjLy3MIKPPnSlir1UKn04lAiwsXLojg\nooYGnwDx+uFpTb3BE0sJf6ehoaHo16+fiAQFIH4uLS11Sw6dTger1So6RB7I4onMQO2rZW5JmDx5\nMmRZxtdffw3AtoeZ37e6r7Sqqgrz58/H8OHDAdgGtJSUFFHWnTt38Oijj4o96YwxsZJbvHhxve0V\nVqvV4gzsl19+GXFxcfj73/8uZJg+fbrw1fO64IGdhw4dUnwff+x15vEswcHBYmcI4N2uD1eYzWa3\nYmH+9Kc/iZ9Pnz4txpTafOvr1q1zSIPK35OvIJ8wQRAEQQSIRrES5vCZFJ+l6nS6gK6GTSaTWKXO\nmjVL+PK4mUgpBoMBSUlJYiXcqlUrtGrVCoDN1Oar/X+PPPIImjVr5rCa5bPHH3/8sUYmKZVKJfy8\n586dE0ep2cNXvEajUWQW4s/EZ+y1Zd/yFbGxsXjqqafEUXVVVVUwGAzCV/3gwYMGtyUJsJnPunbt\nKn63WCxeRfrzmbr9lpq6rqu+GuI6UFZWhoqKChExr1ar8fjjjwt5lUTg2svx8MMPC53485//7PI7\nznC2gn/jjTcc/NfOVkRWqxX5+fn429/+Jv6vbdu2eOeddwDY3kObNm3EqpMx5rYVQCnPPvus8Jtr\nNJoa9WD/zPYuHb4DgruV3Fnd1rdVSq/XY9myZeL3kpISn2yf8uXWzJ49e4q+ljGGBQsWYNCgQbWW\nM2zYMIejT48ePepUv+zbl9K6blSD8JYtW5CWliYUMtBbYsxmszCVJyYmilRnM2fOxP/+9z/F5Zw+\nfRqjR4/G+vXrAdh8qnzrEw9KU4ozk09+fn4NxeAD5apVq2pcP3ToUIc6tg+4Onz4MF577TUxQbBY\nLDWOieP/epscIyIiAp999pnwme7YsUOU/dFHH6F37941Atl4x7lr1y7Islzvx9C5i8ViEUlnAO87\ne14f3hwNp1arHd4VY0zosdItMNwHfPjwYZSVlWHUqFEAbDpdF0o60Nqu4TIVFBRg06ZNHvtuMzIy\nxDnIgwYNwp49e/DGG28AsOnRwoULFcupFI1Gg82bN9d6jKPRaIRWqxWJT+bPn4+IiAgxMX/w4AHM\nZrNH+lLf5uiEhAQRGyJJEsxms08SnPgyIDUiIkK427KysrBly5Y6jwtNT08XiYoAW8pcX8lh/6V6\nB4BPPqdOnWIWi4UVFhaywsJCplarvSrPF3KrVCqmUqlYXl6ew3dHjBjhcJ0syzW+K0kSkySJhYeH\ns5KSEvHdqqoq1rp1a9a6dWsmSZJbctd2Pf+MHj2aWSyWWr937949FhQUxOLj41l8fDy7f/++kKWq\nqoodOnSIjR8/ng0ePJgNHjyYqVQqxfWsVqvdruugoCBWUFDACgoKGGOMWa1WIcunn37KjEYjMxqN\ndZZnsVgcntVqtTKr1cpWrFjBVCoVa9KkCWvSpIlXOuKN7nXu3JmZzWZmNpsZY4yZzWYmy3KteuLO\nR6PROJU5KCjIqS5nZGSw/fv3s/3797OKigq2du1atnbtWhYbG6vo3qWlpay0tJQVFxezvLw8Fh4e\nzsLDw+u1LZaXl7P79++LtuhunQ0YMIBt27aNbdu2jR09epStWbOGlZSUsJKSErZ48WKm1WqZVqv1\nqX5kZ2fXuJ7rQ3JyMuvZsycLCgpiQUFBLDc3l5WXl4vruG7379+f9e/f3+1+0Ju6dvXJysoSbY0x\nxo4cOeJVeUrldue9h4aGsu3bt7Pt27eznTt31vj7zJkzWffu3Vn37t3Z7du32Y0bN1hRURErKipy\n2r/W1p6UQD5hgiAIgggUioZqL4GPZkLXr19njP08EwwODq63mZW7cp85c8bhu0ajkYWEhLCQkBCW\nnJzMWrZsyYKDg1lwcDBLSEhgWq2WhYWFsbCwMLZp0yaH7xYXFzOdTsd0Op1PZ98hISHs2rVrYkVp\nX5fXr19nW7duZffv3xerYP4cRqPRqxWaJElu17VarWbff/89+/7775nFYmEZGRmssrKSVVZWskuX\nLolVA59137p1i926dYutWLGCrV27Vjwjn5FXh6+yU1JSPNYRb3SvY8eODiuGBw8e+KydOEOj0Th9\nT+np6ezcuXPs3LlzrLKykl27do1du3aNjRkzxukqQK/Xs6KiIvFMRqORJScnC2tPfbZFvlLp1q0b\n69atm3gWZ/dWq9VMrVYzWZbZt99+y65evcquXr3KioqKWI8ePdjAgQPZwIEDXa6wPJV5zJgxog0y\nZmtnM2bMYDNmzGCAzarAP8XFxayiokJcW1ZWxkwmE5s3bx6bN28e27RpE9NoNH6pa1ef0tJSoQNV\nVVVO9c2Xeu3OSjg+Pp5NnDiRTZw4kQ0ePJjp9XrWtGlT1rRpUzZ79myWm5vLTpw4wU6cOMG+/vpr\nlpiY6LGVRQmNyifM85Byv+a0adOEvybQDBgwAPn5+QBs8uXk5Iigpps3b8JgMAi5uX9n165dAOCQ\nNxiwPScPmb9+/XqNdGzOcLYPtKysDLt378bo0aMB2JIxcJ9zRkYGTCYT0tLSxPVms1kE+njjU61L\nHmeYzWaxVzMmJgYtW7YU5xXzIy0BoLy8HJ06dXIICgMgthEcOnSoxmEPRqNR7At01+fuK6rLxNMk\neourFJCucpyfPHkSn3zyCQDgL3/5iwjGW79+PW7fvi1iDk6cOAG9Xo/U1FQAwIYNG6BWq4Wfdu7c\nubh8+bKid+9psBB/1qioKBgMBhw5cgSA7XARvp+9vLwc69atE0FbAwYMwJgxY0TglU6nc/DN5+Tk\n4NKlSyK4sr7yoW/cuBH5+flij+/IkSPFoR3BwcE4d+4cEhISANTcZ19YWIi4uDiHRDtTp04VZ5sH\nSqdjY2MdDpU4c+aM39JpuqNDQ4cOFUGRY8eOxbJly0QCDl4OP5t6ypQpIjlUfckkMU96SDfxVUTe\nmTNnROcJAJ06dcLZs2c9Ls/Vo9t3aEpOJ+L74RYtWoTZs2eLQbhDhw7o3bu3aHATJkxASEgIfvOb\n3wCwBS5ptVqxL/DgwYMimUD1oAZXh7YHBwc7jRjX6XTo1asXANsgzPcYlpeXw2q1CpkB3w0MgOu6\nVnJo+7Zt2wAAycnJWL58OQBl5+5mZ2eLPcy/+tWvsHPnTp8cgO6NXr/22mvifUuShBkzZjiNIHYH\nZzIrOfycP9fSpUtFBG5wcDAKCwtFezt48CA6dOggBo5WrVrhwoUL+O677wDAIee4K5SchuasrmfN\nmoX33nvPIVEJf0aLxYIDBw6IYKHWrVs7DBSAbaDl9x8yZIjTE5+q461+8EnOlClT8Lvf/Q6ALUCO\n9yX28PbYtWtXDB48WFyfmJgIk8kkJnauAra8bYt10aVLFxw9elRkXOOnxPkKb/Wa89Zbb4mEJ3Pn\nzsWpU6fw6KOPinLMZrPITlZXBi1fyMwhnzBBEARBBApFRmsvgY98Art372YWi0X4HNLT092KVqv+\ncYVWqxXRnTqdTrFfVK/XM61WK/xKRqORWSwWtnfvXrZ37162fft2tnXrVuGX5PfR6/VMr9ezCRMm\nCH9Q9bJd+VeTk5OdyhYeHs4iIiJYREQE02q1zGAwMIPB4JG/w5d17er73bt3F+89NzeXxcbGKorY\nrU+5vSmXR31z9u3b5xeZ3fHRybLMsrKyRMRraWmp0Nn8/Hzh+zUajezjjz9mSUlJLCEhgSUkJLgl\nb1BQkFf60aZNGwefqRLsfdc3b95kM2fOZDNnznRLbldtUemzBwUFsS1bttRZjsViYQsWLHCINO/W\nrRtbvXo1W716tYiZCJRPmN93586dzGQysaSkJJaUlOTXthgREaG4HK1WyzIzM1lmZiazWq2spKRE\n6HFlZSUzGo2K69LZR2l0dKMyR7/55puYM2eO8A2npqbi4sWLHu9Dc/Xo9mkdVSoVQkNDhanHVQII\nnU4n9tDyFIrc/Lh9+3bk5OQIU9Tx48fd8ps6uzYsLEzk1m5IuHo+JTryyiuvAPj5jGN/4Exub/R6\n+/btIkEAADz00EPiPFxvcSZzdbOdq9zkzz//PABg5cqVKC0tFSlPZVmGJEn4/PPPAdjapqeJc0JD\nQ126PlzVdfPmzZGRkQEAwkTujNzcXAC2/Okmk0m4mtxph65cVO7oh0qlEvEte/bsQVJSkugvXOFu\nfnlftEV7uOk8OzsbwcHBwsfui/3B9jiTOzIyUvE+cVmW0aFDBwC2nAO7du0S5wVXVFQgIyPD6b52\npciyrKgOyBxNEARBEAGiUa2EW7dujdGjR+P69esAgC+//BJWq9XjLDCuvld91aBWq0WkorPMVCqV\nChqNRkTcxcfH44svvkBmZqbDdXwl7G7mG2dya7XaBnPAN3/vsiy7lCkQxx0qob5WwtHR0WJVqdPp\n8MMPP3hcVnXcWQm7gludUlJScPHiRRF13KJFCxQXFwud9iZ6vmnTpi4DYJTUNQ+kHD9+vAhyY4yh\nsLBQBOLcu3cP69evF4F9nsBlCQsLc7r6aow6DbgvNz/9bc+ePSgqKsJTTz0FwPtMedVxJnd0dLRb\nQVT8GQ0Gg0P/yzOY+SLDXkREhKLjLxvVIOxrfK2M9t+zj6z2dcrE+hoY6gslkeUNUW6g8dU14Fxm\nnndYKfwZ1Wq1wzvk9/BF9+Eq4t9eDnfK5P8GBQWJ9Ku+NJH60hztT3zd79kPaCaTqd62dTmTm9+7\nIWG/Zc8Zv1hzdH03EIvFIj7+hPuxffV8npblar8qERjCw8Mhy7Lw6cqyDJVKpWh/MWNMHE/oh7m7\nVzDGhLzl5eU+y2HM24Mv21hjh9e1fW55f5OUlAS1Wu3yPO36hi/AVCqVYv2gnpIgCIIgAkTAM2bx\nGbi/Z1C1nV6iBGfy8r9xM5VSs6C7EZnOiImJESfp8JONajMjcvhsrS4Z7P9eWxSmfX3Yz/xUKpV4\nPloVew9ftQJwqluu9Lp37944duwYAFssgiRJInq/oKCghh+PH0PJ/frO2qlarRZy8VUnX5lIkuSg\nh/Y/K9EPtVrt1kq2ut66C69Hq9VaZ9IeT/uQ/6+4ax2RJAkajUa8V2e65UpHZFkWumY2mx36Kh4H\nYX/Km/21now9tfV7fBXsdj34wydMEARBEERNaIlCEARBEAGCBmGCIAiCCBA0CBMEQRBEgKBBmCAI\ngiACBA3CBEEQBBEgaBAmCIIgiABBgzBBEARBBAgahAmCIAgiQNAgTBAEQRABggZhgiAIgggQNAgT\nBEEQRICgQZggCIIgAgQNwgSFizLKAAAAX0lEQVRBEAQRIGgQJgiCIIgAQYMwQRAEQQQIGoQJgiAI\nIkDQIEwQBEEQAYIGYYIgCIIIEDQIEwRBEESAoEGYIAiCIAIEDcIEQRAEESBoECYIgiCIAEGDMEEQ\nBEEEiP8DV8/CEBq6NRgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f1d91a8cb70>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "mWaLcSa2CHx7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "### Bibliographie\n",
        "\n",
        "**Generative Models from the perspective of Continual Learning**\n",
        "*Timothee Lesort, Hugo Caselles-Dupre, Michael Garcia-Ortiz,  Andrei Stoian  and David Filliat*\n",
        "\n",
        "```\n",
        "@ARTICLE{2018arXiv181209111L,\n",
        "       author = {{Lesort}, Timoth{\\'e}e and {Caselles-Dupr{\\'e}}, Hugo and {Garcia-\n",
        "        Ortiz}, Michael and {Stoian}, Andrei and {Filliat}, David},\n",
        "        title = \"{Generative Models from the perspective of Continual Learning}\",\n",
        "      journal = {arXiv e-prints},\n",
        "     keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition},\n",
        "         year = 2018,\n",
        "        month = Dec,\n",
        "          eid = {arXiv:1812.09111},\n",
        "        pages = {arXiv:1812.09111},\n",
        "archivePrefix = {arXiv},\n",
        "       eprint = {1812.09111},\n",
        " primaryClass = {cs.LG},\n",
        "       adsurl = {https://ui.adsabs.harvard.edu/\\#abs/2018arXiv181209111L},\n",
        "      adsnote = {Provided by the SAO/NASA Astrophysics Data System}\n",
        "}\n",
        "```\n"
      ]
    },
    {
      "metadata": {
        "id": "kUlZfg7t_msK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "XRYxFh1TqGFw",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}