chapter11_part02_sequence-models.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/matthewshawnkehoe/deep-learning-with-python-notebooks/blob/master/chapter11_part02_sequence-models.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G3r4YcGcjdws"
      },
      "source": [
        "This is a companion notebook for the book [Deep Learning with Python, Second Edition](https://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff). For readability, it only contains runnable code blocks and section titles, and a bunch of edits by Matthew Kehoe.\n",
        "\n",
        "This notebook was generated for TensorFlow 2.6."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: Previous material (summarize diagram, bag of words, and TextVectorization layer)."
      ],
      "metadata": {
        "id": "XLys_Axtjkzf"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IqDfDN6djdwu"
      },
      "source": [
        "### Processing words as a sequence: The sequence model approach"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "These past few examples clearly show that <font color='blue'>word order matters</font>: manual engineering of order-based features, such as bigrams, yields a nice <font color='blue'>accuracy boost</font>. Now remember: the history of deep learning is that of a move away from manual feature engineering, toward letting models learn their own features from exposure to data alone. What if, instead of manually crafting order-based features, we exposed the model to raw word sequences and let it figure out such features on its own? This is what <font color='blue'>sequence models</font> are about.\n",
        "\n",
        "To implement a sequence model, you'd start by representing your input samples as\n",
        "<font color='blue'>sequences of integer indices</font> (one integer standing for one word). Then, you'd <font color='blue'>map each integer</font> to a <font color='blue'>vector</font> to obtain vector sequences. Finally, you'd feed these sequences of vectors into a stack of layers that could cross-correlate features from adjacent vectors, such as a 1D convnet, a RNN, or a Transformer.\n",
        "\n",
        "For some time around 2016–2017, bidirectional RNNs (in particular, bidirectional\n",
        "LSTMs) were considered to be the state of the art for sequence modeling. Since you're already familiar with this architecture, this is what we’ll use in our first sequence model examples. However, nowadays sequence modeling is almost universally done with Transformers, which we will cover shortly. Oddly, one-dimensional convnets were never very popular in NLP, even though, in my own experience, a residual stack of depthwise-separable 1D convolutions can often achieve comparable performance to a bidirectional LSTM, at a greatly reduced computational cost."
      ],
      "metadata": {
        "id": "5X6HjRHUlQUg"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hpBO8Uwxjdwu"
      },
      "source": [
        "#### A first practical example"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's try out a first sequence model in practice. First, let's prepare datasets that return <font color='blue'>integer</font> sequences."
      ],
      "metadata": {
        "id": "vA6W62xgmybN"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QpiOx_cLjdwv"
      },
      "source": [
        "**Downloading the data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OGvK58hsjdwv",
        "outputId": "c6039281-6ac8-4f59-d41b-06856df4f919"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "100 80.2M  100 80.2M    0     0  8564k      0  0:00:09  0:00:09 --:--:-- 12.2M\n"
          ]
        }
      ],
      "source": [
        "!curl -O https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\n",
        "!tar -xf aclImdb_v1.tar.gz\n",
        "!rm -r aclImdb/train/unsup"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "TODO: Make some comments in code below."
      ],
      "metadata": {
        "id": "c_gZQ1UCnEQX"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cn0Q6i5Ujdww"
      },
      "source": [
        "**Preparing the data**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_JwuTAnfjdww",
        "outputId": "ad3d822a-1e24-46ca-ca03-9dca124027cf"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found 20000 files belonging to 2 classes.\n",
            "Found 5000 files belonging to 2 classes.\n",
            "Found 25000 files belonging to 2 classes.\n"
          ]
        }
      ],
      "source": [
        "import os, pathlib, shutil, random\n",
        "from tensorflow import keras\n",
        "batch_size = 32\n",
        "base_dir = pathlib.Path(\"aclImdb\")\n",
        "val_dir = base_dir / \"val\"\n",
        "train_dir = base_dir / \"train\"\n",
        "for category in (\"neg\", \"pos\"):\n",
        "    os.makedirs(val_dir / category)\n",
        "    files = os.listdir(train_dir / category)\n",
        "    random.Random(1337).shuffle(files)\n",
        "    num_val_samples = int(0.2 * len(files))\n",
        "    val_files = files[-num_val_samples:]\n",
        "    for fname in val_files:\n",
        "        shutil.move(train_dir / category / fname,\n",
        "                    val_dir / category / fname)\n",
        "\n",
        "train_ds = keras.utils.text_dataset_from_directory(\n",
        "    \"aclImdb/train\", batch_size=batch_size\n",
        ")\n",
        "val_ds = keras.utils.text_dataset_from_directory(\n",
        "    \"aclImdb/val\", batch_size=batch_size\n",
        ")\n",
        "test_ds = keras.utils.text_dataset_from_directory(\n",
        "    \"aclImdb/test\", batch_size=batch_size\n",
        ")\n",
        "text_only_train_ds = train_ds.map(lambda x, y: x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BMYDGkqGjdww"
      },
      "source": [
        "**Preparing integer sequence datasets**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "wPa8Z6jnjdwx"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras import layers\n",
        "\n",
        "max_length = 600                                                                # In order to keep a manageable input size, we’ll truncate the\n",
        "max_tokens = 20000                                                              # we’ll truncate the inputs after the first 600 words. This is\n",
        "text_vectorization = layers.TextVectorization(                                  # a reasonable choice since the average review length is 233\n",
        "    max_tokens=max_tokens,                                                      # words, and only 5% of reviews are longer than 600 words.\n",
        "    output_mode=\"int\",\n",
        "    output_sequence_length=max_length,\n",
        ")\n",
        "text_vectorization.adapt(text_only_train_ds)\n",
        "\n",
        "int_train_ds = train_ds.map(\n",
        "    lambda x, y: (text_vectorization(x), y),\n",
        "    num_parallel_calls=4)\n",
        "int_val_ds = val_ds.map(\n",
        "    lambda x, y: (text_vectorization(x), y),\n",
        "    num_parallel_calls=4)\n",
        "int_test_ds = test_ds.map(\n",
        "    lambda x, y: (text_vectorization(x), y),\n",
        "    num_parallel_calls=4)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, let's make a model. A straightforward to convert our integer sequences to vector sequences is to <font color='blue'>one-hot encode</font> the integers (each dimension would represent one possible term in the vocabulary). On top of these one-hot vectors, we'll add a simple bidirectional LSTM."
      ],
      "metadata": {
        "id": "l38lZluGn3lO"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QCFDJKmtjdwx"
      },
      "source": [
        "**A sequence model built on one-hot encoded vector sequences**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UJ0MDrEwjdwy",
        "outputId": "b2215c6e-806a-4235-8650-ceb5228783cc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_1 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " tf.one_hot (TFOpLambda)     (None, None, 20000)       0         \n",
            "                                                                 \n",
            " bidirectional (Bidirectiona  (None, 64)               5128448   \n",
            " l)                                                              \n",
            "                                                                 \n",
            " dropout (Dropout)           (None, 64)                0         \n",
            "                                                                 \n",
            " dense (Dense)               (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 5,128,513\n",
            "Trainable params: 5,128,513\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")                              # The input is a sequence of integers\n",
        "embedded = tf.one_hot(inputs, depth=max_tokens)                                 # Encode the integers into binary 20,000-dimensional vectors.\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)                             # Add a bidirectional LSTM\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)                              # Finally, add a classification layer\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.summary()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, let's train our model."
      ],
      "metadata": {
        "id": "_AdT9sX9qNeu"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7bJsIAgTjdwy"
      },
      "source": [
        "**Training a first basic sequence model**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GcB23nULjdwy",
        "outputId": "f4cea130-7ba4-4b42-f10c-642da5035537"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            " 66/625 [==>...........................] - ETA: 3:16:07 - loss: 0.6928 - accuracy: 0.5142"
          ]
        }
      ],
      "source": [
        "callbacks = [\n",
        "    keras.callbacks.ModelCheckpoint(\"one_hot_bidir_lstm.keras\",\n",
        "                                    save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)\n",
        "model = keras.models.load_model(\"one_hot_bidir_lstm.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "A first observation: this model trains <font color='blue'>very slowly</font>, especially compared to the light-weight model of the previous section. This is because our inputs are quite large: each input sample is encoded as a matrix of size `(600, 20000)` (600 words per sample, 20,000 possible words). That's 12,000,000 floats for a single movie review. Our bidirectional LSTM has a lot of work to do. Second, the model only gets to <font color='blue'>87%</font> test accuracy—it doesn't perform nearly as well as our (very fast) binary unigram model.\n",
        "Clearly, using one-hot encoding to turn words into vectors, which was the simplest thing we could do, wasn't a great idea. There's a better way: *word embeddings*."
      ],
      "metadata": {
        "id": "iQ8EQDguqXDL"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xd3eMSXljdwy"
      },
      "source": [
        "#### Understanding word embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Crucially, when you encode something via one-hot encoding, you're making a feature-engineering decision. You're injecting into your model a fundamental <font color='blue'>assumption</font> about the structure of your feature space. That assumption is that the <font color='blue'>different tokens you're encoding are all independent from each other</font>: indeed, one-hot vectors are all orthogonal to one another. And in the case of <font color='blue'>words</font>, that assumption is clearly <font color='blue'>wrong</font>. Words form a structured space: they share information with each other. The words <font color='blue'>movie</font> and <font color='blue'>film</font> are interchangeable in most sentences, so the vector that represents “movie” should not be orthogonal to the vector that represents “film”—they should be the same vector, or close enough.\n",
        "\n",
        "To get a bit more abstract, the <font color='blue'>geometric relationship</font> between two word vectors should reflect the <font color='blue'>semantic relationship</font> between these words. For instance, in a reasonable word vector space, you would expect synonyms to be embedded into similar word\n",
        "vectors, and in general, you would expect the geometric distance (such as the cosine distance or L2 distance) between any two word vectors to relate to the “semantic distance” between the associated words. Words that mean <font color='blue'>different</font> things should <font color='blue'>lie far away</font> from each other, whereas <font color='blue'>related</font> words should be <font color='blue'>closer</font>.\n",
        "\n",
        "<font color='blue'>Word embeddings</font> are vector representations of words that achieve exactly this: they map human language into a structured geometric space.\n",
        "\n",
        "Whereas the vectors obtained through one-hot encoding are binary, sparse (mostly\n",
        "made of zeros), and very high-dimensional (the same dimensionality as the number of words in the vocabulary), word embeddings are low-dimensional floating-point vectors (that is, dense vectors, as opposed to sparse vectors); see the figure below. It's common to see word embeddings that are 256-dimensional, 512-dimensional, or 1,024-dimensional when dealing with very large vocabularies. On the other hand, one-hot encoding words generally leads to vectors that are 20,000-dimensional or greater (capturing a vocabu-\n",
        "lary of 20,000 tokens, in this case). So, <font color='blue'>word embeddings</font> pack more information into far <font color='blue'>fewer dimensions</font>."
      ],
      "metadata": {
        "id": "KlcM2yX6qTC0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![word_embeddings.jpg](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCAF0AkMDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD9/KKKKACivh7/AIKh/wDBff4N/wDBJX4ueH/BnxI8PfEnWNU8SaONatZfDmnWVzAkPnSQ7XM93CwfdExwFIwRz2r5l/4jT/2Vv+hH+Pf/AII9K/8AllQB+vdFfLX/AAS1/wCCuXw1/wCCuPw/8U+I/htpHjXR7LwjqEWm3sfiSytraWSSSPzFMYguJgVx6kHPavkj4q/8HhX7Mvwf+J3iLwnqngr45S6l4Z1O40q6e20XS2heWCVo2KE6iCVJUkEgHGMgdKAP1cor8hP+I0/9lb/oR/j3/wCCPSv/AJZV95ftIf8ABTLwF+y//wAE97b9pTXtL8WXvgO60nSdZSz061gk1XyNRaBYAY5Jki3L9oTePNwMNgtgZAPomivyE/4jT/2Vv+hH+Pf/AII9K/8AllXrv7CX/B0B8AP+ChX7VXhX4QeCvCnxd07xN4vN0LO51rStPgsY/s9rNdP5jxXsrjKQOBhD8xXOBkgA/R6ivhb/AIKf/wDBwT8F/wDgk18ctF+H/wAR/DnxM1fWdd0KPxDbzeHNOsrm2S3kuLi3Cs093Cwk327kgKRgrzkkD5t/4jT/ANlb/oR/j3/4I9K/+WVAH690V8y/8Evf+CrPw6/4K0fCbX/GPw40jxlo+meHNV/si5i8R2dvbTyS+Uku5BBPMpTDgZLA5B4xgn6aoAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP5ov+D2j/AJP4+FH/AGIA/wDTjd1+MFfs/wD8HtH/ACfx8KP+xAH/AKcbuvxgoA/o9/4Mhv8Ak1n44f8AY1WP/pIa/Bj9uv8A5PW+Ln/Y46t/6WS1+8//AAZDf8ms/HD/ALGqx/8ASQ1+DH7df/J63xc/7HHVv/SyWgDymv6i/wDgrz/yqY6J/wBiD4F/9GaXX8ulf1F/8Fef+VTHRP8AsQfAv/ozS6AP5dK++/8Ag1//AOU5fwP/AN/Wv/TJqFfAlfff/Br/AP8AKcv4H/7+tf8Apk1CgD6O/wCD1v8A5SZ/Dn/smNn/AOnXVK/Hav6MP+DiL/gkF8Vf+Csn/BUTTrb4XzeGYZPAnws0aXUf7YvmtQwu9W1wR7MI2f8Aj3fPTGRXxZ/xBz/tZf8AP58MP/B7J/8AGaAP0A/4Mn/+TEvin/2Oo/8ASOGv2fr8sv8Ag2E/Y/8AGH/BPnwX8efgz48Glt4o8LeKNOvrqXTbk3Fq6XmmxSxhWKqchQM8dTX6m0AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAH80X/B7R/wAn8fCj/sQB/wCnG7r8YK/Z/wD4PaP+T+PhR/2IA/8ATjd1+MFAH9Hv/BkN/wAms/HD/sarH/0kNfgx+3X/AMnrfFz/ALHHVv8A0slr95/+DIb/AJNZ+OH/AGNVj/6SGvwY/br/AOT1vi5/2OOrf+lktAHlNf1F/wDBXn/lUx0T/sQfAv8A6M0uv5dK/qL/AOCvP/Kpjon/AGIPgX/0ZpdAH8ulfff/AAa//wDKcv4H/wC/rX/pk1CvgSvvv/g1/wD+U5fwP/39a/8ATJqFAH9O3gn/AJSzfFL/ALJJ4O/9PPiqvoevxk/4Ll/8FqvFP/BHH/gqWl34a8E6B4yPxC+FehQ3C6ndzQfZPser68VKeWOd32k5z/dFfL//ABG6/FP/AKIh4A/8G13/AIUAfs5+yH/ykV/a3/7CXhX/ANMUVfUdfmV/wbh/tua1/wAFJ1/aD+N+u6Hpfhq+8V+JNKsG02wmkmig+xaZHCGDPydw2n65r9NaACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD+aL/g9o/5P4+FH/YgD/043dfjBX9UH/BV7/gjx4W/4LDf8FUE0HxX4u1/wnB4D+FOlahaPpcEUrXDXOr6tG4fzB0AgTGPevJf+IJP4Q/9Fl+If/gvtP8ACgCt/wAGQ3/JrPxw/wCxqsf/AEkNfgx+3X/yet8XP+xx1b/0slr+of8A4IUfsC6R/wAEwvjz+0j8GfD3iDUvEuiaXJ4a1qK91CFIrgy3drd+YpCcYHkpj8a/l4/br/5PW+Ln/Y46t/6WS0AeU1/UX/wV5/5VMdE/7EHwL/6M0uv5dK/qL/4K8/8AKpjon/Yg+Bf/AEZpdAH8ulfff/Br/wD8py/gf/v61/6ZNQr4Er77/wCDX/8A5Tl/A/8A39a/9MmoUAfR3/B63/ykz+HP/ZMbP/066pX47V+xP/B63/ykz+HP/ZMbP/066pX47UAf0u/8GT//ACYl8U/+x1H/AKRw1+z9fjB/wZP/APJiXxT/AOx1H/pHDX7P0AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAHzL4b/wCUxnjf/sjXh7/0965X0tX4Bf8AB1V+3d8W/wBgr/go94H1f4SeN9V8F3/if4bW1nqclmI2+1RQapqDxgh1YcNK/wCdfmx/xER/tm/9F38Vf9+rb/43QB/UP+zH/wApPv2o/wDsFeC//SXUa/j6/br/AOT1vi5/2OOrf+lktf0P/wDBo7+0n46/bE8EfH34i/ErxJf+K/GGoa1pGlz6hdhA7wW1rMYlwqgcec/b0r+eD9uv/k9b4uf9jjq3/pZLQB5TX9Rf/BXn/lUx0T/sQfAv/ozS6/l0r+ov/grz/wAqmOif9iD4F/8ARml0Afy6V99/8Gv/APynL+B/+/rX/pk1CvgSvvv/AINf/wDlOX8D/wDf1r/0yahQB9Hf8Hrf/KTP4c/9kxs//TrqlfjtX7E/8Hrf/KTP4c/9kxs//TrqlfjtQB/S7/wZP/8AJiXxT/7HUf8ApHDX7P1+MH/Bk/8A8mJfFP8A7HUf+kcNfs/QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAfzRf8AB7R/yfx8KP8AsQB/6cbuvxgr9n/+D2j/AJP4+FH/AGIA/wDTjd1+MFAH9Hv/AAZDf8ms/HD/ALGqx/8ASQ1+DH7df/J63xc/7HHVv/SyWv3n/wCDIb/k1n44f9jVY/8ApIa/Bj9uv/k9b4uf9jjq3/pZLQB5TX9Rf/BXn/lUx0T/ALEHwL/6M0uv5dK/qL/4K8/8qmOif9iD4F/9GaXQB/LpX33/AMGv/wDynL+B/wDv61/6ZNQr4Er77/4Nf/8AlOX8D/8Af1r/ANMmoUAfR3/B63/ykz+HP/ZMbP8A9OuqV+O1fsT/AMHrf/KTP4c/9kxs/wD066pX47UAf0u/8GT/APyYl8U/+x1H/pHDX7P1+MH/AAZP/wDJiXxT/wCx1H/pHDX7P0AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAH80X/B7R/wAn8fCj/sQB/wCnG7r8YK/Z/wD4PaP+T+PhR/2IA/8ATjd1+MFAH9Hv/BkN/wAms/HD/sarH/0kNfgx+3X/AMnrfFz/ALHHVv8A0slr95/+DIb/AJNZ+OH/AGNVj/6SGvwY/br/AOT1vi5/2OOrf+lktAHlNf1F/wDBXn/lUx0T/sQfAv8A6M0uv5dK/qL/AOCvP/Kpjon/AGIPgX/0ZpdAH8ulfff/AAa//wDKcv4H/wC/rX/pk1CvgSvvv/g1/wD+U5fwP/39a/8ATJqFAH0d/wAHrf8Aykz+HP8A2TGz/wDTrqlfjtX7E/8AB63/AMpM/hz/ANkxs/8A066pX47UAf0u/wDBk/8A8mJfFP8A7HUf+kcNfs/X4wf8GT//ACYl8U/+x1H/AKRw1+z9ABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB/NF/we0f8n8fCj/sQB/6cbuvxgr9n/wDg9o/5P4+FH/YgD/043dfjBQB/R7/wZDf8ms/HD/sarH/0kNfhZ+2T4H1zxb+118UdT0zRdYv9PvvFmqTW9zBYyvHMhu5SGVguCK/dP/gyG/5NZ+OH/Y1WP/pIa/Rj/gjB/wAo2Ph5/wBddX/9O97QB/GTL8KvFEMbO/hvX1RQWZm0+UAAdz8tf01f8Fc5luP+DSvQZEIZH+H/AIEZWHQgyaXg1+p/xZ/5JX4l/wCwVdf+iXr8p/8Agq1/yqJ+Fv8AsnPgH/0LSqAP5f6++/8Ag1//AOU5fwP/AN/Wv/TJqFfAlfff/Br/AP8AKcv4H/7+tf8Apk1CgD6O/wCD1v8A5SZ/Dn/smNn/AOnXVK/Hav2J/wCD1v8A5SZ/Dn/smNn/AOnXVK/HagD+l3/gyf8A+TEvin/2Oo/9I4a/Z+vxg/4Mn/8AkxL4p/8AY6j/ANI4a/Z+gAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA/mi/4PaP+T+PhR/2IA/8ATjd1+MFfs/8A8HtH/J/Hwo/7EAf+nG7r8YKAP6Pf+DIb/k1n44f9jVY/+khr0j9gL/g4D/Za/Y0/ZX0P4YfEXx5qGg+NfB1/q1lq2n/8I/fT/ZZTql24XfHEyH5XU8E9fWvN/wDgyG/5NZ+OH/Y1WP8A6SGvwY/br/5PW+Ln/Y46t/6WS0Af0+fED/g6J/Ys1/wHrdjbfFG/e5vbCeCJT4Z1IBnaNlUZMGByRXl3/BV2Nov+DRfwwrKVZfh14CBBGCDu0rrX8v1f1F/8Fef+VTHRP+xB8C/+jNLoA/l0r77/AODX/wD5Tl/A/wD39a/9MmoV8CV99/8ABr//AMpy/gf/AL+tf+mTUKAPo7/g9b/5SZ/Dn/smNn/6ddUr8dq/Yn/g9b/5SZ/Dn/smNn/6ddUr8dqAP6Xf+DJ//kxL4p/9jqP/AEjhr9n6/GD/AIMn/wDkxL4p/wDY6j/0jhr9n6ACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD+aL/g9o/wCT+PhR/wBiAP8A043dfjBX7P8A/B7R/wAn8fCj/sQB/wCnG7r8YKAP6Pf+DIb/AJNZ+OH/AGNVj/6SGvwY/br/AOT1vi5/2OOrf+lktfvP/wAGQ3/JrPxw/wCxqsf/AEkNfgx+3X/yet8XP+xx1b/0sloA8pr+ov8A4K8/8qmOif8AYg+Bf/Rml1/LpX9Rf/BXn/lUx0T/ALEHwL/6M0ugD+XSvvv/AINf/wDlOX8D/wDf1r/0yahXwJX33/wa/wD/ACnL+B/+/rX/AKZNQoA+jv8Ag9b/AOUmfw5/7JjZ/wDp11Svx2r9if8Ag9b/AOUmfw5/7JjZ/wDp11Svx2oA/pd/4Mn/APkxL4p/9jqP/SOGv2fr8YP+DJ//AJMS+Kf/AGOo/wDSOGv2foAKKKKACiiigAooooAKKKKACiiigAooooAK+O7z9or4+/En45/FXSfBuvfCDQPDvgHxOvh20i1nwhqOp3twP7NsLxpZJYtUt05a8KgCIYCDk5r7Er43/Z//AOS+/tJf9lN/917RKANL/hL/ANqT/oe/gD/4brV//l5Vnwn+2t44j/4JY/FD4xaza+FLvx58ObPx2BHZWVxb6RfXHh/UdVs4H8l55JUjlGnxs6ecSN7AMOMd7XzXa/8AKBv9p7/c+NH/AKkHiWgD8kP+I1v9pn/onPwK/wDBXqv/AMsK7/8AZR/4PBP2ivjv+1B8OvBOq+APgrbaX4u8S6fo95Naabqa3EUNxcxxO0Za+ZQ4VjglSM4yD0r8N69o/wCCcf8AykC+CP8A2PWi/wDpdDQB/Tp/wcVf8Fi/iN/wSD+HXww1f4eeH/BWv3PjbUr6zvV8R211MkKQRQupjEE8JBJkOdxPQcCvys/4jW/2mf8AonPwK/8ABXqv/wAsK+n/APg+F/5Ij+z9/wBhzWP/AERa1/OvQB/Uv/wbvf8ABev4rf8ABXX44fELwv8AEPwt8PtBs/CehwapaS+HbW8gkkke4ETLJ59zMCuDkYAOR1NevfEL9vz9oC/+PfxL0Twne/B3TPD/AIM8SzaFZpq/hTUr68lWOCCTfJLFqcKEkzEcRr0r8wf+DIr/AJO7+NX/AGJ9r/6WrX32n/Jyfx6/7KJef+kdlQB2v/Dbf7Uf/QwfAP8A8ITVv/lzX1V/wTx/aF8R/tS/sk+HvGni2DRbfxFeX2raferpEEsFk7WWqXdkHjjlkkdAy26thpGwWPNfHVfR/wDwRt/5R++HP+xi8Vf+pJqdAH1DRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB/NF/wAHtH/J/Hwo/wCxAH/pxu6/GCv2f/4PaP8Ak/j4Uf8AYgD/ANON3X4wUAf0e/8ABkN/yaz8cP8AsarH/wBJDX4Mft1/8nrfFz/scdW/9LJa/ef/AIMhv+TWfjh/2NVj/wCkhr8GP26/+T1vi5/2OOrf+lktAHlNf1h/trfBWT9pn/g3g+B/wxj1JNGPxK0/4Z+FzqDW5uPsH2u70mHzvL3Lv2lgdu4ZwRkZzX8nlf2A+Jf+USP7Hn/Yd+EP/pz0egD88P8AiBzvf+jkLX/wiG/+Tq7v9ir/AINzbn/gj5/wUi/Zw+Jknxag8ejXPF9/4X/s9fDh07yfP8N61P53mfaZc4FqV27R9/OeMH9y6+dv24v+S7fso/8AZXJf/UQ8TUAfgz/wet/8pM/hz/2TGz/9OuqV+O1fsT/wet/8pM/hz/2TGz/9OuqV+O1AH9Lv/Bk//wAmJfFP/sdR/wCkcNfs/X4wf8GT/wDyYl8U/wDsdR/6Rw1+z9ABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABX5cax4J+IOtftfftFzeGfjL4w8Daf/wsFQ2n6do+i3cTyf2FpBMm+7sppM4KrjfjCDjJJP6j1+dvh3/k6T9o3/soi/8Apg0agDnv+FZ/GP8A6OY+I/8A4TXhj/5WVtfCiwu4v+DeX4+2upanc6zfrp3xfiutRniiimvpRrniMPO6RKkas7ZYhEVAWIVQMCu1rl/h3/ygG/aF/wCvb4w/+n7xHQB8oaR/wZ0fs+6lpNrcHx/8WFM8KSEC6seMgH/n296xviN/wbGfB39huHw18VfCnjf4jXniDwl4w8OS2cOoTWclszS61ZW7b1WBScLKxHPUD6V+xPhX/kWNO/69Yv8A0AV45/wUW/5Nek/7Gvwr/wCpFptAHzR/wcb/ALGegftzeOP2ffBnibU9Y0zSxJ4hvd2nNGsjSJDY7cl0bjBYY/2vYV+fv/ENT8Hf+hw+IX/f+1/+M1+sX/BWD/k5b9n7/rh4l/8ARFjXmlAHk/8AwQL/AOCeHhj/AIJ9f8FBvGum+FNa13VLPxB8PBc3K6mYmYSJqUSqVKIuABnj/aPtj5p/bj+MfxO+HX/BQL496f4X+JOseH9MPjaef7JFpOl3Ch2trYk7prZ36bV+90UdySf0X/4J+/8AKSLV/wDsmrf+nSKvzS/4KG/8pFvjz/2OEn/pJbUAcX/w0/8AHL/osuv/APhP6J/8hV+0n/BA3UL3Vf8AglN8OLrUrx9R1C5v/EU1zdPGkbXEjeINSLOVQKoyT0UAegr8M6/cn/ggL/yid+Gf/X54g/8AT/qNAH2PRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQB/NF/wAHtH/J/Hwo/wCxAH/pxu6/GCv2f/4PaP8Ak/j4Uf8AYgD/ANON3X4wUAf0e/8ABkN/yaz8cP8AsarH/wBJDX4Mft1/8nrfFz/scdW/9LJa/ef/AIMhv+TWfjh/2NVj/wCkhr8GP26/+T1vi5/2OOrf+lktAHlNf1Vf8FHvidrfwM/4Nlvhj498M3aWPiXwJ4c+HfiHSZ3hSZIru2uNKkiZkcFWAcA4IIOMGv5Va/qL/wCCvP8AyqY6J/2IPgX/ANGaXQB+Sf8AxFffts/9FF8P/wDhJab/APGa+jP+CO3/AAWw/aH/AOClv/BWr9nzwJ8XPFmma54b0nXdU1+3t7XQ7OxdbuLw9q0KMXijViNlxKNucHI9BX4qV99/8Gv/APynL+B/+/rX/pk1CgD6O/4PW/8AlJn8Of8AsmNn/wCnXVK/Hav2J/4PW/8AlJn8Of8AsmNn/wCnXVK/HagD+l3/AIMn/wDkxL4p/wDY6j/0jhr9n6/GD/gyf/5MS+Kf/Y6j/wBI4a/Z+gAooooAKKKKACiiigAooooAKKKKACiiigAr+TH/AIOAv2sPip8AP+Cx/wAddF8EfEz4geE9IuNWsr6Sy0nxDd2cDTPplmGfZHIFyQqjOOiqOgFf1nV/Hl/wcw/8ptfjf/196d/6bLSgD55/4eQ/tCf9Fz+Lv/hXX/8A8dr+ir/gm/4k1Hxl/wAGinijV9V1C91HV9V+HPxIvLy+uZ2lubqeS9115JXkYlmdmJYsTkkkmv5bK/qG/wCCXX/KnlrH/ZMPiJ/6V65QB/PtD/wVY/aet4lRP2ifjeiIAqqvjjUgFA6Afvq9J/Y5/wCCgvx7+Of7YPwj8KeMPjd8XPEvhvWPG+hx3um6l4uv7q1ulXUYHUPG8pVsOqsMjgqD1FfHNe0f8E4/+UgXwR/7HrRf/S6GgD93P+Dz74oeJ/g78Pf2e9c8I+JNf8L6umq63bfa9J1CWzlMbwWZZS0bAkfKP19a/Bv/AIeDfHj/AKLP8U//AAqb3/45X7kf8Hwv/JEf2fv+w5rH/oi1r+degD93f+DOD42+Nfjr+2n8XL/xr4w8U+LLnTfBMNvavrGqz3vkI99GzBfMZscqOnqfWvhP/gvp8TvE/wAP/wDgsP8AHqx0bxN4h06zfxF9oMMGoyogd7eEnADY9h6AAdq+xv8AgyK/5O7+NX/Yn2v/AKWrXw7/AMHE3/KZ347/APYcT/0mhoA+W/8Ahobx7/0Onir/AMGs/wD8VX9cH/BtHdzah/wRF+B1xczTXNxcQaxLLLK5d5GbW9QYkk8k5Nfx3V/Yb/wbMf8AKDr4Ef8AXpq3/p5v6APu+iiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD+aL/g9o/5P4+FH/YgD/wBON3X4wV+z/wDwe0f8n8fCj/sQB/6cbuvxgoA/o9/4Mhv+TWfjh/2NVj/6SGvwY/br/wCT1vi5/wBjjq3/AKWS1+8//BkN/wAms/HD/sarH/0kNfgx+3X/AMnrfFz/ALHHVv8A0sloA8pr+ov/AIK8/wDKpjon/Yg+Bf8A0Zpdfy6V/UX/AMFef+VTHRP+xB8C/wDozS6AP5dK++/+DX//AJTl/A//AH9a/wDTJqFfAlfff/Br/wD8py/gf/v61/6ZNQoA+jv+D1v/AJSZ/Dn/ALJjZ/8Ap11Svx2r9if+D1v/AJSZ/Dn/ALJjZ/8Ap11Svx2oA/pd/wCDJ/8A5MS+Kf8A2Oo/9I4a/Z+vxg/4Mn/+TEvin/2Oo/8ASOGv2foAKKKKACiiigAooooAKKKKACiiigAooooAK/Jzxz+xP8Jvj1+2Z+0VrXjP4f8AhnxLqyePktlu7+zE0qxDQ9IcICe253OPVjX6x1+WmvftP/DT4Tftg/tF6X4q+Ifgbw1qTfEBJhaarr1rZzmM6Fo6h9kkitt3KwzjGVI7UAZv/Drz9nn/AKI94E/8FqV6D8FPCuneDf8Ag3d+Omh6XZwWGkaTpfxcsLO0hXbFbW8et+Io441HZVRQoHoBWD/w3b8EP+iyfCr/AMK2w/8AjtdH8HdfsfE//BvV8edU029tNQ0zUNP+L11a3ltMssFzC+ueImSRHUlWRlIIYEgggigD2rw3/wAEaP2Vrvw7YSyfAX4avJJbRuzHSE+YlQSa8t/bm/4Ja/s7fBP4Ax+J/CXwc8B+HvEOl+K/C5tNQstNSKe3L+INOjfaw6ZR2U+zGvoXwv8A8FGf2e4fDWnI/wAd/g0jpaxqyt4100FSFHB/fV5J/wAFAv25Pgp8Sv2bTonhz4wfC3X9ZvfFfhb7PYab4rsLq6uNviHTnbZHHKWbCqzHA4Ck9BQB8yf8HTXgbSPHV3+z5BrGn22owpd6/tSdNyj9zZHp+A/Kvyo/4Zs8Bf8AQqaL/wCA4r9Yf+DobxBY+Hr/APZ9lv720sYjd6/h7iZYlP7iz7sR6j86/Lf/AIWr4X/6GTQf/BhF/wDFUAfd3/Bs38M9A8Bft5fEJtG0mz00yeAVDeQm3dnUYs5/75X8q+s/hP8A8E8Pgf8AtF/Gv4+eIvHXwu8H+KtcPxM1CD7dqVgs83li0smC7j23Oxx6sa+Wf+DbDxVpfiP9u/4g/wBn6lYX+3wEmfs9wkuMahHn7pPqPzFfYfwJ/bK+EHwa+Lvx90bxh8Vfhv4U1hfidqEpsdY8TWVjchDZ2IDeXLIrYyrDOOqn0oA6H/hzx+y5/wBEK+HH/gpSu4/4JE+EtN8BfsL6RomjWVvpuk6T4p8W2lnaQJtitok8TaoqIo7AAAAe1Uv+HkP7O/8A0Xr4L/8AhbaZ/wDHq0v+CS/iGw8W/sR6bqmlX1nqemah4r8XXFrd2kyzQXMTeJ9UKujqSrKQQQQSCDQB9JUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAfzRf8HtH/ACfx8KP+xAH/AKcbuvxgr9n/APg9o/5P4+FH/YgD/wBON3X4wUAf0e/8GQ3/ACaz8cP+xqsf/SQ1+DH7df8Ayet8XP8AscdW/wDSyWv3n/4Mhv8Ak1n44f8AY1WP/pIa/Bj9uv8A5PW+Ln/Y46t/6WS0AeU1/UX/AMFef+VTHRP+xB8C/wDozS6/l0r+ov8A4K8/8qmOif8AYg+Bf/Rml0Afy6V99/8ABr//AMpy/gf/AL+tf+mTUK+BK++/+DX/AP5Tl/A//f1r/wBMmoUAfR3/AAet/wDKTP4c/wDZMbP/ANOuqV+O1fsT/wAHrf8Aykz+HP8A2TGz/wDTrqlfjtQB/S7/AMGT/wDyYl8U/wDsdR/6Rw1+z9fjB/wZP/8AJiXxT/7HUf8ApHDX7P0AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFfx4f8HL0Yj/AOC2vxw2gDN5p5OB1P8AZlpX9h9fyS/8HCv7OPxB+M//AAWW+Omq+EfA3i/xNpkOp2NpJdaZo9xdQpMmmWZaMsiEbgGU49GB6EUAfm7X9Rf/AASsG7/gz/1IH/omfxC/9LNcr+cn/hhb41f9Ek+JP/hN3n/xuv6QP+CaGj3fg3/g0X8Q2Gq2txp19pfw5+I0F5bXERjmtZI73XFdHQjKspBBUjIIIoA9S8P/AAZ8HvoFkT4U8Nkm3jJJ0yDn5R/s1wv7Ufwq8L6P8NNNuLTw3oNrcR+LvDO2WHT4kdc69YA4IXPIJH415Xon/Bc39lS10a0ik+Lmnq8cKKw/sfUuCFAP/LvWB8WP+CvH7Onx40vw94T8JfEqz1nxDq/i/wANrZ2celX8bTlNbsZGwXgVRhEY8ntQB+hv7d/hTS/F/wC2f+z7b6tpthqduLHxQwiu7dJkB8mw5wwIzW1/wonwR/0JvhX/AMFNv/8AEVwH/BUD9oTwn+zd+1d+z5rvjDULrTtMe08TwLJb6bdX7lzDYYHl28cj/jtxXK/8PdPgH/0NXiH/AMInXf8A5DoA734A+A9D8G/8FPpP7I0bStK834XS7/sdpHBv/wCJtD12gZ6D8q/me/4OIoli/wCCznx42qFzrqMcDGSbaHJr+jD9ib9rHwJ+1B/wU0vpvBWq32pJp3wxlS4+06Pe6cyH+1YDwLmGMsORyoIr8Af+C7v7PHj/AOMf/BXv49av4T8D+L/Eelr4kNqbvTtHuLmESx28IdNyoRkH+Y9aAPz5r+w3/g2WUL/wQ5+BGBj/AEXVjx/2Gr+v5Q/+GIvjL/0Sn4i/+E7d/wDxuv6w/wDg2s0u50P/AIInfBKxvba4s72yi1m3uLeeMxywSJreoK6MpAIIYEEH0oA+6KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAP5ov8Ag9o/5P4+FH/YgD/043dfjBX7P/8AB7R/yfx8KP8AsQB/6cbuvxgoA/o9/wCDIb/k1n44f9jVY/8ApIa/Bj9uv/k9b4uf9jjq3/pZLX7z/wDBkN/yaz8cP+xqsf8A0kNfgx+3X/yet8XP+xx1b/0sloA8pr+ov/grz/yqY6J/2IPgX/0Zpdfy6V/UX/wV5/5VMdE/7EHwL/6M0ugD+XSvvv8A4Nf/APlOX8D/APf1r/0yahXwJX33/wAGv/8AynL+B/8Av61/6ZNQoA+jv+D1v/lJn8Of+yY2f/p11Svx2r9if+D1v/lJn8Of+yY2f/p11Svx2oA/pd/4Mn/+TEvin/2Oo/8ASOGv2fr8YP8Agyf/AOTEvin/ANjqP/SOGv2foAKKKKACiiigAooooAKKKKACiiigAooooAK+ff2J/wDkq/7S3/ZVW/8AUd0KvoKvyX8ff8HBnwL/AOCYH7aH7Rfw6+JWm/EOTX7zx8mtxNo+kwXNubeTQtIhT5nuEO7dA/G3pjnngA/WSvgzW/8AlB9+1d/12+Nn/qReJq8S/wCIyX9kr/oFfGL/AMJ61/8AkuvQvg/8Y9K/aO/4Nzfjz8QPD0d6uh+PNI+L3iHTEu4xHcC2vNa8RXEQkUFgr7JFyAxAOeT1oA/klr2j/gnH/wApAvgj/wBj1ov/AKXQ12cP/BGz9pu4hWRPg/4oZHUMpAi5B6fx16D+yd/wTN+PP7PP7WXwo8ZeL/hh4l0jw5ofjfQ5b27dI2EKtqNvGvAYk5Z1HA70Af0d/wDBWD/k5b9n7/rh4l/9EWNeaVT/AODiH9sXwr+xJ8Rv2e/F/jG31ufSHfxFZ5022SeRZHhsdoIZ0HPPfsa+E/8AiI3/AGf/APnw+Iv/AIKIP/kigD9If+Cfv/KSLV/+yat/6dIq9f8A2P8A/kePj3/2VLUf/SKwr4t/4IT/APBQXwN+3t/wUJ8YX/ge28RxW+hfDv7PdNqdmkGHbUoWAG13HIPcjofSs/xr/wAHAvwN/wCCcv7WHx9+HXxD034ht4gHxDvNSB0vSYLi3aGS1tEQhmnQ8+Wx6dCKAP1OrzX/AIJb/wDJoEP/AGOPjD/1KNVr8/f+Ivj9lP8A6Bnxc/8ABBbf/JVfbn/BFH4raZ8df+CcnhLxpoqXcej+LNe8UavYrdRiOYQz+JNTkTeoJAbawyAT9aAPqyiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigD+aL/g9o/5P4+FH/YgD/043dfjBX7P/wDB7R/yfx8KP+xAH/pxu6/GCgD+j3/gyG/5NZ+OH/Y1WP8A6SGvwY/br/5PW+Ln/Y46t/6WS1+8/wDwZDf8ms/HD/sarH/0kNfgx+3X/wAnrfFz/scdW/8ASyWgDymv6i/+CvP/ACqY6J/2IPgX/wBGaXX8ulf1F/8ABXn/AJVMdE/7EHwL/wCjNLoA/l0r77/4Nf8A/lOX8D/9/Wv/AEyahXwJX33/AMGv/wDynL+B/wDv61/6ZNQoA+jv+D1v/lJn8Of+yY2f/p11Svx2r9if+D1v/lJn8Of+yY2f/p11Svx2oA/pd/4Mn/8AkxL4p/8AY6j/ANI4a/Z+vxg/4Mn/APkxL4p/9jqP/SOGv2foAKKKKACiiigAooooAKKKKACiiigAooooAK/jy/4OYf8AlNr8b/8Ar707/wBNlpX9htfyb/8ABf39lbxl8cf+Cxvx21fRrWw+yQ6tZWmZbsKzMul2TZxj0YUAfmXX9Rn/AASr/wCVQHUv+yafEL/0s1uv55f+HeXxI/59dJ/8Dl/wr+if/gmr4euvCH/BpH4k0m6VPt2l/D34j2kyq+V8yO+11GAPpkHmgD3Lw5/yL1j/ANe8f/oIrgP2tP8Akk+n/wDY3eGP/T/p9aHh7wd+08dAsdnwB8PMn2ePa3/Cx7UZG0c4+zVwv7Uvh79oDSfhjp1x4p+C2j6DoMXi3wybq+g8c217JAP7esNuIRApbL7V+8Mbs9qAPBf+D4X/AJIj+z9/2HNY/wDRFrX869f03/8AB25+yl4y/bK0v9nvwb4ItbC41U3+u3zG7uxbxqkcFmCMkdTu/Svxw/4h7v2jf+gb4T/8Hif4UAfa/wDwZFf8nd/Gr/sT7X/0tWvh3/g4m/5TO/Hf/sOJ/wCk0NfqB/waofsM/EP9hb9uf4m6T49sdNgl8QeBI7y2ksr5blQseoRJhsYIJ3H/AL596+T/APgrf/wSk+MX7aX/AAVH+PfjLwdpuhHRl8WPp4N3qqQyM8VtbknbjoQyn8aAPyar+w3/AINmP+UHXwI/69NW/wDTzf1/Ol/xD3ftG/8AQN8J/wDg8T/Cv6RP+DdbwRqPwz/4I6fCLw5q8ccWqaBJrunXaRyCRBLDruoxthhwRlTzQB9sUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAfzRf8HtH/J/Hwo/7EAf+nG7r8YK/Z//AIPaP+T+PhR/2IA/9ON3X4wUAf0e/wDBkN/yaz8cP+xqsf8A0kNfgx+3X/yet8XP+xx1b/0slr95/wDgyG/5NZ+OH/Y1WP8A6SGvwY/br/5PW+Ln/Y46t/6WS0AeU1/UX/wV5/5VMdE/7EHwL/6M0uv5dK/qL/4K8/8AKpjon/Yg+Bf/AEZpdAH8ulfff/Br/wD8py/gf/v61/6ZNQr4Er77/wCDX/8A5Tl/A/8A39a/9MmoUAfR3/B63/ykz+HP/ZMbP/066pX47V+xP/B63/ykz+HP/ZMbP/066pX47UAf0u/8GT//ACYl8U/+x1H/AKRw1+z9fjB/wZP/APJiXxT/AOx1H/pHDX7P0AFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFfzt/8FOv+UnHx9/7GW1/9M+nV/RJX4f8A7V3/AATk+KP7WX7f/wC0L4m8KeI/AOl6Zb+NIdPMOrQ3bTl00XS3zmPK7dsi++QfagD4vr9T/wBiX/lV7+I3/Yp/FD/066/Xyl/w5G+O/wD0Ovwk/wDAfUP8K+vP2a/B2o/Cr/g2j+Mfh/UpbK61bw34d+LGn3UtsGFvNPBq/iGN2Td8wQspIzzgjPNAH6PeE/8AkVdM/wCvSL/0AV4h/wAFN/8Ak0a7/wCxq8Kf+pJplfhdpf8Awev/ABj0zTLe2Hwe+GbC3iWIMbq+52gDP+s9qs6X/wAHU/xT/b/8e+CPhBq3wx+H+iaf4z8ZeHoJb20ubszQeVq9pcDbuZhy0IByDwTQB+s3/BWD/k5b9n7/AK4eJf8A0RY15pXnn/B09+2drn7CV1+z3440PSNJ1qb7Xr9g9vftIq/vILLBGwg8bT39K/Kb/iJq+I3/AETnwT/3/uv/AIugD9xf+Cfv/KSLV/8Asmrf+nSKvPE/5OT+PX/ZRLz/ANI7KvBv+DZf/gop4k/4KHft4/EPVdd8PaHoK+HPAK2irYSysZTJqMTgkOT6Nnn04r0LxX4S+NGtftWfH2bwdP8ADCPRv+Fh3iqNZS+N1uFpZ9fKO3G3b+OaAPUa+j/+CNv/ACj98Of9jF4q/wDUk1Ovib/hXf7SX/P18D/+/eqf419of8EVINStf+Cc/hSPWPsX9qx694oW7+xljb+aPEmp7vL3fNtznGecUAfVVFFFABRRRQB8Xft6/tneNPgl+1j4Y8E6N408I+BdA1fw+2pXOp69p5uYophLMoXIO75gigAd6xfBP/BSHXPhD438Zf8ACwvEFn448PeGfC1hrUNz4a0ZYVvnvJoVieNnkGE2zKDv2gcnJ4FfQ/xC/Y30j4lftaeHPilqd6lz/wAI/o0ujjRp7FJre5DmU+YzMeCDJwNp+71rxf8AbZ/4Jva58S73xv4n8BarEmr+K7HTNLOikrZW0VtavGxVXwyvu8pf3bqqep4AMQvGEebe7v6Nyt+Dj6ddi7KU32srevu3/wDbvUguf+CuTad8QNStLvwOIdI0i903R7gQ6vFeXJvL4u0TLJDvtjEscZ3DzN4Y4xw2N/x//wAFZND8CyazGPBPiTVZ9J8azeCFt7OSN5ry5QMQ8a9wxXAXrkivPvAf7APxi+NHgq+8MfEfxBpXhTwzZXVtq2iLpulaYt9balC/ErJZxRwtEY2cEMxYsVPABz33hj/glRbaHq1hf3fj/VtWvrXx5H48nuLjTolkvLhR80bbGCgM2SWVRjPC1qoraT/rmh+S9p9667ZJvV76P7+WX4OXJ+PTV2dJ/wCCuHga98f2ujXWia5p9o2qw+H7zUJprUCy1KQDMJg80TtGjZRphHtDDHcGvq3Pzf1r5Zh/4JV+GLD423fiWz8RanZ6DqGujxJdaGlhal5LzJYqt4U89LcscmEHB9c4I+psHd14x09Km94RbXvdfuX63tbpa/UqWlRpfD0+9/pa/newtFFFABRRRQB4N/wUY+OPi/8AZt/Z2PjXwibUvouq2h1RJ7cTB7J38uTHo25o+e3NfLviz/gr34msfiF8Yrey/s3+yNP09x4FZrUE3dyl1DanJz+9y0jNj0U193fHj4Q2Px8+DfiTwZqUjQ2fiOwksnmVNzQFh8sgHcq21gPVa+bLL/gjz4QtdE+Edm2u3Up+F95NdzyGxQHXxJdLcbJfm+QBlwPvcGppL95ap8L/ACas/wDwG115y9Sqj9xOHxL8bO6/8Cu0/JI1rr/gp5oPw51OTSPEGj+IdSi8KXVjoHizxLaRQJYafqs6coIjIJWQOkgZlTC7cDdkZzvEn/BXXQPDXiDxxZP4D8X3MXgzWH8Pi7txG0Go3/niGKCNiQA0gEj4PRYz1JArY+JX/BLbw78R/inruqt4p12w8K+LdYttf1/w1DBCYdRvYN+1xMV8yNGLsWQdSx5Hy7V8Qf8ABMDRfEfgb4g6RJ4p1GC48aeMf+E0s72C0RX0W7DFkVVJIkUBmHO3IPaqi3K0p6d0vWG3p77j5KN9SLWfLF/N/wDb2/r7vN6uxqfsD/tP+Lf2j/FPxZg8U2I0r/hEvEh0yy094oxPYR4YmKV42ZZHUjG4Eg4yOtcY3/BYfwtZyeIzeeCvFkFvomsv4dt7kGHyNT1AS7FgSRmVF+UGRi7AIoGc5Ar139kv9kk/sv3/AI3vZ/FN94s1Dx1qw1i9urqzjtnExUh+I/l5JJ4CgdAK841D/glR4b1r4R634avPEeoSXd94yn8a6fqS2UWdNupAB5Zifek0WBgq2N2e2KHo49lGN7d/cTtf/t5q+7tcuNuV9+Z29Pea28+VPyvYrn/grT4b1K10C30fwf4g1rxDrWvz+GZNHtru0aS3vY1QqFmEhhkjfeNsivtwCe1U1/4K/wDhzUV8IQ6X4F8Xanqniqe8spNPi8rz9NubVlEySAEgqA2/eOigkgYxXMfFD/gmV4p0PW/hcfBfiG6n1LTPFVxruu+IWt7K2OnF0iWN7ezCiEImw4iVT35wRj0H4R/8EtvD/wAJvHHgXXI/Euo6nN4SOqzX63Vqn/E8n1BCk0jlSPLAGAFAPA696fW/9X5Y/O3NfXfa2zJ+w+9tP/Anv0vy2/G/QztA/wCCxvw21fVWM2m6/aaJcRX8mmaiWtpW1A2aM8o+zpKZotwX92ZFUP7YOLfhP/grV4G1n4WeJPFuo6NrGmad4ftba8RYbyx1B71bh9kMQ8idhFcbh80Uu0oOSap/Dr/gkf4b+HOun7J4y8Qx6DZLeHSbK2s7O3urCS5UqXku1j8248vJMaycKQOCMg5mk/8ABGTwbd6H4jt/EfiO91W71vTotMt7nT9JstIFisUqypMY4Iws0+5FDSSAllyD1yJ6fL/P8f8AyUrTm8r/APDfK2/W5d1D/gsF4R07w7JI3hTX5detdcsdFudHtru0upP9LjkkhlilikeKUYj2lQwIY4PrXIfGP/gqz4h1bwZ4Tu/A/gvWdO1Gbx4vhXXLW+a1kdJYyC9nGS+3zJQ42yfdXY2SOCezT/gk7ptzolkL7xvf3er2/iPTtelu4tHtbO3ZLNJEjt47aAIke7zCWf5iSM4yTm1rP/BLLT9Q8HatZWvjfVNP1a68et4/0/UorCJ/7OuiMCIxOSsijrk45A4xkF6JpvuvuTp//b79Uls9Y1cWvL9J/rybdH3vbT+Jn/BSy2+D3iu2tPEfw18d6Tpf2y10271O6SCGOO6nUkLBG8gkuYkYFWmiBTKnGcru8D1z/gp58VoPBPxN1KXQzof/AAjfjSLRbW4mtra4j01DKI2sZEWQGSbG5jICV+XjrivSfHf/AAR5sviB4w1jW7/4jatd3+uapa6zdXF1otpNcm5i5ZVmwJEgYk4hRlVflzu2itf4h/8ABKmLxrc+O7a3+IerWHh7x34hTxRcaW2lwTC2vhKJC6y5VypG5QucAN0JGadLScXP59t4P/5Nfcnu7uezUfl901/8jf5tdDoJP+Cofg23+Iz6Y2h+JD4ag8Sr4Ol8UBIPsCaqRnytnmebswGzJsxx0wc1zHgz/gsH4Y8Qa9p0Wo+CfFukaPqd3qNhBqp8meGS4sk8yWNVVt5+Upzjq4HODjauf+CWPhu5+KsupHxRrg8GT+J18Zy+FBBD9mfVACPM87b5nlcn936HrWd4Z/4JR6b4csfCEC+ONXH/AAifiDVNeSa3skhmla+SNCituYIU8vKthuTyOKmN7R5vn90P157eXLfUbtd22/8A2v8A7S/ne2h3n7In7fOhftf65e2Wk6HqOmiCxXUoZpb6yuleBpPLCyrBM7QTZGTFIAQMHuK96r5u/ZY/4Jx6T+zV8XJfGcniOXXtVTTX0m08vR7PS0WFnDs8/wBnRTczHAHmyc/pj6RrSXLZcv8AWr/S3zIjzXd/60X63CiiioKCiiigAooooA/mi/4PaP8Ak/j4Uf8AYgD/ANON3X4wV/Ul/wAHBv8Awb+fE7/grl+0j4J8a+BfGPgTw7ZeGvDX9iXNvr0l2kskguppg6eTDICuJQOSDkV8Cf8AEE5+0P8A9FR+DH/gTqf/AMiUAfU3/BkN/wAms/HD/sarH/0kNfgx+3X/AMnrfFz/ALHHVv8A0slr+pr/AIN6P+CQfjr/AIJC/Br4h+HPHfiTwl4jvPGGtW+pWsmgvcPHDHHB5ZDmaKM7iT2BGO9fmz+0d/wZyfHv4x/tBeOPFun/ABL+ENvYeJtevtVtorifURNHFPO8iK4W1IDBWGcEjOcE0AfhrX9Rf/BXn/lUx0T/ALEHwL/6M0uvgv8A4gnP2h/+io/Bj/wJ1P8A+RK/Yj9ub/gmL4v/AGoP+CKNj+zNomv+G7HxdZ+GvDmiHVL5pl015NNezMrZSNpArC3fb8mcsuQOcAH8cFfff/Br/wD8py/gf/v61/6ZNQr6c/4gnP2h/wDoqPwY/wDAnU//AJEr6W/4I/8A/Brj8Zv+Cd//AAUU+Hnxh8VePfhjrOg+EGvzdWelT3zXc32jT7m1XYJLZE4edScsOAe/FAHy3/wet/8AKTP4c/8AZMbP/wBOuqV+O1f1Cf8ABwN/wb0/FH/grb+1p4U+IXgbxn4B8O6doXhGHw7Pba7JdpO8sd5eXG9fJgkUoVuVHJByp4r4S/4gnP2h/wDoqPwY/wDAnU//AJEoA+xv+DJ//kxL4p/9jqP/AEjhr9n6+BP+Dfb/AIJP+OP+CSH7OPjPwb468QeFPEV/4j8QjV7ebQZLh4Y4vs8cW1/OijO7cp6AjGOa++6ACiiigAooooAKKKKACiiigAooooAKKKKACviD4Of8nE/tIf8AZSh/6j+iV9v18n+Kv+Cf/wAR7b40fEHxN4M+MOh+HdM8f62mvTaZqHgc6nJaTixtLNgs4vodylbRGwUGCx5NAG5XhWk/8oD/ANpn/rj8Zv8A0/8AiWvVf+GI/jr/ANF78Gf+Gyb/AOWldX4M/YJfRf8Agnp42+BWreMG1S48e2fiuHUfEMGli12Ta/eahdzyR2vmuAIn1BwqGQ5EYyeaAP4jK9o/4Jx/8pAvgj/2PWi/+l0Nftd/xA4aB/0cfrH/AIRMf/ybXYfs9f8ABmZonwB+PfgrxzF+0DqupyeDtdstaWzbwdHELv7POkvll/th27tmN2DjOcHpQBg/8Hwv/JEf2fv+w5rH/oi1r+dev7Hv+C1n/BF7T/8Agsl4G8B6NffEC88AHwNfXV7HNBpC6j9r8+ONCpUyxbceWCDk9TxX57/8QOGgf9HH6x/4RMf/AMm0AeQf8GRX/J3fxq/7E+1/9LVr9LfAP/Jd/j3/ANlIvf8A0isaf/wRZ/4N+NN/4I6fFjxn4psvijfePn8X6RFpTW0+grpwtQkwl3hhPLuzjGMD616d4w/4Jn/EB/i/478ReE/jLo2gaZ4312TXn02+8D/2jJZyvDDEyicX0W5f3IIygIyetAGFXpn/AASG/wCTE9F/7GfxX/6kuqVw/wDw7j+M/wD0X3wn/wCG0P8A8s69+/Yx/Zwl/ZM/Z10fwLca8PE11p95qN/camLH7Et1Le6hc30m2HfJsUPcsoG9uFHNAHqVFFFABRRRQB41+0L+3N4M/Zw8YR6Bqlv4h1jWBpsmtXdro1j9qfTbCM4e6nyyhYxg9MtweOmfG9f/AOCs2meDf2hPENjdaZc658NLDw7p+uWeraFpslxdQx3IhJmuS0gVIV83BwgcEgYJzXa/tUfsCX3xz+LV54x8M+MYvCep674Yn8Ia0txpX9oR3VjMTuaMebGUlAJGeQeOmDnzjX/+CQd5bPrNl4X+IceiaH4g8N2HhW+t7nRBdzSWkAjaSRX85AsryRA9MAOwxwDUw5rXe93/AO3/AIfA++suxUuXmsttP/bL/P4/KyRveCP+CpUOs/tL/EHw/qXhvWoPAfhXTo7601eDRpmlVPKMnnTkSH9zONv2crGDJ5iZxmqPxq/4K26ZpfwY8Val4R0HV9J8X+Hraw1GPTfF+lPbpd2V1NHGtwgilJKkSLj5lPzDgjNamt/8E0de0nx94uvvBPxPuvCGleMPDtj4fuEj0zzb60Wzto4IXjuBKhGRGpbAUkMwBHWvO77/AIIr6jqGj+I4G8eaNBN4i0Oz0mR7bQZUVZILmGYztvuXaR5BD8xLfeckYUBRS+xHppf75X/C3yf3FNq/NL+tI/rzfd9/r03/AAVZ+Hmj/EKXwzqOleM4Lmx1uHw9f6mulK2l2l3KPk3SiQkI5D7eN+EY7cDNb3h//gpV8MvEXjqPSUm1y2027vbnTbHxDcaeU0fUbq3XdNDFNksWUdygU9ieK828Vf8ABKu78SnxZ/xWltGPE3jfT/FwzpjHyEtVmUwH97yW8373GMdDmqfw5/4I56T4D8YsreINEu/Ckd7dXcUL+FLV9adJkKrA9/IXISPOVZI1fJPK8YLvt/Vo/q5JenpfOmmorm8vzl/9r9/zW/8ACL/gpl/w0F+1x4N8IeFtBvLXwV4l0m9vv7Q1jTpLa6vGgMoWW2YSlGgJjxll3ZVgQtM/al/4KPat8Cv2ttP8Baf4dFxoWlaQ+veI72ewuZrh7VEaRxaBCqZCKR5kh8vedpK7Tlf2Zf8AgmtrvwE+MvgXxHqHxAtfEOm/D7Sb3RdNsk0T7JIbed5XXfIJWy4aZsnHIAAxyT2P7Qf7Ddx8b/jzq3jOPxHDp0ep+AL3wSLRrIymN7jzsXG/eMhfN+5jnb1GamrdKLp625vn8bjf191eV+yLhq5Kel+W3l8PNb094wNF/wCCuXw18ReG7G/sdE+IN1NqENxeQ2Eeig3Rs7eMPNeY8zaYE+ddwYktFIFBxy7xl/wVy+F3hS6k+z2PjPxBZQ6Pba7Lf6Tpaz20NnOyoJHLSKybXZVYMoIY4GTxWFrX/BLvUYPDngVvDvju20fxH4W8IT+Cr68n0X7Vb6jZTLIrOsXnKY5F82Qg7mBJGenOKv8AwR4h0PQPFGk6P4zEFjr3gu28KRG404vJDJHdRXMlyxEgBDtG3yDGN45OObqWV+TWzdvO3Pb77Q9G30Wkwu7c3W3yu43+68vuXXf3j9m/9t7wh+0/4y1/QdCs/Eum6noEEF48WsaabI3dtMMxXEQJJMbKVI3BSQ6nHNY/jr/go78OPh78Ub/wzfPrrx6NqMGkarrUFgZNK0m8nz5cE027IY7WyVVlGDkjFJ+zn+xfcfAj496540k8QQ6lHrHhrStAFotmYjEbOCKIy795yH8vO3HGepxXnvxT/wCCVo+IvxH8WSQ+Nf7P8D+PPEFn4l1zRjpQlu5LmDeSkNz5oCJIZGJzGxHGM1Xu88F06/8AgVv/AEm70629BK/JJvfp/wCA/wDyWnpc6jwf/wAFUvhv4w+Kln4UXTvGeny32u3HhyPUrzTETTRexHBiMyyMMtxjAyAwLBQc10Hwt/4KIfD34u/EXTfD+mf8JBDF4gnurXQtXu9OMOma9Nbf65LaXOSV/wBtVB6DJ4ryzR/+CUsthc6KJ/GFtPa6X8Q7vxtLENNZfPhnES/ZQfN+UgRn5/f7vFUPhR/wRn0DwRrsFnrOsaFr3g6zkuWS0HhiC31e+SbO2KfUQ5l2x54MQjYgkZAwKil05+y++yv32d0t9vvurbXk7v7ryt2vpyt3tv8AdvfGX/gp0P2df2tPGfhLxVoF9eeDPDWk2V6t7oumyXN3bPP5QaS5YyiNYQZMA7QcsoGSa7XxB/wU1+F/h/xxJpLT67c2FpdWthf69b6cW0nTbi5UtDFNKSGVmAOcIQuCCQQa838af8Eh9Kml+I1p4S1238M6B418P2miWVg1tNdnSzDdQ3DyGSSYtIHaNuCRjf1wMVDrH/BIn+0tQ1PTIvHf2fwR4n1LTdZ13S/7ID3Vxc2iFWENx5oEaSl2YgoxXIAJxy6NtFU7/hzPffaO3nZCqdXH+vdX5yvf5s7n9jv9u/V/2lf2gfiR4Rv/AAfrOkWHha8MWn3b6W8It0X5WivXaVgtwx5RVUZVXPaul+K//BQ74ffB/wCI2q+HtSHiC6HhxrZdf1Sx04z6d4eNwwWEXMgIILFl+4r4zzjml+AX7JOr/AT9o/4ieLLTxVbXnhn4hXX9oz6O+mbLi1uR90rceYcoAZBjYM7h6ZPmXxm/4JRWnxJ+PviXxjp3iHQbO18ZTwXOpW+q+FbbWLmzkjxvNpLM2yPzQMMGjYDOecABR3pp7WXN66X/AF2v0D+d9b6enl/wbHV2f/BVH4b3Xxd/4RBtP8ZQyr4mfwm+qPpiHS0vQ21AZ1kPyyEHbxuwCSqjmrXg/wD4KifC3xl45bSkfxFYaZI93FZeIr3S2h0XU3tULzrBPkliqjPKjt3IB4G6/wCCUMlxfXJHi+1W0uPiUnjsQjSyNsK7h9k+WQDODjeABx92tP4Rf8E0tf8Ahlos/gyT4ralP8KmGpIvh+10tLa4uo71CrRz3O9t4TORhBk56A4qY35Nd7fe+WL17Wbkla+3bcdr/wBd5ffoov5/d1vgv/gp/wDC/wAXxXsszeItBt4NFuPEVlPqumm3TWbCAssk1rhmL4KMArBWOOFxzXgPxS/4LH+JbfUvGN14V8K2ljomjeHdP1rTV8TabPBeTm4ureLc6xz7WiaObehTrlcnqK7HQP8AgkB/aGlxaf4u8fHW7LQvC114U8Oi00cWb2MUzyOJ5iZX82RfMIwNoIArE8Rf8EefEvjPRtai1j4n6deXep+GdO8NRSx+HzCtvFZTwPGxUTncTHAFPI+ZienFU7e0Vtrr7ry387cnldv0Kg1b3v6+H8L83noj1Wb/AIKsfD3SPiHJ4Z1HSvGcFzY63D4ev9STSlbTLS7l+5ulEhIRyH28b8Ix24Gah8Tf8FSfDUutaVp/hPwv4h8SNrniX/hFbDUbgx6fpVzeKyiRBMxaQFQykZhAbI5HJHM+Kv8Aglbe+Ij4sK+NLaL/AISXxvp/i9c6YT9nW1WZTAf3nJbzfvcYx0Oa8g/Z+/Zj+PHwk+K0Gqj4beHdU16bxDNfXWp+IdJ0u5+zwTTFnmTU47z7SzhCCAtuCCSMYGKqnaVSMZd9e32P1c15WT23xXNGi297f/J/ftF/O259tftCftX+Gv2cb3QdP1S21vWdd8TyyR6Xo+jWgur688td0jKpZVCqMZJYdeM815yf+CqHwyvte8D6fpNv4r8QT+P4PO0z+zdNEpjInaB45lLq6OjK5b5SAqk5PGej/ao/ZGvvjl8QvBHjbw14lh8K+MvAb3Isrm50/wC3208U6bJEePzEOQOQwbucg5BHAfs5f8Ev7X9m/wCLXw88R6f4oOoQ+C9IvrG7imsdkmp3F08rNOCHIjUeYAEw3C/e5qaGr/e7X/z/AD013v0tqaVtF+77L776/d07rsz6vooooAKKKKACiiigAoor5h/4KS/Hfxn8PdT+C3gDwLrw8G6t8Z/HUXhi58TCzhuptEs0s7m9na3SdHgNzItt5UZlR1BkJ2MRil1jFbtpfNtJfix9HJ7JNv0Sbf4I+nqK/Nz9oj/goF8Sf+CXn7Uet+BNV1XWv2hNE1rwlpfiLQItcl0/R9W0y9uNetdGa2kurOzjhaB2vElQvBvBikXcRyOk1z/gtnrXhz4hN8NL74VaHafF638Waj4audPk8ZTN4chistOs9QkvBqSaa1wytHf2yLGLHdvc52qpenBqcFOGqfN/5LJRl9zavbo77A4uMuSW+n4x5198U/utuff9FfA37Sf/AAV20vXv+CYfjzxl4Z1C++GfxlTwNqmsad4c1W1zqej3ttFMSdk0XlyIDC7o7LtkTa20BsUzSv8AgsT4z8M+OLX4WeJPhDpa/GbUr/w3puiada+NfO0fUxq9leXa3NzfGxR7byY9PujKkdtP8wQRmQNkOz5nDqnFerk3FJdHqrafkJtKKm9rSfoo8t2+3xL8ux9+UV+fnwe/4Kd/E/8AaA/4KL/Dr4c2/hXQ/CWjWa+LtG8eaRJrf21hqmlSaeBJa3H2IGWBYryF4jmEy/aZBIsRhUSaH7TX/Bdfw5+yf+0tc/DLxX4IurfU7Px5pfhu4ul1YeTb6HfWsMw8QPmH5YI5ZhA8ecBx/rO1JNOMJ30nqvNcyinfazbTT7a7KVhpqUo9Y7/JNteqs1bvZbtX+86K/O3wx/wXa1v4i/Cvxf468M/BL7d4U+Gvhm28Z+LJLzxeLS9h0i7kuXtJLGD7Gy3U0lhb/a2jlkt1USpGJHbmtjwj/wAFmvFvxN1+K18MfBvQ7u38UfEfUPhp4Knv/HT2h1u7sVmmuby7RdOkNlarbQO4KmeUyYTy8HzKEm5ciWvbqveUde15Oyvv02YS92PM9u/pHmdu9o6u2yPvqivhf9lD9vX4m2v/AATQ+LPxi8aeG5PGHi/4f+K/FYn8OW1/DG0Nlp2pzRm0iuI4AsvkW8bBXMW6XyhnBbIzfGX/AAXT0u++KSeGfh74IsfF0Gsa2mh+Htf1DxHJp2j6nJFpMWp6hJJJDZ3MsaW8dzaxDZFMZJZWGEWNmpXVlb+WMu9lNJq/32vtvrZDaabT7yj6uDd7fde29uh99UV8N/Cz/grh45/aQ1OLR/hn+z/qWteKtE8MxeJPFmi694nTw/Pp3m39zZR2li01qyXksjWVzLG8xtYWiEbGRS+1eA/ZW/4Kt/G3x/8AEbxF8Orn4deFPGPxHvfGnjJdKs5vF39k6fo+i6LdxWzwzXEemyM8iyzwxREROZtzvI8IXFF/e5fKUvVRaUrd7b32sm77XT0V33UfRyTkr9trW3u0rH6R0V+ePgn/AILm+Jvjb4f8S698OfgDqvizQvBngnT/ABbrMX/CRMmsrNePeQpY29jDZTm4Mc9lMJJFkGIlZ1Vztjb3L9ir9vbxB+2d4fj1HSPCPgSWz0/X5dI1260Tx+mqw6dALUTRzJG1pDdCYyMIntbu3tJUwX+Zcbr5G5OPVb/+Bcv5/etdtROSSTfX/wCR5vv5WnbztvofTlFFFSMKKKKACiiigAooooAKKKKACivx1/aM/wCCnf7QPw1uNf8AAXh7xZ9v8c/Anx14m8WeOpW0y0ebVPBGmSWV1bWzJ5W2Mz2mqQoJECuTaP8ANnca9h/Zv/4KzfEf4xfGb4h2fgbw3p/xYg1/UNd8SeELbUNfi8P6ZpvhXR5YNL8yG5js7iS4nu9QS6ZFkXaAOZUUCphOMoKp0cedf4bJ/rL/AMFz7a1ODjJxfSXJ/wBvXa/SNu/PDvp+lNFfnPpX/BfW48c+FvEXi3wr8H/7S8D6DpnheVby/wDFa2OpXupeIY4hp9ilqLSRAFnlEc0zThUQF0WU/JXqv7EP7WfxJ8ZfHj9qq3+L1pp+gW3wq1TSlstF0rUF1a00m1bRYruYw3X2W2lnEpJl/exKyl9oAAFVP3OZz0UU2/Kzin8/eT9HfZq8w99xUdea1vmrr/L1ut00fYdFfm7H/wAHAd9b/Au/+IV18Er19CvfCL+MPDZsPEMtybmEXEEaWmpSNYpDYXUkVwkqLHJcoQsi+YGXnqbX/gsL8QdB+J+paB4t+B+haLY+GfiBoXw78Qajpvj9tRa0vNZigks5rWJ9Nh+0RoLmITb2hKFvk84AkKpeGjWvb/t7k69ebT5ro0Lmjyud9Ff8I8z/APJdf+CffNFflT+z3/wXQ8cfs7/sQ/DPxZ+0H4Fk1Wbxz4P1PXPD+v6RrcU954lubO4jQQXdoLSCGwaZbiJkZJJl2AltrZWvZNI/4LJ+J9Q8Zad4Ak+DloPij4k8Q2Oh+H7eLxRc/wDCLaolxZXN5LOdWl02OVTbx2c6yxpZyMHMQXcsm4aSg1Lk31t/5Ny/mtt0tXZajeibeyv+Cu/lZ77PZan3lRX5TfBj/grt8bvFvwO+Jvhi08H6f4l+MHhy08c+KNRe98RwafbeE9M0/VbmxtLe2eOykW+njeNlQSRwpIsG6SVS+K1f2Wf+CvfxF1nxB4N8FWnhe7+J3xA8dL4a01H8QeKLXQ9ItrmfwdDrdzcobbS5JYo2xKXRvPYyElNiYiEU/fh7SO3LCXym2lt2aab2vs2k7FT3Jum90538vZ2bv6qSa623S0P1Eor847X/AIL26v47+C/iPxb4J+C1vrMnw28DS+N/HdhqfjMac2kIl3e2n2SydLKYXkpbT7l9zi3XywhyWbYPQPht/wAFedW+IP7Vlv8ADSb4ZaX4bfU5nt9KGveMBpesav8A6ALqO7tbO5tI4ruykkJiElpdTzLje9uoBAI+82o62/za076xdrb2utAn7mstP+GT+Wkl96PtyisfwBqmta14J0q78RaTa6DrtxbJJf6da3/2+GymI+eNJ/Lj81VPAfYueu0VsU2rOzEndXCiiikM+cP2iP8Agov4f/Zf/aesvBHiyze00C58NnXZNYhSa5ljfzpIxF5EUbHbiMsXJAHOa3vHX/BRv4P/AA8ls1vvFMsy3em2+stJZaXd3kdpZz7fKnnaKJhErb1xvwfmHHIrif2sf2GvFHx6+P2t+K9K1PQLWw1P4c3fg+KK7llWZbqWSV1kYLGw8oBxkgluD8teS/En/gl38YPGngWDw2njrw3Nosfg7T9BhspL/ULW3sLm3jiEjiOAKlyrvG2HnViofiPKrWcXLkXNvr+dSy9bKHyffUppc+m3/Ap/5z+7tofUVh+3Z8MNY+IEXhuy8QzXl9NeR6alxBpl3Jp/2uRPMjtjdiLyBKychS/Y9+K8j+BX/BXTwd8VfBN//akKaD40jj1OSy0dhcSW94LONpNouvJEe5lXJUZIBHBJAPEeG/8AglX410L45+HvEltrXg3QU068sby91bRptRt9TuUgiRZLVoCxtnVyhBlIVmBJKclaq6H/AMEpvHumeB/AOmPrnhEz+Fb3xLc3TLPcbJV1KBY4An7nJKlfnyBgdN1aW95ro0/k9NV+O/3dXMXrr5flK6/Bff8AI9X8Hf8ABWf4ZR/DXw7qnjK+k8PazrekDWnsLSyu7+OGAzyRBhIkIyP3ZJyBjv2z1fxN/wCCk/wo+Guk6pc/2tqevNpGn2+pXSaRpNzdR28Vwge282cIIYTLuUL5ki8sM4r588M/8EovHui+GLiyl13wg0svwwuPBKlZ7jaLyS8acSf6n/U7Tgn72f4cc14140/Zu+Knww+LWqWF38KrT4g2ttpej6ZD9o0vVmsdUFlaRoAr2LpbyoZFOGvRuGFyEAIpys6jitFd/dzSWnnZRfzCKtFS30X38sG/ldyXk0j9JfF/7Rfhz4X/AARtvHnjG4PhfSJreCaRJyLmWJ5gNkQFuZBI5LAfuywPJBI5rz3U/wDgp38GdE8Iz61feJb2xtrPU10e7iudGvIbqyuWR5FSWFohIuVRiDjHBHXirX7Rvwm+Iv7Q37Jem6N4dv7D4ceMLuKzuL6285mhtwEzLZrPEC0YBIHmRjOEwODXzlon/BKDx/H42fVNQ1vwhPDc+O9H8VTRNf39232e1SYTQl7hHkkcmUBTI53AEkrnaBq9WUdldJenNFN+ejl933zF/uoy3dm+2vK2vS7S6dfu+mdJ/wCChPwk1rxPYaTB4oYz6hJa2yzHTbr7HDcXMYkgt5bny/JjmdckRu4YFWBAIxXY/Gn9orwp8AItK/4SS9u47rXp2tdMsrLT7i/vL+RV3MscMCO7YXknGBxzyK+TfEP/AASU1fUf2gdb1NNX8N3fg/xD4pXxRPJezah/admxcySW8cMcq2z5Y8Sv84HY9K9n/wCCgH7Iup/tY+GPDdvplt4PvZdBvJLhrfXmvLcOHTbmO4tHEsZBAONrK2FzjbzDb9lGSXvN6rstL/jdfK+2pokvaOL2to+71t+n3lvxV/wUq+D/AIQg0iS48RX0ya7pT61ZG10a8naW1R3R3KrEWTaY5NwYAqEJOAKf4x/4KS/BvwSLNrnxY9zFd6dDq7S2Gl3d5HaWkxAjmnaKJhCrFgMPhuRkcivk3Uv2HvjL4G+Nvgvw1o17b+IbyP4ealot74j1m3vJNHtPtU9zm2WdQzBo45R5YfJO0ZUA8dB47/4JNfFG88CnwjpXj/Rb7wunhm30m1tr27v7FLO6RlklkMNv+7nWSQPgz79iuMISi1T+G67v7k5q34R121fkTHWVn/Xwf5y08vkfRniD/gpr8FfCvjLVNC1DxgLa90aEXF0fsFy8SxtGkisrrGQ+4SJgLksWAAJraP7eXwwg8DXuv3WuXun2+najFpE9peaPeW+oi8lUNFAto0Qnd3UgqFQ5HPY48I8L/wDBMzxRp3g34vadf3vge9l+IFhotrYrcx3NxChsYVWRZtoidAzqCrRsSOGxkYrkbf8A4I/eMLn4XSW03ijQrPUtP8Q22t6LoEF/qU+hWaxRskkXnu4ukMu4EunzLsAGc5Vuy0/w/i1f7lf5r5M6/N/lp97Ptr4OfG/wz8fPCsms+FtR/tCzguZLK4WS3ktp7S4jOHhlilVZI3U9VZQfwrzH9r39uOw/ZE+JXw60rV9NM+jeNJb1Ly/QyyS6etukTApDHG7SljIBgYxXBfAD9nr4k/si6r4T0bQrfwpcw+OPE13qfjJrSC/nt9OthEpQRXE8jMW+UpulO52dcKdrtXfftUfssa78c/j38J/FWl32k21h4Dk1N76K6kkWacXMEcaeUFRgcFTncV4xjNTUupQcNr6/+A3fyvommFOzup9nb8UvnpcwIf8Agpr4K1T4reH7SxvNNk8C6z4VuvE8viCaSaF7ZLeWWJ0MBizkNERyQ2eApyM4fxw/4K0eFfA3hnVbjwl4f1/xNe6Zoqa+32+1k0a3NnJNFFFMPtKpNIkhlG1oonHTJA5HhOt/8Ev/AIofBz4PQ3llceH/ABBqOgeCNV0BrHTY2u57ya7vp5h5Uc8aI6iKbByQwIO1WIXPOfCj9mH4pfGL4e+JfAEXwo8P+GZPEOgQadL4t1KPWrOSyW3lhkjgJvzI7IxixstkCA4PAAondpqG6T/9Kn8tEo+vqxx0d5bc1vl7mvzvL+kfev7MH7Zfgj9q2yuo/DWpNLq+k29vNqdjJaz272pmTcpXzUXemcgMvBwD3GfLfi1/wVS8J+Gfi74e8I+EYB4ruNQ1q40XU7krcWtvYyQqpby5TEY5yCcMEbjg9CM53hD/AIJqX958dvFniLxL4lv7LSNZ0vRrWzXw1rd1p94JrOyS3k85kVAYyVJUbj2JANcFYf8ABLD4jaD4t0CytfEvgy48G+FvF1/4jsPO+1Lqcsd2ihklOxk3L5agAdcsSw4WniPiTht+vMtPuv39R4a2ntPL8Y3f3PTp6Hp/w3/4KyfDy/8AhR4b1vxlNN4f1fxDptxqy2FhZXepIltDcywM+9IQcDymY5UYAJ6DNdv4P/4KQ/BjxzLrYsfGcKxaBpn9s3c91Y3NtEbQMqmWNpI1Eg3sq4TJLHABr53+HX/BKfx54Q0DRbW41zwjJJpvw71vwjIY57ghrq9lu3ikGYR+6UXCbj97hsKeMs1j/gkJ4o8baTa6dqvibQbK2g+G9t4S860aeVxfwXkdysmwogaA+Xg5YNz93iipu+Xzt99S34Rh683QzpXaXP8A1rD9HL/wHqfQ97/wUn+D2m+ELzW7nxJd2lrp9za2tzFdaPeW11CblS9vIYZIlk8qRBuWQLtwRzS6V/wUj+EOu+HItQsvEN/eST6nJo8OnQaJey6lNdIgdo1tViMxwpyTtx754r5qu/8AgkP401jwjdST6j4FsvEVxqOlNi1vNTngNraFjIzTXJlkMjttIQKEXBAbHFbHxL/4JW+LfGuveOdQZ/h1rC+IvGlx4jsrTU5dQhMcEqMoBuLfZJDIpIJVRIjYGT8vLlZXt/WsPzTk/K2pUbtq/wDWkvyaS87ntE3/AAUU8KzfGvw9pFrcaa3gvW/Clz4qfxBPLNbtbxQPKjqYXiBGDEc7iGzxtrSH/BSv4OHwhq+tnxPcxWeiQ211dJcaReW8/wBnuXVILhIpIleSJyy4dQVwwJIyK8BT/gkb4u8RR6JY+I/HNlqVla+CL3w5c3bSTy3MFxNcyzw+Urr88EW5F+eQMQpGACAMvxz/AMEmPiT8UvBWtSa74m8EjxL/AMI3pfhLSI7M3SWItLOaGQzTuyF/NYQj5VQqCx5otqo/j/2/P8o8n33HDXWX9aR/+2fyS0PpXTf+Ck/wb1XwXq2vR+K5BZaNfw6ZPG+l3a3T3EwYwxxwGLzJDIEYrtU/dOcYrnta/wCClPhjWPGnw303wbB/wkFv418TTeGdRe5WbT7nQ54xGWDwSxhy37zodo469cecfEH/AIJ1/Fa88e/EXWvCvjXQdDXxhrmk6lHDHc3VvLc29pC6SQyTxRiS3JZlIaEkkDqtY/wo/wCCWfj7wH8U/Dev3mu+EpodH8fT+Lp1ju72aR4JI4h5atMju8gKHmSQkjBLkk0Q5filsuV/jDmX3OX3ffDcrO2+v5St+Kj9/wB33rRXm37KHiLx/wCLPg5baj8SbTTbHxJc3VwRBZW0tsi24kKwlo5fnVmUbsNggMMgHIHpNElZ2KCiiikAVwv7Q/7Nng39qn4ejwx440l9U0yK8g1K1eC8nsbvTruB98NzbXMDpNBMjcrJE6sMkZwSD3VQanfDTNOuLkxzTC3jaUxwoXkk2gnCqOSxxwO5qZWteXTX7tRxvfQ+frL/AIJafBw2Ot/2tpHiHxVqviS70y71PW/EHiXUNT1a7Om3SXdlF9qlmaSOCKdFcQxlYycllJY5tePP+CZfwj8feOdS8VvpOu6N4w1PXpfEreINF8RX+m6nb30tjDp8rxTQzKY0ktYIo3iX922xWKlgGrc8SftDW3jr9mP4g+JIW8WfC6PRNIvmOr+JvD09nJpJS1Z/tot5Bulji++cDB2EV+auiftCeMf+Cb3w68N+K/Es3jrU/GF98Mtc1Pw9r8PxT1Dxv4E+Kl3Z6UNQN1eWl+yXenybYjMn2eONB5rx+aQVWhyUefn0UVf703b5+zj5t2snyu1Rg5cqhvJ2+6yv8lN67JXu0mr/AKcN+xZ8N2/ZMu/gf/YEg+Gt/pM2iXOmHULlpZ7abd5oa5MhnZ3LsWkMm8liS2ea5r4o/wDBNT4P/F7xbfeIdW8PahF4kvDpDx6vYa1e2V9p8ulLcLYTW0sUqtBJGt1cKWTHmLKyvvXivlPxb/wVX+MPwh8Zj4e+Kr/4NXXjXxFo/hbXNB1jRvDWuXlmBq0t2ktgumW09zcXt4gtJHhxcW0csYZ3aAIc5XwD/wCCs3x7/aF+KvhjwNp9j8JtE1aLWPGtl4i1LV9Dv4IrmDw/LYBDDbf2hmylmW9KuJp5lhKlsttKG5Xc5X3XM360/ifrBv1/luZxacIpbO1l5TV4/KSivLRc1rH1/wDDn/gmj8IPhP408J+JNB0PWrLxH4OvNU1C11U+I9Rku7+41Nonv5L2Rpyb0zNDEWFx5gHloFChQAn7Q/8AwTE+CH7VXxF8Q+LPHfgiDW/EHirwi3gbUrv7dcwNPpJuBceSBHIqq4lUESqBIAMBgOK+HrH/AILh/GXwr4LtrLxV4f8AAMHxE8ZDQY/D2jr4d1O3i086lqC2j3i3CXdzZa3p6CSPy7izvIHlkZVMUIfcv23/AME//wBpTxz8ftE8e6b8Q9K0Wy8ReA/Er6L9s0qH7FDqsBghnjmexa6uZrGUCXY0Eszn5A4JVxh+z54tbqPMvLpGSXT7aTto03a6UrP2nJLzdvXrKLfXeLavqmk3a6vT+IH/AASh+CHxHnUXnhrVrGym0Cx8K6lp2leIdQ02x17SrIEWlnfQQTIl1HECVHmAkqSjEoSp2NX/AOCbfwi1LwIdAg0C/wBIt4/F9z49s7rS9ZvLK+0vW7hnaW7tp45RJDu82RfLUiPY5XZtOK92opNtvme71/FS/wDSkpeqvuJJJWWy/wAuX/0n3fTTY82/Zy/ZL8C/sp/CGfwL4N0u6t/Dd3eXmoXMGoajc6pLdz3cjS3Mks1zJJLIZHdi25j9415t4Z/4JL/A3wB8F/BvgXwt4Z1TwjpXw81K81fw1eaJr9/Zapo91d+YLmSO8SbzyJFldWR3ZCu1SuFUD6Sopf5JfKOy9F07D2082/m73fq7u/e7PmDxB/wR3+Auv+G7LSR4c8Q6fp8OjN4dv49P8V6rat4h01riS5e11GRLgPeo08srsZ2ZiZZBu2uwMs3/AASJ+CEHia51vStJ8WeGdauNV1DWEvtD8Yatp09pLqGDfJAYrhfJhuCql4owqbkRgoZVYfTVFFl+f4u7+96vuw/r7lb8tPTQ+fvD3/BLf4GeEfh/4m8LaV4JGmaD4u8Pab4X1K1tdUvIt9jpxma0VHWUPHIjzyv5yMJWdtzOzAGtr4PfsGeAvgd4ll1zQ5/G7a7fasNb1TUrzxhql1ca9craCzj+3F5yLqKOBVVIZQ0SFQ6oHG6vZ6KfM3Jzb1e7763/AD19dRWVrf1tb8tPTQKKKKQwooooAKKKKACiiigAooooA8kb9hb4VyfGr4h/EN/CdtJ4u+KuhReG/FF89xMf7TsI4zEsJj37EymFLIoZgiZJ2jHDL/wSU+B+m+AvBXh3RdA17wrZ/D/Rbrw3o9x4f8T6lpd6umXUgkubOa5gnWWeKVxvIlZiHyylWOa9d+KHxxt/hZ4j0XTZvDXjbW31t9iXGiaFPqFtafMq5nkjBWIfNnLdgx7Gvk/9pPwT/wAL7/4LReDfh9r3iDx5a+DT8GtT11tL0Hxhqvh+KS+j1i0hS4c2FxAzOscsijcTw3sMRGKcoUorSXMvL3Y1Jv8ADnVv777sptxUqsnrHl9felCC/wDbflFdke1Wf/BMD4G6b8OPF3hG18C29p4b8b2Gk6dqljb391Enk6XEsWn+SyyBrd4FRCskRV96hyxYZro/2ef2JfAP7Ms3jWfw9a65fX/xGuo7zxNfa9rl5rV1rEscIgUyPdSSHAiATauF2gDGBX5v/s4f8FOfjfoevaz8GfBV7aeNj4U1XxndaN428X6ddeIp9d0TSdQitLWEmG7tWkdZZZIpb6SV9qwBjHIzEjt3/wCCx/xv1zRPF/jmz8PfCzSPBnw8XwE2r6DcQ3Gp6pqLeITbJcJBqNveC1QQNOxjkEMyyLjIX7xuDVdKotVUUdX1U3ZX/wC3lZrWzWulm4kvZSdF6Om5aLo42WnykmttH0aaX1FZ/wDBHP4C2/ha80Gfw94j1HQJtGfw7ZaZf+LNVurTQ9NeeK4e1sY5Lgi1QyQxH93hgsaoCEASu88R/sB/CvxZrWu6hf8AhyWa78S+LNK8b6i41K6Xz9W0xIEsp8CQBRGttD8i4RtnzK2Tn4T+JH/Bcn4qfADS/HGseNfDfgJI4rPW5PCWi2+nX4i1V7S/S2glt9cgnutP1GJY3El1HtsriAgqImwa+tP2DP2o/ib8Xfif8QvBPxP0vw4uo+EINMv7HVtI09tHF7DeRyFopdOmvbu4gaN4m2ytJsmRwVClWFEP3kVLyuv/AALmt6qS5n567jqWjzJ92n9yj9zTS815FXwn/wAEYP2evCnhGXw+3hHWNb0AaFdeG7HTdb8T6nqltothdSJLcxWST3D/AGVpJI43MkW1wUXaygAVrt/wSl+D9xZzyXVt401HxBJqlnrMPia98Y6pca/Y3VpFJBatBfPOZoljimlTYrBHEj71cuxP0hRR/X3Pm/PX113G23v5/irP71o+6PlVf+CL3wBtfB0WiWWg+LNMh+zapY3d1Y+M9Ygv9VtNTn+0X9td3S3PnXMU03zssrthixUrubPWfDT/AIJhfBX4Q/Efw/4s8P8AhOey13wvNaz6bcHV7yUQPbaP/YsJKPKVbbYDyvmByfnOX+avfqKItxVo+S+S1S9E9V56iaTd35/jv9/Xv1Pzx/a4/wCCG1l8TvDmqeFPho3h3wd4V8ReFrjwvc3LavrttqVitxfz3s0kv2a7WLVYfMuZXjtbsKsblvnMcjRV9AeHv+CVvwj0Dxjpuu/ZPFU15pWoDXLa2PijUU06DVvsZs21OO0WcQR3RhZgHRBtZi6gP81fR1FKCUI8sf61b/Nt+uvRWc25y5pf1dJP70kvS/d3yvA3g+2+H3g7TNDs7jVbu10m2S1hm1PUZ9RvJVQABprmd3mmc45eR2ZjySTWrRRVNtu7JSSVkFFFFIZ558bP2r/h1+zldWVv428W6VoFzqCl7eCdmeaRAcF9iBmCZBG4gLkYzWB4x/4KA/BnwBrcen6v8QtBs7qaG3uYwWd0liuEDwyK6qVZGUg7gcAEZIryD9vP9k74rfHb4s3F74THhv8A4R2+8K3GiMyXMem6ks0m8kT3H2aSWW2OQBAkiKSfm2jJPi1v/wAEufih/wAK+8a2EljoJvda8A6R4dsib9Ti7tri1klBOPlXbC2D3wKmLfJzNa328vf/AMo6/wB4ppc6in0/WP5Xlp/dPqL9qX/gpV4A/Ztu4dKgvbHxR4oe8tIJdHtLwJLDFcfdlL7WTgFTtzuwwPTmum/a6/bE0n9kCHwbca3Y+fp/irW00ia7a6EEemIylmnfKncqgEkDHTrXyF4//wCCZvxbP/CQ6TpVp4V1LSfEmuaJ4ie/n1Joru0e0heOW2CGMhgDIxDbsbVHUsQv1B+3F+zj4g/aA8SfCOfRIbGe28H+MrXWtUFzMExbR/e2gg7j7Vo4pWSf20v+3fdvp2Xva9bXTtYhP4m19htetpW+b93T5PqdHqH7eXwf0v4aaf4vn8f6Cvh/VZ5LW0uBIzPPLHjzEWIL5mVypb5eAyk8MMuvP26/hFYX3h63k8eaJ5niuCO50nYzut8kkxgUqyqRnzAVIJBUqcgYNfInhb/gnj8bPB2n2MGnxeEobYeKNY1Gcw30VrqkcN3GiRPFem2meGPAPmJFtZhkc5BV/wCzX/wS38ceFvFHgn/hLbTQlsdD8Jaxo086XK3L2l3dXF28Msa45KrOp3cEH6VnFtx5redv+3W7et0kn57DmuV8qfW1/na/pbdfifRHh/8A4KXfD7xz+0jpfgLw3f2GuWN3p15fXuvRXojtdNNsrs6OGUZXamfMDbcHOTiu38Dftt/Cb4k6Nr2oaL4/8OXln4YjM2qSm58pbSMHHmHfjKZIAYZUkgA5NfFfhr/glv8AFrXY/Duha2nhTRNN8N+DdX8KRanYai8z3TzvcSRTOnlqwRmmCsoydoYnk7R1ep/8E8fip8Y/2etQ8HeIdJ+FnhW60nwzYaBpGoWCvcahqUlpOkoM9wEBjtpApzGAx37WI+XmnotP696WvySj7u+vkxrWWumv6R0+bcve2VvNH1TZ/tt/Ce/+GupeL4vHegt4f0eZLe9ufOIa2kf7iNGR5m5udo2/MASMgVw/wf8A+CmHw9+JPhnxjr2pX9j4a8PeGNfbQrS/ubzzRrRxuSSFAoc7l5CAM2M+lfPfjv8A4Jq/FT4p3/iTx7ew+E9I8VXur6HqFn4Zg1F5bGePToTE6yzeWAGk4ZcKQOQTzWXd/wDBMP4t6xrl74wmGi2OuwePbvxTHo+m+IJbJbiC5WMsILxI90MsbJhWKDgk8fdL669vu1hr56OWi2trcS2X9dJaeWqirve+h9g/FD9vL4W/DD4W2fiqbxjoV3Z61bzy6MkN1uOqyRKSY1Kg7fmwpLABGIDYPFdL+zF8c4v2lfgP4c8cwadJpMPiGBp1tJJhK0AWR0wWAAP3M9O9fFFl/wAEwvib8PYvD2taJa+Db/VZdP13TtT0mbVro29h/aMLxrMk8yu0rqGzJgIGbAAwS1fQ37FPw1+K37P3gb4eeA9X0fwz/wAIpo2gyrq19HeNJeRX5uJ2WONQQrR7GjJbHUt6U4pWd99Pl8V157LXZ3SWopPVW21+fw29N3p5N7Enxj/4KHaT8Dvi54+8Ka3oFzE3gzwt/wAJRa3X2tQusplE8lFK/I3mPtzk/dJxVH4Rf8FL/D3xeuvhyltpBso/G+kalreoT3OoIsPhy2smdHaZioDAsjDPygcGuZ/4KIfsCa/+1R8dPhx4h0GW1isLMjS/FHmziNpNOFzFMAowd5z5vH0rzr4a/wDBJXxJp198dbG/1Gy0/S/FNhc6R4MkWfzRaW81211iRQMou5IlYDJId6yg3yXlulL7435f/AuaO38r7suajeyfWP3Oyfzi1J/9vI+q9L/bm+Emt+Ctc8Q2fjvRLnSfDaxvqc0bOzWiO6ojmPbvKMzABwpU54NcT8bP+CpPwl+FXwtl8Qab4m0vxPey28lxp2l21w0UupLHN5T7WKEKAwfBYYbYdua+dfEv/BNP4sfEPwL4sutTtvC+k60/gbTPBGladY6m0sV/9kntne6lkaNQgZYSVXkjOD0ydL49f8E1/iJLca7H4I0rwhcad4w8Baf4Zuobi8+yHRri1eCR2iURlXEjQ9cj5nZj0G6ql7S5fl90rP52j6c1nqTT1a5vK/8A5Jf7ry9eXTc+2LH4z6Lb/BSw8da5eWnh3RbrS4NUuJr24VYrNJY1cKznAJy4UepxgZOK4+b9vb4PQeCoPET+PtDGjXF8NMFxuf8Ad3JUuI5F27oyVUkbwBgZzXE/tUfsc+If2gv2DdD+HVhqFnpniPRrTTHHnSsLW4mtY1V4mdAWCk5wwB5VTivA4P8Agml8SdV8Q6N4ju7HR7bUJvHGj63qFvceJLrVrsWdmsiu89zMAssh3/KqIMKACTWjSdWUVpHmSXo2lfz0b9LXehKbVJSers215pPT8vW9lqfVHh//AIKFfBvxlqVhp+i+P9B1PVNWikksbOORlkuGTd8h3KNjkqcK+CeMA5GcT9kz/gpF8Pf2pvD9pjU9O8OeJZ4bi6k0O7vQ00EMLsGcyFVQ/Ku8gcheSMDNfPGmf8E2fiPZ3miSfY9DU2PxUv8AxZMwvVyNPlWIR445b5G+XtWJ4H/4JT/FLxR4H8G+D/EcvhfwxpngzTddEer6ffvcXd/PqcRCxPH5a4WNjhjuIZc461lBtptrovv5eb8/dfp0uaTilon1evkm0vvsn5X6pH2r4V/bT+FHjfTNdvNL8e+HLu18NQNd6lILkKLeAHBlG7G+PPG9MqSQAckCvPPh9/wVG+G3jzxt47h/tKysvCPgi0s7tvEstyfs999oAwiRFA4ZXJXHJJHAr5q07/gkr8SfFPw01XSdXl0LTNR07wsvh/TLuTxHfal9vkjuUmUIrKkdraME/wBXsdgzEjAPEvir/gmp8YPiVqHjHXr7TfB2j6nenw/eafpkGptLaXT6dEInhkbyxt3KN2duN3GcfNWkFH2lpP3bfq9e+2vLv0fQnXkb6/8A7On4tX20ue2eD/8AgrP4f8ffEG00jR/Dj3unX/jBPCltqa6oixzq4DLdKjIGKkEHZ96vWfh/+3x8Hfil4iXSdD+IGgXmoPbS3nks7Q4ii3GRiZFUDaEZiCc7QWxjmvk/w1/wTx+LN38c7Lxbq2m+G7ETfE5fGN3Baap5yW9swBZVJRS7Akj7ozjOBmuUm/4Jz/FD4T/BrwXdtomkajd+CtP8VS6laWx+3SXRvbd0t0SJQDMWyAVHT36VlzOMXzLaKfz5E2u3xX2812tXLzN8r9P/AAKX6KP3n2Kf+Cj/AMIrrxZY6Tp/iZdVa7hmu5ru1gb7Fp9tEPnuZpn2oIc/LvUsCcjtXf8Awa/aK8EftB2V7P4M8R6fryac6pdLAWWS3LAlCyMAwDAEq2MNg4JxX5m/s4/D3U/iroVp4H/4VFf+IfEdx4Zl0CLXtc1XWDpvhqMKZCkkL28SwK75A8qRmVum4fKfsD/gnZ+yz8Rf2evEHim88W6lcxaJqlraW1ho91r7a7NbyQqVaQXBhi2RYwqRgNgcEnaCejkjdq/TR99X+iXzb6K5i5Oya3vqvu/W/wAl3Z6xrn7Z/wAK/DXxXTwPfeOdBt/FMk62v2FpjlJm+7EzgbEkPTYzBskccis+H9vn4Oyza5H/AMJ/ocT+Gy66kszPEbVlk8oqdyjLbztCrktzgHFfKvx3/YA+NXxO+KGuXWzwq+jT+NbbxDaf2fexafBLbI7nMtqlsrS3QVvmmlmY8kDdksItY/4JV+Ntb+EPiLyv7EsfFw+JU/i+y23zQnU7HH7uFrmNS8MgLMVOCFYk8ZyMINuEZS67/wDknTy5pXXXl0N5KKbS+X3Tf48sUn05tT7r+Fvxd8NfGzwomueFNasdd0t3aLz7V93lyL95HU4ZHGRlWAIyOOa6OvA/2CP2atc/Z68L+K7nxHFZ2mq+L9ZfVZLaHVbnVZYgVwDPczEebOed7IiqcA89vfK0kkrW7L77ar5PQyi29/P89AoooqSgooqDU7iW0064lgtzdzxxs8cCuEMzAEhAx4GTxk8c0N2VwWrsOvYYbmzljuEjkt5EKyrIAUZSMEMDxjHXNfNPxL/4JWfAfXfgx488N+E/AfgH4cah8RPDV/4YOvaJoNpDdWNtdxMsggwFCoc7zGhVWIyR3rofjR468T+Pf2IPjDc+KfA974EvIfCurxxWd1qdrftcJ9glPmBrd2UDJIwTnjpX5p/E22+Kfhn9mH9gDUvFPiL4d6z4ZjWCbRtP07wpcW13bkeDdQaMXUs97cQ3CiMbWUW8YcnOAPkPPWcVGoprRezT81Vc1bTso7db2djelzKUHB6+/JetNRa/F79LaXP0t8Df8E0vgL4V+DcPg6H4VfDy60WVrO7uduhWy/brq1j8uC6JVciRFLBGB+QOwUgE11nwq/Y9+EHwduLa58FfDbwB4bksDdpbyaRolta/Z/tXlLdBfLQbfN8iESY+/wCUm7O0V+RHg7/gqR8UPCv7LGlX0nxf0b4S6/4Y0v4baf4Q+H9l4d0K1tfGVhqtrpzX17FbyWpl2Az3aRrZGKG3FmVKE9PTP2D/AIw/FL9nLxT8O/8AhHvE+reLfDHxU+MPxP0efwFLZaZb2sZtLjV72F7a7MSXCztPbNuM1w0W2XARdu4+hWjL20+Z/wA133tyK1l3U0rdLcrSscNKUVQjKK0VtOy5Zyvr25Hrs73TfX9ENF/4J4fAHwb4a1/TbL4O/DGx0jxHbpb6tbL4dtFt7uCOTzkidSm3ykl/eKnCq/zABua7j4J/BTwP8AvB76B4A8N+H/C+jfaXupbTSLWO3jkuJMF5ZNg+eVuNztljxk1+W/8AwWB/bI8feLvA/wAffAnjnxbJ+zpoml/BqTWNG8OXT6NfS+P7u6nu7ae2a4kSYuipHBGsdm8UqveKzMflWuX+FP7Ufxan+N+q+AfBPjyH4Y6XNqvi69vrvw/4T0Q3uonSfCnhu6tPOe4tJRI/m3MgeaRXlePCbhtjMfHCsuWpJ6KMXL5RlJP7uW6Wr7pM6pU/3lOmt5yt83CMl96dr7ebW37QUV+MOv8A/BTz9o/4S/CbwtqV18Sv+Eif4ofCnwj46vdUv/D+l20Pw/fVNZhs725t/LgRDbQ284IN75wR0DuxTKV6H8Mv2zfjj8a/2gPAfwt0v48KfD+t+Pdd0OTxjoNjoeq6vcWlpoUeoC0lm/s7+y2uIrglTJawFfLcK6iRTjacXFTf8rkn/wBuzUJfc3vs1td6GKqRaT7pP5OHOvw6b38tT9W6ZBcR3Sbo3SRQSpKtkZHBFfixon/BX740R+Evhx4o8T/FyytbVPD2lXGq6T4f0vRv7V1G4bW7ywlubjTNQjhe/huUgjQLpF7FLBIXJhbKAp8OP+CjHjzRLP4paTc/Fnw58ANF8FaZ4z8VeGns/DGiWUHjjVYPE+q2/wBlkSe2KTCNLe381LYR3c0l55jSlmBMSajv/f8A/Ker+9Xt1e1jXke3W8F/4M2+7S/RXvex+1dMnuEtYWkldI41GWZjgL9TX5LeCf8AgoZ+0Jr19d/EXXPHF94fs9J+LfgbwRd/D7/hHdONhBaaxpulS38csrW/2/zo5r6QxkTqVKbWD5AX5m/bH/4KjeOP2gP2cPFvh1/ild+NvCnxQ+F/i3VrvS9TsvD1jLp7WksDWj29npxmu7FSplTyr+7nlcRsSqMpAc1KKfdJ/hTVXpfeLVvO/YVC1ScY9G0vvm4fhKLv8u5/QIrbhkcg8gjvS1hfDD/kmvh7/sGW3/opa3a0qw5KkodnYxoVPaU41O6T+8KKKKzNQooooAKKKKACiiigAooooAK8v+OP7HHwe/aY8TWV98Rfht4B8cavp9q1taT67ottf3FvbltzIhlVmCFjkgcZrZ+KHxC8WeD/ABHotr4e+H9/4vsdQfbfX0GrWdmmlDco3Mkzq8nBZv3YJ+UjqRXyB+0P8c/BP7P/APwXi8Daz488YeFvBOjz/A3VbSK+1/VoNNtpJm1yyZYlkmdVLlUchQckKxxwalcsqkIP7Tlr2cYSl+PLb56X2Kd405yXTl+fNOMfwvf5dD6f8c/sKfBb4leB/D/hnXfhR8PdR8PeE4nt9E02TQLYWukQuAJIreMIFijcABkQBWHDAitmb9lb4Zz6Tqtg3w+8GGx1waeNRtv7Gt/JvRp+z7CJE2YYW/lx+UCP3exduMCvzq+NX/BTnx7ffE74lat4U+Muk2Wp+G/iP4c8K+BvhpZWWlXkXjzQ9RWwP9pxyNFJeztMl3czRTW0ywItn8yON9eF2v8AwWV/awm+I1v4H3x/2pLdt8DTqX9hWuB8Qv7R/wCQqV8rb5A0/wDe+Xjysj7nerpc1SzW8mvvajJX87zhHXac4p21tMlGN77R0v2V3f5WjKV1o4xbV9E/1s0v9gn4I6L4m17WLb4SfDqLU/FEF1batOPD9qWv4ro5ukfKYKznmUYxIQC+410XwO/Zl+Hn7M2l39n8PvBXhnwbbarMtxerpGnx2pvJFXYjSFQC+1cKu4naoCjAAFflP4d/4KofHiSy1XWZ/ifoQ8TO/jiy1DwFeQ6YZ/DqaTa3zWs8FrBaG+jnia2t5JXv5TbyrOdqpujU/TUf7Uv7QfwJ/wCCZ9145TRfEfxm8VXPw5PjaPxVfDR7S1tbyXTBdC1TTrNIJngik4A2PIwbmRu2aqRVCVZfDGMX20fM0u32W/5fO5fJKVaNF/FKUvvTUW/xS0u/Kx94y6jBBN5bzwpJx8rOAeTgce56VNX4Z6t8e5pf27H1zTfjBofx+n1SP4RWlzreo6Lod6unm78Q3DSQIltbLbxyp5plikCC4h85P3m5Q9dh8Kv2zf2nPiJ8H/DGvXH7Q/iC2vfE3wK8S/FiYQ+EfD3l2l/pd4kFvaxBrEn7NIsymbeXkYxny3hDYFSajSdSfRzT/wC3FKT/APJYN7eSuRT/AHlTkh2ptf8AcRpL7pNL8e9v2cor8fvhH/wVc+N/xS+KWmajqHxA8NeGNS/4TDwzoNt8PrtLCKDX9Nv7CynmuFg+zS6tJPP9pmkhmhlW2j8kK6lVkeqGi/8ABRH9orwj+z98OvGt/wDG/T7i7+LvgXx1qzt4j8MWMejeEbnR7qCO0uYvsFn9pKiORvMM32hM/P5YUFaqpFwTcul/whz29XHbz0dhUWqslGO7t+Lcb/JrX8Ln7GyXEcUiIzorSEhFJwXI5OPWn1+J3ij/AIKh/E+G40G5069/4Wt438Aa/wCIZLP+2NC8O64yTL4Kvr+CCx1XSFWG4VpI1Zmt4bO58uXyZUIbnpvH3/BSv4v6Iuk+Hvh7+0Pb/FC18WaJ4J1HUPF8GhaJcP4M1HVvEFrp89ki21sttsmt5pnjhuUeePyCTI2c04wcqqoR+JuCXrNNr8nd7dr3RMpqNP2r25ZS+UZcr/G1v+Az9i6YtxG8zRh0MiAFkDcqD0yK/JPx7/wUj+JHw28d+I/hZ4g+OPiLT9Q8MfELxJotp4l/sjwtpl7qmmWGk6ZeR/bL6/RNOtzE+oHctvp8s06qNqxbXY9h/wAEDf2i/Ef7Wvx9+JPxF8XXdrqHibxT8LPAFzqd1bQrDHcTrJr8bPsUBVJ2ZIUAAk4AHAzjLmjGcdnyv5TjKS/9JafZm3LZyjLdc3/ks1B/ndeXa5+n9FFFUQcP8dv2iPCf7OXgy81nxPqtpai3tZruGx+0xJe6iIl3OlvG7r5j47A96j8G/tLeDfF3gbwlrsmt6focXjezivNJtNWvIbW7uVkUMqrGXO5+RkKW5r5U/wCCjn7J/jr4r/tAXfiHRfBC+PdI1PwDdeHbNDdWsR0O/MjyLcbZmXseGXLZOOOK8Z8Sf8E7vi3K9oL7wprGv2Wt+BNI0JLaw1DSYn0ie3ihWW3mlu0kaFN8byeZbAkk4ycnE022ryXX/wCT/O0XfZX771NJSsu3/wAh+XNLTd8r6H6RTfHfwPb+Kf7Dk8ZeFE1r7ULH+z21a3F19oOMQ+Vv3eZyPlxnnpV3S/in4Y1vxfdeH7LxHoN3r1iCbnTYNQikvLcDqXiDb1/EV+bfjj/gnN8RriTx7dp4LfU9cfxfoFzo+py3ltNdTWcEMq3UiTko2C3llvlQuQp2cYGt4L/YR+MOh/HHU18N+Hbjwppl9NrYkvdaudM1SCwF3bypHPaXsYW9DszLx5SlQBuLfNmnou7s/wAotfi2vl6k9/l+cr/kvv8ARH6I+GPiX4c8bapf2OjeINE1e90t/LvbeyvoriWzbptkVGJQ+zYrQ1zXbLwxo9zqOpXlrp+n2UZmuLq5lWKGBAMlndiAqgdSTivzl/ZZ/Zu+Iv7JV7a+O9d8Ca1FN8MfCmox3Il17R7Ky1Q5d/Jj+zwNLMhz5nm3MgKEYy2AD9efGhNf/aO/YL1/7J4duNP8SeMfCUjw6LJOplgnngyIC7BBuBbGSF/ClW92m5Q1aX/yVvv5b/0m6opSqxhPRN2/9Jv93N8/vt3L/tCeAY9Ci1RvHHhAaZPcfZI7s6zb+RJNtVvLD79pfa6naDnDA9xWHpv7V3hW/wDj34n+HrteWep+EtJh1q/v7oRR6cLeTYVIl35yA4zuUDrzXwd4m/4Js+PPB+l/De80fwZeX1iPAz6Trek6XNpQu7PVZkbz5H+2rJCRLuVGmjBkQJwQFUF2o/sH/F3wnD42tbDwBF4mi1DwRo+jW0Gra5FcpJJDJA8kYkRrfzXhCfKGCIfLH38ANUlZ6d2vwlr6XSa8nZ23cw961+qT/GOnrZteq0v0+1/jR+3r8MPgr8Kv+Ewm8S6Z4i0s3sdhGuhaha3ks0rEAhR5qqdoIZvmyq807wx+3b8N/FHxS1/wuuuWtkdBjsn/ALWu7u2i0vUjdxiSFLafzSJWIPTA5BxnFfBHiP8A4J0fFPxR4T+Kkn/Cvt13qV/oWo6Nb3CaVZyTpCJBcrHHbv5ELbWXKhuRwWds59H8VfsJ+LvjJqvx4u0+H58FWviXw9otx4TsRNaqttf2UMbfZ0EL7UIZGjzgL855xRGyleW1m/8AyWL+bvzW+5i1k+VaaL73Jfgk9fmz7t1f4z+D/D1rqU9/4r8NWMOjXC2moSXGpwRrYzMNyxSlmAjcjkK2CR2rjPF/7aPgXwL8aNK8F6pqH2SXWNDbxDFrEs0CaRHah2TL3DSDBJHHykHI55r4z8WfsHfFufwZ8L/F15pd7rHiIazqviDxlpNk+myXcd3ekBJY0vA9rKUjRFKsG2/w+oi+Hf7DOtfC3xRoV38QdIu9A+HNl8P9Z03W7vUNYtdQXQ/tM1wwjLxJGMqJA4EaMqlvvHHEu6Tb6c1+11Bvfs5JWfbTdlL3mkutvWzklt3S6fpv+iWj+P8AQvEOqixsNa0m+vms49QFvb3kcsptpP8AVz7VJPlt/C+Np7GuC+HH7Yvgr4ijxzIbx/D9p8Pdcl0DVbzWpIbO2NxGSCUcyEFCRwW2k+leD/8ABGH4W6no3wS1fxjrt2dSvdeuItI0y7IOH0vT0+z25TIB2E+YR6gKa8o1v9h/4oQfFLWvFb+DZdd0bT/i9d+KH8Pm7tQ2vadIRsnRZJBGSoBwkhU/vDx1qprlq8vS179ryh+UZO67p9ETFtwv1vb7oyv98kkn2a7n343xs8GpPDEfFvhgS3Ng+qQodUg3S2ibi9wo3cxLtbLj5RtPPFWvDvxQ8NeL762ttJ8RaFqlzeWQ1K3itL+Kd57UtsE6KrEtFuBXePlzxnNfmrq3/BMb4sW/we8HXFjpf2HxBfeIdWs9S0yC9hY6LoepKqNGXD7CEAlYrGTgz8c5r0r9iX9gb4yfDbS/E8reMpPhvrVjdR6Do99JpVrrRu9HgeeQAI77UV5JlbPDfu8YGaIq+/8AX+duaPradlpYb8v61/Xlk/nC+59R/GX9tXwH8E/HHh7w3f6j/aeu+IdZg0RbHTJoJ7jT5puY2uYzIrxxn1wT6A1V+D/7efwz+Lvw6tvEreIbDwpbXbTeXa+I7220+7KRSeU0uwyn93v4DZx+NfFyfsJfFPSPjnpr3fgP+2biy+LA8U3XjNbqzVr7TZGQ42b/ADQFKs7JjAJAANYHwm/4J0fEuGx8NweIvh+J4dO8IeJbN47qW0nWG9uPtRtABvI3lnjKsPukg5GKzpyfJOUukbr15U7d97r1HJe/yra6X3ykr9tlF6bI/TLw98XvCfi7VLux0rxP4d1O9sIVubq3tNShmltomAKyOqsSqkEEEjByKqw/HrwNceGJ9bj8Z+E30a1nFrPfrq9ubWGYnAjaTftDkkAKTmvzd/4dn/FKbwzpGn6B4Wg8MX158LG0rVblLi3gW51P7e8zwTMjZZ5IVRN5yuCoLADg8Sf8E+Pij4j+CfjD7N4G8RWeqalY6JposbzVNHjF81tNGZGS3s4o4wsSowEskvmOG5UkkjWy5+Xzt/5NKN/SyT9Hfa11DWN32v8A+Sp2+9tfK3p+kafHfwPJ4WutcXxl4UbRLGc2tzqA1a3+y28w6xvLv2q/I+UnNc9cftYeFF+OfhjwFbNd6lfeLtJm1nT9RsvKm01rePfnMwfOTsONqkdOa+FPHv8AwTy8b23jD4hvpHw91aDRP+EwsdV8P/2HqWn232WGOCZGmjs5v3M6gsAY2aJskEEjdUOi/sDfHLxPc+HbbUNGstCli8G6/pCXtotlZpatcvcNBHOls2wSS+Zh2jDACTkkgmovo5dlf19zmt682nytqwWtr9Wl6e/a/wD4Dr8+2/6JW/x28EXnh7UdWi8ZeFZdK0dxHf3qatbtb2LE4Alk37UJJAAYjmpNM+Nfg3WtF1TUrPxb4Zu9O0Rtmo3UOqQSQ2DekzhtsZ/3iK/NrWv2A/i34o+HPiV7H4fSeEng+Hum+FG0pL+yZ/EeoQ3cDyXA8qQrtCxs2+QgnAAzzWp44/YP+JXhjVfiDB4W+FekX2iazaeGVgtJJreODfb2gF3PFAJ40kmSbdxKdrl2b5qpL33G+nf5tfilfyuriTvG/wDW0dPVOTXnyux9v+N/21fAfg/UfA0NtqQ8SwfEDWhoOm3mhTQXtrHc/LnzZBIAFG4Z27j7V6zX5ffCn9hb4t6F8e9C1e68GX1rpVt8S7PxC5eTT4PJtAnzymK2fyUI4BSIEAjA3AZP3l+zb8dNY/aH+EV/r934UPhi8jv7zT7ayl1EXSXXkMY/NEyIo2M4YZUH7pIJpSsoNrVrX5csX/6U2u/kNXckntt8+aX6JP8A4c6WP46eCZtd1HS18Y+FW1PR4pJ7+0GrQGexjjGZHlTfuRVHJLAAd6Xw18cvBXjM3n9j+MPC2q/2dbC9u/seqwT/AGWAjcJX2udqEc7jgY71+ZSfsJfGjUNc0K6Pwxj8PSWKeIIrqDTX0+O2jN1ayJEqTCd7m4ViwBadyF3YUAbidAf8E1fifH4U02w0PwfHoN5efCmTSdVljuLaEXWqfb3maCVlf53kiVF3nK4KgsAOJ15OZ72/HllL7vdS9X10KsuflW1/w5lH8m36elz77+FH7Yfgv42/GnxF4I8M3UmrXfhuxgv5tStnhn025jlxt8maORt5BOD8oAIIzxXqdfFn/BOr9nLxf8Lv2l/F/iXWPh3/AMK/0LVfC2ladbW4uLWRXuII4o5jiBiAWZGfJHO7JO4kV9p1tOKjGNnd2d//AAJr8rGUZNt38vyT/O4UUUVmWFFFQambkadcfYxC135beQJiRGXwdu4gEhc4zgZxQ3ZXBauxF4g1mx8OaDe6hqlzbWem2FvJcXdxcuEhghRSzu7HgKFBJJ4ABrmZvjV4COm+DLiTxN4Y+yeO5I4fCsjXsOzXXeBpkWzOcTFoFZwI85RSegrz/wCKVh8SfE/7GvxXsvGel+Fz4hvPDWq2+nWnhaa6vVuQ1jIqLiWJHMrOSAqqc8dzivz38Zfsw/GL4U/AT9hrxC2t/Gn4gS+DzBe3fhOLw5pkUvhR4/Cl8FVDDp6TRyLKUt0+2NIu91Dh2NRzqLlz7RdL7puSl80oq3rqVytxjy7tVPvgotfJt/hofoH8Yf8Agnl4H+Onj2fV/EOpeM7nSb6/07U7/wAMrrcg0K+ubB0e1c25yY1V4onaOF445WiUyI5zn1fx3448M/CfQI9W8Sapo3h7TEuobVLu/mjtoVuLiVYYow7EDfJLIqKOrM4AyTX4kx+Nv2jfFfh/xDYeD7v9pnT7HxJ8PtAmu5TqfizVL2x1lvE+mR3Yjvr+1t1ivlsZroTx2EENuqK+E2odvoX7bPwa8Zx+NfGvgjVT+0h4h1XQPid4KHw1sIr/AMR67oV/4biudPnurm7kzLbXLpP9seae8ZpomjgUOqbAdYxceSD2crW9JRhK3d2ace8It6ctjOU01Oovsr/2znXpu4vtKy63P2K1LQ7LWHha7s7W6a3YvEZolcxMQVJXI4OCRx2Jp66ZbK+4W8AJzkiMZOQAfzAH5Cvxu1Txx8aof2a/O0/UP2xp/wBoq5S8X4l21jb6jLoGk2X9t263T2cNyht4rhbHzfsH9jjzJEDuwcgNXRXniX4jr+0YbKz1f9ptv2NG8YWwm1hp/FH/AAkiE6HO7pHdt/xOzpn9oCANJu2CUlN/l7lqV8PN/W6V/wDC7+7L7TTWljSStJrtf8E3b1026Jp9T9VfAHjXwv8AFbwwNX8M6loviDSHeewF3p80dxAzQSvDNDvQkZSVHRl/hZGBGQaj8TeKfCXwa0XSjq17oPhnT576DSdO+0yRWkL3VzIIobeLOB5krsFVF5YkAAk1+HfwQ0L44fDT4AaHomozftE+B/BsHhPxLfeDjYaV4lh1e48QT+J9UkSS8t7BUknvmtmsnit9QxbSiWQlTlmX034peHviz8UviX4Xg+KkXx61f4raX8dvBd5pun2OnawvhL/hGYP7PlkvXtrffpaBJftDyyszTRTDYZNgAN0Y+0qQhteSTv2dRU3/ANvK7lZ7Q9671RFZ+zVR78vPa392Lkvk9I3X2/d7M/Y+Xw5p89zazPYWbzWJLWztApa3J4JQ4yue+KW68P2F6YPOsrSX7LL58G+FW8mTn51yOG5PI55q5RUDIWsIGJzDEcuJDlByw6N9eBz7VVt/Cul2hm8rTbCP7RI0suy3Qea7feZuOScDJPXFaFFAAOKKKKACiiigAooooAKKKKACiiigAooooAK5nU/G/hG6+Jlv4QvNT0CTxdcaa+rQaRNNE1/JZJIsT3CxH5zEsjqpcDaGYDOTWd8ULn4jw+I9FHguz8E3Oks//E3fW7y6huY13L/qFiidWO3f98ryF7E4+UP2jvGX/CiP+C0vgv4ga9oHjq48G/8ACnNS0FtV0PwhquvQxX0ms2kyQP8AYbeZkYxxSN8wAwvXkZUWnUhF6KXN+EJSX3tJfMck1CcluuX8ZRT+5Nv5H22NBsRqEN39itPtVtEYYZvJXzIozjKK2MhTgcDjipRp9uH3eRDu3+bnYM7+m76+9flJ8R/iT8bPG/7aHibWLXWvjbot7eeOvD7/AA6sbLQfEX9k3fhmSC0ebzrD9zpuxi94LqTUMXNsQuAhWNG8G+JHxj/aS162v9U8DW/7T+k6zr/g7x1/bVjeX3ifVbzT9QWB5NLSQPZ22m2V2JFxBHpcK4XaDJJuVnSk3DntrZu3XSKnb115Wt4zVvMpRTqcifWKv01bjf00un1i0/I/c5fD9guoz3gsrQXd1GIZpxCvmTIOis2Mkcng8c1bRBGgVQFVRgADgCvyU/ac/Z6+Mnw8Hxqk8GeK/wBp+6l+Hfwk8O6z4HKeMPEF8NQ8RyXl9Je7lMrLfzbUiV7aQSIsbqPKUCPbRHin9pu5/aP1+e31n4tWnxZ/4WJrEenaO2m63c+GLjw4ttMLFWiZk0SGzZPs7faxuuhcEqd5LIKqPkbjvZSenlLl073d5W/lTe+hnD3oqW17b+cFPXto1H/FptqfqlD4i8HaF4/t/CMd14es/E19Zy6zBpCtFHeT20cqrLdLEPmZFllQM+MBpBk5aqnhf4zeAPGXxK1zwVo3iXwrqnizwtCo1jRbO9hmvdJjcKyieFSWiDB0IDAZDKe9fm9/wS68K65rX/BQ/wCGnivUYvj7qWsx/Am6svHeoePrbW2tNO8SS6jpslxawSX6CGF2KSsYLUiEJGhRAMk4nhT9kTx2f279f0yzk+PfhfwX48/aC1y58Q3WjeJtf02DUNJi8Kwz20rXEcw2QPfjyhMjLv2JCHxGiLfJyyUZa+7Nu23u1VTVu6afOu623FKfuymr6OO+9nSdR3XdNctuktGz9YzoFi2pw3psrQ3lvGYop/JXzYkPVVbGQDgcD0qZdOt0VQIIQIwVUBB8oPUD696/GPVPFP7RUHwat5vD1/8AtKS/HqTS/HsfxMsLsa5Jounwpp+onS205ZB9iilFymnCzfTsSzBnJMnzEbXjT4GfHbwD4Z+JmqaF4r/ajvNW8HeEvh7r3heOfxVr98t7rV5c41wNC8jJdDy4l820ZWhgEjlYo9wImmud2e3uq/8Ai5tV3S5Xd93buyq16fm9fwcY2fZ3knb+XXyP13h0vSvCek5jtrDTrGwV5vkiSKK3HJduAAvck/Ws/wCHGt+FvHvgfT9e8J3Gh6r4d19F1Syv9MaOW01BZMOtwjp8rhuCHBOeDmvjH/gqvc+J7j49eFtP1y6+Mun/AAaufAviAmX4cf2uLubxPm3WxiujpamcoYWuPKilzBLLw6tgCvgKz8UfHf4afsreB9A8PeH/ANo7wt418AeCvAltotnbTeJEtpm2W73/ANn0qws0snjRWljuX1SedgybUjj+UMsP+9fa7il85Ti7+nIpW/kkpdbBWXItNbJv7lC1v/A2r/zJrpc/c3SNR8L/ABHS6awn0DXl0jUnt7k27w3Qsr6EgOj7c7J0JAIOGXvitm10+Cxz5MEUO4AHYgXIGcdPqfzr8XNH0fxR8C9Z8UeC7fQP2ktJsdX+LvjvVb6+ttY8Y2GmSwGYNpjGXT7aa+vJZ0kd4Rb3MEczoxneZgij9If+CTPiLx54r/4JxfCK9+J//CS/8J8+hJHrR8Q20tvqhnSR0/0hJQJPM2quS43N94k5ySj79BVfKD+co3a+TVvSwVPdqun5zXyjLlT+e/3n0RRRRQBxt38evDNj8dLX4cSXso8W3ulHWorT7PIUa1DtGX8zGwHcpG3Ofauujuo5kZlkRlQlWIYEKR1B+lfC37ffwM+JXxH/AGvtYvvAelaystz8K5tMttUhjeK3Nwb2RnthccIsrxFgAWB+cdOteC+E/wBnH4k+Hfg/44m074Z+MdR0u/0HTdKvvD8uiyeH4ru7jkQtOIILkz3hjCEM67fO35PfMwk5QTe+v/pc0vTSK+b6KxUlaWm2n/pMX+cn93U/V4a1ZtbLMLu2MLtsWTzV2s3oDnGabBr1jdSBY7y0kZjtAWZSSeuOtfknqn7Knj/X/CXjjRo/h54rsND1jx14dvrWysfDdzplutt5F0s80NvvmMSqGTcfMYqSu4qflG18Tv2Gtb8L+MPiNeeGPh34pjk0Dx9oUfhSe3sruVotNjSZZXgbkui+XBuk5+6uWrSEeaST0TaX38l/u5/wZk5NJtdL/wDtzX/pP4o/Va6W3vt1rMIZt67mhfDblz1KntmuR8SftCeFPCnxf8O+BbzUSviXxUlxJp1tHC8iyCBd8u5wCqEDsxBNfnX4H/ZR+MSftULNe6b4mtPGEXjibV5PE0ehI1nNp5XGW1RrgEwso2/ZRGSAemflqx+w7+zb4t8C/tVfBy71P4ZeMND1Pw+2vQ+LNfu9Pf7LqNzIJzDL52SGXY6qHOAxOFLYqaH7xxvpdX/8lv8Ag9H56W7XiP3cZ8urW33/AOSuvz2v+gHxG/ai8FfCv4peHvBesapLH4m8UYawsobOacmMvs82RkUrHGGyCzkDg9ga7mLVrWaAyJc27RqQC6yAqM4xz75H518I/tzfsl3PxH/blu/EVp4M1zVrOb4cajK19bQXLwNqscM8dsm5Tt8zaIwI/wCLjKkmvI9M/Ys1v4YfC74N6pqHwz8WeKPDU2i6pJ4n8OQ2ct3dw63PBJDBcS2rnK4UQKCFAj8nPBOTEZP2fM1r/wAGaS/8k++Ue+luPv8ALF6f8CLf/pX3J7ta/qTc6vaWUpSa5t4nChiryBSATgHBPTPFSxXcU8siJJG7xHDqrAlD7jtX5JTfsRfErVNEmtvG3gvxD4g1PSvhDNFp8n2OW6FvqAvma1tlkXKtcRwuAEBJAyMcV9C/8E5/2eNe+C/7WdzeSeFdf0HRtV+Gulvf3F1azpBc6sfs7Th3fgz7jIWXOR83A5rZRXM4t9/w57f+kf8Aky3MnL3b27f+2f8Ayen+F7H1z8c/j/4V/Zx8Dy+IfFuonT9MhliiZkheeTdI4RPkQFsbiBnGBXUS31ndgQSyW7efHv8AJkIy6eu09q/MT/goL+zR408XfHf4t3b/AA38W+L9R1250S58K63p1g93bafZw4W5h3KflYkgFApJwWwB81Zvxd+BXj/xn+0JNqkfwg1/SLyP4i2l1NqMGk3l/c3Vhko0x1AzlBAR/wAsIYQiA8su0AxQvUUVs5P7vhWvo5O/o9LK5de1ObS2S+96vT5LT1XXQ/SH4PftBeEvjbba83hrUBcweGNSm0e8LxGERzQhd+0MASg3AbsYODiuztrqO9gWWGRJY2+66MGVvoRX5UWX7OGufCz4V/ErwtD8BPE+r6xN4588Xg0+5OnS6KJcwhBbyo96iMu7yVOPnUk8YHtX/BPbUvF/7L3/AAmenar8MvHzad4r8dRwaZBYeHns7TTbeZObswNI6wWw+Xdtd9mACTinStUin5J+rfKrffL8GruzYqnuSa7N/cubXz0XzunZbH3jRRRSAKKKKACiiigAooooAKKKKACo7W1jsrdIYY0iijUKiIoVUA6AAcAVJRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAH/2Q==)"
      ],
      "metadata": {
        "id": "tVUt72wZswQg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Besides being <font color='blue'>dense</font> representations, word embeddings are also <font color='blue'>structured</font> representations, and their structure is learned from data. Similar words get embedded in close locations, and further, specific <font color='blue'>directions</font> in the embedding space are meaningful. To make this clearer, let's look at a concrete example.\n",
        "\n",
        "In the figure below, four words are embedded on a 2D plane: <font color='blue'>cat, dog, wolf</font>, and <font color='blue'>tiger</font>.\n",
        "With the vector representations we chose here, some semantic relationships between these words can be encoded as geometric transformations. For instance, the same vector allows us to go from <font color='blue'>cat</font> to <font color='blue'>tiger</font> and from <font color='blue'>dog</font> to <font color='blue'>wolf</font>: this vector could be interpreted as the “from pet to wild animal” vector. Similarly, another vector lets us go from <font color='blue'>dog</font> to <font color='blue'>cat</font> and from <font color='blue'>wolf</font> to <font color='blue'>tiger</font>, which could be interpreted as a “from canine to feline” vector."
      ],
      "metadata": {
        "id": "vbUCExaAszfD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![cat_dog_wolf_tiger.jpg](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0KCgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCADHAW8DASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD9/KKKKACiioYNRt7q6nhinhkmtSFmjVwWhJAYBh1GQQRnsaAJqKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAMzxp4hk8JeD9V1WHTr/AFibTbSW5jsLFA9zesiFhFECQC7kbRkgZIyQOa/Df/gkJ8Uv219B/wCC2HxS8WfFL4UeLk8KfFS5trPxpp63EEkHg8MhOkTqvmElIIVMJ29YpZGYFlWv3br5/wD2d/8Ak+39ov8A7lr/ANN8lAH0BRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRXhnxN/wCChHgH4Y/tkfD/AOBc39s6l468fvcCNbGyL2ejrFZz3am7nJCI0sdvJsjUtIcbioX5qN2ord/8OG0XJ7JXfoep/FLS/FGs+BL628Ga1ofh7xLJ5f2PUNY0eTV7K3xIpfzLaO5tnk3Rh1GJ02sysdwUq3xf8CPh1+0ZL+2f8eY7X4ufB6DUov8AhHvt9xL8LNQlhus2LmPy4hrymLauQ2Xk3HkbelfeFfP/AOzv/wAn2/tF/wDctf8ApvkoA+gKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK8J/ac/Zp1/4x/tSfs8eNNKn0uLSfhT4h1XVdYS5ldJ5ornR7uyjEChCGYSzoSGKgKGOSeD7tRR5g9YuPdNfJqz/MK+f/ANnf/k+39ov/ALlr/wBN8lfQFfP/AOzv/wAn2/tF/wDctf8ApvkoA+gKKKKAPzW/4KNftc/Fv4GftZeOvhB4W8ZXOna58ddG8LQfCuZoYn/4R6+k1N7DV5Igy/P5duY7ohs4wcV5j+zb/wAFjfib4y8aeC5Xhl8W6xouh6B8LNZ8PS3semWF78QdQ1G7guZ7y4WGV4Et7fS5pSEjc4uVAQllI/TP4lfsofD34wfG/wAAfEfxJ4attU8a/C5rx/DGpvPMjaWbuIRXGEVxHJuQADzFbaRldp5rib//AIJjfA2/8LeNdIHgWG1tviD4sHjrWZrPU722vH11WR11CC4jmWa1mVkBU27xhSWwBvbKoXgo8+r5rvzjeyj5fu5T1351B/ZuqrWk5cuiaSXk7avzvKENP5XUXXX85/8AgoV/wWG+NuieOvEWieFIbP4eeM/hp4X8eWXiTSrbV11LSpr6xsNKvLPUra5ksw8uy3vjKkUkEeZMxvgfvK/SjwrqXjb4c/sM2WqXGt+Hk8baZoC6pdah4q1WS90wyAefKLi8SK2ZYim5POEK+UCHMUgQo/JX3/BH/wDZ51fRprPUPAl1qjXlprFne3l74k1W4v8AUk1dIY9Ra5unuTNPLMkEK+ZI7OixqEZAMV75r/w20DxZ4JHhvV9G03V9A2RRnT7+3W6t5FiZWjDJIGDbWRCN2eVB61dO0aKg9ZWjd+ac27eT5l91tkiJ3lU5lteX3Pk5fmuV/ffqzw79gL9tbVP257TxH4mGgx+DfDmmi0srbQdUz/wkUNy0InkuLpA2yK2lSWI2/BM0Y87cFkVR9GVmweDdItvFtxr8elabHrt3aR2E+pLbILue3jd3jhaXG9o1eSRlUnAMjEDLGtKpGFFFee/tWfETX/hF+z14p8V+GpfCMWq+GrM6oP8AhJ7trPS5YYWWSeOW4U/uN8KyKsxDLG7KzI6qVIB6FRXzh/wTq/bg1H9vzwv4m8aJoUXhDwxZ3kWlaf4e1MFfFGnXEab7h9TiDlLbzPMiMMIBYxBZi5E6onj/AO01afE/Sv8AgqN8I/h1o/x++Kmg+EPino/ibW7+xs7Pw+/9mvYi0aCG1ebTJHWL/SHB80yMQF+YYORa1I0+9/wTl+SY9ouXa34tL82j7vor8x/hn/wW88SRfs7+KNV8L/DvUPiTN8KPCmoeM/Gd34o8Z22m30NmmqajawQ25tdLEVzMUsJn2mK3VI1jBklkJJ9H8Ff8Fsn+IPxeXTtJ+Fl5f+DLbxda+CNR1O11aefV7S+lijMlyunpZGNrCGaVI3na6SQAM4hKjJmM4yV4+vyu43+9WFO8E3PS1/wSk/nZp27H3jRXgH/BN/8Aa/8AFX7df7N2l/E7XfAOl+ANG8Tr9o0G1g8SnWbq6tg7oZLgfZYEgYsmVRWlyrAkqcrXv9aTg4S5ZbgFFFFSAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFfP/7O/wDyfb+0X/3LX/pvkr6Ar5//AGd/+T7f2i/+5a/9N8lAH0BRRRQAUUUUAFFFFABRRRQAVj+Ovh9oXxP8PHSfEej6brulm4guzaX9us8LSwTJPC5RgQSksaOpxwyA9q2KKAMXS/hxoGh+OdX8T2ei6XaeItft7a01PU4bZEutQit/M8hJZANziPzpdoYnbvbHWsLxV+zv4R8afHTwl8SNS0x5/GHgeyvtO0a9F1Ki2kF6IhcqYwwjfcIY+XUldvGMmu3oo636/wCat+Ta9A6W/rufnj+1x/wQ2svid4c1Twp8NG8O+DvCviLwtceF7m5bV9dttSsVuL+e9mkl+zXaxarD5lzK8drdhVjct85jkaKvoPw9/wAEsfhB4e8a2OvR6b4g8+21S18QT6eniK/i0bUNXt4I4YtSm09JhbPcBY0JPl4ZgGYFgGH0XRShFQjyx/rVy/Nt+uvRWKn7x3n/AFdJP70kvS/d35D4B/Ajwv8AsyfB7QfAXgvT20nwt4ZtvsmnWjXEtwYI9xbBkkZnbljyzE8119FFU227vcAooopAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXz/wDs7/8AJ9v7Rf8A3LX/AKb5K+gK+f8A9nf/AJPt/aL/AO5a/wDTfJQB9AUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFfP8A+zv/AMn2/tF/9y1/6b5K+gK+f/2d/wDk+39ov/uWv/TfJQB9AUUUUAeFft/ftgar+xp8MvCeraH4T0/xlrXjPxnpHguwsb/W30e1jn1GYwxzS3CW1yyojYLBYWOCcdMHi/Dv/BTa1+GfxM8beC/j7oOifCHxD4N0S08TJdWXiF9e0bW9Nubg2kclpcG1tp3nF0BCbc2yuXkj2b94rrf+Chn7Jevfte/DPwZpvhrXNI0DWfBXjnRvGttNqdpJc2tw+nTmYQusbo2HOASG4Ga8I+Of/BJTx1+1D8S9S+KXjX4j+FV+KemtoH/CHppXh64i0HRItJ1P+0xDcRSXTTXX2mfHmOHj2BE2qSvM0b3/AHm3O7+UOSFmkt2p8zae6uv5R1bW9zflVvOXPK6fZcltVs7Pue7T/wDBT/4GQeDLfWz443Lc6tNoS6Ymjag+uJfQxedPbyaYIDfRvFFiVw8A2RkO2FINdnpX7YHw2139mpfjDY+KrO9+GbWrX/8Ab9vDNJbLbLKYnnIVCwiRlYvIV2oqs7EIpYfB/wAZv2MPiB+zH+1rp37SU17f638TvFOsao2oJ4Y8BX/iTw3olpNo9nYxW0trazjUXkP2GJhdpGUZ9ySJCrq4+g/+Ccn7OvxL+Fn/AASQ8BfD6/u4PA3xIGjTJcXF5p6Xh0l7m7lnYtbiTZ5qxSnCF2RJMBg4Uq1ws4OT3XLfybTuvO1t9N+q95zK6kl0d/wdk/LT176fCvovwj8c/CXj/wAe6r4Z0PXLPV9Y0Oztb+/is900VrFchmg3TKDFudF3hN2/Yyvt2upPWV8//sJ/sE6b/wAE+dK1nwp4J1qWX4a33k3lhot7aq95pV/t2XUi3akGSCbbG4hdCYX8wI4iMcMX0BSGFFFFABRRWbH4y0ibxE+jpqumtq0a72shcobhVxnJjzuxgg9O9AGlRSE4FCsHUEHIPII70ALRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXz/8As7/8n2/tF/8Actf+m+SvoCvn/wDZ3/5Pt/aL/wC5a/8ATfJQB9AUUUUAFFFFABRRRQAUUUUAFfD37en7aHxG+Bnx31fwv4C1W01TQrvRrW68WavLpD3S/BaB5Vi/tabylxdRyxNJILZyHiNu07N9m8zy/uGmG3jJc7E/eDD/AC/fHTn1oAyLprq9+Hkh0q/XUL2XTibO+JQrcyGL5JcqNmGOG+UbeeBivyj/AGGvFv7L3hf9kz4IaL458NQeIf2orDxhbDWtF0koPiPD4sF45vL67IlhvBbq3mSzSyv5LW2Ad6lUP66ogjQKoAAGAAOAKYLaMXBl8tPNK7S+0biPTPXFKF4z5l/d9VyttWfS99fNRaty6k/ep8nr6e8rarrbp5OS+1p+EPxh/wCCmXxD/aM8I+IvC9h8adV1vQ/iT8OviLLqmjTR+H4tT017CwuZrSH+zra1lm0wlYnh2XN7NPLH5jbY5FDp+wP/AAT61mx8QfsOfCa703xOnjKyk8K6eI9YWa2mF0RAisN1siRHYwMfyqMbMHLAk+vRQJAWKIqb23NtGNx9T70+qp2hBx7qOv8AhdR/+32/7d89Cr781LonN2/xcmny5Px8tSiiikAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXz/wDs7/8AJ9v7Rf8A3LX/AKb5K+gK8U+B3gfV9B/bG+Oms3mnXdtpWvjQDp13JGRFe+TZSJLsbodjYB9CRQB7XRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRXgv8AwUP/AGrNb/ZA+CumeI9A0/R9RvtQ1y30ny9TZ1gRZUkO8lCCMFB+GamUlG1+rS+92X4jSv8An92p71RXxlp/7e/xI8H/ABR8CaX4wPwZudD8X6pLZSXvhnVbnUPsqQxiSUs33VYK6EBuxNdNrH/BV/wW1/oA0Pw94q1bTtfW8vLfUri2Fha3NlZpI9zcW5lIaYoI2xHhSx9KrTl5vl+F/wAvyfZi1PqeivnG/wD+Co/wz07R7G+ki8TeTqHhZvGEWLFMixE7QHI8z/Wb0Py+mOarWP8AwVe+FWqaDa39qnjC6+0Ws2oyW8Ghyyz2dhE2176ZVzst85+bOTtOBim007P+rNp/c07+j7AtVdf1on+q+9H0vRWR4c8d6V4w8O6Lq2lXJ1DTPEEaTWNzbwvJHLG8RlV2IHyKVU8vtGSq/eYA69J3Ts90Cd1dbMKKKKACiimXFwlpbvLK6xxxqXd2OAoHJJNJtJXYb6IfRXyR+wp/wUmvf2tPjv4p8K6noVlotjHayar4Znj3iTU7JLl4C77iQTwp+UAcP6V0Pxp/b3u/hP8AtveFfhePC2r3ej6rYtc399FpkssoLD5JIWDBDBH1mkIIQI+elP8Alv8Aa2/Hftswejkv5d/w/wA0fS1FfO2lf8FQvhZrr3n2OXxHcRpZ3l/p039kyRxeIIrQMbg2bvtWUoEY4JUnHGa7T4HftjeDf2ifEVrpvhaTUL2S40GDxC8rQBYrWGaRo0ikO7ibKt8gB4U800m9v63/APkX9zB6b/1sv1X3nqtFeI3n/BQj4bWPxVl8Kte6sWt9XXw/Pq66bKdIg1Ns7bJrnG0SnB/2eDlhVDwX/wAFH/h943+IN34Vgt/E1pr+lTXqatZ3eniNtFitEVpri4O8qsPzYVgWLEEAdMymmrr1+Wjv9zT9Gnswaadn/V7/AOTXqrbnvtFfM1v/AMFY/hVNZahcSR+MLOC10uXWbJ7vRJLddbtI5Cjy2hcgOAQfvbfunuCKoXX/AAWA+FFha3ktxYeP7f7BFbXc6yeHJgYrScKY7tuywHfGAxI3GRNobNP1/rVr80/ufYP6/L/Nfej6oor5v1T/AIKofCzRvG99o0//AAlIh0vUbfTLzVU0h3020e4GYXkmUkKj84yATg8cGue+D/8AwU903VvD+tv4v0+X+218Zah4Y0LR/DtjNe32rJarGxkEeTyA+WOQvTFHr/W3/wAkg6X87fhJ/wDtrPrGiuW+DHxn8O/tAfDqw8U+Fr7+0NH1EMI5DG0bxurFXjdGAZXVgQQR29MGupptNOzEmnqgr82P2rv2z/i94J+Inx98faJ49vNK8O/AX4g+EPCVl4Ki0ywksdftL9dLa+ku5JIGuvNk/tQiJoZo1T7Mvytls/pPXhnxH/4JyfCf4rfHCTx/rOiapNrN7fadqmpWkOt3sGla1eadj7Bc3lkkot7iS32rsZ0P3I927Ym0ptKrGUtk1+Eot6dbxUo2envBPWm4rf8A4D0v01s7rVW0PYvFvh6Hxd4W1HS7i4v7SDUbaS2knsbySzuYVdSpaKaNlkjcZyHRgykAggivyN/ZZ/bD+IXx6/az8N/s4eJPjXqsnwf8P+IdRXw98VLJrqw1b40GwaFl0KPUkCRiS2Z3W5uLeQvdi32oVzKx/Wn4i/D7Sfix4F1bw1r1tJeaLrls9nfW6XElubiFxh0LxsrgMCQcEZBI6GvNPGH7OfwO8a+CfDvwV1PQ/BZ03wslpqeg+FoZ0trnSBaMPs9zaxxss0PlsMCWPBGSM8nKo3hXVR6rTTv1v6w3iut2nZbup71FwW+uvbS1v+3tm+lrq8rOKfEP42fF7w1401Cx0D4If8JLo9tIFtdT/wCEysrP7YuAd3lOpZOSRgntWL/w0R8dv+jdP/L+0/8A+Ir36NPKjVRnCjAyST+JPJp1AHz/AP8ADRHx2/6N0/8AL+0//wCIo/4aI+O3/Run/l/af/8AEV9AUUAfP/8Aw0R8dv8Ao3T/AMv7T/8A4ij/AIaI+O3/AEbp/wCX9p//AMRX0BRQB8//APDRHx2/6N0/8v7T/wD4ij/hoj47f9G6f+X9p/8A8RX0BRQB8/8A/DRHx2/6N0/8v7T/AP4ij/hoj47f9G6f+X9p/wD8RX0BRQB8/wD/AA0R8dv+jdP/AC/tP/8AiKP+GiPjt/0bp/5f2n//ABFfQFFAHz//AMNEfHb/AKN0/wDL+0//AOIrw/40fHf4z3H7cfwOuJ/gN9nv7fTfEotbP/hNrF/tgaGy8w7wmE2ALwfvbuOlfeFcp458I+Ex4x8PeMfEH2K21Xw0ZrLSr+5vDAtu16YonjA3BGaVkiUBgTnAXk8gHlX/AA0R8dv+jdP/AC/tP/8AiK/KLWf+CvP7a/w+/wCC8PxP+GHgX4YX3xH0Oa50me++G93qKXUXhyE6VZM8sGqIBFZLIW8wtIGh3SnKFzk/u7Wdo/hHSvD2q6pfWGm2Fle65Ot1qNxBAsct/KsSRLJKwGXYRxxoC2SFRR0AoAl8PXt3qWgWNxf2X9m309vHJc2nnCb7LIVBaPevD7SSNw4OM1coooAK8P8A29/2Tbv9sb4SaT4ZtNSsdM+wa7barM93C0sc0cSyK0e0d239+Ote4VzXxX+MPhn4G+EJNe8Waxa6JpUciQ+dNkmSRjhURVBZ3PZVBPB44qZW0cujT+ad1+I1fVLqn+K1Pn39or/gnpaX+p+CtW+F2n+H/CH/AAg1xqOq/wBnWFnHbHUrqa2SOPacCNXzGo3Px0zwK+af2Uf2Vvi9oD2nhK9+E/h/T4NVsLnRdd8S6lo8UV/pVvcJIks1tci8cSv+8PCRKHHXAOR9LftC/wDBWLwB8KPDXhe68KtaePL7xVetZwW0eoLpq2gQhZGneVMxEMVG10B55xxn2/40/tH+C/2dPD1lqXjbXbbQodRl8i2RkknmuJMZKxxxKzvjPJVTjIz1oSSi+b4W7O/e2q+aaX5WYNttRW6V19+/33ep8b2H/BIPxzNoTWWqfEDw5eLa+CpvBun+TpcsP2eJrkzIzned5yz5PHUDHGT1Hxd/4JKT+ONW8L6tpmveGhqen+ErTwrqUWs6M1/a/uIljF5bIJE2yrt4RwyHv3B+ofD/AO0d4K8WfBaf4h6Xr1vqPg62tZbybUbaKSURRRAmQmMKZAV2nK7dwx0rlD+398HxE7/8JtZbY9AXxOx+y3HGnsQom/1fUlh+7/1nP3aud27S3X/2719eab+99NFDb3fT/wBIX5qC+5dde++D/wAN4Pg/8LdA8L2souIdCsYrNZRBHAJdi4LeXGAiZOTtUYGa6SvK7f8Aba+Fdz8RrTwmvjPTRr988UMVs6SoPNlUPHC0hQRpMwIxEzB88bc8Vgj/AIKT/BD7HrU58fWKp4ecRX4azug8LmUxBApi3OxdThUBJAJxjmnOUpSvLd3/AOCTBJRtHZW/4B7lRXi/7Hv7ZWnftdeF/F+s2enJpGmeF9dn0iO4e881L2KNFcXPKJ5YZWztOcY5NSaf/wAFBfg3quna7d23jvTJ7fw5bNe3rpDOf3Cv5bSxDZmdA/BaHeAe9S9N+1/la9/uKWu3e3zva33nslcd+0H4G1n4nfBHxR4c8P6jb6Rq2u6dLYQXs6syW3mrsZ8LzkKWxjvivMG/4Km/AKOB5G+I+mKEljhIazug2ZBlWC+Vkpjq4G0ZGSMisrwb/wAFQPAGo/HfxZ4G8R3Nn4Xl0PVoNK0u9lunuIdbaVSQ+5YgkC8Lgu+Du4PFKVP2idNrf/gf5rzGp8lqnb+v09DlPhb/AMEsB8AfjN8LfGHg/wAVXqzeELR7DXINUuJbpNQgdNrJbAnEK7nmcJ93LA+td9+0d+xzrPxb/aY8H/EPRdf0uxXRtMuNB1TT7+zeVbuxuC4m8t0dSshSRwMjAO09sHt7T9sT4ZX3xbbwLF4x0p/FCXDWZtBv2G4UZaATbfKMw7xh946YzXmi/wDBUT4fa9+0b4d8CeG7uy1+x1OPUH1PW1unt7fRzaRNK2Q8WJUIRv3ivtGDycU3JTtfX4mvRp3+Vm/LXzFy8ilLayV++j0+d166dkcF8Lv+Caep/A/WfDWra5r0Xjbwz8IbTVZfDOhafpXk6hqZut8jRzu8uxny21QAAxIyQM113/BK/wDZNvP2Z/hh4nvdV0m90TUPFmsyXNtYXssclzYadGWFrDIULKHAd2IB4L+tc18RP+CyvhDSPEHiOHwfoy+NdJ8P+HV13+0U1B7BZ3+1xWzW+yWDK480Pv5BHQc5r1K1/wCCk/wdtdasNF1jxjY6N4jujaxTadNBcH7LPcRJIiNL5Qj24cfvMhfUiqpuTvy66W++T17ttrf/ADFUt16O/l8N7dlZSvb/ACZ4/wD8Oil0/wCO2pa7Z674ck8Pan4i/wCEldb3Qhc6vA5cu1nHO7tEsLMQfMMRkXAx33afhL/gnR8QfB3xlvviFY/E6K28W+MrbUrTxTOlq/klLjItmtF3fK1sBGVDHkxAZAJr1P4k/wDBRP4XfDrWZdKTVr7xFrkeppoq6fo1jJcNNeu4T7OszBbfzRnJQyggA8cYru/jJ+0Z4M/Z90qwu/GGtxaMuqyGKzhMMtxcXLKu5gkUSvI20csQpAyMkZrKPKqat8K0XzSTXzVv+HLlKTm1L4nq/k3r8nf+lp8W6n/wRv8AGmvSNcan8QND1HUZdDvdHuL6W2vJLm9eYsVuJnlmkLPzyF2KAAAM5J7Dxl/wSt17xNpfjKBPFekRHxP4H0bwnEWtpD5EtibQvM3PKt9nbAHI3DNe3eMv+Ci3wW8AJpjap47sIk1nTF1myeK1ubhLm1aRohIrRxsOHRwV+8uxiQACas+PP+CgHwc+GmqadZ6z490i3m1W1hvrcxJLcIIJsGKWR4kZYkcMpBkKggg9KpxUlyvvb5+9p6+9LT/IlSs+Zev/AKTr/wCSx/pnyP4O/wCCbnxE+JHxB+KPhjVr+28NeA9V17SLi5uZ9OMs+txWcbYa1cSAICww25T95SOhDdPqX/BG27uozqQ8R+HbvWrPxXqWu2tpqWmy3Ol3VreCIfZ50WRHDJ5eQyN/iPpzxL+3H8JvB/xVtPBWo+OtFt/El6YlitdzsgaQAxq8yqYkLAggO4JyPUVcs/2w/hnqHxcbwLD4w0uTxQty1l9kG/YbhRlrcTbfKMw7xh946YzVO82n1/P4U/8A0lem3UHpe+35X5rfL3n677oP2S/2eU/Zf+Clh4U+3Wup3EMstzc3Ntp0WnwySyNk7IY+FUDA5JJxknmvS6KKcpOTuxJJKyCiiipGFfj74zsNPT9p3xD4VaKA/tHXn7Vel+JtMjkg3ay/hhY7b/TYjxIdOTTFuIWcHylO+Mnc2D+wVFFL3K8a/a2npOFT86aXo2FT36MqPe/4wnD8pt+qQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAV8Pf8Fvfghpvjf4afCrxhdS69PqfhH4oeEF020h1KePT43n1+xjknktkYRzSiIuivKreWrvt2lia+4aKI+7Vp1f5JRl/wCAyTt87W+YPWE4fzRlH/wJNX+V7hRRRQAUUUUAFeE/t4/swa3+0l4Q8JTeGbvTYfEHgfxHa+IbO31KR47O/MJOYZGRWZc54baemO+R7tRR1TW6afzTuvxQ0913TXyas/wPg7wd/wAEyvHen/Frwn401W88LPqEnxEu/Gev2VtcytbWcE2wrBAWiBlYYbO4KPu9cZr2z9uT9nv4h/HDXvAlz4Jv9DtbLQLyafUori6bTr+QOECG3vUgmkgAw27ywpIxyeMfQtFKytGPSLTXySivwihXd3Lq0197bf4tnz5+wN+yhrH7PP7LF74C8ZnTL6e/v7+Sf7HM80MsFxxgs6hslc5yPxNfJvhL/gih4302/wDDh1DXdFmhttfe21XbdyEz6ArW7RRKPL/1mY5crwBvHzV+mtFXzNVFU6pJf+A7fhp6NiavFx82/v3/AB19Un0PhHxZ/wAEwvHOs/EHXtGttW8Mr4A8RfECDx3LqUlzP/a9tsEm61WLyyjHMh2uZBwM9TiobD/gmN440j4BtY2svhL/AITHTvib/wAJxapLcSfY9RtlxsgmkEW4Ny2PlIGTz81felFTD3ElHpb8OS33ckV8vMcrSvfrf8ea/wD6XL+kfEXwe/4JeeK7rwl8QIfGPj7xF4WvvFmuajqMdn4P1uRNKuI7qNR/pMLxKZcMGG3IyvGRnjz/AEv/AIJL/Ed/hldaLqEng2S+0fw3daJpF83iHVrx7h5pMgpFIRBaQ7GO5Fjky+SAM5H6O0UuVcvKtrW+VnH8myuZ83M973+d7/mkfAXjv/gmR8Q/Eek+Oobe48MeZ4j+H2g+GLQveSALd2RsjMW/dcRn7O+0jJPy5Azxn6t/wS7+KetDxhoX2zwPB4e8eazpF9e3xvLhr3TorBBnykEW12cswALLjbyfm4/Q2irjNqTl3d/nzc35mfKuTk/r4VH8kfAXgf8A4JReMPDPxGFhe3+gar4Oh8WS+JoNRuNd1UXcSt90LYRslsLoED98WbI6hh8tYnhP/glL8UhH4R8P6tqPgu20Dwjo/iHRIdQsLu4N5dJqEU4jmkRogOGmUFQ33Vbk5r9GKKz5Vy8nS1vw5fy0/PUtu9/663/PU/NnWf8Agll8Y/FXhq8tL+T4f28sXw+tvBtl9k1C4KyPBfQTLLJugGNyRsTjPJA9cb/jn/glz8RfEehfEe3gufCvn+KtS8OXViz3kg2x2EDJOHPlZBLEbQM574r9B6K1jNxk5rdu/wA+bm/MlxTh7Pp/lFRX4JH5UfD/AEDxd4S+L+q3GtfBfWvGk7eOp9ctNFaHXNLSOVptsd4IEgbS/lUbvMaQ5XqT1r7H/wCCgf7KPiH9pu38H3Hh+x0Ce68OXE8zSXWrXmk6la+Yirm1uoA6KcqNwkicEKMEHr9IUVlyr2Uab+zb8El+S33+5FX/AHkqi63/ABbf6n5l6z+yP8a/Dvx38N+FENj4w1yT4U33h++17VXu/wCzIRcXV2vli6ERLyRxyptDqCwXoOK1vH//AASn+MN14HvfCmneIPCd7oj+G7DSrV0v5tMbzYPLaX7SkduWu1Mit5fmy4Rdp25ULX6PUU7XWvn+Lk/w52JaO6/r4f8A5FeXQ+F/Av8AwT3+Knwv8YeIbHTH+F2seFvG2o6ZqupXHiC0fULnT2t0Ikjjt3iMTkMzeW5YYwD8tVfhf/wS48Y+AvjHaC7u/Dus+ENP8Zt4tgvrrXNVFyg3BlVbCNkthcggAzFmyDyGHy195UVp7R86qdV/9rb7uVW9PNkuKcHDo/8Ag/nzO/8AwwUUUVBQUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAf/Z)"
      ],
      "metadata": {
        "id": "9kuy3bJVt7cc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In real-world word-embedding spaces, common examples of meaningful geometric\n",
        "transformations are “gender” vectors and “plural” vectors. For instance, by adding a <font color='blue'>female</font> vector to the vector <font color='blue'>king</font> , we obtain the vector <font color='blue'>queen</font>. By adding a <font color='blue'>plural</font>  vector, we obtain <font color='blue'>kings</font> . Word-embedding spaces typically feature thousands of such interpretable and potentially useful vectors.\n",
        "\n",
        "Let's look at how to use such an embedding space in practice. There are two ways\n",
        "to obtain word embeddings:\n",
        "\n",
        "* Learn word embeddings jointly with the main task you care about (such as document classification or sentiment prediction). In this setup, you start with random word vectors and then learn word vectors in the same way you learn the\n",
        "weights of a neural network.\n",
        "* Load into your model word embeddings that were precomputed using a differ-\n",
        "ent machine learning task than the one you're trying to solve. These are called\n",
        "<font color='blue'>pretrained word embeddings</font>.\n",
        "\n",
        "Let's review each of these approaches."
      ],
      "metadata": {
        "id": "IJ-qZFO0uB9f"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "haFy0Ef_jdwz"
      },
      "source": [
        "#### Learning word embeddings with the Embedding layer"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Is there some ideal word-embedding space that would perfectly map human language\n",
        "and could be used for any natural language processing task? Possibly, but we have yet to compute anything of the sort. Also, there is no such a thing as <font color='blue'>human language</font>—there are many different languages, and they aren't isomorphic to one another, because a language is the reflection of a specific culture and a specific context. But more pragmatically, what makes a good word-embedding space depends heavily on your task: the perfect word-embedding space for an English-language movie-review sentiment-analysis model may look different from the perfect embedding space for an English-language legal-document classification model, because the importance of certain semantic relationships varies from task to task.\n",
        "\n",
        "It's thus reasonable to <font color='blue'>learn</font> a new embedding space with <font color='blue'>every new task</font>. Fortunately, backpropagation makes this easy, and Keras makes it even easier. It's about learning the weights of a layer: the <font color='blue'>Embedding</font> layer."
      ],
      "metadata": {
        "id": "lcFISJxkux3E"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jh7mx0Qujdwz"
      },
      "source": [
        "**Instantiating an `Embedding` layer**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "9mZ7UQTUjdwz"
      },
      "outputs": [],
      "source": [
        "embedding_layer = layers.Embedding(input_dim=max_tokens, output_dim=256)        # The Embedding layer takes at least two arguments: the number of\n",
        "                                                                                # possible tokens and the dimensionality of the embeddings (here, 256)."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Embedding layer is best understood as a dictionary that maps integer indices\n",
        "(which stand for specific words) to dense vectors. It takes integers as input, looks up these integers in an internal dictionary, and returns the associated vectors. It's effectively a dictionary lookup (see figure below)."
      ],
      "metadata": {
        "id": "jXv2gYZXvqdt"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "![embedding.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAbsAAABOCAIAAAA2Iy9XAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAABOlSURBVHhe7Z1/TJRXusfnv2lCiOUPskmJmRJjNhti2o39g1zS3EK8bRoTYxOzZhPNvblkzA53ZOusVtFeFFs6UhUNO15wuYLUXiuKyuqMsuMdxcWCpQXFlWJbKYKGsi5Ld1f+eP9jn+c55/01887M4UfdwX0+f5T3PXN+Puec7/s8531NXTMMwzCMGqyYDMMwqrBiMgzDqMKKyTAMoworJsMwjCqsmAzDMKqwYjIMw6jCiskwDKMKKybDMIwqrJgMwzCqsGIyDMOoworJMAyjCismwzCMKkkV89tPz5w5c/1rTd5ODXQk3ncMTMlbNbDOcN+EvBM4pTmjnnNRQYaOxzR0cmgKVDLacS5mGHd+Vn5G52hWTI90h49Uer3eLTUX+x9Py9TFwAItggXkB+jI9OP5zUlSxRxrKHa5llb0iLupE2tcyPo2udNim3Nc2evbZqeY7RtdrmxfVN4JfrfF4yl497q8S4VT6WcAGtaKtbDDLFSFx+TPyblb85LLVRQalreqOBczjKs+H048o3OkzEQ0sCLL5c579ecwiWvg0pX1esNXs32o/aNYoEWwgCz0evo8uCJrnvUlj8ppZ71y6Cu81to3ul05OTmuHF+Upr9n51JX9uqW9NvaxvzG/4zuxjkP64dRzPnxjM6RIhMn1sA2KQ4N6BI50bY+x+V6qeauvM9wMm/2FrpHC1BfinNMdCOlU4mXBdUh+K/wOskBpV030VWzNh8epS6Xe8nyDU2D6PDSE6oAU935WzqGLgVWLnFDhqz8/NzE/uqPs3v1b3o8q1b9K2V1L1kZiKIvrj10Ko2phbnUalZuYWUXNDrcvG65x/OLdrjUBmpXeTzLS9ueTlTx/fffy6u5kXQSySKFhWL47ryfhZoMU2xoAb+FpC8nL1+mrQ31UbThYBwIRfpCYprcS/LzYFpp7hyNm3o+wLgNcr6z8gsKEjwRYzDa122/kL2glQEdHg69nutZ9WupHsPH1no8axqHnbpLbRcsz4Oms1Y1zvaRMHdGRkbk1ZygXaH7FBIIK2P9j6YgKd1O+Y+3/806aJyx5dL4ZD7IbhofrF+yD21FNRQWyrldGbj0kFpP1ljhqlUiGdZLAwl7ykWQrEyqRfCHuhKPZ+0xnLWptv/0eDzrjqNjRddv1t/DhSHao/m+hqaxz/eQU490/hb2L/cUbI/RKKnSvBUVXZqTtWgNGk2tqum7Vf8mLbSsXOxTuo4c6voTtRFPCsUUjuXmmPAo4YISUDK1tvUuUNCBmeGGYrdr2cbWoWnYk9WvuF3uje0abRt41B76tD/WcQ5Tl/lgIkWGBHHQ9xjtfz3nHtQCaPlurUNprWfnMld20Z7Oh5r2sG1jrsu9uuU78J0wr3t9680aKiM2+FPg1q1b0PHXXnvt7bffvnbtmkxVh6xV9I48wBTQMaNpkemHv3kDLmHMXRPaQLVuHKvF0Dq4WR2NMxUpzcUJ6ZsWaUIxHY2bej602GZZEezJnUYZE730cKgIfKttXTAJ04MfwjWWpkRcNgBWn/1G05DjXOptX+jvDn/6LVX8VIBJhHbXrl176NChOTwIxUzGe+8ChZ2CRtYH3Yve6tL1HxvZl1X0aPdqXwGbHQU5EDOCvqteA87IlbJlov3UjeEqaEXft7hhLM0icCyTbhGgWrhWt/xJCAiAXhddF1R/LspWG/NNg7DN9+8ce2RAFdF4QDA/eYsEiXz7eGvNfLEbzPHSNmzqqxAYBAsZA5ODSNGR+86CmUoxxeElxOW0uvHMUk9AlxOVE6dQD9wB3SmlbokzTnsOOvy0jT9uh+o5B6oLMNG5NP1d0/iHUeL6rgIx9zIoQlT1EnyK+ROJRKhRE9hyu3fvBiWVzaSGDBB3jknHmFaL0LXcjLodrGliIUGqo3GOl8JvpZfEc1mPD5JMTar5mIlYK6LVQGVMjBU5/Xh0FE/Y4W//8XU/EolUgqIUXM14quM8l9S2lFYVQN3kZMyPwsJCOYXEiy++CE/B8+fPy2bSgYMnqUgk/U4RBpeDpjla/k5MmGW0eR09csaOvQ77/selTd1DxssLrMFwa6mV4oZrKRqTk0c+T/pFIC7iyqRdBHqDdHSXnY39m476SDJsZcUwYcjWoSfpkYmGVZFkUtsgdM7Wsh0calOPRkfB1TcGlr4jyUilmKKi1fv2FetNi4SKClBQbI0aeKNJVye6BZMa3ZJJRg68ixu/MTl6YUqUGZ1LYwn0rC38tKobc2iXSrPht4I9fdISaQBdg10xT1544QXcXgk8//zz4LOk102LtexYLRJ/rVvMLEg5ihv+18k4NZD4o21dMic2CFVRgcSpSTUffzn5lnUrOvRcTzJCHuhK4asr9WUvnIKtXT0gmDkb26cof+Jc2tpWABRTTsb8eO6552jq4nn55ZdVdJOcq7j9NjXQEesfffy5zdiGca0WtA6art1L8qRJiJ//3zj4dKFNJSL6lEcYWINpKbzL2XwsfWN6Ofot+SIwLxBR5nbaRUA6l+MLQTC0tGIfXJsHepjbXIl64YShJ/bICtl5fdtXGLJgKSqSYK1mh46ZvU3fkWSkVEzyLdxumCC9FqoSEApKzwNjiZD44xPT6JYp3pTB6YlhjMHWWWko59L0CDMe5bQk6aCIYk9Xbi6EdsWhp/d+UkTlAqGSwsFUDess1rJjtUj8dYLFyAGAJ6ujcTowEWMjhIwKxZJMTar5EKbXK0rhY1LdS0sviEM1fJ7LqmiNLC0qWioEM8lc2tp+ioioXABTCbFCc3OzaqwAiDiw2njxA8E47moYam/anWIzuLAfvTJAvv00DG7ln6ce9cdiEHmCx3S/FQNJ0C2swfAxqQrpY6ZujCYq/SKwL05ZJu0ikOd54kUxBH4gZ6Ah8KDUxFLQyxrHNAlDT+yRDbJzUVERnutAIUdrTRuHA4jWU0+RmzGe9B1JRmrFFCvaMgKh70atdGDiLj40OK097MQjjWWbY8Jx0IcpXh+u/3hk2vFUwpwTW2fxBhOdS5PBcn0XHuvHcnhsMdEOV9lFhwam8YTCvm5/UGBHgQ8C4Rvsrrm8BSID/PtpGVLoQNhltUj8tWExsMOlqWlpfOP0Js44f8XTQtdLe25CTkqjqpyNm3I+xLFX7sbWodH+prewIko1kaXpUZpTemFK06ZuUt16VbR/Ado/gONc2tp+ioBEzvqBZ4MWoSvrX3b//xDMYX9r6Y/BwvTuPP1OsQ2anHHIPjCln1GvbhkmWy3b1glWpUNFdFuwBlHv9BDOCOZTaIxu0i8C5zLpFoGUTEhHv4r0TJd1Oj6kNavJ/hY3yONDfehpFQMghZP1A47WErWKpigRVxYJ5brm0UffdaTtSBLSKKYW8YOLWxj8XN6Lt2fmG88ZfElVgG+gIEpYU9lGH0Ybb9oIPYd7ycqSkgL7ezVAz0yvqcRrNXBe4UZkdCytPbxUWUKtupcsLwlAq9+F/T/1FOiv8mLbCzyeVz7oxaoyHjJoPPEWib/Wfy8oKaEXzVn5ZAb8OdE4IjFQmAfhHGQsKdSrcjJumvnAwHADVAAtb3oLlpf5aCf00vg2laJHXBb7AhuMgxP5zNUF07m7trYXFyBRYB+0NA5o5ZpKfP1CpNspcYOe7muS9UD+DSF8hzZDHyDoaWv2XXooXudkL1uBtjbzpW+MbtIvgmRlUi8CQLwlF2/M7/56FeTUX28bKxGne5Pob8LQHfa8HfrSosAf0b8Hd7KWWFpiFeaXbGpBbdB6qrDf9NI+fUccSaOYDGNhLFzl9daDJ4uQyyPeuc0C8gcWzReKGY/NCXw6LMAiWMywYjLqaD27fwIBWeGmmprAq3kQ6MzqIy7cavjvYChmYhaEf4Bizm8RLHpYMZlZoT3sDYuPRsPdI/o3LorgvxI+c4ZeXjALA5n0aX6zSsxnESx2WDEZhmFUYcVkGIZRhRWTYRhGFVZMhmEYVVgxGYZhVGHFZBiGUYUVk2EYRhVWTIZhGFVYMRmGYVRhxWQYhlGFFZNhGEYVVkyGYRhVWDEZhmFUYcVkGIZRhRWTYRhGFVZMhmEYVVgxGYZhVGHFZBiGUSWZYv7tdiwSz+XYFUrsvPtXmesZ4343jdNO590HwhY/+LD15rvvy4TZMGlMWOz2pEwjzEFZ6xXZ59SSE7L1Z3ZlMIwkmWI+OLHTG8/Wg7WUWHMxYV88GewERf1sVN4uUjpqaZx2ai7eFLZwGHZKTA2zEqdnVq7WB8rLoKHaDpkwG748XR0IbPH7vN6dJ+7INAJqDSTWe4fGNKeWnBDVzdpEDLPYSKOYm4MfyZ0OxDrO1AeB4520L7TJsZE+EMpIZ09b0OctK9/z8WXyZ0g5LV6H8HI6b9/rjUpRfTIx2BONRHu+nhT/D88MQbpjv9mNQ/fuOnIe77rv63LQPnqHhjv4ROZPPRDQMFIwn38LaBbgpGdxkGbPXceop04tJNT7INpcV1fXtlD/S/c4xdQmyVIRsMwwGmv0s8t4J11QNPPlriG8hjWES6izb0LaVCyby5/dg0cwe6xM5pFGMbc2fSETEHNfaLdPVoLb4vPvCgYr0H/x7m6FCE84aVTIsodEanl5Of00fqNJFC0HAfFt2X824/aFdDUNh0kOxecDwaOL7Y034CeFgdgVTNwdOY3iASrR0xMFQZlEcZHCoitbeELIyJhdh6W8RHsGdX2RGLr9WYu1PYsehS2KafF9zahcT4z1DctG7A8BvYnBr3tlRpuvbJltuTQMyipP3dGuhvxw6T98BUx0/Yjf66u5OKnda9+/BUxaVo4ecFll4+/haSoq8peXww/GBDBMxpBGMQMHPqH9AcAT39wXVw6jegQab0JWLbIfU7cfv5VSMWFTVASbL507AsLp/WX977UZsY9875/NsGA+iWKW1Zwd7G6iSxihdl1hIFQSbUhua9Vh8Oua/+egiJG9vrIyIcFlZXBffuQ6KJRuKpAR8kjLKk/eJuEy5HkLlQaBbr8nfug4jLIjdJsQiqkLl9AjQiomxe/xcbruEBOijO/dkyS92mgkvgnoRvXpL6mgwJxtraMWM21tvKnNjJ99H1NhZVgkUxfMb05XYaVVZ0C1vzmF9tl69Ia+xGCAew4smAfMMAuG+jkm7AZzX0xeOYx6sbW2NdJ6+B1c+YHGG6l9TNwlMzM3GwN4s/U9iAnrPtyF28i3P5JZvkQSxaR7Y4RKA6GSusqZ+kSV7I9ot1t2eL07Wm5TNhI6+qW8NjKKajh+LbRVWFMIzraDpy7Qw+t8Y9V/CXtOXqzxef3BNjooePIF6TlV9EUTFN15tAsr0kbD+7F/1qhctGRLoV749/+W3F3qGtX04Oz74EJXfZLQhBWriQBt7Jsv8cRGnG/QehD28h9uqg9Q1yfbg/ibf9eHaMD3oLckrbIiKsIwmYf6OSZEcOa+0O5FareiH4DLvf5EWEaVhp44KKbYBOLaW15JJQWZ5kvIPqZWTKWBUEmUF8q9/7djIyMYadMdqJX+s5HN+EViy2Cch0rQyyNfPzH/zJcn35U1CuLqBRJTzF7YbpJ0Sd5KKFWYCLzhKnKPy3fsrSQhFDNPGk6Q1ssSvneqpfGA5ugDS0UMk4HM7Rzz7ukq+Buoi5IvZEK7yxsIXZscj5BbQ0tfpIqqHlBJ784WDDbHLx4M7Nh76FyfKJ0piP6au9a6i42xKA2ESqK8PJkYGRmhkB7vdBHSfzayiV9EnRhYn3gXPMjQVU2c/QXP6ubWJu+ET1yA/FSyPHRVnGuOtwdBqrAiio19VaeGREU3G1Gt5qaYdEVBNmI2YYMykYmwpwAda0s/Ui4imUc/vRAZ/fsj49jB45WBymBL7JFZERZhmExjbor5AINBC2XlgcqDp26Mi/MoQr4niVdMKQNUhELVspqLsGUyCtFfc9dad7E5lrQDeRBtpmhdBp4ydt9ZWy9iUIjn62ugYv+u+o+P0N///mCviE7BndwVDO5CT628toNqFceVZeWVwbpgJbXnq/gIBFr2oqwieGDvrzCZ2gNnbbyjthwyQRggKgKgRfCCe9uoM3ofAMht7WxztNd6AzVh09ilOr2qFIopXUloOLjnV346qdUXkVgdxnGv6CG6zuKTgu1NoMqsmExmk0wx/0hfn9BmMhGfpMC2G7/aCHtHBuV1dQf24JsBOuEf7TpVDynHwoM3aGdioCr2qFkVeEjRE5Sr1fqlTsYgNcWIsY1hw7VtLKkHIovF0XysSV7BdbMwclsb/X3vg734p+5UuLP1GB52nLPVqo32RmW65XMc6kXnOehG/Ylw9Cz8Doj+PRnuoQLQu/ApSrcopgXMbeksKaa8NmsKy4G21ekHnFasJnoyKLsfHRyi5KPhQcpEp7HiOFsC9VKPoeu9wn+2VsQwmUcyxUxNVwP6L+h1EKSYZRUfWf1R5plBm5wwvjTSbhwNeL3B9qSf4TsjXUdwhg9f0atimEXI3BSTHIkIehLEsVbjg0LmmYO+HvP5t+zYe0BE/r6q09/I31QRruOxs9KTZJjFylwVk/mnQZscw6/XMfCnhyOrHvNPDCsmwzCMKqyYDMMwqrBiMgzDqMKKyTAMoworJsMwjCqsmAzDMKqwYjIMw6jCiskwDKMKKybDMIwqrJgMwzCqsGIyDMOoworJMAyjCismwzCMKqyYDMMwqrBiMgzDqMKKyTAMoworJsMwjCqsmAzDMKqwYjIMw6jCiskwDKMKKybDMIwaMzN/B8Ajz0mBhnr2AAAAAElFTkSuQmCC)"
      ],
      "metadata": {
        "id": "vbQ3qvn7vyCQ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The *Embedding* layer takes as input a rank-2 tensor of integers, of shape `(batch_size,sequence_length)`, where each entry is a sequence of integers. The layer then returns a 3D floating-point tensor of shape `(batch_size, sequence_length, embedding_dimensionality)`.\n",
        "\n",
        "When you instantiate an *Embedding* layer, its weights (its internal dictionary of token vectors) are initially random, just as with any other layer. During training, these word vectors are gradually adjusted via backpropagation, structuring the space into something the downstream model can exploit. Once fully trained, the embedding space will show a lot of structure—a kind of structure specialized for the specific problem for which you're training your model.\n",
        "\n",
        "Let's build a model that includes an *Embedding* layer and benchmark it on our task."
      ],
      "metadata": {
        "id": "tGWlh6I0v77P"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xQDdU3jgjdwz"
      },
      "source": [
        "**Model that uses an `Embedding` layer trained from scratch**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XPW1QVqijdwz",
        "outputId": "74a25aff-765a-460d-bbf7-31bf2eae102e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_1\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_2 (InputLayer)        [(None, None)]            0         \n",
            "                                                                 \n",
            " embedding_1 (Embedding)     (None, None, 256)         5120000   \n",
            "                                                                 \n",
            " bidirectional_1 (Bidirectio  (None, 64)               73984     \n",
            " nal)                                                            \n",
            "                                                                 \n",
            " dropout_1 (Dropout)         (None, 64)                0         \n",
            "                                                                 \n",
            " dense_1 (Dense)             (None, 1)                 65        \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 5,194,049\n",
            "Trainable params: 5,194,049\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "Epoch 1/10\n",
            "625/625 [==============================] - 725s 1s/step - loss: 0.5296 - accuracy: 0.7391 - val_loss: 0.3803 - val_accuracy: 0.8542\n",
            "Epoch 2/10\n",
            "625/625 [==============================] - 645s 1s/step - loss: 0.3777 - accuracy: 0.8534 - val_loss: 0.3608 - val_accuracy: 0.8566\n",
            "Epoch 3/10\n",
            "342/625 [===============>..............] - ETA: 4:23 - loss: 0.3036 - accuracy: 0.8893"
          ]
        }
      ],
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = layers.Embedding(input_dim=max_tokens, output_dim=256)(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "callbacks = [\n",
        "    keras.callbacks.ModelCheckpoint(\"embeddings_bidir_gru.keras\",\n",
        "                                    save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)\n",
        "model = keras.models.load_model(\"embeddings_bidir_gru.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It trains much faster than the one-hot model (since the LSTM only has to process\n",
        "256-dimensional vectors instead of 20,000-dimensional), and its test accuracy is comparable (<font color='blue'>87%</font>). However, we're still some way off from the results of our basic bigram model. Part of the reason why is simply that the model is looking at slightly less data: the <font color='blue'>bigram</font> model processed <font color='blue'>full reviews</font>, while our <font color='blue'>sequence</font> model <font color='blue'>truncates</font> sequences after <font color='blue'>600 words</font>."
      ],
      "metadata": {
        "id": "9QNSS5v6whr8"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ri5Zb9A0jdw0"
      },
      "source": [
        "#### Understanding padding and masking"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "One thing that's slightly hurting model performance here is that our <font color='blue'>input sequences</font> are full of <font color='blue'>zeros</font>. This comes from our use of the `output_sequence_length=max_length option` in `TextVectorization` (with `max_length` equal to 600): sentences <font color='blue'>longer</font> than <font color='blue'>600 tokens</font> are <font color='blue'>truncated</font> to a length of 600 tokens, and sentences <font color='blue'>shorter</font> than <font color='blue'>600 tokens</font> are <font color='blue'>padded with zeros</font> at the end so that they can be concatenated together with other sequences to form contiguous batches.\n",
        "\n",
        "We're using a bidirectional RNN: two RNN layers running in <font color='blue'>parallel</font>, with one processing the tokens in their <font color='blue'>natural order</font>, and the other processing the <font color='blue'>same</font> tokens in <font color='blue'>reverse</font>. The RNN that looks at the tokens in their natural order will spend its <font color='blue'>last iterations</font> seeing only vectors that <font color='blue'>encode padding</font>—possibly for several hundreds of iterations if the original sentence was short. The information stored in the internal state of the RNN will gradually fade out as it gets exposed to these meaningless inputs.\n",
        "\n",
        "We need some way to tell the RNN that it should skip these iterations. There's an API for that: <font color='blue'>masking<font color='blue'>.\n",
        "\n",
        "The *Embedding* layer is capable of generating a <font color='blue'>mask</font> that corresponds to its input data. This mask is a <font color='blue'>tensor</font> of <font color='blue'>ones</font> and <font color='blue'>zeros</font> (or True/False booleans), of shape `(batch_size, sequence_length)`, where the entry <font color='blue'>mask[i, t]</font> indicates where time step <font color='blue'>t</font> of sample <font color='blue'>i</font> should be skipped or not (the timestep will be skipped if mask`[i, t]` is 0 or False, and processed otherwise).\n",
        "\n",
        "By default, this option isn't active—you can turn it on by passing <font color='blue'>mask_zero=True</font> to your *Embedding* layer. You can retrieve the mask with the `compute_mask()` method:"
      ],
      "metadata": {
        "id": "-rLx8D0Jw-cT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "embedding_layer = Embedding(input_dim=10, output_dim=256, mask_zero=True)\n",
        "some_input = [\n",
        "    [4, 3, 2, 1, 0, 0, 0],\n",
        "    [5, 4, 3, 2, 1, 0, 0],\n",
        "    [2, 1, 0, 0, 0, 0, 0]]\n",
        "mask = embedding_layer.compute_mask(some_input)\n",
        "<tf.Tensor: shape=(3, 7), dtype=bool, numpy=\n",
        "array([[ True, True, True, True, False, False, False],\n",
        "[ True, True, True, True, True, False, False],\n",
        "[ True, True, False, False, False, False, False]])>"
      ],
      "metadata": {
        "id": "qQNwlT7Vx980"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In practice, you will almost never have to manage masks by hand. Instead, Keras will <font color='blue'>automatically</font> pass on the mask to <font color='blue'>every layer</font> that is able to process it (as a piece of\n",
        "metadata attached to the sequence it represents). This mask will be used by RNN layers to skip masked steps. If your model returns an entire sequence, the mask will also be used by the loss function to skip masked steps in the output sequence.\n",
        "\n",
        "Let’s try retraining our model with masking enabled."
      ],
      "metadata": {
        "id": "_GYcZ_HByN4U"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xaujp8Gdjdw0"
      },
      "source": [
        "**Using an `Embedding` layer with masking enabled**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DtmQGHsujdw0"
      },
      "outputs": [],
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = layers.Embedding(\n",
        "    input_dim=max_tokens, output_dim=256, mask_zero=True)(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "callbacks = [\n",
        "    keras.callbacks.ModelCheckpoint(\"embeddings_bidir_gru_with_masking.keras\",\n",
        "                                    save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)\n",
        "model = keras.models.load_model(\"embeddings_bidir_gru_with_masking.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This time we get to <font color='blue'>88%</font> test accuracy—a small but noticeable improvement."
      ],
      "metadata": {
        "id": "4ddOdbTRKmC9"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5YGjh-IIjdw0"
      },
      "source": [
        "#### Using pretrained word embeddings"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Sometimes you have so little training data available that you can't use your data alone to learn an appropriate task-specific embedding of your vocabulary. In such cases, instead of learning word embeddings jointly with the problem you want to solve, you can <font color='blue'>load embedding vectors</font> from a <font color='blue'>precomputed embedding space</font> that you know is highly structured and exhibits useful properties—one that captures generic aspects of language structure. The rationale behind using pretrained word embeddings in natural language processing is much the same as for using pretrained convnets in image classification: you don't have enough data available to learn truly powerful features on your own, but you <font color='blue'>expect</font> that the <font color='blue'>features</font> you need are fairly <font color='blue'>generic</font>—that is, common visual features or semantic features. In this case, it makes sense to <font color='blue'>reuse</font> features learned on a different problem.\n",
        "\n",
        "Such word embeddings are generally computed using <font color='blue'>word-occurrence</font> statistics (observations about what words co-occur in sentences or documents), using a variety of techniques, some involving neural networks, others not. The idea of a <font color='blue'>dense, low-dimensional</font> embedding space for words, computed in an <font color='blue'>unsupervised</font> way, was initially explored by <font color='blue'>Bengio et al.</font> in the early 2000s, but it only started to take off in research and industry applications after the release of one of the most famous and successful word-embedding schemes: the <font color='blue'>Word2Vec</font> algorithm (https://code.google.com/archive/p/word2vec), developed by Tomas Mikolov at Google in 2013. Word2Vec\n",
        "dimensions capture specific semantic properties, such as gender.\n",
        "\n",
        "There are various precomputed databases of word embeddings that you can down-\n",
        "load and use in a Keras *Embedding* layer. Word2vec is one of them. Another popular one is called <font color='blue'>Global Vectors for Word Representation</font> (GloVe, https://nlp.stanford.edu/projects/glove), which was developed by Stanford researchers in 2014. This\n",
        "embedding technique is based on factorizing a matrix of word co-occurrence statistics. Its developers have made available precomputed embeddings for millions of English tokens, obtained from Wikipedia data and Common Crawl data.\n",
        "\n",
        "Let's look at how you can get started using <font color='blue'>GloVe</font> embeddings in a Keras model. The same method is valid for Word2Vec embeddings or any other word-embedding database. We'll start by downloading the GloVe files and parse them. We'll then load the word vectors into a Keras Embedding layer, which we'll use to build a new model.\n",
        "\n",
        "First, let's download the GloVe word embeddings precomputed on the 2014\n",
        "English Wikipedia dataset. It's an 822 MB zip file containing 100-dimensional embedding vectors for 400,000 words (or non-word tokens)."
      ],
      "metadata": {
        "id": "o_Fk0FChKxAU"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZwMAFa74jdw0"
      },
      "outputs": [],
      "source": [
        "!wget http://nlp.stanford.edu/data/glove.6B.zip\n",
        "!unzip -q glove.6B.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's parse the unzipped file (a .txt file) to build an <font color='blue'>index</font> that <font color='blue'>maps words</font> (as strings) to their <font color='blue'>vector</font> representation."
      ],
      "metadata": {
        "id": "-L-N4516MgzO"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kxtHCVlkjdw0"
      },
      "source": [
        "**Parsing the GloVe word-embeddings file**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8ddEmRRbjdw1"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "path_to_glove_file = \"glove.6B.100d.txt\"\n",
        "\n",
        "embeddings_index = {}\n",
        "with open(path_to_glove_file) as f:\n",
        "    for line in f:\n",
        "        word, coefs = line.split(maxsplit=1)\n",
        "        coefs = np.fromstring(coefs, \"f\", sep=\" \")\n",
        "        embeddings_index[word] = coefs\n",
        "\n",
        "print(f\"Found {len(embeddings_index)} word vectors.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, let's build an embedding matrix that you can load into an *Embedding* layer. It must be a matrix of shape `(max_words, embedding_dim)`, where each entry `i` contains the `embedding_dim`-dimensional vector for the word of index `i` in the reference word index (built during tokenization)."
      ],
      "metadata": {
        "id": "TJKhaPdMM8li"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OAuhDc_4jdw1"
      },
      "source": [
        "**Preparing the GloVe word-embeddings matrix**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Y6G4_ARfjdw1"
      },
      "outputs": [],
      "source": [
        "embedding_dim = 100\n",
        "\n",
        "vocabulary = text_vectorization.get_vocabulary()                                # Retrieve the vocabulary indexed by our previous TextVectorization layer.\n",
        "word_index = dict(zip(vocabulary, range(len(vocabulary))))                      # Use it to create a mapping from words to their index in the vocabulary.\n",
        "\n",
        "embedding_matrix = np.zeros((max_tokens, embedding_dim))                        # Prepare a matrix that we’ll fill with the GloVe vectors.\n",
        "for word, i in word_index.items():\n",
        "    if i < max_tokens:                                                          # Fill entry i in the matrix with the word vector for index i. Words\n",
        "        embedding_vector = embeddings_index.get(word)                           # not found in the embedding index will be all zeros.\n",
        "    if embedding_vector is not None:\n",
        "        embedding_matrix[i] = embedding_vector"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we use a <font color='blue'>Constant</font> initializer to <font color='blue'>load</font> the <font color='blue'>pretrained embeddings</font> in an *Embedding* layer. So as not to disrupt the pretrained representations during training, we <font color='blue'>freeze</font> the layer via `trainable=False`:"
      ],
      "metadata": {
        "id": "GJZ3pbusNu2E"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lJGo-INmjdw1"
      },
      "outputs": [],
      "source": [
        "embedding_layer = layers.Embedding(\n",
        "    max_tokens,\n",
        "    embedding_dim,\n",
        "    embeddings_initializer=keras.initializers.Constant(embedding_matrix),\n",
        "    trainable=False,\n",
        "    mask_zero=True,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We're now ready to train a new model—identical to our previous model, but leveraging the 100-dimensional pretrained GloVe embeddings instead of 128-dimensional learned embeddings."
      ],
      "metadata": {
        "id": "EhOa_ki_N4CR"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "el3TQAWGjdw1"
      },
      "source": [
        "**Model that uses a pretrained Embedding layer**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cF6vn70-jdw1"
      },
      "outputs": [],
      "source": [
        "inputs = keras.Input(shape=(None,), dtype=\"int64\")\n",
        "embedded = embedding_layer(inputs)\n",
        "x = layers.Bidirectional(layers.LSTM(32))(embedded)\n",
        "x = layers.Dropout(0.5)(x)\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x)\n",
        "model = keras.Model(inputs, outputs)\n",
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"binary_crossentropy\",\n",
        "              metrics=[\"accuracy\"])\n",
        "model.summary()\n",
        "\n",
        "callbacks = [\n",
        "    keras.callbacks.ModelCheckpoint(\"glove_embeddings_sequence_model.keras\",\n",
        "                                    save_best_only=True)\n",
        "]\n",
        "model.fit(int_train_ds, validation_data=int_val_ds, epochs=10, callbacks=callbacks)\n",
        "model = keras.models.load_model(\"glove_embeddings_sequence_model.keras\")\n",
        "print(f\"Test acc: {model.evaluate(int_test_ds)[1]:.3f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You'll find that on this particular task, pretrained embeddings aren't very helpful, because the dataset contains <font color='blue'>enough samples</font> that it is possible to <font color='blue'>learn</font> a specialized enough <font color='blue'>embedding space</font> from <font color='blue'>scratch</font>. However, leveraging pretrained embeddings can be very helpful when you're working with a smaller dataset."
      ],
      "metadata": {
        "id": "8u-zMJBxOApe"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Ns3vzk-XOFGf"
      },
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "colab": {
      "name": "chapter11_part02_sequence-models.i",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}