GPT2_Training.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qHsu0d1oWOkm"
      },
      "source": [
        "**CS Interview QA Chatbot Usage**\n",
        "\n",
        "This workbook contains the code for training and running the GPT-2 Transformer model that answers question about CS interview questions.\n",
        "\n",
        "Please ensure that you have ran the preprocessing.ipynb notebook before running this, and store it in your google drive\n",
        "- ensure that you have combined.csv stored in your directory\n",
        "\n",
        "To train from scratch:\n",
        "1. [Import libraries](#scrollTo=ZNjjbPXpWiqa&line=1&uniqifier=1)\n",
        "2. [Load the preprocessed dataset](#scrollTo=ErDcd3pqnL5N&line=1&uniqifier=1)\n",
        "3. [Run Utility Functions](#scrollTo=-NASuplC3w-N)\n",
        "4. [Define Model Architecture](#scrollTo=3a-zptK6DXDJ)\n",
        "5. [Define Training Function](#scrollTo=SFjeoyzLfRmz)\n",
        "6. [Initialise and Train Model](#scrollTo=3HQZcsVhKU8I)\n",
        "7. [Run evaluation functions](#scrollTo=03W05HyA5Sxh)\n",
        "    - Fallback Mechanism with BERTScore\n",
        "8. [Evaluate with BERT, ROGUE, BLEU Score (quantitative)](#scrollTo=-ABw4DJ089aC&line=3&uniqifier=1)\n",
        "9. [Host on Telegram](#scrollTo=-ABw4DJ089aC&line=3&uniqifier=1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNjjbPXpWiqa"
      },
      "source": [
        "### Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oB5MazNu379Q",
        "outputId": "5f87d265-bd1d-4a08-ac3c-5bb7863a6db2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive', force_remount=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dYHCm3AHS1Gc"
      },
      "outputs": [],
      "source": [
        "FOLDER_PATH = '/content/drive/My Drive/Colab Notebooks/nlc/project'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pHkIyAN5U4j0"
      },
      "outputs": [],
      "source": [
        "from __future__ import absolute_import\n",
        "from __future__ import division\n",
        "from __future__ import print_function\n",
        "from __future__ import unicode_literals\n",
        "\n",
        "import torch\n",
        "from torch.jit import script, trace\n",
        "import torch.nn as nn\n",
        "from torch import optim\n",
        "import torch.nn.functional as F\n",
        "import csv\n",
        "import random\n",
        "import re\n",
        "import os\n",
        "import unicodedata\n",
        "import codecs\n",
        "from io import open\n",
        "import itertools\n",
        "import math\n",
        "\n",
        "\n",
        "USE_CUDA = torch.cuda.is_available()\n",
        "device = torch.device(\"cuda\" if USE_CUDA else \"cpu\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ErDcd3pqnL5N"
      },
      "source": [
        "### Loading Datasets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "VWYGjdz_tjVW",
        "outputId": "c57143e9-6bc7-4ecb-b6f6-e40dc67fde56"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"df\",\n  \"rows\": 3775,\n  \"fields\": [\n    {\n      \"column\": \"Question\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3299,\n        \"samples\": [\n          \"define bidirectional search\",\n          \"what do you mean by divide and conquer algorithm\",\n          \"explain accumulatorbased architecture in computer systems\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Answer\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1817,\n        \"samples\": [\n          \"inner join returns matching rows from both tables left join returns all rows from the left table and matching rows from the right\",\n          \"the basic operations carried out in a linked list include creation of a listinsertion of a nodedeletion of a nodemodification of a nodetraversal of the list\",\n          \"there is no concept of folder in ubuntu everything included in your hardware is a file\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "df"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-ffb3f238-1964-4f27-8902-218b57a9a9b4\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Question</th>\n",
              "      <th>Answer</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>how does randomised algorithm work</td>\n",
              "      <td>the algorithm typically uses uniformly random ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>what do you mean by bestfirst search</td>\n",
              "      <td>bestfirst search is a search algorithm which e...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>how do you explain a daemon</td>\n",
              "      <td>daemon disk and execution monitor is a process...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>what is phonetic algorithm</td>\n",
              "      <td>a phonetic algorithm is an algorithm for index...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>what do you mean by uniform costsearch</td>\n",
              "      <td>a tree search that finds the lowestcost route ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3770</th>\n",
              "      <td>explain biasvariance tradeoff</td>\n",
              "      <td>biasvariance tradeoff is a concept in machine ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3771</th>\n",
              "      <td>what is stochastic gradient descent sgd in mac...</td>\n",
              "      <td>stochastic gradient descent sgd is an optimiza...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3772</th>\n",
              "      <td>explain stochastic gradient descent</td>\n",
              "      <td>stochastic gradient descent sgd is an optimiza...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3773</th>\n",
              "      <td>what is the backpropagation algorithm in machi...</td>\n",
              "      <td>the backpropagation algorithm is a widely used...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3774</th>\n",
              "      <td>explain backpropagation algorithm</td>\n",
              "      <td>the backpropagation algorithm is a widely used...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3775 rows × 2 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ffb3f238-1964-4f27-8902-218b57a9a9b4')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-ffb3f238-1964-4f27-8902-218b57a9a9b4 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-ffb3f238-1964-4f27-8902-218b57a9a9b4');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-7029e08a-bc98-480a-8fd0-61b203922c8a\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7029e08a-bc98-480a-8fd0-61b203922c8a')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-7029e08a-bc98-480a-8fd0-61b203922c8a button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "  <div id=\"id_273a8973-6961-4a58-ac93-19df6fc0975c\">\n",
              "    <style>\n",
              "      .colab-df-generate {\n",
              "        background-color: #E8F0FE;\n",
              "        border: none;\n",
              "        border-radius: 50%;\n",
              "        cursor: pointer;\n",
              "        display: none;\n",
              "        fill: #1967D2;\n",
              "        height: 32px;\n",
              "        padding: 0 0 0 0;\n",
              "        width: 32px;\n",
              "      }\n",
              "\n",
              "      .colab-df-generate:hover {\n",
              "        background-color: #E2EBFA;\n",
              "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "        fill: #174EA6;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate {\n",
              "        background-color: #3B4455;\n",
              "        fill: #D2E3FC;\n",
              "      }\n",
              "\n",
              "      [theme=dark] .colab-df-generate:hover {\n",
              "        background-color: #434B5C;\n",
              "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "        fill: #FFFFFF;\n",
              "      }\n",
              "    </style>\n",
              "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('df')\"\n",
              "            title=\"Generate code using this dataframe.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "    <script>\n",
              "      (() => {\n",
              "      const buttonEl =\n",
              "        document.querySelector('#id_273a8973-6961-4a58-ac93-19df6fc0975c button.colab-df-generate');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      buttonEl.onclick = () => {\n",
              "        google.colab.notebook.generateWithVariable('df');\n",
              "      }\n",
              "      })();\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                               Question  \\\n",
              "0                    how does randomised algorithm work   \n",
              "1                  what do you mean by bestfirst search   \n",
              "2                           how do you explain a daemon   \n",
              "3                            what is phonetic algorithm   \n",
              "4                what do you mean by uniform costsearch   \n",
              "...                                                 ...   \n",
              "3770                      explain biasvariance tradeoff   \n",
              "3771  what is stochastic gradient descent sgd in mac...   \n",
              "3772                explain stochastic gradient descent   \n",
              "3773  what is the backpropagation algorithm in machi...   \n",
              "3774                  explain backpropagation algorithm   \n",
              "\n",
              "                                                 Answer  \n",
              "0     the algorithm typically uses uniformly random ...  \n",
              "1     bestfirst search is a search algorithm which e...  \n",
              "2     daemon disk and execution monitor is a process...  \n",
              "3     a phonetic algorithm is an algorithm for index...  \n",
              "4     a tree search that finds the lowestcost route ...  \n",
              "...                                                 ...  \n",
              "3770  biasvariance tradeoff is a concept in machine ...  \n",
              "3771  stochastic gradient descent sgd is an optimiza...  \n",
              "3772  stochastic gradient descent sgd is an optimiza...  \n",
              "3773  the backpropagation algorithm is a widely used...  \n",
              "3774  the backpropagation algorithm is a widely used...  \n",
              "\n",
              "[3775 rows x 2 columns]"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "# read from csv\n",
        "df = pd.read_csv(os.path.join(FOLDER_PATH, 'combined_data.csv'))\n",
        "df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tsHfppjCpu-a",
        "outputId": "e5a6e854-ced7-442d-f8d6-4778e42027f3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(3775,\n",
              " ['how does randomised algorithm work',\n",
              "  'the algorithm typically uses uniformly random bits as an auxiliary input to guide its behavior in the hope of achieving good performance in the average case over all possible choices of random bits'])"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# convert the df to 2d list\n",
        "qa_pairs = df.values.tolist()\n",
        "len(qa_pairs), qa_pairs[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-NASuplC3w-N"
      },
      "source": [
        "### Functions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pY5GHdd0Tovm"
      },
      "outputs": [],
      "source": [
        "# Default word tokens\n",
        "PAD_token = 0  # Used for padding short sentences\n",
        "SOS_token = 1  # Start-of-sentence token\n",
        "EOS_token = 2  # End-of-sentence token\n",
        "UNK_token = 3  # Unknown word token\n",
        "\n",
        "class Voc:\n",
        "    def __init__(self, name):\n",
        "        self.name = name\n",
        "        self.trimmed = False\n",
        "        self.word2index = {}\n",
        "        self.word2count = {}\n",
        "        self.index2word = {PAD_token: \"PAD\", SOS_token: \"SOS\", EOS_token: \"EOS\", UNK_token: \"UNK\"}\n",
        "        self.num_words = 4  # Count SOS, EOS, PAD, UNK\n",
        "\n",
        "    def addSentence(self, sentence):\n",
        "        for word in sentence.split(' '):\n",
        "            self.addWord(word)\n",
        "\n",
        "    def addWord(self, word):\n",
        "        if word not in self.word2index:\n",
        "            self.word2index[word] = self.num_words\n",
        "            self.word2count[word] = 1\n",
        "            self.index2word[self.num_words] = word\n",
        "            self.num_words += 1\n",
        "        else:\n",
        "            self.word2count[word] += 1\n",
        "\n",
        "    # Remove words below a certain count threshold\n",
        "    def trim(self, min_count):\n",
        "        if self.trimmed:\n",
        "            return\n",
        "        self.trimmed = True\n",
        "\n",
        "        keep_words = []\n",
        "\n",
        "        for k, v in self.word2count.items():\n",
        "            if v >= min_count:\n",
        "                keep_words.append(k)\n",
        "\n",
        "        print('keep_words {} / {} = {:.4f}'.format(\n",
        "            len(keep_words), len(self.word2index), len(keep_words) / len(self.word2index)\n",
        "        ))\n",
        "\n",
        "        # Reinitialize dictionaries\n",
        "        self.word2index = {}\n",
        "        self.word2count = {}\n",
        "        self.index2word = {PAD_token: \"PAD\", SOS_token: \"SOS\", EOS_token: \"EOS\", UNK_token: \"UNK\"}\n",
        "        self.num_words = 4 # Count default tokens\n",
        "\n",
        "        for word in keep_words:\n",
        "            self.addWord(word)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SEwUmgIBUn_1"
      },
      "outputs": [],
      "source": [
        "# Turn a Unicode string to plain ASCII, thanks to\n",
        "# http://stackoverflow.com/a/518232/2809427\n",
        "def unicodeToAscii(s):\n",
        "    return ''.join(\n",
        "        c for c in unicodedata.normalize('NFD', s)\n",
        "        if unicodedata.category(c) != 'Mn'\n",
        "    )\n",
        "\n",
        "# Lowercase, trim, and remove non-letter characters\n",
        "def normalizeString(s):\n",
        "    s = unicodeToAscii(s.lower().strip())\n",
        "    s = re.sub(r\"([.!?])\", r\" \\1\", s)\n",
        "    s = re.sub(r\"[^a-zA-Z.!?]+\", r\" \", s)\n",
        "    s = re.sub(r\"\\s+\", r\" \", s).strip()\n",
        "    return s\n",
        "\n",
        "# Returns True iff both sentences in a pair 'p' are under the MAX_LENGTH threshold\n",
        "def filterPair(p, max_length):\n",
        "    # Input sequences need to preserve the last word for EOS token\n",
        "    return len(p[0].split(' ')) < max_length and len(p[1].split(' ')) < max_length\n",
        "\n",
        "# Filter pairs using filterPair condition\n",
        "def filterPairs(pairs, max_length):\n",
        "    return [pair for pair in pairs if filterPair(pair, max_length)]\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nVcfYDvrqYDg",
        "outputId": "ae7108d0-55a0-4acf-e725-db2d842ba290"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "before filtering, no. of qa pairs: 3775\n",
            "after filtering, no. of qa pairs: 3145\n"
          ]
        }
      ],
      "source": [
        "voc = Voc(\"qa\")\n",
        "MAX_LENGTH = 50  # Maximum sentence length to consider\n",
        "\n",
        "# filter pairs based on MAX_LENGTH\n",
        "print(f'before filtering, no. of qa pairs: {len(qa_pairs)}')\n",
        "qa_pairs = filterPairs(qa_pairs, MAX_LENGTH)\n",
        "print(f'after filtering, no. of qa pairs: {len(qa_pairs)}')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zbcg0ggv41Hs"
      },
      "outputs": [],
      "source": [
        "# normalise each qa pair and add word from question and answer into vocab\n",
        "for i in range(len(qa_pairs)):\n",
        "    question, answer = qa_pairs[i]\n",
        "    question = normalizeString(question)\n",
        "    answer = normalizeString(answer)\n",
        "    voc.addSentence(question)\n",
        "    voc.addSentence(answer)\n",
        "    qa_pairs[i] = [question, answer]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v5i-L5GjTo0Q",
        "outputId": "c4b0a600-7019-4cd0-d71c-e8d54f2d7f77"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "4980"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "vocab_size = voc.num_words # words + 3 tokens (PAD, SOS, EOS)\n",
        "vocab_size"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U4LwPRCGVJ6A",
        "outputId": "1082fcf0-620c-46ea-f4c2-4525305e3858"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[['how does randomised algorithm work',\n",
              "  'the algorithm typically uses uniformly random bits as an auxiliary input to guide its behavior in the hope of achieving good performance in the average case over all possible choices of random bits'],\n",
              " ['what do you mean by bestfirst search',\n",
              "  'bestfirst search is a search algorithm which explores a graph by expanding the most promising node chosen according to a specified rule'],\n",
              " ['how do you explain a daemon',\n",
              "  'daemon disk and execution monitor is a process that runs in the background without users interaction they usually start at the booting time and terminate when the system is shut down'],\n",
              " ['what is phonetic algorithm',\n",
              "  'a phonetic algorithm is an algorithm for indexing of words by their pronunciation'],\n",
              " ['what do you mean by uniform costsearch',\n",
              "  'a tree search that finds the lowestcost route where costs vary']]"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "qa_pairs[:5]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bNTjxk94OEKw",
        "outputId": "fe58b6ff-162c-4323-bd5f-d1fa81de2124"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(3145, 3145)"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "questions = [pair[0] for pair in qa_pairs]\n",
        "answers = [pair[1] for pair in qa_pairs]\n",
        "len(questions), len(answers)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3a-zptK6DXDJ"
      },
      "source": [
        "## Transformer Model Architecture"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xgTB9JENDWdI",
        "outputId": "13cc09a9-75cf-421c-cae7-3374753f1209"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.46.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers) (3.16.1)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.26.2)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (1.26.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers) (24.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (6.0.2)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2024.9.11)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers) (2.32.3)\n",
            "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.5)\n",
            "Requirement already satisfied: tokenizers<0.21,>=0.20 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.20.3)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers) (4.66.6)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.2->transformers) (2024.10.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.2->transformers) (4.12.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.4.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2.2.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers) (2024.8.30)\n"
          ]
        }
      ],
      "source": [
        "!pip install transformers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7rGUNAQ_JCBx"
      },
      "outputs": [],
      "source": [
        "from transformers import GPT2LMHeadModel, GPT2TokenizerFast\n",
        "import torch\n",
        "\n",
        "class GPT2QA:\n",
        "    \"\"\"GPT-2 Model for Question Answering.\"\"\"\n",
        "\n",
        "    def __init__(self, model_name=\"gpt2\"):\n",
        "        self.tokenizer = GPT2TokenizerFast.from_pretrained(model_name)\n",
        "        self.model = GPT2LMHeadModel.from_pretrained(model_name)\n",
        "        self.device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "        self.model.to(self.device)\n",
        "\n",
        "    def generate_response(model, input_text):\n",
        "        \"\"\"Generate a response using the GPT-2 model with appropriate configurations.\"\"\"\n",
        "        inputs = model.tokenizer.encode(input_text, return_tensors=\"pt\").to(model.device)\n",
        "\n",
        "        # Generate response with the appropriate configurations\n",
        "        outputs = model.model.generate(\n",
        "            inputs,\n",
        "            max_length=150,\n",
        "            do_sample=True,  # Enable sampling for using top_p\n",
        "            top_k=50,\n",
        "            top_p=0.95,\n",
        "            num_beams=2,  # Use beam search to leverage early stopping\n",
        "            early_stopping=True,\n",
        "            pad_token_id=model.tokenizer.eos_token_id  # Set padding token id\n",
        "        )\n",
        "\n",
        "        response = model.tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
        "        return response.strip()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SFjeoyzLfRmz"
      },
      "source": [
        "### Training\n",
        "\n",
        "Hyperparameters were adjusted based on computational resource limits."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fSOQg14rXdOV"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from torch.optim import AdamW\n",
        "import time\n",
        "\n",
        "class QADataset(Dataset):\n",
        "    \"\"\"Dataset class for question-answer pairs.\"\"\"\n",
        "\n",
        "    def __init__(self, qa_pairs, tokenizer, max_length=50):\n",
        "        self.qa_pairs = qa_pairs\n",
        "        self.tokenizer = tokenizer\n",
        "        self.max_length = max_length\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.qa_pairs)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        question, answer = self.qa_pairs[idx]\n",
        "        inputs = self.tokenizer.encode_plus(\n",
        "            question,\n",
        "            answer,\n",
        "            add_special_tokens=True,\n",
        "            max_length=self.max_length,\n",
        "            padding='max_length',\n",
        "            truncation=True,\n",
        "            return_tensors='pt'\n",
        "        )\n",
        "        return {\n",
        "            'input_ids': inputs['input_ids'].squeeze(),\n",
        "            'attention_mask': inputs['attention_mask'].squeeze(),\n",
        "            'labels': inputs['input_ids'].squeeze()  # For supervised training, labels are the same as input_ids\n",
        "        }\n",
        "\n",
        "def train_model(qa_pairs, model, batch_size=8, num_epochs=3):\n",
        "    \"\"\"Train the GPT-2 model on the question-answer pairs.\"\"\"\n",
        "\n",
        "    tokenizer = model.tokenizer\n",
        "    dataset = QADataset(qa_pairs, tokenizer)\n",
        "    dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "    optimizer = AdamW(model.model.parameters(), lr=5e-5)\n",
        "\n",
        "    model.model.train()\n",
        "    for epoch in range(num_epochs):\n",
        "        total_loss = 0\n",
        "        for batch in dataloader:\n",
        "            optimizer.zero_grad()\n",
        "\n",
        "            input_ids = batch['input_ids'].to(model.device)\n",
        "            attention_mask = batch['attention_mask'].to(model.device)\n",
        "            labels = batch['labels'].to(model.device)\n",
        "\n",
        "            # Start timing for total training time\n",
        "            start_time = time.time()\n",
        "\n",
        "            # Forward pass\n",
        "            outputs = model.model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n",
        "            loss = outputs.loss\n",
        "\n",
        "            # Backward pass\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            total_loss += loss.item()\n",
        "\n",
        "            # End timing for current batch\n",
        "            end_time = time.time()\n",
        "            print(f\"Time taken for batch: {end_time - start_time:.4f} seconds\")\n",
        "\n",
        "        # Average loss for the epoch\n",
        "        avg_loss = total_loss / len(dataloader)\n",
        "        print(f\"Epoch {epoch + 1}/{num_epochs}, Average Loss: {avg_loss:.4f}\")\n",
        "\n",
        "    # Save the fine-tuned model\n",
        "    model.model.save_pretrained(\"fine_tuned_gpt2\")\n",
        "    model.tokenizer.save_pretrained(\"fine_tuned_gpt2\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 6126,
          "referenced_widgets": [
            "011fb883f7774528b22fa0b34a9b3e40",
            "2f38f15ce1b14e58a366b8ca21aead8f",
            "a453b9c07c7b479e81adadf2dc2e7098",
            "7c2a0ed083ec4d5884caff6764d9b149",
            "736df333921f4513ac4866c204b98884",
            "851fe6f3404846cb99e1bf1db0c7142b",
            "e952eba68ece4c6b864f438d8d42911d",
            "92d2ae5a3f354d02a0e9afeac5a1676a",
            "5e196603a4c14a50a051a122904ac089",
            "ad59ce55998b4b8ea451ab422c9d5dc2",
            "eb7419cdc0a748549ee56223ae667acc",
            "1000a5f574904ea181cb42cc7a423b28",
            "e37527644fb14daa9ba20129acbdd943",
            "5455a53eb3e14f55a771241306134200",
            "6c4d98c7d2d94b03a3ccf464c9d646f9",
            "44838c56d35e48c3a0344f83434e64a2",
            "b091c384f23743bbb7eb578ecde09bb4",
            "b75d47c7f07e47f1972112842f25a565",
            "93192a852192454580ec57ca901f748f",
            "949099bccb9c4f298ed868eb177dc73f",
            "bb28b0289c9748c4ae6b217edc631b86",
            "f5ca910ea9334267b3924c2cd8c559b8",
            "f36515bfd5e44dffa6a69f778be80772",
            "9992fa1f43494fc2bbe474773903b78a",
            "dedd003de05d4bdb9033a0b1b40a1bcc",
            "779dd1b4114d4a2c96de5d304a4febf6",
            "0b0772747f364fec9ea5cee6082bb697",
            "dc28651be51745eca94664bbb71ad861",
            "bd2c8c995a8e4bc4a5d4fbf122e37740",
            "4f0cfadfd255445dae90afcf10c3d751",
            "e5d1c331340e4381be2a8d5ed25fdf60",
            "6cd737c50b84495aa6381d17d5e001da",
            "e6852d226c07498ba6519985b74f7ab7",
            "ca842850c6704e0093c229357acee9f0",
            "03eb561d1f63450f900cd8ec8677ce80",
            "dce048c43a4348c4a09aa2b46a33f1f3",
            "aadb3c07f7d1487188baa1285a40a420",
            "c5260793c0f44875a435918454139e68",
            "2bc8bca1a528472e96949584c08dcd4d",
            "71529b0d406d44168144727ba51e77ee",
            "c95f26ac0139443eba7f2dd7614bc817",
            "45cd7100b77a44f8b6181740e20f42fd",
            "a7b3720526a04006859104ed7d01633a",
            "3d4fa04a46ac4ca587c84a3e27fb3ebc",
            "9ac178dcf5b74427825a35c9daf00020",
            "4c287a64e0404f24aad86f868b2db58c",
            "954e8e5c89844fa2b03b2baf1c9ee25e",
            "58fa86de43b749e09a698b8cf7398dab",
            "9c2a57d2ad0e424bbad9196ce659c3cd",
            "5943e25a3559420abdab0272af46edd1",
            "df5b52e7d83b454585333790d844237f",
            "efc83d30cd024a158ada903f0eb2836d",
            "c7a156eb0fa3426ebe21fa855fdf47fe",
            "a77884af85264f80bab7781bc38b9d40",
            "a64f38afb26746b6b62c8209a41d8d2f",
            "f942289393664f5d98ebbc7686d6261b",
            "c7884273ab104cfb822965ee145dd879",
            "c70b53da90834f7aa84b7927c1864b4a",
            "5bbdc5dae16446e5b9024e533b7ae5ca",
            "37fe86c55e574d608d21a382c94bbb03",
            "b503f0b42c82409697e5bd05a3a7fc15",
            "aee261858dd24782a41d7978082c7ab3",
            "467b9e1ca277401da2d2fdbdaa65a669",
            "d8e3030d673a428895a73342c0b9c598",
            "25df876373a14c19b5800bbd024a098d",
            "a11c25686f6845bf8ba8b30da865e7fa",
            "6a16330e3ec24fa98f9e906c9df21d25",
            "cafb3801fa764d23b6a65bee141e044c",
            "ff3a59cf02b5436cb980e3191da0c2fb",
            "1783f66ad7cf482b8bcdf548eff924a1",
            "176b5aed89224adead7172e95a311409",
            "53aeda84c2ab4ad091751016af363a53",
            "ebdd95042703441da39390f258891d49",
            "de0547f1e48444d2bcfad75b1cf77e26",
            "bb7f7cb49f5449248fe260f40f9cfc70",
            "d41eb2e1e05444e8a181b71559ae6112",
            "20746af24ae94d85a8f0ee29868da5fc"
          ]
        },
        "id": "6VG7bg-CKVCY",
        "outputId": "b852545a-13ff-4bfc-e8a2-9142561d61f8"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "011fb883f7774528b22fa0b34a9b3e40",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/26.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1000a5f574904ea181cb42cc7a423b28",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f36515bfd5e44dffa6a69f778be80772",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ca842850c6704e0093c229357acee9f0",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9ac178dcf5b74427825a35c9daf00020",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f942289393664f5d98ebbc7686d6261b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "6a16330e3ec24fa98f9e906c9df21d25",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "The new embeddings will be initialized from a multivariate normal distribution that has old embeddings' mean and covariance. As described in this article: https://nlp.stanford.edu/~johnhew/vocab-expansion.html. To disable this, use `mean_resizing=False`\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Time taken for batch: 1.4572 seconds\n",
            "Time taken for batch: 0.1630 seconds\n",
            "Time taken for batch: 0.1539 seconds\n",
            "Time taken for batch: 0.1367 seconds\n",
            "Time taken for batch: 0.1358 seconds\n",
            "Time taken for batch: 0.1297 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1360 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1358 seconds\n",
            "Time taken for batch: 0.1359 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1344 seconds\n",
            "Time taken for batch: 0.1365 seconds\n",
            "Time taken for batch: 0.1355 seconds\n",
            "Time taken for batch: 0.1357 seconds\n",
            "Time taken for batch: 0.1346 seconds\n",
            "Time taken for batch: 0.1392 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1355 seconds\n",
            "Time taken for batch: 0.1376 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1362 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1367 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1380 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1374 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1389 seconds\n",
            "Time taken for batch: 0.1363 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1354 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1355 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1370 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1354 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1354 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1376 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1407 seconds\n",
            "Time taken for batch: 0.1356 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1389 seconds\n",
            "Time taken for batch: 0.1358 seconds\n",
            "Time taken for batch: 0.1374 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1375 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1380 seconds\n",
            "Time taken for batch: 0.1356 seconds\n",
            "Time taken for batch: 0.1376 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1405 seconds\n",
            "Time taken for batch: 0.1358 seconds\n",
            "Time taken for batch: 0.1417 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1390 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1375 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1367 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1393 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1354 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1378 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1397 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1391 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1357 seconds\n",
            "Time taken for batch: 0.1357 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1356 seconds\n",
            "Time taken for batch: 0.1399 seconds\n",
            "Time taken for batch: 0.1358 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1359 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1360 seconds\n",
            "Time taken for batch: 0.1312 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1413 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1376 seconds\n",
            "Time taken for batch: 0.1389 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1367 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1380 seconds\n",
            "Time taken for batch: 0.1390 seconds\n",
            "Time taken for batch: 0.1355 seconds\n",
            "Time taken for batch: 0.1367 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1376 seconds\n",
            "Time taken for batch: 0.1360 seconds\n",
            "Time taken for batch: 0.1369 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1360 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1370 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1351 seconds\n",
            "Time taken for batch: 0.1397 seconds\n",
            "Time taken for batch: 0.1397 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1365 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1396 seconds\n",
            "Time taken for batch: 0.1371 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1378 seconds\n",
            "Time taken for batch: 0.1380 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1379 seconds\n",
            "Time taken for batch: 0.1378 seconds\n",
            "Time taken for batch: 0.1377 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1416 seconds\n",
            "Time taken for batch: 0.1424 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1378 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1402 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1412 seconds\n",
            "Time taken for batch: 0.1387 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1392 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1412 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1392 seconds\n",
            "Time taken for batch: 0.1401 seconds\n",
            "Time taken for batch: 0.1401 seconds\n",
            "Time taken for batch: 0.1361 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1402 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1412 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1373 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1391 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1382 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1397 seconds\n",
            "Time taken for batch: 0.1355 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1394 seconds\n",
            "Time taken for batch: 0.1391 seconds\n",
            "Time taken for batch: 0.1398 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1405 seconds\n",
            "Time taken for batch: 0.1405 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1392 seconds\n",
            "Time taken for batch: 0.1413 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1393 seconds\n",
            "Time taken for batch: 0.1383 seconds\n",
            "Time taken for batch: 0.1393 seconds\n",
            "Time taken for batch: 0.1388 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1390 seconds\n",
            "Time taken for batch: 0.1392 seconds\n",
            "Time taken for batch: 0.1407 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1366 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1393 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1386 seconds\n",
            "Time taken for batch: 0.1402 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1398 seconds\n",
            "Time taken for batch: 0.1384 seconds\n",
            "Time taken for batch: 0.1407 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1401 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1370 seconds\n",
            "Time taken for batch: 0.1396 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1406 seconds\n",
            "Time taken for batch: 0.1397 seconds\n",
            "Time taken for batch: 0.1399 seconds\n",
            "Time taken for batch: 0.1399 seconds\n",
            "Time taken for batch: 0.1424 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1417 seconds\n",
            "Time taken for batch: 0.1385 seconds\n",
            "Time taken for batch: 0.1391 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1427 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1372 seconds\n",
            "Time taken for batch: 0.1398 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1393 seconds\n",
            "Time taken for batch: 0.1401 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1423 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1427 seconds\n",
            "Time taken for batch: 0.1406 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1412 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1420 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1395 seconds\n",
            "Time taken for batch: 0.1401 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1428 seconds\n",
            "Time taken for batch: 0.1423 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1391 seconds\n",
            "Time taken for batch: 0.1417 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1398 seconds\n",
            "Time taken for batch: 0.1406 seconds\n",
            "Time taken for batch: 0.1425 seconds\n",
            "Time taken for batch: 0.1396 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1428 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1381 seconds\n",
            "Time taken for batch: 0.1341 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1404 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1349 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1421 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1419 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1405 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1402 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1417 seconds\n",
            "Time taken for batch: 0.1399 seconds\n",
            "Time taken for batch: 0.1416 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1421 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1408 seconds\n",
            "Time taken for batch: 0.1428 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1403 seconds\n",
            "Time taken for batch: 0.1406 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1427 seconds\n",
            "Time taken for batch: 0.1398 seconds\n",
            "Time taken for batch: 0.1428 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1425 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1420 seconds\n",
            "Time taken for batch: 0.1429 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1405 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1416 seconds\n",
            "Time taken for batch: 0.1421 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1413 seconds\n",
            "Time taken for batch: 0.1412 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1409 seconds\n",
            "Time taken for batch: 0.1420 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1407 seconds\n",
            "Time taken for batch: 0.1420 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1416 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1425 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1400 seconds\n",
            "Time taken for batch: 0.1417 seconds\n",
            "Time taken for batch: 0.1420 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1413 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1415 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1414 seconds\n",
            "Time taken for batch: 0.1418 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1425 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1411 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1476 seconds\n",
            "Time taken for batch: 0.1368 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1410 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1429 seconds\n",
            "Time taken for batch: 0.1312 seconds\n",
            "Epoch 1/3, Average Loss: 2.9267\n",
            "Time taken for batch: 0.1422 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1429 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1428 seconds\n",
            "Time taken for batch: 0.1426 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1427 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1424 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1517 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1531 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1476 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1514 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1517 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1512 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1526 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1525 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1528 seconds\n",
            "Time taken for batch: 0.1514 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1524 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1521 seconds\n",
            "Time taken for batch: 0.1524 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1521 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1519 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1530 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1530 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1514 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1522 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1526 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1538 seconds\n",
            "Time taken for batch: 0.1514 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1521 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1525 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1523 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1512 seconds\n",
            "Time taken for batch: 0.1514 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1522 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1530 seconds\n",
            "Time taken for batch: 0.1520 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1529 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1522 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1503 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1503 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1433 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1336 seconds\n",
            "Epoch 2/3, Average Loss: 1.7645\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1476 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1433 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1429 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1427 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1434 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1437 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1431 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1503 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1433 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1441 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1436 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1430 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1483 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1476 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1487 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1447 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1438 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1476 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1494 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1486 seconds\n",
            "Time taken for batch: 0.1489 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1496 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1513 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1449 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1515 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1503 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1508 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1482 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1475 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1484 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1470 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1473 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1471 seconds\n",
            "Time taken for batch: 0.1490 seconds\n",
            "Time taken for batch: 0.1485 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1491 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1454 seconds\n",
            "Time taken for batch: 0.1463 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1479 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1480 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1512 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1521 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1503 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1464 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1443 seconds\n",
            "Time taken for batch: 0.1450 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1466 seconds\n",
            "Time taken for batch: 0.1495 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1452 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1459 seconds\n",
            "Time taken for batch: 0.1512 seconds\n",
            "Time taken for batch: 0.1448 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1456 seconds\n",
            "Time taken for batch: 0.1518 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1516 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1511 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1468 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1488 seconds\n",
            "Time taken for batch: 0.1453 seconds\n",
            "Time taken for batch: 0.1501 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1507 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1460 seconds\n",
            "Time taken for batch: 0.1477 seconds\n",
            "Time taken for batch: 0.1467 seconds\n",
            "Time taken for batch: 0.1498 seconds\n",
            "Time taken for batch: 0.1474 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1509 seconds\n",
            "Time taken for batch: 0.1465 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1451 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1510 seconds\n",
            "Time taken for batch: 0.1444 seconds\n",
            "Time taken for batch: 0.1505 seconds\n",
            "Time taken for batch: 0.1461 seconds\n",
            "Time taken for batch: 0.1499 seconds\n",
            "Time taken for batch: 0.1442 seconds\n",
            "Time taken for batch: 0.1502 seconds\n",
            "Time taken for batch: 0.1435 seconds\n",
            "Time taken for batch: 0.1478 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1500 seconds\n",
            "Time taken for batch: 0.1440 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1458 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1457 seconds\n",
            "Time taken for batch: 0.1492 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1446 seconds\n",
            "Time taken for batch: 0.1493 seconds\n",
            "Time taken for batch: 0.1432 seconds\n",
            "Time taken for batch: 0.1462 seconds\n",
            "Time taken for batch: 0.1481 seconds\n",
            "Time taken for batch: 0.1455 seconds\n",
            "Time taken for batch: 0.1504 seconds\n",
            "Time taken for batch: 0.1472 seconds\n",
            "Time taken for batch: 0.1506 seconds\n",
            "Time taken for batch: 0.1469 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1445 seconds\n",
            "Time taken for batch: 0.1497 seconds\n",
            "Time taken for batch: 0.1439 seconds\n",
            "Time taken for batch: 0.1320 seconds\n",
            "Epoch 3/3, Average Loss: 1.3269\n"
          ]
        }
      ],
      "source": [
        "import pandas as pd\n",
        "from torch.utils.data import DataLoader\n",
        "from torch.optim import AdamW\n",
        "import time\n",
        "\n",
        "def load_data(file_path):\n",
        "    \"\"\"Load and preprocess the QA dataset.\"\"\"\n",
        "    df = pd.read_csv(file_path)\n",
        "    qa_pairs = df.values.tolist()\n",
        "    # Normalize each pair\n",
        "    qa_pairs = [(q.lower().strip(), a.lower().strip()) for q, a in qa_pairs]\n",
        "    return qa_pairs\n",
        "\n",
        "def main():\n",
        "    # Load the QA data\n",
        "    file_path = '/content/drive/My Drive/Colab Notebooks/nlc/project/combined_data.csv'\n",
        "    qa_pairs = load_data(file_path)\n",
        "\n",
        "    # Initialize the model\n",
        "    gpt2_qa_model = GPT2QA()\n",
        "\n",
        "    # Add a padding token if not already set\n",
        "    if gpt2_qa_model.tokenizer.pad_token is None:\n",
        "        gpt2_qa_model.tokenizer.add_special_tokens({'pad_token': '[PAD]'})\n",
        "        gpt2_qa_model.model.resize_token_embeddings(len(gpt2_qa_model.tokenizer))\n",
        "\n",
        "    # Create DataLoader\n",
        "    dataset = QADataset(qa_pairs, gpt2_qa_model.tokenizer)\n",
        "    dataloader = DataLoader(dataset, batch_size=8, shuffle=True)\n",
        "\n",
        "    # Set up the optimizer\n",
        "    optimizer = AdamW(gpt2_qa_model.model.parameters(), lr=5e-5)\n",
        "\n",
        "    # Training loop\n",
        "    gpt2_qa_model.model.train()  # Ensure model is in training mode\n",
        "    num_epochs = 3\n",
        "\n",
        "    for epoch in range(num_epochs):\n",
        "        total_loss = 0\n",
        "        start_epoch_time = time.time()  # Track epoch time\n",
        "\n",
        "        for batch in dataloader:\n",
        "            optimizer.zero_grad()  # Reset gradients\n",
        "\n",
        "            input_ids = batch['input_ids'].to(gpt2_qa_model.device)\n",
        "            attention_mask = batch['attention_mask'].to(gpt2_qa_model.device)\n",
        "            labels = batch['labels'].to(gpt2_qa_model.device)\n",
        "\n",
        "            # Start timing for total training time\n",
        "            start_time = time.time()\n",
        "\n",
        "            # Forward pass\n",
        "            outputs = gpt2_qa_model.model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n",
        "            loss = outputs.loss\n",
        "\n",
        "            # Backward pass\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            total_loss += loss.item()\n",
        "\n",
        "            # End timing for current batch\n",
        "            end_time = time.time()\n",
        "            print(f\"Time taken for batch: {end_time - start_time:.4f} seconds\")\n",
        "\n",
        "        # Average loss for the epoch\n",
        "        avg_loss = total_loss / len(dataloader)\n",
        "        print(f\"Epoch {epoch + 1}/{num_epochs}, Average Loss: {avg_loss:.4f}\")\n",
        "\n",
        "    # Save the fine-tuned model\n",
        "    gpt2_qa_model.model.save_pretrained(\"fine_tuned_gpt2\")\n",
        "    gpt2_qa_model.tokenizer.save_pretrained(\"fine_tuned_gpt2\")\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "03W05HyA5Sxh"
      },
      "source": [
        "### Fine-tuning\n",
        "\n",
        "A custom dataset class, CustomQADataset, prepares the input sequences by tokenizing and encoding the combined questions and answers, ensuring compatibility with GPT-2’s requirements. The fine_tune_gpt2 function manages the model training process, utilising an AdamW optimizer to minimise loss over a specified number of epochs. After training, the fine-tuned model and tokenizer are saved locally and to Google Drive, allowing for future deployment in answering user queries."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rSRA2k_pYL4-",
        "outputId": "a9b5a4c0-cd60-4a82-f9dc-9f486434e171"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:591: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1/3, Average Loss: 1.2321\n",
            "Epoch 2/3, Average Loss: 0.8862\n",
            "Epoch 3/3, Average Loss: 0.6758\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "from transformers import GPT2LMHeadModel, GPT2TokenizerFast, AdamW\n",
        "\n",
        "class CustomQADataset(Dataset):\n",
        "    def __init__(self, qa_pairs, tokenizer, max_length=100):\n",
        "        self.qa_pairs = qa_pairs\n",
        "        self.tokenizer = tokenizer\n",
        "        self.max_length = max_length\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.qa_pairs)\n",
        "\n",
        "    def __getitem__(self, idx):\n",
        "        question, answer = self.qa_pairs[idx]\n",
        "        text = f\"Question: {question} Answer: {answer}\"\n",
        "        inputs = self.tokenizer.encode_plus(\n",
        "            text,\n",
        "            add_special_tokens=True,\n",
        "            max_length=self.max_length,\n",
        "            padding='max_length',\n",
        "            truncation=True,\n",
        "            return_tensors='pt'\n",
        "        )\n",
        "        return {\n",
        "            'input_ids': inputs['input_ids'].squeeze(),\n",
        "            'attention_mask': inputs['attention_mask'].squeeze(),\n",
        "            'labels': inputs['input_ids'].squeeze()\n",
        "        }\n",
        "\n",
        "def fine_tune_gpt2(qa_pairs, model_name=\"gpt2\", num_epochs=3, batch_size=8, learning_rate=5e-5):\n",
        "    tokenizer = GPT2TokenizerFast.from_pretrained(model_name)\n",
        "    model = GPT2LMHeadModel.from_pretrained(model_name)\n",
        "\n",
        "    # Set the padding token\n",
        "    tokenizer.pad_token = tokenizer.eos_token\n",
        "\n",
        "    # Save paths in Google Drive\n",
        "    save_path_pretrained = '/content/drive/MyDrive/Colab Notebooks/nlc/project/models/pretrained_gpt2'\n",
        "    save_path_finetuned = '/content/drive/MyDrive/Colab Notebooks/nlc/project/models/fine_tuned_gpt2'\n",
        "    # Create directories if they don't exist\n",
        "    os.makedirs(save_path_pretrained, exist_ok=True)\n",
        "    os.makedirs(save_path_finetuned, exist_ok=True)\n",
        "\n",
        "    # Save the pre-trained model to Google Drive\n",
        "    model.save_pretrained(save_path_pretrained)\n",
        "    tokenizer.save_pretrained(save_path_pretrained)\n",
        "\n",
        "    # Prepare dataset and dataloader\n",
        "    dataset = CustomQADataset(qa_pairs, tokenizer)\n",
        "    dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)\n",
        "\n",
        "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "    model.to(device)\n",
        "\n",
        "    # Set up the optimizer\n",
        "    optimizer = AdamW(model.parameters(), lr=learning_rate)\n",
        "\n",
        "    # Training loop\n",
        "    model.train()\n",
        "    for epoch in range(num_epochs):\n",
        "        total_loss = 0\n",
        "\n",
        "        for batch in dataloader:\n",
        "            optimizer.zero_grad()\n",
        "\n",
        "            input_ids = batch['input_ids'].to(device)\n",
        "            attention_mask = batch['attention_mask'].to(device)\n",
        "            labels = batch['labels'].to(device)\n",
        "\n",
        "            outputs = model(input_ids=input_ids, attention_mask=attention_mask, labels=labels)\n",
        "            loss = outputs.loss\n",
        "\n",
        "            loss.backward()\n",
        "            optimizer.step()\n",
        "\n",
        "            total_loss += loss.item()\n",
        "\n",
        "        avg_loss = total_loss / len(dataloader)\n",
        "        print(f\"Epoch {epoch + 1}/{num_epochs}, Average Loss: {avg_loss:.4f}\")\n",
        "\n",
        "    # Save the fine-tuned model\n",
        "    model.save_pretrained(\"fine_tuned_gpt2\")\n",
        "    tokenizer.save_pretrained(\"fine_tuned_gpt2\")\n",
        "\n",
        "    # Save the fine-tuned model to Google Drive\n",
        "    model.save_pretrained(save_path_finetuned)\n",
        "    tokenizer.save_pretrained(save_path_finetuned)\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    fine_tune_gpt2(qa_pairs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YwF6hNKycetd"
      },
      "outputs": [],
      "source": [
        "from transformers import GPT2LMHeadModel, GPT2TokenizerFast\n",
        "\n",
        "def load_fine_tuned_model(model_dir=\"fine_tuned_gpt2\"):\n",
        "    \"\"\"Load the fine-tuned model and tokenizer.\"\"\"\n",
        "    tokenizer = GPT2TokenizerFast.from_pretrained(model_dir)\n",
        "    model = GPT2LMHeadModel.from_pretrained(model_dir)\n",
        "    model.to(torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\"))\n",
        "\n",
        "    return model, tokenizer\n",
        "\n",
        "def generate_response(model, tokenizer, input_text):\n",
        "    \"\"\"Generate a response using the fine-tuned GPT-2 model.\"\"\"\n",
        "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "\n",
        "    # Alter the input to guide the model toward a more appropriate response\n",
        "    prompt = f\"Q: {input_text}\\nA:\"\n",
        "    inputs = tokenizer.encode(prompt, return_tensors=\"pt\").to(device)\n",
        "\n",
        "    outputs = model.generate(\n",
        "        inputs,\n",
        "        max_length=150,\n",
        "        do_sample=True,\n",
        "        temperature=0.7,\n",
        "        top_k=50,\n",
        "        top_p=0.95,\n",
        "        pad_token_id=tokenizer.eos_token_id,\n",
        "        eos_token_id=tokenizer.eos_token_id\n",
        "    )\n",
        "\n",
        "    response = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
        "\n",
        "    # Process the response to remove any leading prompts, if necessary\n",
        "    response = response.split(\"A:\", 1)[-1].strip()\n",
        "    return response"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t541Po20lUww"
      },
      "source": [
        "### Begin Interaction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JXBnTYXPdh6E",
        "outputId": "dac75c53-280f-4ef6-ccb8-7636ca15993a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "What can I answer for you today? Type 'q' or 'quit' to exit.\n",
            "You: define recursion\n",
            "Bot: recursive algorithm is one that invokes some or all of its outer nodes in a recursive program\n",
            "You: please explain recursion\n",
            "Bot: recursion is a recursive algorithm used to solve a class of computational problems by enclosing an accumulator with an upper bound on the number of possible solutions that may be made by taking the solution of a recursion as a whole and dividing it into parts to solve the bigger problem\n",
            "You: what is a decision tree?\n",
            "Bot: a decision tree is a tree which represents a decision in terms of its children\n",
            "You: explain binary search\n",
            "Bot: binary search is a search algorithm that finds a shortest path between two vertices or edges using binary search algorithms often used in computer graphics and cryptography\n",
            "You: which furniture would you describe yourself as?\n",
            "Bot: a computer programmer\n",
            "You: q\n",
            "Exiting chatbot! All the best for your job search :)\n"
          ]
        }
      ],
      "source": [
        "def chat():\n",
        "    model, tokenizer = load_fine_tuned_model()\n",
        "\n",
        "    print(\"What can I answer for you today? Type 'q' or 'quit' to exit.\")\n",
        "    while True:\n",
        "        user_input = input(\"You: \")\n",
        "\n",
        "        if user_input.lower() in ['q', 'quit']:\n",
        "            print(\"Exiting chatbot! All the best for your job search :)\")\n",
        "            break\n",
        "\n",
        "        user_input = user_input.strip()  # Simple text normalization\n",
        "        response = generate_response(model, tokenizer, user_input)\n",
        "        print(\"Bot:\", response)\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    chat()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n2WXvkcxkj6z"
      },
      "source": [
        "#### BERT Score evaluation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ef67ZIwvnOVt",
        "outputId": "a6b24a44-ae2c-4208-c4ee-2eaa8dcc1538"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting bert_score\n",
            "  Downloading bert_score-0.3.13-py3-none-any.whl.metadata (15 kB)\n",
            "Requirement already satisfied: torch>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from bert_score) (2.5.1+cu121)\n",
            "Requirement already satisfied: pandas>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from bert_score) (2.2.2)\n",
            "Requirement already satisfied: transformers>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from bert_score) (4.46.2)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from bert_score) (1.26.4)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from bert_score) (2.32.3)\n",
            "Requirement already satisfied: tqdm>=4.31.1 in /usr/local/lib/python3.10/dist-packages (from bert_score) (4.66.6)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from bert_score) (3.8.0)\n",
            "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from bert_score) (24.2)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0.1->bert_score) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0.1->bert_score) (2024.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.0.1->bert_score) (2024.2)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (3.16.1)\n",
            "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (4.12.2)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (3.1.4)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (2024.10.0)\n",
            "Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.10/dist-packages (from torch>=1.0.0->bert_score) (1.13.1)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy==1.13.1->torch>=1.0.0->bert_score) (1.3.0)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /usr/local/lib/python3.10/dist-packages (from transformers>=3.0.0->bert_score) (0.26.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=3.0.0->bert_score) (6.0.2)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers>=3.0.0->bert_score) (2024.9.11)\n",
            "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers>=3.0.0->bert_score) (0.4.5)\n",
            "Requirement already satisfied: tokenizers<0.21,>=0.20 in /usr/local/lib/python3.10/dist-packages (from transformers>=3.0.0->bert_score) (0.20.3)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (1.3.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (4.55.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (1.4.7)\n",
            "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (11.0.0)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->bert_score) (3.2.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->bert_score) (3.4.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->bert_score) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->bert_score) (2.2.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->bert_score) (2024.8.30)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas>=1.0.1->bert_score) (1.16.0)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.0.0->bert_score) (3.0.2)\n",
            "Downloading bert_score-0.3.13-py3-none-any.whl (61 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.1/61.1 kB\u001b[0m \u001b[31m2.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: bert_score\n",
            "Successfully installed bert_score-0.3.13\n"
          ]
        }
      ],
      "source": [
        "!pip install bert_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 278,
          "referenced_widgets": [
            "5e08c6480ac746d6a72eba00171f204d",
            "68e9a5a9f8f84d3e981c18a5de56804a",
            "953be174df8944389170e2f9fb31922b",
            "96dd521de1ff4696bbea88ee9e706e1a",
            "ad3399f6d5814dc6a754db343ebbcab2",
            "5e2345e0c9824bf98b1955ecd779d08b",
            "203cf15939be4b6e8d7fbe022e93b528",
            "07e4ca8525f747b0b2d5c67ae0d855dd",
            "73dae1509eff4a0280c2f478d577d76e",
            "5153518412254b649b883bfe889aebaa",
            "05982dffdb1f4156b56be7638920f807",
            "4aab0ca51aa9421c90062ca20ac63ce6",
            "df0b22866b3f4b1dbd96d28c01db0bb0",
            "99ba1f1dac764cb1840f7c6a88c02657",
            "22c1d8a62390494e9b9982072ab8622b",
            "7b1467950c1d4f808e3d84743288a70f",
            "62ed9c71c4b845e0b36de4553f6ca24e",
            "601edd4e281b4ba0a781579e250778c1",
            "a52cede4aaee4758b84b6d36f6806eb0",
            "867aa27b84124ffd8483c5277d0d31ce",
            "bc93e99d1cb6494092b8c729698fc316",
            "344458ea08b74b5a8da2bb0abeb6dd3d",
            "05e7394e0a0f44da905d0549bc2590fd",
            "818105dfe21a43b18e426f7f223fe489",
            "b02eb3c09dc84d7aa3be65e181f203e8",
            "c86b4b71af7043cb87719a74466619da",
            "0749d23d6ae44c5f928d014a7542f44b",
            "fad9f2cb18d641b298dd37b1c6b6b380",
            "8463a49a80d141688ba502bee849001b",
            "e52f2d5a6fbe475599a5e0f8f8234ec2",
            "6d73c398c9e940f8ace079d0f0840a10",
            "b95ffcee9789452faad7f8996f0de9c0",
            "9be5b84ce8e642d5b9a1b859e07a1c13",
            "126a14cefea549cd8c8ea9e8382968dd",
            "eba7d68e70e846c8a27847112fe1f3d0",
            "5b495a63b2454553b555d840c05a1dc1",
            "881edc30d11f4defb07802cff6dbbfb4",
            "f5677eb2124b452dbbc80cfb83f0e205",
            "e6ae61471dd24bf690acfb41fd084f97",
            "0bb17b9632a247fbb7284519ffd76a96",
            "c342fa93d42a4dde813a70e4479e5594",
            "431388ea5f05445f99d872236205d84a",
            "ca07523f11b441919ab18e915becd77e",
            "7c74b66f3def4ae2bd81b29332026916",
            "f86152e29a7245b7859f5109a31cfbe8",
            "0eddc67139e143cba46520d3172bc4c4",
            "feeeb285817d46619f41466ad9179847",
            "d80bde9aac1f48a5934e309215005048",
            "6eae644feb624b05955c432578ca8929",
            "5048fc1bc40d4a9aa0e70d45331db387",
            "8aee443415fb404a8848464c25b05b1c",
            "4960f591b8584185b21d4e95b1ed211a",
            "ab87eabe28e64cd7b0c3d2f60ede3c70",
            "ddee46817f22425c8b95da586a885f0f",
            "3cd27df4c009414a94cdd497cf241b57",
            "907c8e92875f45c4a178e092e825e6fc",
            "9b04b753f0484daa9a4af6f1f9b0c246",
            "aed402b5b1514b94894a3d06bf8e2bb7",
            "c849f7a839514e6ea031356ea835f5f8",
            "b2233f7ae27e48d9994f6ebaeca04b8b",
            "d774401650aa4cccb293b6e7c42b52a5",
            "b9ed6d72b3ce4459b41d5052ad81d941",
            "b529e365ac524cae8d0217f02fb950f7",
            "1006fb1c29c64d2da832216160ebd30f",
            "eb67368acc594f20991fd7af7951ef09",
            "7a1cf708aa3b4d938f2afddd032cbd8d"
          ]
        },
        "id": "GuC4Whsmk1df",
        "outputId": "6d551d88-d168-427b-e853-3eb2d5686c49"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5e08c6480ac746d6a72eba00171f204d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/25.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4aab0ca51aa9421c90062ca20ac63ce6",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/482 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "05e7394e0a0f44da905d0549bc2590fd",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "126a14cefea549cd8c8ea9e8382968dd",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "f86152e29a7245b7859f5109a31cfbe8",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "907c8e92875f45c4a178e092e825e6fc",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/1.42G [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Warning: Empty candidate sentence detected; setting raw BERTscores to 0.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Average Precision: 0.8790\n",
            "Average Recall: 0.8724\n",
            "Average F1 Score: 0.8755\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "from bert_score import score as bert_score\n",
        "import random\n",
        "from transformers import logging\n",
        "\n",
        "# Set logging level to suppress informational logs\n",
        "logging.set_verbosity_error()  # Only errors will be shown\n",
        "\n",
        "model, tokenizer = load_fine_tuned_model()\n",
        "generated_responses = []\n",
        "all_results = []  # For storing results\n",
        "\n",
        "for question, answer in zip(questions, answers):\n",
        "    generated_response = generate_response(model, tokenizer, question)\n",
        "    generated_responses.append(generated_response)\n",
        "\n",
        "    # Prepare candidates and references for BERTScore calculation\n",
        "    candidates = [generated_response]  # Generated response\n",
        "    references = [answer]              # Ground truth answer\n",
        "\n",
        "    # Calculate BERTScore\n",
        "    P, R, F1 = bert_score(candidates, references, lang='en', device='cuda' if torch.cuda.is_available() else 'cpu')\n",
        "\n",
        "    # Append results to the list\n",
        "    all_results.append({\n",
        "        \"Question\": question,\n",
        "        \"Ground Truth\": answer,\n",
        "        \"Generated Response\": generated_response,\n",
        "        \"Precision\": P.mean().item(),\n",
        "        \"Recall\": R.mean().item(),\n",
        "        \"F1\": F1.mean().item()\n",
        "    })\n",
        "\n",
        "# Create a DataFrame for results\n",
        "results_df = pd.DataFrame(all_results)\n",
        "\n",
        "# Save the DataFrame to a CSV file\n",
        "results_df.to_csv('/content/drive/My Drive/Colab Notebooks/nlc/project/gpt2_bertscores.csv', index=False)\n",
        "\n",
        "# Calculate average BERTScores\n",
        "avg_precision = results_df[\"Precision\"].mean()\n",
        "avg_recall = results_df[\"Recall\"].mean()\n",
        "avg_f1 = results_df[\"F1\"].mean()\n",
        "\n",
        "# Print average scores\n",
        "print(f\"Average Precision: {avg_precision:.4f}\")\n",
        "print(f\"Average Recall: {avg_recall:.4f}\")\n",
        "print(f\"Average F1 Score: {avg_f1:.4f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Fallback Mechanism with BERTScore"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import time\n",
        "import random\n",
        "import torch\n",
        "import warnings\n",
        "from bert_score import score as bert_score\n",
        "from transformers import logging\n",
        "\n",
        "# Suppress FutureWarnings from transformers library\n",
        "warnings.simplefilter(action='ignore', category=FutureWarning)\n",
        "\n",
        "# Set logging level to suppress informational logs\n",
        "logging.set_verbosity_error()\n",
        "\n",
        "def evaluateInput(encoder, decoder, searcher, voc, max_length=MAX_LENGTH, threshold=0.7):\n",
        "    print(\"What can I answer for you today? Type 'q' or 'quit' to exit.\")\n",
        "\n",
        "    known_questions = [pair[0] for pair in qa_pairs]  # Assuming this is pre-loaded from your data\n",
        "\n",
        "    while True:\n",
        "        # Get input sentence\n",
        "        input_sentence = input('> ')\n",
        "\n",
        "        # Check if it is quit case\n",
        "        if input_sentence.lower() in ['q', 'quit']:\n",
        "          print(\"Exiting chatbot! All the best for your job search :)\")\n",
        "          break\n",
        "\n",
        "        # Normalize sentence\n",
        "        input_sentence = normalizeString(input_sentence)\n",
        "\n",
        "        # Prepare to calculate BERTScore\n",
        "        batch_size = 128\n",
        "        all_bert_scores = []\n",
        "        candidates = [input_sentence] * len(known_questions)  # Create a list of candidates matching reference list size\n",
        "\n",
        "        # Start timing for total time per input\n",
        "        start_time = time.time()\n",
        "\n",
        "        # Process known questions in batches\n",
        "        for i in range(0, len(known_questions), batch_size):\n",
        "            refs_batch = known_questions[i:i + batch_size]\n",
        "            # Create a corresponding batch of candidates\n",
        "            cand_batch = candidates[i:i + batch_size]  # Match candidates to the size of reference batch\n",
        "\n",
        "            # Print the current batch number\n",
        "            print(f\"Computing BERTScore for batch {i // batch_size + 1}\")\n",
        "\n",
        "            P, R, F1 = bert_score(cand_batch, refs_batch, lang='en', device='cuda' if torch.cuda.is_available() else 'cpu', rescale_with_baseline=True)\n",
        "            max_f1 = F1.max().item()\n",
        "            all_bert_scores.append(max_f1)\n",
        "\n",
        "        # Get the maximum BERTScore across all batches\n",
        "        max_bertscore = max(all_bert_scores)\n",
        "        print(f\"Maximum BERTScore: {max_bertscore}\")\n",
        "\n",
        "        # Check if BERTScore meets the threshold before generating a response\n",
        "        if max_bertscore >= threshold:\n",
        "            # If the score is sufficient, proceed to generate a model response\n",
        "            output_words = evaluate(encoder, decoder, searcher, voc, input_sentence)\n",
        "\n",
        "            # Filter out EOS and PAD tokens\n",
        "            output_words[:] = [x for x in output_words if x not in ['EOS', 'PAD']]\n",
        "\n",
        "            # Print the bot's response if valid\n",
        "            if output_words:\n",
        "                print(\"Bot:\", ' '.join(output_words))\n",
        "            else:\n",
        "                print(\"Bot: I generated no valid response, but your question was similar enough.\")\n",
        "\n",
        "        else:\n",
        "            # Provide fallback response if BERTScore is below the threshold\n",
        "            resources = [\n",
        "                \"Perhaps this article might help: https://www.linkedin.com/advice/3/what-should-you-research-before-computer-science-r6tuc\",\n",
        "                \"You might find this helpful: https://medium.com/@andreimargeloiu/the-definitive-guide-to-the-coding-interview-2704d166664c\",\n",
        "                \"Here is a more light-hearted video to help with your preparation: https://youtu.be/1t1_a1BZ04o?feature=shared\"\n",
        "            ]\n",
        "            selected_resource = random.choice(resources)\n",
        "            print(\"Bot: I'm sorry, I didn't quite understand your question.\")\n",
        "            print(selected_resource)\n",
        "            print(\"Alternatively, please specify more details or clarify your query!\")\n",
        "\n",
        "        # End timing for total time per input\n",
        "        end_time = time.time()\n",
        "        total_time = end_time - start_time\n",
        "        # print(f\"Total time taken for computing BERTScores: {total_time:.4f} seconds\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uDZ-k_5DYgFq"
      },
      "source": [
        "### BLEU, ROUGE, BERT Score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bAIs3V0MY5aH"
      },
      "source": [
        "1. Install Necessary Packages and Libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qtPbUjTNYhsK",
        "outputId": "52dcd1dc-b4bf-442f-aabc-4037c17e1523"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: nltk in /usr/local/lib/python3.10/dist-packages (3.9.1)\n",
            "Collecting rouge-score\n",
            "  Downloading rouge_score-0.1.2.tar.gz (17 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from nltk) (8.1.7)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.10/dist-packages (from nltk) (1.4.2)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.10/dist-packages (from nltk) (2024.9.11)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from nltk) (4.66.6)\n",
            "Requirement already satisfied: absl-py in /usr/local/lib/python3.10/dist-packages (from rouge-score) (1.4.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from rouge-score) (1.26.4)\n",
            "Requirement already satisfied: six>=1.14.0 in /usr/local/lib/python3.10/dist-packages (from rouge-score) (1.16.0)\n",
            "Building wheels for collected packages: rouge-score\n",
            "  Building wheel for rouge-score (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24935 sha256=df3fb932adff1d336111be9ebc90bd99580c49610971fd9471bce4f4c6f1f44f\n",
            "  Stored in directory: /root/.cache/pip/wheels/5f/dd/89/461065a73be61a532ff8599a28e9beef17985c9e9c31e541b4\n",
            "Successfully built rouge-score\n",
            "Installing collected packages: rouge-score\n",
            "Successfully installed rouge-score-0.1.2\n"
          ]
        }
      ],
      "source": [
        "!pip install nltk rouge-score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2QePRtwdY76s"
      },
      "outputs": [],
      "source": [
        "from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction\n",
        "from rouge_score import rouge_scorer\n",
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YL-tY5r_Zuj6"
      },
      "source": [
        "2.  Evaluation with BLEU and ROUGE"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DPC_cR6lebIN",
        "outputId": "c37e42a5-1f10-4c1b-d5a1-43c4e5ccd07a"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import nltk\n",
        "nltk.download('punkt')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "d7I9iAO9ZsIz",
        "outputId": "8e006b8c-35ec-415d-9a8e-2a136b381437"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Average BERTScore Precision: 0.8803\n",
            "Average BERTScore Recall: 0.8761\n",
            "Average BERTScore F1 Score: 0.8780\n",
            "Average BLEU Score: 0.1822\n",
            "Average ROUGE-1 F1 Score: 0.3889\n",
            "Average ROUGE-L F1 Score: 0.3415\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "from bert_score import score as bert_score\n",
        "import random\n",
        "from transformers import logging\n",
        "import nltk\n",
        "from nltk.translate.bleu_score import sentence_bleu, SmoothingFunction\n",
        "from rouge_score import rouge_scorer\n",
        "import pandas as pd\n",
        "\n",
        "# Set logging level to suppress informational logs\n",
        "logging.set_verbosity_error()  # Only errors will be shown\n",
        "\n",
        "# Load the fine-tuned model\n",
        "model, tokenizer = load_fine_tuned_model()\n",
        "generated_responses = []\n",
        "all_results = []  # For storing results\n",
        "\n",
        "# Initialize BLEU smoothing function\n",
        "smoothie = SmoothingFunction().method4\n",
        "\n",
        "# Initialize ROUGE scorer\n",
        "rouge = rouge_scorer.RougeScorer(['rouge1', 'rougeL'], use_stemmer=True)\n",
        "\n",
        "# If you haven't already, split your data into training and testing sets\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# Assuming questions and answers are your data lists\n",
        "train_questions, test_questions, train_answers, test_answers = train_test_split(\n",
        "    questions, answers, test_size=0.2, random_state=42\n",
        ")\n",
        "\n",
        "for question, answer in zip(test_questions, test_answers):\n",
        "    generated_response = generate_response(model, tokenizer, question)\n",
        "    generated_responses.append(generated_response)\n",
        "\n",
        "    # Prepare candidates and references\n",
        "    candidate = generated_response.strip()  # Generated response\n",
        "    reference = answer.strip()              # Ground truth answer\n",
        "\n",
        "    # Calculate BERTScore\n",
        "    P, R, F1 = bert_score(\n",
        "        [candidate], [reference],\n",
        "        lang='en',\n",
        "        device='cuda' if torch.cuda.is_available() else 'cpu'\n",
        "    )\n",
        "\n",
        "    # Tokenize the texts for BLEU and ROUGE using the model's tokenizer\n",
        "    reference_tokens = tokenizer.tokenize(reference)\n",
        "    candidate_tokens = tokenizer.tokenize(candidate)\n",
        "\n",
        "    # Calculate BLEU score\n",
        "    bleu_score = sentence_bleu(\n",
        "        [reference_tokens], candidate_tokens,\n",
        "        smoothing_function=smoothie\n",
        "    )\n",
        "\n",
        "    # Calculate ROUGE scores\n",
        "    rouge_scores = rouge.score(reference, candidate)\n",
        "    rouge1_f1 = rouge_scores['rouge1'].fmeasure\n",
        "    rougeL_f1 = rouge_scores['rougeL'].fmeasure\n",
        "\n",
        "    # Append results to the list\n",
        "    all_results.append({\n",
        "        \"Question\": question,\n",
        "        \"Ground Truth\": reference,\n",
        "        \"Generated Response\": candidate,\n",
        "        \"BERTScore_Precision\": P.mean().item(),\n",
        "        \"BERTScore_Recall\": R.mean().item(),\n",
        "        \"BERTScore_F1\": F1.mean().item(),\n",
        "        \"BLEU\": bleu_score,\n",
        "        \"ROUGE-1_F1\": rouge1_f1,\n",
        "        \"ROUGE-L_F1\": rougeL_f1\n",
        "    })\n",
        "\n",
        "# Create a DataFrame for results\n",
        "results_df = pd.DataFrame(all_results)\n",
        "\n",
        "# Save the DataFrame to a CSV file\n",
        "results_df.to_csv('/content/drive/My Drive/Colab Notebooks/nlc/project/gpt2_scores_w_bleu_rogue_bert.csv', index=False)\n",
        "\n",
        "# Calculate average scores\n",
        "avg_bert_precision = results_df[\"BERTScore_Precision\"].mean()\n",
        "avg_bert_recall = results_df[\"BERTScore_Recall\"].mean()\n",
        "avg_bert_f1 = results_df[\"BERTScore_F1\"].mean()\n",
        "avg_bleu = results_df[\"BLEU\"].mean()\n",
        "avg_rouge1_f1 = results_df[\"ROUGE-1_F1\"].mean()\n",
        "avg_rougeL_f1 = results_df[\"ROUGE-L_F1\"].mean()\n",
        "\n",
        "# Print average scores\n",
        "print(f\"Average BERTScore Precision: {avg_bert_precision:.4f}\")\n",
        "print(f\"Average BERTScore Recall: {avg_bert_recall:.4f}\")\n",
        "print(f\"Average BERTScore F1 Score: {avg_bert_f1:.4f}\")\n",
        "print(f\"Average BLEU Score: {avg_bleu:.4f}\")\n",
        "print(f\"Average ROUGE-1 F1 Score: {avg_rouge1_f1:.4f}\")\n",
        "print(f\"Average ROUGE-L F1 Score: {avg_rougeL_f1:.4f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ABw4DJ089aC"
      },
      "source": [
        "### Telegram Implementation\n",
        "- Replace token with your own API token obtained by BotFather on Telegram\n",
        "- application = Application.builder().token('**Replace token here**').build()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lj1jJKxy-wd8",
        "outputId": "aee7ffcb-6ce0-46e1-ca7a-5c2a1aa502a2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting python-telegram-bot\n",
            "  Downloading python_telegram_bot-21.7-py3-none-any.whl.metadata (17 kB)\n",
            "Requirement already satisfied: httpx~=0.27 in /usr/local/lib/python3.10/dist-packages (from python-telegram-bot) (0.27.2)\n",
            "Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx~=0.27->python-telegram-bot) (3.7.1)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx~=0.27->python-telegram-bot) (2024.8.30)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx~=0.27->python-telegram-bot) (1.0.7)\n",
            "Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx~=0.27->python-telegram-bot) (3.10)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx~=0.27->python-telegram-bot) (1.3.1)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx~=0.27->python-telegram-bot) (0.14.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx~=0.27->python-telegram-bot) (1.2.2)\n",
            "Downloading python_telegram_bot-21.7-py3-none-any.whl (654 kB)\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m654.9/654.9 kB\u001b[0m \u001b[31m9.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hInstalling collected packages: python-telegram-bot\n",
            "Successfully installed python-telegram-bot-21.7\n",
            "Requirement already satisfied: nest_asyncio in /usr/local/lib/python3.10/dist-packages (1.6.0)\n"
          ]
        }
      ],
      "source": [
        "!pip install python-telegram-bot --upgrade\n",
        "!pip install nest_asyncio"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "emtICBcb9jko",
        "outputId": "c97cd297-5eac-4037-86e6-988931e66a3d"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n",
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n",
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n",
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n",
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n",
            "ERROR:telegram.ext.Updater:Exception happened while polling for updates.\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 743, in _network_loop_retry\n",
            "    if not await do_action():\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 737, in do_action\n",
            "    return action_cb_task.result()\n",
            "  File \"/usr/lib/python3.10/asyncio/futures.py\", line 201, in result\n",
            "    raise self._exception.with_traceback(self._exception_tb)\n",
            "  File \"/usr/lib/python3.10/asyncio/tasks.py\", line 232, in __step\n",
            "    result = coro.send(None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_updater.py\", line 367, in polling_action_cb\n",
            "    updates = await self.bot.get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 647, in get_updates\n",
            "    updates = await super().get_updates(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 4421, in get_updates\n",
            "    await self._post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 617, in _post\n",
            "    return await self._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/ext/_extbot.py\", line 351, in _do_post\n",
            "    return await super()._do_post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/_bot.py\", line 646, in _do_post\n",
            "    result = await request.post(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 202, in post\n",
            "    result = await self._request_wrapper(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/telegram/request/_baserequest.py\", line 385, in _request_wrapper\n",
            "    raise Conflict(message)\n",
            "telegram.error.Conflict: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running\n"
          ]
        },
        {
          "ename": "KeyboardInterrupt",
          "evalue": "",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-23-336e7f0a1665>\u001b[0m in \u001b[0;36m<cell line: 68>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     68\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0m__name__\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"__main__\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     69\u001b[0m     \u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload_fine_tuned_model\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 70\u001b[0;31m     \u001b[0masyncio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/nest_asyncio.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(main, debug)\u001b[0m\n\u001b[1;32m     28\u001b[0m         \u001b[0mtask\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0masyncio\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_future\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmain\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     29\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 30\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mloop\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_until_complete\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtask\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     31\u001b[0m         \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     32\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mtask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdone\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/nest_asyncio.py\u001b[0m in \u001b[0;36mrun_until_complete\u001b[0;34m(self, future)\u001b[0m\n\u001b[1;32m     90\u001b[0m                 \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_log_destroy_pending\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     91\u001b[0m             \u001b[0;32mwhile\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdone\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 92\u001b[0;31m                 \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_run_once\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     93\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stopping\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     94\u001b[0m                     \u001b[0;32mbreak\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/nest_asyncio.py\u001b[0m in \u001b[0;36m_run_once\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    113\u001b[0m                 scheduled[0]._when - self.time(), 0), 86400) if scheduled\n\u001b[1;32m    114\u001b[0m             else None)\n\u001b[0;32m--> 115\u001b[0;31m         \u001b[0mevent_list\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_selector\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mselect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    116\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_process_events\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mevent_list\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    117\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/lib/python3.10/selectors.py\u001b[0m in \u001b[0;36mselect\u001b[0;34m(self, timeout)\u001b[0m\n\u001b[1;32m    467\u001b[0m             \u001b[0mready\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    468\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 469\u001b[0;31m                 \u001b[0mfd_event_list\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_selector\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpoll\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtimeout\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_ev\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    470\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mInterruptedError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    471\u001b[0m                 \u001b[0;32mreturn\u001b[0m \u001b[0mready\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ],
      "source": [
        "import logging\n",
        "import asyncio\n",
        "import nest_asyncio\n",
        "from telegram import Update\n",
        "from telegram.ext import Application, MessageHandler, filters, ContextTypes\n",
        "from transformers import GPT2LMHeadModel, GPT2TokenizerFast\n",
        "import torch\n",
        "\n",
        "# Apply the nest_asyncio patch for Jupyter environments\n",
        "nest_asyncio.apply()\n",
        "\n",
        "# Configure logging\n",
        "logging.basicConfig(\n",
        "    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',\n",
        "    level=logging.INFO\n",
        ")\n",
        "\n",
        "# Load the fine-tuned GPT-2 model and tokenizer\n",
        "def load_fine_tuned_model(model_dir=\"fine_tuned_gpt2\"):\n",
        "    tokenizer = GPT2TokenizerFast.from_pretrained(model_dir)\n",
        "    model = GPT2LMHeadModel.from_pretrained(model_dir)\n",
        "    model.to(torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\"))\n",
        "    return model, tokenizer\n",
        "\n",
        "# Generate a response using the fine-tuned GPT-2 model\n",
        "def generate_response(model, tokenizer, input_text):\n",
        "    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
        "    prompt = f\"Q: {input_text}\\nA:\"\n",
        "    inputs = tokenizer.encode(prompt, return_tensors=\"pt\").to(device)\n",
        "\n",
        "    outputs = model.generate(\n",
        "        inputs,\n",
        "        max_length=150,\n",
        "        do_sample=True,\n",
        "        temperature=0.7,\n",
        "        top_k=50,\n",
        "        top_p=0.95,\n",
        "        pad_token_id=tokenizer.eos_token_id,\n",
        "        eos_token_id=tokenizer.eos_token_id\n",
        "    )\n",
        "\n",
        "    response = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
        "    response = response.split(\"A:\", 1)[-1].strip()\n",
        "    return response\n",
        "\n",
        "# Define the Telegram bot's response function\n",
        "async def respond(update: Update, context: ContextTypes.DEFAULT_TYPE):\n",
        "    user_message = update.message.text.strip()\n",
        "    try:\n",
        "        response = generate_response(model, tokenizer, user_message)\n",
        "        await update.message.reply_text(response)\n",
        "    except Exception as e:\n",
        "        logging.error(f\"Error generating response: {e}\")\n",
        "        await update.message.reply_text(\"Sorry, something went wrong!\")\n",
        "\n",
        "# Main function to initialize and start the bot\n",
        "async def main():\n",
        "    # Replace 'YOUR_TOKEN' with your bot's API token\n",
        "    application = Application.builder().token('YOUR_TOKEN').build()\n",
        "\n",
        "    # Add handler for text messages\n",
        "    application.add_handler(MessageHandler(filters.TEXT & ~filters.COMMAND, respond))\n",
        "\n",
        "    await application.initialize()\n",
        "    await application.start()\n",
        "    await application.updater.start_polling()\n",
        "    await asyncio.Event().wait()\n",
        "\n",
        "# Run the bot\n",
        "if __name__ == \"__main__\":\n",
        "    model, tokenizer = load_fine_tuned_model()\n",
        "    asyncio.run(main())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eXj_OaZTesNs"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "BMH_9ap-KVHD"
      ],
      "gpuType": "T4",
      "machine_shape": "hm",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.5"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "011fb883f7774528b22fa0b34a9b3e40": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2f38f15ce1b14e58a366b8ca21aead8f",
              "IPY_MODEL_a453b9c07c7b479e81adadf2dc2e7098",
              "IPY_MODEL_7c2a0ed083ec4d5884caff6764d9b149"
            ],
            "layout": "IPY_MODEL_736df333921f4513ac4866c204b98884"
          }
        },
        "03eb561d1f63450f900cd8ec8677ce80": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2bc8bca1a528472e96949584c08dcd4d",
            "placeholder": "​",
            "style": "IPY_MODEL_71529b0d406d44168144727ba51e77ee",
            "value": "tokenizer.json: 100%"
          }
        },
        "05982dffdb1f4156b56be7638920f807": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "05e7394e0a0f44da905d0549bc2590fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_818105dfe21a43b18e426f7f223fe489",
              "IPY_MODEL_b02eb3c09dc84d7aa3be65e181f203e8",
              "IPY_MODEL_c86b4b71af7043cb87719a74466619da"
            ],
            "layout": "IPY_MODEL_0749d23d6ae44c5f928d014a7542f44b"
          }
        },
        "0749d23d6ae44c5f928d014a7542f44b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "07e4ca8525f747b0b2d5c67ae0d855dd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0b0772747f364fec9ea5cee6082bb697": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0bb17b9632a247fbb7284519ffd76a96": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0eddc67139e143cba46520d3172bc4c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5048fc1bc40d4a9aa0e70d45331db387",
            "placeholder": "​",
            "style": "IPY_MODEL_8aee443415fb404a8848464c25b05b1c",
            "value": "tokenizer.json: 100%"
          }
        },
        "1000a5f574904ea181cb42cc7a423b28": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e37527644fb14daa9ba20129acbdd943",
              "IPY_MODEL_5455a53eb3e14f55a771241306134200",
              "IPY_MODEL_6c4d98c7d2d94b03a3ccf464c9d646f9"
            ],
            "layout": "IPY_MODEL_44838c56d35e48c3a0344f83434e64a2"
          }
        },
        "1006fb1c29c64d2da832216160ebd30f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "126a14cefea549cd8c8ea9e8382968dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_eba7d68e70e846c8a27847112fe1f3d0",
              "IPY_MODEL_5b495a63b2454553b555d840c05a1dc1",
              "IPY_MODEL_881edc30d11f4defb07802cff6dbbfb4"
            ],
            "layout": "IPY_MODEL_f5677eb2124b452dbbc80cfb83f0e205"
          }
        },
        "176b5aed89224adead7172e95a311409": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1783f66ad7cf482b8bcdf548eff924a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d41eb2e1e05444e8a181b71559ae6112",
            "placeholder": "​",
            "style": "IPY_MODEL_20746af24ae94d85a8f0ee29868da5fc",
            "value": " 124/124 [00:00&lt;00:00, 8.76kB/s]"
          }
        },
        "203cf15939be4b6e8d7fbe022e93b528": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "20746af24ae94d85a8f0ee29868da5fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "22c1d8a62390494e9b9982072ab8622b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bc93e99d1cb6494092b8c729698fc316",
            "placeholder": "​",
            "style": "IPY_MODEL_344458ea08b74b5a8da2bb0abeb6dd3d",
            "value": " 482/482 [00:00&lt;00:00, 8.31kB/s]"
          }
        },
        "25df876373a14c19b5800bbd024a098d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2bc8bca1a528472e96949584c08dcd4d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2f38f15ce1b14e58a366b8ca21aead8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_851fe6f3404846cb99e1bf1db0c7142b",
            "placeholder": "​",
            "style": "IPY_MODEL_e952eba68ece4c6b864f438d8d42911d",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "344458ea08b74b5a8da2bb0abeb6dd3d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "37fe86c55e574d608d21a382c94bbb03": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3cd27df4c009414a94cdd497cf241b57": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3d4fa04a46ac4ca587c84a3e27fb3ebc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "431388ea5f05445f99d872236205d84a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "44838c56d35e48c3a0344f83434e64a2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "45cd7100b77a44f8b6181740e20f42fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "467b9e1ca277401da2d2fdbdaa65a669": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4960f591b8584185b21d4e95b1ed211a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4aab0ca51aa9421c90062ca20ac63ce6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_df0b22866b3f4b1dbd96d28c01db0bb0",
              "IPY_MODEL_99ba1f1dac764cb1840f7c6a88c02657",
              "IPY_MODEL_22c1d8a62390494e9b9982072ab8622b"
            ],
            "layout": "IPY_MODEL_7b1467950c1d4f808e3d84743288a70f"
          }
        },
        "4c287a64e0404f24aad86f868b2db58c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5943e25a3559420abdab0272af46edd1",
            "placeholder": "​",
            "style": "IPY_MODEL_df5b52e7d83b454585333790d844237f",
            "value": "config.json: 100%"
          }
        },
        "4f0cfadfd255445dae90afcf10c3d751": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5048fc1bc40d4a9aa0e70d45331db387": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5153518412254b649b883bfe889aebaa": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53aeda84c2ab4ad091751016af363a53": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5455a53eb3e14f55a771241306134200": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_93192a852192454580ec57ca901f748f",
            "max": 1042301,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_949099bccb9c4f298ed868eb177dc73f",
            "value": 1042301
          }
        },
        "58fa86de43b749e09a698b8cf7398dab": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a77884af85264f80bab7781bc38b9d40",
            "placeholder": "​",
            "style": "IPY_MODEL_a64f38afb26746b6b62c8209a41d8d2f",
            "value": " 665/665 [00:00&lt;00:00, 55.6kB/s]"
          }
        },
        "5943e25a3559420abdab0272af46edd1": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5b495a63b2454553b555d840c05a1dc1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c342fa93d42a4dde813a70e4479e5594",
            "max": 456318,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_431388ea5f05445f99d872236205d84a",
            "value": 456318
          }
        },
        "5bbdc5dae16446e5b9024e533b7ae5ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_25df876373a14c19b5800bbd024a098d",
            "placeholder": "​",
            "style": "IPY_MODEL_a11c25686f6845bf8ba8b30da865e7fa",
            "value": " 548M/548M [00:02&lt;00:00, 216MB/s]"
          }
        },
        "5e08c6480ac746d6a72eba00171f204d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_68e9a5a9f8f84d3e981c18a5de56804a",
              "IPY_MODEL_953be174df8944389170e2f9fb31922b",
              "IPY_MODEL_96dd521de1ff4696bbea88ee9e706e1a"
            ],
            "layout": "IPY_MODEL_ad3399f6d5814dc6a754db343ebbcab2"
          }
        },
        "5e196603a4c14a50a051a122904ac089": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "5e2345e0c9824bf98b1955ecd779d08b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "601edd4e281b4ba0a781579e250778c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "62ed9c71c4b845e0b36de4553f6ca24e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "68e9a5a9f8f84d3e981c18a5de56804a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5e2345e0c9824bf98b1955ecd779d08b",
            "placeholder": "​",
            "style": "IPY_MODEL_203cf15939be4b6e8d7fbe022e93b528",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "6a16330e3ec24fa98f9e906c9df21d25": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_cafb3801fa764d23b6a65bee141e044c",
              "IPY_MODEL_ff3a59cf02b5436cb980e3191da0c2fb",
              "IPY_MODEL_1783f66ad7cf482b8bcdf548eff924a1"
            ],
            "layout": "IPY_MODEL_176b5aed89224adead7172e95a311409"
          }
        },
        "6c4d98c7d2d94b03a3ccf464c9d646f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bb28b0289c9748c4ae6b217edc631b86",
            "placeholder": "​",
            "style": "IPY_MODEL_f5ca910ea9334267b3924c2cd8c559b8",
            "value": " 1.04M/1.04M [00:00&lt;00:00, 2.02MB/s]"
          }
        },
        "6cd737c50b84495aa6381d17d5e001da": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d73c398c9e940f8ace079d0f0840a10": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6eae644feb624b05955c432578ca8929": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "71529b0d406d44168144727ba51e77ee": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "736df333921f4513ac4866c204b98884": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "73dae1509eff4a0280c2f478d577d76e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "779dd1b4114d4a2c96de5d304a4febf6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6cd737c50b84495aa6381d17d5e001da",
            "placeholder": "​",
            "style": "IPY_MODEL_e6852d226c07498ba6519985b74f7ab7",
            "value": " 456k/456k [00:00&lt;00:00, 27.0MB/s]"
          }
        },
        "7a1cf708aa3b4d938f2afddd032cbd8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7b1467950c1d4f808e3d84743288a70f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7c2a0ed083ec4d5884caff6764d9b149": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ad59ce55998b4b8ea451ab422c9d5dc2",
            "placeholder": "​",
            "style": "IPY_MODEL_eb7419cdc0a748549ee56223ae667acc",
            "value": " 26.0/26.0 [00:00&lt;00:00, 2.30kB/s]"
          }
        },
        "7c74b66f3def4ae2bd81b29332026916": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "818105dfe21a43b18e426f7f223fe489": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fad9f2cb18d641b298dd37b1c6b6b380",
            "placeholder": "​",
            "style": "IPY_MODEL_8463a49a80d141688ba502bee849001b",
            "value": "vocab.json: 100%"
          }
        },
        "8463a49a80d141688ba502bee849001b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "851fe6f3404846cb99e1bf1db0c7142b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "867aa27b84124ffd8483c5277d0d31ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "881edc30d11f4defb07802cff6dbbfb4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ca07523f11b441919ab18e915becd77e",
            "placeholder": "​",
            "style": "IPY_MODEL_7c74b66f3def4ae2bd81b29332026916",
            "value": " 456k/456k [00:00&lt;00:00, 6.34MB/s]"
          }
        },
        "8aee443415fb404a8848464c25b05b1c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "907c8e92875f45c4a178e092e825e6fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9b04b753f0484daa9a4af6f1f9b0c246",
              "IPY_MODEL_aed402b5b1514b94894a3d06bf8e2bb7",
              "IPY_MODEL_c849f7a839514e6ea031356ea835f5f8"
            ],
            "layout": "IPY_MODEL_b2233f7ae27e48d9994f6ebaeca04b8b"
          }
        },
        "92d2ae5a3f354d02a0e9afeac5a1676a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "93192a852192454580ec57ca901f748f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "949099bccb9c4f298ed868eb177dc73f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "953be174df8944389170e2f9fb31922b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_07e4ca8525f747b0b2d5c67ae0d855dd",
            "max": 25,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_73dae1509eff4a0280c2f478d577d76e",
            "value": 25
          }
        },
        "954e8e5c89844fa2b03b2baf1c9ee25e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_efc83d30cd024a158ada903f0eb2836d",
            "max": 665,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c7a156eb0fa3426ebe21fa855fdf47fe",
            "value": 665
          }
        },
        "96dd521de1ff4696bbea88ee9e706e1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5153518412254b649b883bfe889aebaa",
            "placeholder": "​",
            "style": "IPY_MODEL_05982dffdb1f4156b56be7638920f807",
            "value": " 25.0/25.0 [00:00&lt;00:00, 603B/s]"
          }
        },
        "9992fa1f43494fc2bbe474773903b78a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dc28651be51745eca94664bbb71ad861",
            "placeholder": "​",
            "style": "IPY_MODEL_bd2c8c995a8e4bc4a5d4fbf122e37740",
            "value": "merges.txt: 100%"
          }
        },
        "99ba1f1dac764cb1840f7c6a88c02657": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a52cede4aaee4758b84b6d36f6806eb0",
            "max": 482,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_867aa27b84124ffd8483c5277d0d31ce",
            "value": 482
          }
        },
        "9ac178dcf5b74427825a35c9daf00020": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4c287a64e0404f24aad86f868b2db58c",
              "IPY_MODEL_954e8e5c89844fa2b03b2baf1c9ee25e",
              "IPY_MODEL_58fa86de43b749e09a698b8cf7398dab"
            ],
            "layout": "IPY_MODEL_9c2a57d2ad0e424bbad9196ce659c3cd"
          }
        },
        "9b04b753f0484daa9a4af6f1f9b0c246": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d774401650aa4cccb293b6e7c42b52a5",
            "placeholder": "​",
            "style": "IPY_MODEL_b9ed6d72b3ce4459b41d5052ad81d941",
            "value": "model.safetensors: 100%"
          }
        },
        "9be5b84ce8e642d5b9a1b859e07a1c13": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9c2a57d2ad0e424bbad9196ce659c3cd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a11c25686f6845bf8ba8b30da865e7fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a453b9c07c7b479e81adadf2dc2e7098": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_92d2ae5a3f354d02a0e9afeac5a1676a",
            "max": 26,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5e196603a4c14a50a051a122904ac089",
            "value": 26
          }
        },
        "a52cede4aaee4758b84b6d36f6806eb0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a64f38afb26746b6b62c8209a41d8d2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a77884af85264f80bab7781bc38b9d40": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7b3720526a04006859104ed7d01633a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aadb3c07f7d1487188baa1285a40a420": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a7b3720526a04006859104ed7d01633a",
            "placeholder": "​",
            "style": "IPY_MODEL_3d4fa04a46ac4ca587c84a3e27fb3ebc",
            "value": " 1.36M/1.36M [00:00&lt;00:00, 7.78MB/s]"
          }
        },
        "ab87eabe28e64cd7b0c3d2f60ede3c70": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ad3399f6d5814dc6a754db343ebbcab2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad59ce55998b4b8ea451ab422c9d5dc2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aed402b5b1514b94894a3d06bf8e2bb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b529e365ac524cae8d0217f02fb950f7",
            "max": 1421700479,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1006fb1c29c64d2da832216160ebd30f",
            "value": 1421700479
          }
        },
        "aee261858dd24782a41d7978082c7ab3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b02eb3c09dc84d7aa3be65e181f203e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e52f2d5a6fbe475599a5e0f8f8234ec2",
            "max": 898823,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6d73c398c9e940f8ace079d0f0840a10",
            "value": 898823
          }
        },
        "b091c384f23743bbb7eb578ecde09bb4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b2233f7ae27e48d9994f6ebaeca04b8b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b503f0b42c82409697e5bd05a3a7fc15": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b529e365ac524cae8d0217f02fb950f7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b75d47c7f07e47f1972112842f25a565": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b95ffcee9789452faad7f8996f0de9c0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b9ed6d72b3ce4459b41d5052ad81d941": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bb28b0289c9748c4ae6b217edc631b86": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bb7f7cb49f5449248fe260f40f9cfc70": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bc93e99d1cb6494092b8c729698fc316": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bd2c8c995a8e4bc4a5d4fbf122e37740": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c342fa93d42a4dde813a70e4479e5594": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c5260793c0f44875a435918454139e68": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c70b53da90834f7aa84b7927c1864b4a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_467b9e1ca277401da2d2fdbdaa65a669",
            "max": 548105171,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d8e3030d673a428895a73342c0b9c598",
            "value": 548105171
          }
        },
        "c7884273ab104cfb822965ee145dd879": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b503f0b42c82409697e5bd05a3a7fc15",
            "placeholder": "​",
            "style": "IPY_MODEL_aee261858dd24782a41d7978082c7ab3",
            "value": "model.safetensors: 100%"
          }
        },
        "c7a156eb0fa3426ebe21fa855fdf47fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c849f7a839514e6ea031356ea835f5f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb67368acc594f20991fd7af7951ef09",
            "placeholder": "​",
            "style": "IPY_MODEL_7a1cf708aa3b4d938f2afddd032cbd8d",
            "value": " 1.42G/1.42G [00:06&lt;00:00, 233MB/s]"
          }
        },
        "c86b4b71af7043cb87719a74466619da": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b95ffcee9789452faad7f8996f0de9c0",
            "placeholder": "​",
            "style": "IPY_MODEL_9be5b84ce8e642d5b9a1b859e07a1c13",
            "value": " 899k/899k [00:00&lt;00:00, 4.77MB/s]"
          }
        },
        "c95f26ac0139443eba7f2dd7614bc817": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ca07523f11b441919ab18e915becd77e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ca842850c6704e0093c229357acee9f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_03eb561d1f63450f900cd8ec8677ce80",
              "IPY_MODEL_dce048c43a4348c4a09aa2b46a33f1f3",
              "IPY_MODEL_aadb3c07f7d1487188baa1285a40a420"
            ],
            "layout": "IPY_MODEL_c5260793c0f44875a435918454139e68"
          }
        },
        "cafb3801fa764d23b6a65bee141e044c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53aeda84c2ab4ad091751016af363a53",
            "placeholder": "​",
            "style": "IPY_MODEL_ebdd95042703441da39390f258891d49",
            "value": "generation_config.json: 100%"
          }
        },
        "d41eb2e1e05444e8a181b71559ae6112": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d774401650aa4cccb293b6e7c42b52a5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d80bde9aac1f48a5934e309215005048": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ddee46817f22425c8b95da586a885f0f",
            "placeholder": "​",
            "style": "IPY_MODEL_3cd27df4c009414a94cdd497cf241b57",
            "value": " 1.36M/1.36M [00:00&lt;00:00, 12.1MB/s]"
          }
        },
        "d8e3030d673a428895a73342c0b9c598": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "dc28651be51745eca94664bbb71ad861": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dce048c43a4348c4a09aa2b46a33f1f3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c95f26ac0139443eba7f2dd7614bc817",
            "max": 1355256,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_45cd7100b77a44f8b6181740e20f42fd",
            "value": 1355256
          }
        },
        "ddee46817f22425c8b95da586a885f0f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "de0547f1e48444d2bcfad75b1cf77e26": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dedd003de05d4bdb9033a0b1b40a1bcc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4f0cfadfd255445dae90afcf10c3d751",
            "max": 456318,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e5d1c331340e4381be2a8d5ed25fdf60",
            "value": 456318
          }
        },
        "df0b22866b3f4b1dbd96d28c01db0bb0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_62ed9c71c4b845e0b36de4553f6ca24e",
            "placeholder": "​",
            "style": "IPY_MODEL_601edd4e281b4ba0a781579e250778c1",
            "value": "config.json: 100%"
          }
        },
        "df5b52e7d83b454585333790d844237f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e37527644fb14daa9ba20129acbdd943": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b091c384f23743bbb7eb578ecde09bb4",
            "placeholder": "​",
            "style": "IPY_MODEL_b75d47c7f07e47f1972112842f25a565",
            "value": "vocab.json: 100%"
          }
        },
        "e52f2d5a6fbe475599a5e0f8f8234ec2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e5d1c331340e4381be2a8d5ed25fdf60": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e6852d226c07498ba6519985b74f7ab7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e6ae61471dd24bf690acfb41fd084f97": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e952eba68ece4c6b864f438d8d42911d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "eb67368acc594f20991fd7af7951ef09": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb7419cdc0a748549ee56223ae667acc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "eba7d68e70e846c8a27847112fe1f3d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6ae61471dd24bf690acfb41fd084f97",
            "placeholder": "​",
            "style": "IPY_MODEL_0bb17b9632a247fbb7284519ffd76a96",
            "value": "merges.txt: 100%"
          }
        },
        "ebdd95042703441da39390f258891d49": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "efc83d30cd024a158ada903f0eb2836d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f36515bfd5e44dffa6a69f778be80772": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9992fa1f43494fc2bbe474773903b78a",
              "IPY_MODEL_dedd003de05d4bdb9033a0b1b40a1bcc",
              "IPY_MODEL_779dd1b4114d4a2c96de5d304a4febf6"
            ],
            "layout": "IPY_MODEL_0b0772747f364fec9ea5cee6082bb697"
          }
        },
        "f5677eb2124b452dbbc80cfb83f0e205": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f5ca910ea9334267b3924c2cd8c559b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f86152e29a7245b7859f5109a31cfbe8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0eddc67139e143cba46520d3172bc4c4",
              "IPY_MODEL_feeeb285817d46619f41466ad9179847",
              "IPY_MODEL_d80bde9aac1f48a5934e309215005048"
            ],
            "layout": "IPY_MODEL_6eae644feb624b05955c432578ca8929"
          }
        },
        "f942289393664f5d98ebbc7686d6261b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c7884273ab104cfb822965ee145dd879",
              "IPY_MODEL_c70b53da90834f7aa84b7927c1864b4a",
              "IPY_MODEL_5bbdc5dae16446e5b9024e533b7ae5ca"
            ],
            "layout": "IPY_MODEL_37fe86c55e574d608d21a382c94bbb03"
          }
        },
        "fad9f2cb18d641b298dd37b1c6b6b380": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "feeeb285817d46619f41466ad9179847": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4960f591b8584185b21d4e95b1ed211a",
            "max": 1355863,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ab87eabe28e64cd7b0c3d2f60ede3c70",
            "value": 1355863
          }
        },
        "ff3a59cf02b5436cb980e3191da0c2fb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_de0547f1e48444d2bcfad75b1cf77e26",
            "max": 124,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_bb7f7cb49f5449248fe260f40f9cfc70",
            "value": 124
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}