12_custom_models_and_training_with_tensorflow.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C7sjDEDiAxrK"
      },
      "source": [
        "**12장 – 텐서플로를 사용한 사용자 정의 모델과 훈련**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YUAFnQdfAxrM"
      },
      "source": [
        "_이 노트북에는 12장의 모든 샘플 코드와 연습 문제에 대한 해답, 부록 C의 코드 예제가 포함되어 있습니다_."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jX7ukQPOAxrM"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/rickiepark/handson-ml3/blob/main/12_custom_models_and_training_with_tensorflow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "sFcMpxP2AxrN"
      },
      "source": [
        "# 설정"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mwy7Yxo_AxrN"
      },
      "source": [
        "이 프로젝트에는 Python 3.7 이상이 필요합니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "xNKQVwanAxrN"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "assert sys.version_info >= (3, 7)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C1IOdJ7mAxrO"
      },
      "source": [
        "그리고 TensorFlow ≥ 2.8:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "0Ddo9OPsAxrO"
      },
      "outputs": [],
      "source": [
        "from packaging import version\n",
        "import tensorflow as tf\n",
        "\n",
        "assert version.parse(tf.__version__) >= version.parse(\"2.8.0\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "JN8_1_vCAxrP",
        "outputId": "1b55ea1d-e6ab-469b-a5a7-bfb59add9c0b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "debconf: unable to initialize frontend: Dialog\n",
            "debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 1.)\n",
            "debconf: falling back to frontend: Readline\n",
            "debconf: unable to initialize frontend: Readline\n",
            "debconf: (This frontend requires a controlling tty.)\n",
            "debconf: falling back to frontend: Teletype\n",
            "dpkg-preconfigure: unable to re-open stdin: \n",
            "Selecting previously unselected package fonts-nanum.\n",
            "(Reading database ... 122349 files and directories currently installed.)\n",
            "Preparing to unpack .../fonts-nanum_20180306-3_all.deb ...\n",
            "Unpacking fonts-nanum (20180306-3) ...\n",
            "Setting up fonts-nanum (20180306-3) ...\n",
            "Processing triggers for fontconfig (2.13.1-2ubuntu3) ...\n"
          ]
        }
      ],
      "source": [
        "import sys\n",
        "# 코랩의 경우 나눔 폰트를 설치합니다.\n",
        "if 'google.colab' in sys.modules:\n",
        "    !sudo apt-get -qq -y install fonts-nanum\n",
        "    import matplotlib.font_manager as fm\n",
        "    font_files = fm.findSystemFonts(fontpaths=['/usr/share/fonts/truetype/nanum'])\n",
        "    for fpath in font_files:\n",
        "        fm.fontManager.addfont(fpath)\n",
        "\n",
        "# 나눔 폰트를 사용합니다.\n",
        "import matplotlib \n",
        "\n",
        "matplotlib.rc('font', family='NanumBarunGothic')\n",
        "matplotlib.rcParams['axes.unicode_minus'] = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y5lX_jAOAxrP"
      },
      "source": [
        "## 넘파이처럼 텐서플로 사용하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PyPC0lBMAxrP"
      },
      "source": [
        "### 텐서 및 연산"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yiDDeZuHAxrQ"
      },
      "source": [
        "#### 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "V1OI3MGjAxrQ",
        "outputId": "9ccc7353-66ed-46df-b378-41f4b829da61",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "t = tf.constant([[1., 2., 3.], [4., 5., 6.]]) # 행렬\n",
        "t"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "U5M77FruAxrQ",
        "outputId": "66d99963-2d01-47cc-aa36-85c322f2d240",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "t.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "CbvAXuawAxrR",
        "outputId": "e2ce4023-ebc6-43c3-d2f4-2dc6a58f58ff",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.float32"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "t.dtype"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DeWM0-UNAxrR"
      },
      "source": [
        "#### 인덱싱"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "_8b8zwp0AxrR",
        "outputId": "3d9d11a7-747c-41ad-8f5b-f7dbd6b72980",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              "array([[2., 3.],\n",
              "       [5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "t[:, 1:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "zr6WTVXWAxrR",
        "outputId": "6566d668-4d6c-4675-a6fc-4d676fd4db41",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 1), dtype=float32, numpy=\n",
              "array([[2.],\n",
              "       [5.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "t[..., 1, tf.newaxis]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r8gzp0IuAxrR"
      },
      "source": [
        "#### 연산"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "A2MP_j5ZAxrS",
        "outputId": "227bd269-efe3-4b43-e76c-c328c2fd867b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[11., 12., 13.],\n",
              "       [14., 15., 16.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "t + 10"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "1pJxYGP_AxrS",
        "outputId": "dc664381-274a-4135-fcb6-810430179cdc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[ 1.,  4.,  9.],\n",
              "       [16., 25., 36.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "tf.square(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "mcLo21_iAxrS",
        "outputId": "8b5753fe-7de0-4786-971f-b3ee4886ef58",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              "array([[14., 32.],\n",
              "       [32., 77.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "t @ tf.transpose(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WkBUbZ7hAxrS"
      },
      "source": [
        "#### 스칼라"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "omn-bldFAxrS",
        "outputId": "f76e48b3-4c68-4ef4-df56-460608d4572c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=42>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "tf.constant(42)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V46tEuLXAxrS"
      },
      "source": [
        "#### 케라스의 저수준 API"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Un53vnAOAxrT"
      },
      "source": [
        "여전히 케라스 저수준 API를 사용하는 코드를 실행할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "L82N01k2AxrT",
        "outputId": "42fcfd1d-d5ef-4c33-c514-0d66640f4056",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[11., 26.],\n",
              "       [14., 35.],\n",
              "       [19., 46.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "K = tf.keras.backend\n",
        "K.square(K.transpose(t)) + 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qkoesP6uAxrT"
      },
      "source": [
        "하지만 케라스는 더 이상 멀티 백엔드를 지원하지 않으므로 대신 TF의 저수준 API를 직접 사용해야 합니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "gBDCe42PAxrT",
        "outputId": "f6b728dc-0596-403c-d3bd-8bc7d0200f39",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[11., 26.],\n",
              "       [14., 35.],\n",
              "       [19., 46.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "tf.square(tf.transpose(t)) + 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lLBJR7srAxrT"
      },
      "source": [
        "### 텐서와 넘파이"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "fi4jcBHHAxrT",
        "outputId": "56eb8f7d-43df-42b5-f0a1-af8e17374d28",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([2., 4., 5.])>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "a = np.array([2., 4., 5.])\n",
        "tf.constant(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "DfgBFTIlAxrT",
        "outputId": "a74644ff-49e4-43fd-a5ee-0fe2be130ad7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "t.numpy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "JBXYk1dXAxrT",
        "outputId": "5421902f-2e34-4a97-853b-1c52861cbaa9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "np.array(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "E555f4-eAxrU",
        "outputId": "e40ddd97-0778-45f7-b180-6cd0c42f56c7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 4., 16., 25.])>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "tf.square(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "u3zp3ymEAxrU",
        "outputId": "f171b309-011e-467b-8d3c-71127ffe12ff",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1.,  4.,  9.],\n",
              "       [16., 25., 36.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "np.square(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bkrb8RMGAxrU"
      },
      "source": [
        "### 타입 변환"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "1T2zObbgAxrU",
        "outputId": "284a1c0c-2437-4d05-d63d-606d35d69c2d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cannot compute AddV2 as input #1(zero-based) was expected to be a float tensor but is a int32 tensor [Op:AddV2]\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    tf.constant(2.0) + tf.constant(40)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "REJyTlwkAxrU",
        "outputId": "cc2a018b-5c21-462c-9577-adf9479015a4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cannot compute AddV2 as input #1(zero-based) was expected to be a float tensor but is a double tensor [Op:AddV2]\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    tf.constant(2.0) + tf.constant(40., dtype=tf.float64)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "voj491m-AxrU",
        "outputId": "027e2a21-6b3f-47ab-cba8-eb4360d2f4cd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=42.0>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "t2 = tf.constant(40., dtype=tf.float64)\n",
        "tf.constant(2.0) + tf.cast(t2, tf.float32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K6E_xZ7FAxrU"
      },
      "source": [
        "### 변수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "ojTQabFEAxrV",
        "outputId": "6263d6e4-3b7e-4011-e3b8-bb552c0b8f6f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "v = tf.Variable([[1., 2., 3.], [4., 5., 6.]])\n",
        "v"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "id": "zk6PmMzbAxrV",
        "outputId": "b7f1ecee-3013-4a4c-cbe7-bb751c81914d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2.,  4.,  6.],\n",
              "       [ 8., 10., 12.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "v.assign(2 * v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "ydDSJtkDAxrV",
        "outputId": "2ebc2e63-2948-4513-dc3d-b2bf4f25d5c9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2., 42.,  6.],\n",
              "       [ 8., 10., 12.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "v[0, 1].assign(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "tags": [],
        "id": "5GbX-IxqAxrV",
        "outputId": "1d6f0556-da32-4ac4-f13b-1d7464ed04e2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2., 42.,  0.],\n",
              "       [ 8., 10.,  1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "v[:, 2].assign([0., 1.])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "4OX5vh4rAxrV",
        "outputId": "0b87c639-69f1-4231-e527-8f347b68b5a2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[100.,  42.,   0.],\n",
              "       [  8.,  10., 200.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "v.scatter_nd_update(\n",
        "    indices=[[0, 0], [1, 2]], updates=[100., 200.])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "EA-7BthwAxrV",
        "outputId": "4078a069-6f31-4537-d4b6-03660e883fe0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[4., 5., 6.],\n",
              "       [1., 2., 3.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "# 추가 코드 - scatter_update() 사용 방법을 보여줍니다.\n",
        "sparse_delta = tf.IndexedSlices(values=[[1., 2., 3.], [4., 5., 6.]],\n",
        "                                indices=[1, 0])\n",
        "v.scatter_update(sparse_delta)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "cXN3oYYjAxrV",
        "outputId": "4a74ab92-0419-45b2-c533-916671b1b4ae",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "'ResourceVariable' object does not support item assignment\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    v[1] = [7., 8., 9.]\n",
        "except TypeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d7G6WFDrAxrW"
      },
      "source": [
        "### 문자열"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zD1fDn0hAxrW"
      },
      "source": [
        "이 섹션과 이후 섹션의 코드는 부록 C에 해당합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "GsDN85PRAxrW",
        "outputId": "3e131d14-d7a1-44c6-9cd7-a536e68b69ee",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'hello world'>"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "tf.constant(b\"hello world\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "bSJynyadAxrW",
        "outputId": "5379d3ae-ef07-41bf-e4c7-3bcb57f77010",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "tf.constant(\"café\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "-Xy-h4QkAxrW",
        "outputId": "54d678ff-e864-4e1a-b5b7-eabfdcefca3b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "u = tf.constant([ord(c) for c in \"café\"])\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "rQL2muPVAxrW",
        "outputId": "c8c70aaa-ca86-4306-9713-ebe7ddafa07a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=4>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "b = tf.strings.unicode_encode(u, \"UTF-8\")\n",
        "tf.strings.length(b, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "2YvVp9bSAxrW",
        "outputId": "dfb09bab-f207-4c91-c3ac-25292cbdf7c0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "tf.strings.unicode_decode(b, \"UTF-8\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6RpxZM16AxrW"
      },
      "source": [
        "### 다른 데이터 구조"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RI0_AWrFAxrW"
      },
      "source": [
        "이 섹션의 코드는 부록 C에 있습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gZNHPgBiAxrX"
      },
      "source": [
        "#### 문자열 배열"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "4jMNXDJoAxrX",
        "outputId": "4e957609-6333-415f-cdee-78d294798096",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'hello world'>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "tf.constant(b\"hello world\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "id": "m2MTywzxAxrX",
        "outputId": "a3c8bf38-59a8-4a57-f372-631182222e9b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ],
      "source": [
        "tf.constant(\"café\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "id": "Zi5cZWcHAxrX",
        "outputId": "f0d6de9c-ea88-4f57-9d77-b0a20517766c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "u = tf.constant([ord(c) for c in \"café\"])\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "Mhgz8OoxAxrX",
        "outputId": "25c820cb-1e3a-46ce-b155-96ec566701eb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "b = tf.strings.unicode_encode(u, \"UTF-8\")\n",
        "b"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "id": "M_hZqjvnAxrX",
        "outputId": "278816e4-47c4-4f23-bd25-9cadbc4054f8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=4>"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ],
      "source": [
        "tf.strings.length(b, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "id": "WB9KciAFAxrX",
        "outputId": "f4f96a8b-b80d-4528-ef72-61b0ac2307b0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ],
      "source": [
        "tf.strings.unicode_decode(b, \"UTF-8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "XYVKG6PCAxrX"
      },
      "outputs": [],
      "source": [
        "p = tf.constant([\"Café\", \"Coffee\", \"caffè\", \"咖啡\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "id": "__ZYEY-IAxrX",
        "outputId": "bf3994c1-0c70-415c-cdb8-671d346fbc00",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([4, 6, 5, 2], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "tf.strings.length(p, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "id": "bw01QOHRAxrY",
        "outputId": "c74ea5ef-62b8-434d-a498-ea83ba5e7f38",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 97, 102, 233], [67, 111, 102, 102, 101, 101],\n",
              " [99, 97, 102, 102, 232], [21654, 21857]]>"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ],
      "source": [
        "r = tf.strings.unicode_decode(p, \"UTF8\")\n",
        "r"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5vRBco4hAxrY"
      },
      "source": [
        "#### 래그드 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "id": "-CM3R_m5AxrY",
        "outputId": "7e07c590-2973-4a54-a5b1-a6c84b798c8d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(6,), dtype=int32, numpy=array([ 67, 111, 102, 102, 101, 101], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "r[1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "id": "3sgYgXB3AxrY",
        "outputId": "c8fba645-b671-4cc2-f7b8-035b8226906c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 111, 102, 102, 101, 101], [99, 97, 102, 102, 232]]>"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ],
      "source": [
        "r[1:3]  # 추가 코드 - 래그드 텐서의 슬라이스는 래그드 텐서입니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "id": "eWw-dmXJAxrY",
        "outputId": "a431eb58-7ed5-4652-9589-52b5e552e5fb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 97, 102, 233], [67, 111, 102, 102, 101, 101],\n",
              " [99, 97, 102, 102, 232], [21654, 21857], [65, 66], [], [67]]>"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ],
      "source": [
        "r2 = tf.ragged.constant([[65, 66], [], [67]])\n",
        "tf.concat([r, r2], axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "id": "IgM0lcTlAxrY",
        "outputId": "a6ba4385-730e-42f7-9940-132a82c47de6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<tf.RaggedTensor [[67, 97, 102, 233, 68, 69, 70], [67, 111, 102, 102, 101, 101, 71],\n",
            " [99, 97, 102, 102, 232], [21654, 21857, 72, 73]]>\n"
          ]
        }
      ],
      "source": [
        "r3 = tf.ragged.constant([[68, 69, 70], [71], [], [72, 73]])\n",
        "print(tf.concat([r, r3], axis=1))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "id": "7Q7rYycBAxrY",
        "outputId": "63bbc532-59ca-4ead-c011-2b636de2307e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 6), dtype=int32, numpy=\n",
              "array([[   67,    97,   102,   233,     0,     0],\n",
              "       [   67,   111,   102,   102,   101,   101],\n",
              "       [   99,    97,   102,   102,   232,     0],\n",
              "       [21654, 21857,     0,     0,     0,     0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ],
      "source": [
        "r.to_tensor()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_-k1oueCAxrY"
      },
      "source": [
        "#### 희소 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "id": "9czxHg_BAxrY"
      },
      "outputs": [],
      "source": [
        "s = tf.SparseTensor(indices=[[0, 1], [1, 0], [2, 3]],\n",
        "                    values=[1., 2., 3.],\n",
        "                    dense_shape=[3, 4])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "id": "LO9aP3-eAxrZ",
        "outputId": "72272654-e98b-4ccd-ff06-7e8d0fe5379c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 1., 0., 0.],\n",
              "       [2., 0., 0., 0.],\n",
              "       [0., 0., 0., 3.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ],
      "source": [
        "tf.sparse.to_dense(s)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "id": "Hx-pnTOXAxrZ",
        "outputId": "48cf11ec-cc5e-4b0c-c382-ddf749c47cf7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SparseTensor(indices=tf.Tensor(\n",
              "[[0 1]\n",
              " [1 0]\n",
              " [2 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([ 42.  84. 126.], shape=(3,), dtype=float32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "s * 42.0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "id": "1AUkxI9zAxrZ",
        "outputId": "6841d790-9769-4a01-c632-19046ee28b73",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "unsupported operand type(s) for +: 'SparseTensor' and 'float'\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    s + 42.0\n",
        "except TypeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "id": "pFgjJfkJAxrZ",
        "outputId": "73c8c666-e98e-46ef-eee0-3ba6b94aae34",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[ 30.,  40.],\n",
              "       [ 20.,  40.],\n",
              "       [210., 240.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서와 밀집 텐서를 곱하는 방법을 보여줍니다.\n",
        "s4 = tf.constant([[10., 20.], [30., 40.], [50., 60.], [70., 80.]])\n",
        "tf.sparse.sparse_dense_matmul(s, s4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {
        "id": "yHory7jKAxrZ",
        "outputId": "eff2dda8-e380-4faa-ce95-3815790de8ed",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{{function_node __wrapped__SparseToDense_device_/job:localhost/replica:0/task:0/device:GPU:0}} indices[1] is out of order. Many sparse ops require sorted indices.\n",
            "  Use `tf.sparse.reorder` to create a correctly ordered copy.\n",
            "\n",
            "\n",
            "\t [[{{node SparseToDense}}]] [Op:SparseToDense]\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서를 생성할 때 값을 \"읽는 순서\"로 제공해야 하며, 그렇지 않으면 `to_dense()`가 실패합니다.\n",
        "s5 = tf.SparseTensor(indices=[[0, 2], [0, 1]],  # 잘못된 순서!\n",
        "                     values=[1., 2.],\n",
        "                     dense_shape=[3, 4])\n",
        "try:\n",
        "    tf.sparse.to_dense(s5)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "id": "OqVH-dpGAxrZ",
        "outputId": "5c4da9d6-bee2-4c72-91e5-2c3382c77e08",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 2., 1., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서 s5의 값을 재정렬하여 수정하는 방법을 보여줍니다.\n",
        "s6 = tf.sparse.reorder(s5)\n",
        "tf.sparse.to_dense(s6)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "To8AbsxmAxra"
      },
      "source": [
        "#### 텐서 배열"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "id": "0BxExIGRAxra"
      },
      "outputs": [],
      "source": [
        "array = tf.TensorArray(dtype=tf.float32, size=3)\n",
        "array = array.write(0, tf.constant([1., 2.]))\n",
        "array = array.write(1, tf.constant([3., 10.]))\n",
        "array = array.write(2, tf.constant([5., 7.]))\n",
        "tensor1 = array.read(1)  # tf.constant([3., 10.])을 반환하고 (0으로 채웁니다!)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "id": "Pa6Wau4mAxra",
        "outputId": "10fa2211-7a9d-4efd-832a-e954ef13936d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[1., 2.],\n",
              "       [0., 0.],\n",
              "       [5., 7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ],
      "source": [
        "array.stack()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "id": "BoO188ZgAxra",
        "outputId": "a8a7498e-15e6-4a14-b4f2-04ac4538e4a3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[ 1.,  2.],\n",
              "       [ 3., 10.],\n",
              "       [ 5.,  7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ],
      "source": [
        "# 추가 코드 - clear_after_read를 비활성화하는 방법을 보여줍니다.\n",
        "array2 = tf.TensorArray(dtype=tf.float32, size=3, clear_after_read=False)\n",
        "array2 = array2.write(0, tf.constant([1., 2.]))\n",
        "array2 = array2.write(1, tf.constant([3., 10.]))\n",
        "array2 = array2.write(2, tf.constant([5., 7.]))\n",
        "tensor2 = array2.read(1)  # tf.constant([3., 10.])를 반환합니다.\n",
        "array2.stack()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {
        "id": "a1bKgzxHAxra",
        "outputId": "350b1a5f-fbea-4240-d76c-78af7af73dde",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[1., 2.],\n",
              "       [0., 0.],\n",
              "       [5., 7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ],
      "source": [
        "# 추가 코드 - 동적 크기의 텐서 배열을 생성하고 사용하는 방법을 보여줍니다.\n",
        "array3 = tf.TensorArray(dtype=tf.float32, size=0, dynamic_size=True)\n",
        "array3 = array3.write(0, tf.constant([1., 2.]))\n",
        "array3 = array3.write(1, tf.constant([3., 10.]))\n",
        "array3 = array3.write(2, tf.constant([5., 7.]))\n",
        "tensor3 = array3.read(1)\n",
        "array3.stack()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z9DLr8QEAxra"
      },
      "source": [
        "#### 집합"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "id": "Lw6V_M4sAxra",
        "outputId": "913dfcd4-a576-441f-879b-c066ac778a52",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SparseTensor(indices=tf.Tensor(\n",
              "[[0 0]\n",
              " [0 1]\n",
              " [0 2]\n",
              " [0 3]\n",
              " [0 4]], shape=(5, 2), dtype=int64), values=tf.Tensor([ 1  5  6  9 11], shape=(5,), dtype=int32), dense_shape=tf.Tensor([1 5], shape=(2,), dtype=int64))"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ],
      "source": [
        "a = tf.constant([[1, 5, 9]])\n",
        "b = tf.constant([[5, 6, 9, 11]])\n",
        "u = tf.sets.union(a, b)\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "id": "JOCIBh5DAxra",
        "outputId": "f1bb3475-0264-40ee-d933-763941641b86",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[ 1,  5,  6,  9, 11]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ],
      "source": [
        "tf.sparse.to_dense(u)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "id": "Ysq--SHTAxra",
        "outputId": "548e62d1-12fe-47fa-d431-e05f4586a6fc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 5), dtype=int32, numpy=\n",
              "array([[ 1,  5,  6,  9, 11],\n",
              "       [ 0, 10, 13,  0,  0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ],
      "source": [
        "a = tf.constant([[1, 5, 9], [10, 0, 0]])\n",
        "b = tf.constant([[5, 6, 9, 11], [13, 0, 0, 0]])\n",
        "u = tf.sets.union(a, b)\n",
        "tf.sparse.to_dense(u)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "id": "_v3UrL4iAxrb",
        "outputId": "8cc36dda-235b-4dbb-c91d-4cef26bdabc3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 5), dtype=int32, numpy=\n",
              "array([[ 1,  5,  6,  9, 11],\n",
              "       [-1, 10, 13, -1, -1]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ],
      "source": [
        "# 추가 코드 - 다른 기본값(이 경우 -1)을 사용하는 방법을 보여줍니다.\n",
        "a = tf.constant([[1, 5, 9], [10, -1, -1]])\n",
        "b = tf.constant([[5, 6, 9, 11], [13, -1, -1, -1]])\n",
        "u = tf.sets.union(a, b)\n",
        "tf.sparse.to_dense(u, default_value=-1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "id": "KSgMPHJDAxrb",
        "outputId": "c2864006-140d-4b6d-e6e6-5ebde0799c0a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[2, 3, 7],\n",
              "       [7, 0, 0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ],
      "source": [
        "# 추가 코드 - `tf.sets.difference()`를 사용하는 방법을 보여줍니다.\n",
        "set1 = tf.constant([[2, 3, 5, 7], [7, 9, 0, 0]])\n",
        "set2 = tf.constant([[4, 5, 6], [9, 10, 0]])\n",
        "tf.sparse.to_dense(tf.sets.difference(set1, set2))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "id": "0eooV96SAxrb",
        "outputId": "7591c2ce-49db-4f98-97f2-4ef8cb7795df",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[5, 0],\n",
              "       [0, 9]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ],
      "source": [
        "# 추가 코드 - `tf.sets.difference()`를 사용하는 방법을 보여줍니다.\n",
        "tf.sparse.to_dense(tf.sets.intersection(set1, set2))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "id": "6SwJyFhUAxrb",
        "outputId": "15ff4c26-8957-4455-da2c-42bd3c24a5db",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1,), dtype=bool, numpy=array([ True])>"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ],
      "source": [
        "# 추가 코드 - set1[0]에 5가 포함되어 있는지 확인합니다.\n",
        "tf.sets.size(tf.sets.intersection(set1[:1], tf.constant([[5, 0, 0, 0]]))) > 0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bsHvqIjtAxrb"
      },
      "source": [
        "#### 큐"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "id": "1_1mwRJ3Axrb",
        "outputId": "dbab3766-c826-45df-c519-9be28a6b58d0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ],
      "source": [
        "q = tf.queue.FIFOQueue(3, [tf.int32, tf.string], shapes=[(), ()])\n",
        "q.enqueue([10, b\"windy\"])\n",
        "q.enqueue([15, b\"sunny\"])\n",
        "q.size()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "id": "5WQ1dE_HAxrb",
        "outputId": "2e46dede-f613-48c7-bedb-d1e6fccdf339",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=int32, numpy=10>,\n",
              " <tf.Tensor: shape=(), dtype=string, numpy=b'windy'>]"
            ]
          },
          "metadata": {},
          "execution_count": 68
        }
      ],
      "source": [
        "q.dequeue()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "id": "udUYygHeAxrb"
      },
      "outputs": [],
      "source": [
        "q.enqueue_many([[13, 16], [b'cloudy', b'rainy']])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "id": "tJlE67eKAxrc",
        "outputId": "a8c7d2a4-50a8-495e-ff8d-89ea137f1b22",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(3,), dtype=int32, numpy=array([15, 13, 16], dtype=int32)>,\n",
              " <tf.Tensor: shape=(3,), dtype=string, numpy=array([b'sunny', b'cloudy', b'rainy'], dtype=object)>]"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ],
      "source": [
        "q.dequeue_many(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dw_LFcqOAxrc"
      },
      "source": [
        "## 사용자 정의 손실"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {
        "id": "h8ITtrkcAxrc"
      },
      "outputs": [],
      "source": [
        "def huber_fn(y_true, y_pred):\n",
        "    error = y_true - y_pred\n",
        "    is_small_error = tf.abs(error) < 1\n",
        "    squared_loss = tf.square(error) / 2\n",
        "    linear_loss  = tf.abs(error) - 0.5\n",
        "    return tf.where(is_small_error, squared_loss, linear_loss)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "id": "sMf7oMPXAxrc",
        "outputId": "4684955b-27e0-4314-daa5-2ef97ccd638b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 373
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 800x350 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqIAAAFkCAYAAAD7WFnHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3fElEQVR4nO3deVhU1RvA8e+wCAiCigqI4pJLmpaaO+5LbuVapmZpm6Wm/krNJXdNM8stbdNcytTKXLPc9wVNXDIXLHMPUFBB1hmY+f1xAkRAGRi4M8P7eZ554t65c+f1dGHeOfec9+hMJpMJIYQQQggh8pmD1gEIIYQQQoiCSRJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEEEIIoQlJRIUQQgghhCYkERVCCCGEEJqQRFQIIfJAbGwsq1ev5uTJk1qHIoQQVksSUSFEgdO3b190Ol2mj7t371rkPW7dukXv3r1ZsWJFjs9RrVo1XF1dc/Te06ZNY8eOHVke06RJE8aPH5/j2IQQwhKctA5ACCHy26BBg2jfvn2mz7m7u2e6/86dO+zdu5fHH3+cxx9/PN1zW7ZswcPDgyZNmlgsxhs3bvDXX39Rv379HL12/PjxfPDBB7Rp0ybTYw4ePIivr29uwxRCiFyRRFQIUSAsWrSIyMjIRx736aefpv7cp08fAgICAAgJCaFbt25MnDiRSZMmpXtN//79efzxx9mzZ4/F4h03bhzJycno9XoSExNxcXHJ9mv/+ecfgNTYhRDCWkkiKoQoED799FNCQkLMek3Dhg01SeZmzJjBsmXLePrppzlx4gQdOnTg+++/x8/PL1uv/+233wAsNsxACCHyiowRFUIUCH/88Qfx8fFmPZo1a5avMYaHh9OnTx/Gjh1Lly5dOHz4MKtWreLIkSPUqFGDhQsXEhcX99BzXLp0ie+//x4HBwcmTpzIsWPH8il6IYQwnySiQogCoVChQri6uhITE8OHH35Is2bN8PPzw8PDg2LFivHEE0/w6quvcuTIEVxdXXF1dcXBIeOfyKCgIObOnZvu8ajk8GEMBgP79+/nnXfeoVKlSqxZs4bp06ezdu1anJ2d6dmzJydPnqRGjRq88847lC1bljfffJP169dnONe9e/d4/vnn0ev1bNmyhYCAANq3b8/Ro0dzHJ8QQuQluTUvhCgw7ty5Q506dbh9+zajRo2iefPmeHt7k5iYyNmzZ5k/fz4tWrRgyZIlvPrqq5me4/Dhw/z555/p9sXFxbF37150Op3ZMUVGRtK9e3cSExN58cUXGTt2LBUrVgTU2NOyZcsydepU9u7dy+7du/niiy/48ccfqVOnTrrz/PPPP3Tv3p1Tp06xYMEC2rZty7Zt22jZsiVNmzZlypQpDBs2LEez8IUQIs+YhBCigPj5559NgGn27NmZPh8XF2dydnY2NWvWLMNzhw8fNgGmqVOnZnjO39/fVKlSJdNnn32W+pg8ebIJMA0fPvyRcV27ds2k1+sz7Hd3dzc1aNAgw/77j01KSjLNnDnT5O7ubnJzczMtWbIk3bG3bt0yde7c2QSY2rVrl7ofMPXo0eORsQkhRF6SHlEhRIHRsGFDihYtyvz58wkICKBZs2Z4e3uTkJDAuXPnmDt3LgaDgY4dO5p9bn9/f955553U7cuXLzNx4sRsvbZMmTJmvZezs3Pqz46Ojuj1eho2bMj8+fOpXr16umNLlCjBhg0b2LNnD1WqVDHrfYQQIq9JIiqEKDBKly7NkSNHmDx5Mq+99hrR0dGpzzk4OFC7dm2WLVtGv3798iWe/v37s3z58ocec+TIkYfe8l+1ahXjxo1j3LhxAOj1evR6fYbj6tatC0BMTAygxpM6OclHgBBCW/JXSAhRoFSpUoXvv/8ek8lEeHg49+7do1ChQvj4+OT7+Ml+/frRsGHDXJ2jXr166banT5/O5MmTs/XaHj16sGbNmly9vxBC5IYkokIIu/b777+bXT/0QX379k39OTg4mMWLF6d7Pqez5lu2bEnLli0BNZFKp9NRtGjRHMcJ0L17dypVqvTI4/Kr11cIIR5GElEhhF1bvnw5CxcuzNU57k9E169fn2nppNyqV68erq6uGWbkm6tSpUqULl36kcflZIa/EEJYmtQRFULYtQULFmAymbJ8uLi40Lx584ceA2qi08OOseTynrnx8ccfU7JkyUc+kpOTtQ5VCCGkR1QIIezRggUL8Pf3f+gxj3peCCHymiSiQghhJW7fvs2CBQseeZyLiwtvvvnmQ4+JiIjA0dHxoceEhYXx1FNPUahQIbPiFEIIS5FEVAghrERoaChDhgx55HFeXl6PTEQnTZqUrffs1auXJKJCCM3oTCkDoIQQQgghhMhHOZ6sNHPmTHQ6XZYD9I1GI+PGjaNMmTKUKlWKDh06cPny5Zy+nRBCCCGEsDM5SkTPnDnDihUrHjrQfebMmaxdu5Zjx44RGhpK9erV6dSpE0lJSTkOVgghhBBC2A+zE9GkpCT69+/PZ599luXycCaTiXnz5jFu3Dh8fX1xdHRk2rRpXL16le3bt+c6aCGEEEIIYfvMTkQ//PBD6tevT4sWLbI85tKlS4SHhxMYGJi6z83NjTp16hAUFJSjQIUQQgghhH0xa9b88ePHWbFiBSdOnHjoceHh4QD4+Pik2+/j45P6XGYSExNJTExM3TYajdy+fRtvb29ZBUQIIYQQwgqZTCbu3btH6dKlcXAwr48z24moXq+nf//+fPnll3h4eDz0WKPRCGRcQs7BwSH1uczMmDGDyZMnZzckIYQQQghhJa5du0aZMmXMek22E9EpU6bQqFEjWrdu/chjvb29AVWc2c/PL3X/7du3qVixYpavGzNmDO+9917qdlRUFAEBAVy4cIHixYtnN9QCz2AwsHv3blq2bImzs3O2XqPbvRvdsWMY33sPHlEE2x7lpM2EtJu5YmNjKVeuHAAXL17Ey8tL44hsh1xr5ivQbRYTg1PNmhjHjME4YIBZLy3Q7ZZDt2/fpkqVKhQpUsTs12Y7ET169ChBQUH88MMPqfuio6N59tlnqV27Nvv370/dX6lSJby8vAgODubZZ58F1CSnEydOMOAhF4SLiwsuLi4Z9hcvXjw1uRWPZjAYKFy4MN7e3tn/JXr+efUooHLUZkLazUyurq6pPxcvXpyiRYtqF4yNkWvNfAW6zYoWhZUroVYtMDN/KNDtlks5GUaZ7Rv527ZtIzo6mrt376Y+AgIC+OWXX9ixYwfPPPMMc+bMAcDJyYm33nqLDz74gNDQUAwGA+PGjcPd3Z1OnTqZHaTIJ1FRMHcu3LundSRCCCFEzjk6wrPPgpm3iUX+y3FB+/slJSUREhLC1atXU/dNmzaNFi1a8NRTT1GqVCl+//13tm7dipubmyXeUuSF6GgYOxaksoEQQghbdeQIvPwy3L2rdSQiG3K11vz9KyVduXIl3XPOzs7MmzePefPm5eYtRH4qWxbCwyEHYzyEEEIIq3D7NoSFgaen1pGIbLBIj6iwI0WKQHKyfJMUQghhmzp0gO3bwcwyQkIb8n9JZNSuHQwdqnUUQgghhHmCglRvqLAZubo1L+zUe+9BiRJaRyGEEEKYZ8AANVP+22+1jkRkkySiIqOOHbWOQAghhDDf3r0QG6t1FMIMcmteZO74cejXT40XFUIIIayd0QjFiknJJhsjiajIXHIynDsnY22EEEJYv6tXoXx5OHZM60iEmSQRFZmrVw+OHgV/f60jEUIIkdeOHYNXXoFKlUCng3HjtI7IPDoddOkCjz+udSTCTJKIioc7fx5u3NA6CiGEEHnp4EE147xJE/Dysuy5f/wROnUCPz917mbN4MABy75H2bLw2Wfg4WHZ84o8J4moyJrBAE2bwoIFWkcihBAiLw0ZAhcuwLJlap12S5o7V1ViWbgQfvpJ3Wlr3RpOnbLM+XfuhB9+AJPJMucT+UpmzYusOTvDtm1QvbrWkQghhMhLeVn8fdMm8PZO227TBmrWVInp119b5vynT8OLL+b+XCLfSSIqHq52bfVfk0mNwRFCCCHMcX8SCirprVEDLl2yzPnnzoX4eMucS+Q7uTUvHu3rr6FFC7ntIYQQIveSk+H339XEqNy6fl39180t9+cSmpBEVDxa5cpqcLnBoHUkQgghbN2CBarc0qBBuTtPZKT6fFq2zCJhCW3IrXnxaC1bqocQQgiRG0eOwOjRqjxUzZq5O5eXF6xcqWb6C5slPaIie2JjVWmMmze1jkQIIYQtunxZ1fp87jmYODH353Nygm7doGTJ3J9LaEYSUZE9iYnqG6yla78JIYSwPS+8oCawPuxx/+fF3buqlmj58rB8ee4nv27eDK++qj6bhE2TW/Mie4oXV4PCixTROhIhhBCWdusW7N2rfo6LU4uZrFkD7u7QoUPG46tVg379Mu6/ehV271bl/558Uu3T66F7d3XeXbssM7EoNladz8Ul9+cSmpJEVGRfkSJgNEJEBJQqpXU0QgghLOXMGdXLmeLnn9WjXDl1S/1BU6Zk3Hf5sqqw4uysVlPy9FT7Bw1SSe6iRapkU0rZJheXtBKB5urZUz2EzZNEVJjntdfgr7/UcnBCCCHsQ25L9KUkof/+q5LQrl3TntuxQ3VivP56+tdkleQ+yrZtUKeOWq1J2DxJRIV5Bg6UMk5CCCHS3J+E/vSTmpD04POWotfDSy+pXtbJky13XqEZSUSFeRo00DoCIYQQ1uJRSailFSoE586Bo2Pevo/INzJrXpjvn3/UH5vwcK0jEUIIoZX8TkKNRkhKUrfkixXL2/cS+UYSUWG+YsXUhKV//9U6EiGEELn1qDJM9z9S3J+ErlmT90koqLGhFStCaGjev5fIN2Ynoj/++COBgYH4+Pjg7+9Px44dOXXqVKbHTpo0CQ8PD3x9fdM9EqXul20rVkxNVsrpbEchhBDWw2SCH36Ajh3B11fNdm/aFPbvV8/d/4C0JDQ0VCWhnTvnT5wVKkD//ipGYTfMTkQ3b97MJ598QlhYGFeuXKF27dq0b98eo9GY6fEjRowgLCws3cNF6n7Zh7//hmPHtI5CCCFEbs2dq255L1yobrP7+0Pr1vBgR9P9SehPP+VfEgpQtaoqG5XbYvjCqpg9WWn58uVpL3ZyomfPnkyfPp2IiAhKSW3JgmXQIHBwgC1btI5ECCFEbmzaBN7eadtt2qi14BcuhK+/Ttvfrx9cuQJVqsDaterxoD594JlnLBvf/PnqPdu3t+x5heZyNWs+NDSUmTNn8swzz+RpEpqUlGenFrmxaJEUthdCCHtwfxIKqpOhRo204vOgJgsFB6ufL1xQj8zcXxjfEkwm+PVXtUyoJKJW6Ztvcj7lKMeJaNOmTTlw4ACNGzdmw4YNWR43b948vvrqK5ydnalZsyYjR46kRYsWmR6bmJiYbvxodHQ0AH37OvDjjwY8PHIabcFi+K/OpyGv632WLq3+GxenVtKwYfnWZnZG2s0897eTwWCQdjODXGvmy1WbJSfj9PvvGNu2xXj/6+/cye6bm/+eD7NpEyQn50sda7nWss9ohDFjHJgzJ+fltHQmU86XUrh58yZjx47l0KFDHD9+HFdX13TP37lzh8KFC+Pi4sLt27dZvHgxEyZMYOfOnQQGBmY436RJk5icaYHaKCpWNDJuXBDFi8tEJ2vifeYM9T7+mD1z5pBQvLjW4Qhh1RISEujVqxcAq1evzvA3UwhrUXHTJp5YupQ9s2dzr3x5zeLQJSfjcf0698qV0ywGkbnERAfmzn2aw4dLA9GAF1FRUXimLO2aTblKRAH0ej1FihRh7dq1dOrU6ZHHt2rVimrVqrFw4cIMz2XWI1q2bFkgCvAkIMDEhg1JPPFEbiK2fwaDge3bt9O2bVuc87qn8u5dHD75BOOwYVCyZN6+Vx7K1zazI9Ju5omNjaXYf/UPb968SdGiRbUNyIbItWa+nLaZ7uhRHNu0wThiBMYJE/IwwmzEsm4dTi++iOHsWahUKV/eU661R7t1C7p3d+TIEXVL3sEhCqOxaI4SUbNuzScnJ+P4wGoGDg4OODo6ZtiflYSEBIpn0XPm4uKS6Yz6MmVMXL8OV6/qaN7cmbVr1WQ+8XDOzs55/0tUsiTMnIm9rHGRL21mh6Tdsuf+NpI2yxlpN/OZ1WaXL0OPHvDcczhOmYKj1jPUu3WDbdtwrlYt399arrXMXbigKn1dvKi23d3hm2+S+e9mj9nMGl16+vRpOnfuzOnTpwHVGzp8+HB8fHxo2rQpvXv3ZsSIEanHjx07litXrgCqt/Ojjz4iJCSEt99+26wgt25N4umn1c/R0Wqs8n2T94U1WLQI1q3TOgohhBA5dfcudOoE5curD1mtk1BQ8w/attU6CvGfgwehUaO0JLR0aVVutk2bnN9cNysRrVmzJm3btqV///74+PhQoUIFrly5wvbt23F3d+fChQv8888/qcd7eXnRoUMHSpUqhZ+fH4cOHWL//v34+/ubFaSPD+zdC889p7aTklRN20mT0urrCo399hsEBWkdhRBCiJzQ66F7dzX5dMMGcHPTOiJ4801VN1RYhR9+UHejb99W2zVqqI/93K5tY9ateUdHR4YMGcKQIUMyfT44pazDf0aNGsWoUaNyHt193N1Vh9uwYaqsGcDkyaqyxKJFUKiQRd5G5NRPP0E2h2cIIYSwMoMGqR6fRYvUB2tK2SYXF+1W0atUSfVECU2ZTPDxxzB6dNq+tm3Vx76XV+7Pn6s6ovnN0RE++wweewyGD1eN8+23cO2aqqkr4/415Oio/oecOAF16mgdjRBCCHPs2KFq8bz+evr95cqpcaNasFBHlsi5pCQYMgS+/DJt36uvwldfWa5qY84rkGpEp4N331WZeErlk927ITBQLfYgNLR+PTz9dNZFjoUQQliny5czritvMmmThMbHw5w5EBWV/+8tUt27p1ZwvT8JnToVvvnGsqXDbS4RTdGjh0pAS5RQ22fPQsOGsvS5pjp1gl27oHJlrSMRQghhq44cgbFjISJC60gKrH//hebN1fQPUInnd9/BuHGWn8Nms4koqMQzKCgt7wkLUw33yy/axlVgFSoELVuqq1RmkQkhhMiJFi3UB/pjj2kdSYF0+jQ0aKBG2oEa9rhtG/TtmzfvZ9OJKKjr9PBhaNJEbcfFQZcu8Pnn2sZVYJlM6mqdPl3rSIQQQtia8HC1lKclZsEIs+3YofKp69fVdvnycOiQ+m6QV2w+EQXw9obt20ktpmo0wuDBMGKE+lnkI51O1XSoWFHrSIQQQtiaPn3IcWV0kStLl0KHDqpeO0DduqqjL6/XErCpWfMP4+oK33+vsvePPlL7Pv1UjbP+7jvrKIlWYNxf40EIIYTIro8+UjVNRb4xmWDiRDURKUXnzrBypSqdmdfsokc0hYMDzJihygqklLT8+Wdo1Uqtiyry0Y0bqvCYjBUVQgiRXfXqqTI4Il/o9fDKK+mT0CFDVEnM/EhCwc4S0RQDBqgJSx4eajsoSE1skqpC+ejCBTVO9L6VtoQQQohMXbsG7drJZ0Y+unNHNfmKFWpbp1NVs+bNy9/1aewyEQW1Hv3+/WodVFDXdqNGcOCAtnEVGC1aqF5RmfUohBDiUSIiVPX0kiW1jqRAuHxZdTzv2aO2XV1hzRr43/8sX57pUew2EQWoVUuVI3vySbV9+7ZaJ/WHHzQNq2DQ6VS/flwc3LypdTRCCCGsWe3asHMnFCmidSR279gxdZf43Dm1XbKkqsvevbs28dh1IgpQpozqGW3bVm3r9WpC3syZMnwxz5lM6iuXLNMmhBAiKzt2wPnzWkdRIGzcqOqth4er7SpV1Mz4hg21i8nuE1EAT0/YvDn9ErqjR8PAgepOgMgjOh188gmMH691JEIIIazVuHHw4YdaR2H3FiyAbt3UjUpQ9UIPHdJ+BJ3dlG96FGdnWLRIlbf84AO176uv1Pr0P/4odwPyTOvWWkcghBDCmu3dm1a8Ulic0QgjR8Ls2Wn7evVSdUNdXbWLK0WB6BFNodOp5Wu//16tRgmwZQs0a6bm1Yg8cuwYNG0qf2iEEEKkMZnU54KLi0xSyiPx8fDCC+mT0DFjVB5kDUkoFLBENEWfPmrd1GLF1PbJk2p8xB9/aBqW/fLzg+LFVa0IIYQQAmDfPvD3h5AQrSOxSzdvqjrqa9eqbUdHdSd4+nRVd91aWFEo+at5czU2onx5tX39uhovsW2bpmHZJ39/2LABypXTOhIhhBDW4vHHYfJkNWNGWNSFC6pkZVCQ2vbwUPXVBwzQNq7MFNhEFNTvQFAQ1K+vtu/dg06dYMkSbeOyWwcOqBoRQgghhI8PvPde/heutHP796skNGVtgNKl1b727bWNKysFOhEF9Xuwezd07aq2k5LU7Ppx46S8k8XNmAGff651FEIIIbQ2blzakj7CYn74Adq0UXXTQdVRP3JE1VW3VgU+EQUoXFitKDBsWNq+Dz+Evn0hMVG7uOzOd9/JagJCCFHQmUxqPFxkpNaR2A2TSdVH79VL1UsHVT99/35VT92aFZjyTY/i6Ahz50KFCvDuu+p/6sqVajb9unVpE5tELhQvrv4bHq66ooUQQhQ8Oh0sW6Z1FHYjKQkGD4avv07b9/rr8MUXqnSltZMe0QcMG6YSTzc3tb13LzRuDJcuaRuX3QgKUpOXgoO1jkQIIUR+u31brTAjY98s4t49eO659EnotGmqbrotJKEgiWimunSBPXugVCm1ff68Ku909KimYdmHunXV17SqVbWORAghRH5buxZ69IBbt7SOxObduKHqoG/ZoradndWw2w8+sK35X2Ynoj/++COBgYH4+Pjg7+9Px44dOXXqVJbHz58/nwoVKlCqVCkCAwM5efJkbuLNN/Xrq867xx9X2zdvQosWsH69llHZAScnePNNVUtCCCFEwfL663DmTFpPj8iRP/5QHWQpKVXRorB9O7z0kpZR5YzZiejmzZv55JNPCAsL48qVK9SuXZv27dtjNBozHLtq1SqmT5/O1q1buXnzJj179qRdu3ZERUVZJPi8VqGCqjXavLnajo+H7t1h3jxt47ILs2er5R2EEEIUDDExqqtO68XNbdy2baru+fXrart8+fS5iq0xOxFdvnw5jRo1QqfT4eTkRM+ePQkLCyMiIiLDsXPmzGHo0KFU+a9Y7bBhw/D09GTlypW5jzyfFCsGW7emfcswmeB//1OP5GQtI7NxtnTfQAghRO4kJakaQrNmaR2JTVuyRNU7v3dPbderp+7eVqumbVy5kasxoqGhocycOZNnnnmGUg90s+v1ek6cOEFgYGC6/Y0bNyYopdS/jXBxUZWHxo1L2zdvHjz/PMTFaReXTXv3XVVXVAghRMEwfrz1VlW3ciaTykFef13l9KDms+zebftFaHJcvqlp06YcOHCAxo0bs2HDhgzPR0ZGkpSUhM8DLeTj48MfWSzqnpiYSOJ9hTujo6MBMBgMGAyGnIZqMRMmQECAjkGDHElK0rF+PTRvbmTdumSruhBS2soa2uyh9Hp0q1Zh6tJFDXDRkM20mZWRdjPP/e1kLX/XbIVca+azujbr00f911riyYK1tVtiIgwY4MiqVWl9h++8k8ysWUYcHa2jOXPTVjlORPfv38/NmzcZO3YszZo14/jx47i6uqY+nzJmVPfALVgHB4dMx5MCzJgxg8mTJ2fYv3v3bgoXLpzTUC2qVCn44IOSfPxxPeLjnTl2zIG6deMZPz6IMmVitA4vne3bt2sdwkO53LlDm8GDORkSwr9NmmgdDmD9bWatpN2yJyEhIfXnXbt2pfubKbJHrjXzad1mJU6dwu/oUc7064exUCFNYzGH1u0GEBPjzIwZ9TlzpgQAOp2J1177kzZt/mHrVo2Du09cLm4P60ym3BXz0uv1FClShLVr19KpU6fU/fHx8bi7u3Pw4EEaNWqUun/AgAHExMRkOk40sx7RsmXLEhoaire3d27CtLjTp6FLFyeuX1eJdtGiJtasSaZZM+1roxkMBrZv307btm1xtvZCYjdvWsXsSZtqMysi7Wae2NhYiv23OsbNmzcpqvGdAFsi15r5rKXNdN9/j8OqVSRv2mQT8wOspd0uXYLOnZ0ICVFt5uZmYvnyZLp21T7PeFBkZCR+fn5ERUXh6elp1mvN6hFNTk7G0dEx3T4HBwccHR0z7Hdzc6N69eoEBwenS0SPHj1K3759Mz2/i4sLLi4uGfY7Oztb3R+eOnXUAOFnn1XlE+7e1dGxoxNLl6bdfdCaNbZbBv7+avBLeDj4+modjW20mRWSdsue+9tI2ixnpN3Mp3mb9e8P/fvbXOFyLdvt999VfnHzptouWRI2bdLRoIF1LoiZm3Yy67o4ffo0nTt35vTp04DqDR0+fDg+Pj40bdqU3r17M2LEiNTjBw8ezMyZMwkJCcFoNLJw4UIuXbqUZSJqa/z9Yd8+6NBBbev1anb9hx/KohFmmTgRGjSwjoEuQgghLGfdOlW2SWTbhg2qFFNKElq1qur4atBA27jyilmpdc2aNWnbti39+/fn+vXrODk5Ua9ePbZv3467uzsXLlxId2t94MCBRERE0KpVK2JjY6latSpbt27F1wp6viylSBHYuBHeeQe++krtGzdOdanbyjqvmnvpJQgMVMXuhRBC2IfQUOjVS603+corWkdjEz77TC01ntKZ1ayZyuWLF9c2rrxk1ie/o6MjQ4YMYciQIZk+H5zJ+uHjx49n/PjxOYvORjg5qaSzYkUYNUrt++YbuHoV1qwBM4dLFDxVq8qSn0IIYW/8/OCvv9R/xUMlJ8OIETB3btq+3r1h6VJVQtKe2dqQDaul08H778Pq1WkXzfbtavWDa9e0jc0mJCSob86ZlAITQghhY+7dA6MRAgLk1uAjxMXBCy+kT0LHjlXrxtt7EgqSiFrciy/Cjh1p3einT6dfD1ZkwdVVPWRwrRBC2L7//Q/attU6Cqt38ya0aqVuvwM4OqqRDB9+CA4FJEOTQXl5oEkTOHwYOnaEixfh33+haVP48ce0iU0iE8uWaR2BEEIISxgwAMLCtI7CqoWEqJzg0iW17eGhhvO1a6dtXPmtgOTb+a9KFZWMNmyotmNi4Lnn4OuvtY3L6oWHw+efax2FEEKI3GjQQK1BKTK1fz80apSWhPr7w4EDBS8JBUlE81TJkrBrF/ToobaTk+Gtt2DMGDV0RmTi8GEYPVoG1gohhC0KC1O9LikZlshg1Spo0wbu3FHbTz6pyjM99ZS2cWlFEtE85uambsnfV16Vjz5SRe/vW+1PpOjcGa5cgbJltY5ECCGEucLC4O5d+G8FMZHGZIIZM9Tnv16v9j3zjOodLVNG29i0JIloPnBwgFmzYOHCtMHHP/ygxnFHRmobm9VxcFB/wBIS1OBaIYQQtqNWLZVZyfK16RgM6o7o2LFp+954A375RUo8SiKajwYNUtWJChdW2wcOqDEiFy9qG5dV6tRJ/dYKIYSwDb/8Ih9ombh3T41WWLQobd+HH6o5I1LZShLRfPfss2pZ0JTFpf76S01oCgrSNi6rM20azJ6tdRRCCCGyw2iEkSNh/nytI7Eq16+rqjlbt6rtQoXg++9Vz6hOp21s1kLKN2ng6adV4tmxI5w9CxER0LKlKl6bMrGpwGvUSP3XZJLfViGEsHYODhAcDPct813Q/fGH+py/cUNtFysG69erZTtFGukR1Ui5cnDwoCpkC2pI5AsvqE5Aqen+nytXVNZ++rTWkQghhMhKXJyaoFS4sExS+s+2baqmeEoSWqECHDokSWhmJBHVUNGi8Ntv8MorattkguHDYcgQVeqpwCtdGp54QjJzIYSwZgsWqOLZcXFaR2IVvvlG9YTeu6e269dXlQkff1zbuKyV3JrXWKFCakGhihVh0iS1b+FCuHpV1Rpzd9cyOo05O8N332kdhRBCiId5+WXV5ZcyE7eAMplg/Hg1ESlF165qTGgBb5qHkh5RK6DTwcSJKiF1+u+rwaZN0Lw5hIZqGpp1OH4c5s7VOgohhBCZ8fNTY8sKsMRE6Ns3fRI6bJhaslOS0IeTRNSK9OunZtZ5eant4GA1o/7MGW3j0tyhQ6ruhawAIIQQ1iM2Vk0sPXBA60g0dfu2Kky/cqXa1ulU38ncueDoqGVktkESUSvTqpWaxBQQoLavXoXAQLVUaIH11ltq+qGrq9aRCCGESBEdrVbBK8DLAl26pD6j9+1T225usHat6g0V2SOJqBV64glV3qlOHbUdFQXt28O332obl2acndXXymvXVFE2IYQQ2vPzU2tYly+vdSSaOHpU3bU8f15tlyoFe/aocaEi+yQRtVJ+frB3r1pgCNTyYP36wZQpBXQSudGoBs3ePwBHCCGENn76CXbu1DoKzaxfDy1awM2bartqVdWBVL++llHZJklErZiHh7rYBw1K2zdxIrz6Kuj1moWlDQcH9Yfv44+1jkQIIcR338Hq1VpHoYl586B7d4iPV9vNm6upDBUqaBuXrZLyTVbOyUmVaKtYEUaMUPuWL1d3qX/+WdUiLTCeflr9V69Xda+EEEJoY8OGtEysgEhOVrW+581L29enDyxZAi4u2sVl66RH1AbodOri/+mntPk6u3apVRuuXNE2tnx38qQaGH/unNaRCCFEwRMXp/7+6nQFqi5RXBw8/3z6JPSDD9TS3JKE5o4kojbk+edVAlqihNo+c0YNlD5+XNu48lX16vDaawWsK1gIIazE8uVQqxbcuqV1JPkmPBxatlRD5UDNnV20CKZNU/m4yB2zE9GjR4/SsWNHSpUqhZ+fH61ateLkyZOZHjtp0iQ8PDzw9fVN90hMTMxt3AVWo0ZqqbDKldV2WJhau/aXX7SNK98UKgQffaRmcwkhhMhfr78OW7ZAyZJaR5Ivzp9Xn7tHj6rtIkXg11/hjTe0jcuemJ2Ivv/++wwcOJDQ0FBu3LhBgwYN6NKlS5bHjxgxgrCwsHQPF+nHzpVKlVQyGhiotmNjoUsX+PxzbePKV5s2wciRWkchhBAFh8GgOgNattQ6knyxf7+Oxo1VrVAAf3/Yv18VrxeWY3YiumPHDp577jkcHR1xcHDg5Zdf5urVq4SHh+dFfCIL3t6wYwf07Km2jUYYPFjlZkajtrHli1u34MIF9YdRCCFE3oqKUrNmN27UOpJ8sXevPx06OHLnjtp+6ik4ckT9V1iW2Ymok1P6ifaHDx/Gx8eHEikDF0W+cXWFVatg1Ki0fZ98An36OJKYaOfDf199Vc3adHbWOhIhhLB/Op0qZl23rtaR5CmTCT76yIE5c+qi16sBoO3aqZWT/P01Ds5O5ap8099//82IESP47LPPcMxiQdV58+bx1Vdf4ezsTM2aNRk5ciQtWrTI9NjExMR040ejo6MBMBgMGKTnK0tTp0JAgANDhzqQnKxj7VoHzpwJpEEDA6VLax1d3tLt34/J21tNYsqFlOtLrjPzSLuZ5/52kr9r5pFrzXwWbTM3N1XIWp0w9+ezQgYDDBniyJIlafnM668bmT8/GWdn2/5nBwfrWLDAgaAgHRcv6hg9OpkpUyx3+zQ315jOZMrZOj137twhMDCQ9u3bM3v27CyPKVy4MC4uLty+fZvFixczYcIEdu7cSWDKAMf7TJo0icmTJ2fYv3LlSgoXoDIRORUcXIpZs+qRkKC+X/j6xjBhQhClS8dqHFkeSU6m9ZAhhNepw58yclzYgISEBHr16gXA6tWrcU2pxyaEFXts/XoM7u5cbdtW61DyTFycE7Nm1eXECZ/UfS+/fJbu3f+yi5nxmzZV5LffKlC16m2OHPGjU6d/eOml8xY7f1xcHH369CEqKgpPT0+zXpujRDQmJoY2bdpQuXJlvv32W3Rm/F9q1aoV1apVY+HChRmey6xHtGzZsoSGhuLt7W1umAXSiRPQtasjoaHq1ry3t4mff06mcWM7XRf0xg0oXTrXNTQMBgPbt2+nbdu2OMvt/myTdjNPbGwsxYoVA+DmzZsUlTJk2SbXmvks1WYOQ4aAtzfGSZMsF5wVuX4dunRx4vRp9TlSqJCJd94JZurU6nZzrRmNaoFCgMqVnejd22jRHtHIyEj8/PxylIiafWs+Pj6eZ599ltKlS7N06VKzklBQPQLFixfP9DkXF5dMZ9Q7OzvbzcWQ1+rXhwMHDLRsGcPVq55ERupo186Jb79Nm9hkV8qXV/+9ehUCAnJ9OrnWckbaLXvubyNps5yRdjNfrtvsyy/BZMLRHroGH3DqFHTqpPo0AIoVg59/TiY6+gbOzk/Z7bXm6OiIs3PmQypzIldfdMw5WK/X061bN1xcXFi9enWGiUu9e/dmRMo6lMDYsWO58t/SP4mJiXz00UeEhITw9ttv5zhg8Whly8KMGftp00Z920lMhBdfVMu052wghpU7dUrN5ty1S+tIhBDCfly/riaFmkx2Wbl961a1QmFKElqhgiqN2KSJPX5QWi+zEtHDhw+zdetWjh07RkBAQLoi9Tt27ODChQv8888/qcd7eXnRoUOH1OL3hw4dYv/+/fjL1LM85+6exIYNybz2Wtq+UaNg0CBIStIurjzx5JOwbBk0bqx1JEIIYT9++kkVsI+J0ToSi1u8WPWEpvzTGjSAoCCoWjXjscuWqTx82bKcv58lzmGvzLo137x5cx42pDQ4ODjd9qhRoxh1f20hka+cndUvW4UKMH682vfll2p9+h9+UCtE2AWdDvr2VT/b6Td3IYTId+++q8Z02c2HhRorOX48TJ+etq9bN7VmvMyJ1oadF5sUOh2MGwfffZdWcvO336B5c/j3X21js7jvv1e9osnJWkcihBC27b9hdfZUPDMxUfVZ3J+Evvuu6viVJFQ7kogWEH37wvbtkDJJ98QJdSvi9GlNw7KsKlXUgJ/7Ki8IIYQw0/nzatz9L79oHYnF3L4NbduqRWBAddLMnw+zZ0MWZdBFPpFEtABp3hwOHUqbaH79usrbduzQNCzLqVcPZs2Sr7ZCCJEblSurO0x2sqj6P/+om2X796ttNzdYtw6GDMn9uV94QSW1D3scOJD797FnuVpZSdieatXUgOznnoPff4foaOjQAb76inQTm2zakiXg4gIvvaR1JEIIYVuMRtVF+N/CC7buyBH1eXfrltouVUp19NarZ5nzV6umVj590NWrsHu3GhL35JPw99+Web+cunUL9u5VP8fFqU7vNWvA3V3lAFqSRLQA8vFRvyAvvaQqcyQlqYmRly7BlCl2MNdn/37w9JREVAghzGEyqV7QTp3U4Ekbt26d+hiIj1fbjz8Ov/6qJvBaypQpGfddvgwtWqgk9Mcf1ceR1s6cUb23KX7+WT3KlVPxakluzRdQ7u7qIhw6NG3ftGnwyit2MMRy8WKYN0/rKIQQwrYkJ0Pr1lCjhtaR5NrcudCjR1oSmjI0zZJJaGZSktB//1VJaNeueft+2dWihfqe8eBD6yQUpEe0QHN0VPlaxYrqy6/JpEpYXL8Oa9eqFSZskqOj+sf89hs0amTD/xAhhMhHTk4wZozWUeRKcjK8956aiJTipZfgm2/UiK28dH8S+tNP0KVLzs/VrRucO2fea779Vq2uaGskERUMG6a65/v0Ud8e9+xRA7stfQsjX0VGqvp3H3+sqvgLIYTI2po1cPEivP++zY7Pio1NG3KWYty4/BlyZskkFNRQuZAQ814TF5e799SKJKICULcPdu9OG9R9/jw0bGjZQd35qkQJOHkSHntM60iEEML6hYTAn3/abBIaHp42CRdU525+TcK1dBIK6uOroJAxoiLVg0uc3bypxtXc/+3SplSqpP6oXr2qdSRCCGHdPvgAVq7UOoocOXdOdZykJKFFisDmzfmfhK5ZY5kk1NIeVV7q/ocWJBEV6VSsqAZ0N2umtuPj1ViVzz7TNq4c27ZN/aP+/FPrSIQQwvpER6sENDnZJntD9+5VQ8lSJt2UKQMHD+ZPCdSUJDQ0VCWhnTvn/XvmhMmklvXu2BF8fdUs/qZNVYGZBycvaUESUZFB8eIqf+vTR22bTGp2/bvv2uDqmS1bwrJlqm6HEEKI9DZvVvX7wsK0jsRs33+vVku6e1dt16ql7urVrJn3731/EvrTT9abhKaYO1eNWFu4UMXr768KJJw6pXVkMkZUZMHFRc2gr1ABPvxQ7Zs7Vy0/vGKFDS1e5Oys1jcFVajZQb57CSFEqt69VfeYDa0pbzKp9eLHjUvb1769KpdUpEj+xNCvn/o8rFJFVZlZuzbjMX36WM/iVJs2gbd32nabNiphX7gQvv5au7hAElHxEDqdqi1avjy8/bbqDV23TnUybtqkVqiwGV99pWpb7N8vyagQQoCaoFS1qrqfbSMMBhg4UJVjSjFggEqonPIpozEaIThY/Xzhgnpk5v4C8lq7PwkF9TFYo4aana81+UQWj/TGG6qUU8o3zaNH1cBwc0tLaKpGDfXVNClJ60iEEEJ7f/0F1aurlU1sRHS0WvTp/iT0o4/gyy/zLwkFlcTFxGReIP7+R6dO+ReTuZKT1eSuSpW0jkQSUZFNzzwDBw6k3b25dEnVit+3T9u4si0wECZOhEKFtI5ECCG0V6mSup/83HNaR5It169DkyawfbvaLlQIVq+GUaNsco6V5hYsUAVlrKHMtiSiItuefBKOHIGnnlLbd+6ogeI2VfFj/nxYtEjrKIQQQjt6vcreunSxiS/nJ0+q8oKnT6vt4sVh50548UVNw7JZR47A6NFqjG1+TOx6FElEhVn8/dUwy/bt1bZer1aymD5du9IPZgkJUauHCCFEQZScrG5nffKJ1pFky5Ytai7Vv/+q7ZQSg02aaBuXrbp8WX3/eO45dZPQGkgiKsxWpAhs3Ahvvpm274MP1IBxg0G7uLJlwQI1qEgIIQoikwlefTWtWLQV+/prePZZNR4T1NyE+xddEea5e1eNWy1fHpYvt54hDZKIihxxdlYT0e/P6RYvVn80oqO1i+uRdLq06r7SMyqEKGicnOCdd6B+fa0jyZLRCGPGwFtvpdWu7t4ddu2CkiXzP55atVTvYa1a2p4jN/R61YZxcWq1RDc3beLIjJRvEjmm06mB4uXKqZpqer0qhN+0qaqRbLUVQRISYMQIGDIE3n9f62iEECJ/fPKJ+vt3fwFOK5OQoDpsV69O2/fee/Dxx+DoqE1MtWrlPoG0xDlyY9AgtQrVokVqsnFK2SYXF6hdW7u4QBJRYQG9eqmxo127wu3b8McfamD55s3a/uJlyc0NTpxQy0wIIURBkZioMj0rFRmpPkcOHFDbDg4wb57qwBW5s2OH6ml+/fX0+8uVS1seVStya15YRNOmagB5xYpq+99/1b6tW7WNK0spSWhQkPrtFEIIe/fBB2qVEit08aJaMz4lCS1cWC2gIkmoZVy+nHmtU62TUMhBInr06FE6duxIqVKl8PPzo1WrVpw8eTLL4+fPn0+FChUoVaoUgYGBDz1W2LaqVVVe17Ch2o6JUQOjrbZa0vnzavbohg1aRyKEEHnnzBn44gurXdAjKEj9KU5ZocjHR91Gtvb124VlmJ2Ivv/++wwcOJDQ0FBu3LhBgwYN6NKlS6bHrlq1iunTp7N161Zu3rxJz549adeuHVFRUbkOXFinkiXVgPLu3dV2crKaTT92rBV2PD7+uPprl8X1K4QQ9sBh1y6YOzdt5o8VWbtWLRt965barlZNJaZ162obl8g/ZieiO3bs4LnnnsPR0REHBwdefvllrl69Snh4eIZj58yZw9ChQ6lSpQoAw4YNw9PTk5U2VQFdmMvNDX76SQ0wTzFjhqo3mpioXVyZatZMDUS6c0frSIQQIk8YhwyB48fVzBQrYTLBnDnw/PNpw1ZbtICDB1V5IVFwmJ2IOj2woOvhw4fx8fGhxAMTP/R6PSdOnCAwMDDd/saNGxMUFJSDUIUtcXCATz+Fzz5TP4OaBdm2rRqQblU2bsSpUiVcU76SCyGEPUhMxOf331XW5+6udTSpkpNh2DDVWZGyEErfvqp4fbFi2sYm8l+uZs3//fffjBgxgs8++wzHB+oqREZGkpSUhI+PT7r9Pj4+/PHHH5meLzExkcT7usyi/ytIaTAYMFh9pXTrkdJW1tBmb70F/v46+vZ1JC5Ox/790KiRiU2bklInNmmuSRNM48ejL1rUKtrMlljTtWYL7m8n+btmHrnWzGf8+Wfqz5hBYp8+ULmy1uEAEBsLL7/syC+/pPWDjR2bzMSJRnQ661gURa418+WmrXQmU84WZrxz5w6BgYG0b9+e2bNnZ3j+xo0blClThvPnz1P1vmUQRo8ezfHjx9m2bVuG10yaNInJkydn2L9y5UoKFy6ckzCFlfj776JMm9aAu3ddAfDySmTs2CNUrWplt8RNJutZbkLYnYSEBHr16gXA6tWrcXV11TgiYe88rl0jpmxZrcMA4M4dFz78sAF//626PR0djQwadIrWra9qHJnIrbi4OPr06UNUVBSenp5mvTZHiWhMTAxt2rShcuXKfPvtt+gy+eCOj4/H3d2dgwcP0qhRo9T9AwYMICYmJtNxopn1iJYtW5bQ0FC8vb3NDbPAMhgMbN++nbZt2+Ls7Kx1OKkuX4bOnZ04f15dL66uJpYvT6ZbN+0XqTcYDJwfPZqn/vwT42+/pY0nEA9lrdeatYqNjaXYf/ceb968SdGiRbUNyIbItWamixcxBARYTZudOwddujhx+bL6++/paeKHH5Jp3Vr7v/8PkmvNfJGRkfj5+eUoETX71nx8fDzPPvsspUuXZunSpZkmoQBubm5Ur16d4ODgdIno0aNH6du3b6avcXFxwSWTwdTOzs5yMeSAtbVb5cpw+LCaUb97NyQk6OjVy4lPP4X//U/7jsg4X19wdMTZZFJrmIpss7ZrzVrd30bSZjkj7ZYNp05B7droNm8GtG+zPXugWze11jlA2bKwebOOmjWte00drdvNluSmnczq9tHr9XTr1g0XFxdWr16dYeJS7969GTFiROr24MGDmTlzJiEhIRiNRhYuXMilS5eyTESF/StaVA1If/lltW0yqQHrw4ZpX1kk8oknMH78McjtUiGELatZE1avxtSypdaRsGIFPPNMWhJaq5Yqz1SzppZRCWti1teRw4cPs3XrVooXL05AQEC651asWMGFCxfS3VofOHAgERERtGrVitjYWKpWrcrWrVvx9fW1TPTCJhUqBMuXq1WYUoYEf/aZunW/apUVTO78/nu1EK8Vr8cshBCZSkhQX6Z79tR05o/JpBZxmjAhbV/Hjqp6SpEimoUlrJBZiWjz5s152JDS4ODgDPvGjx/P+PHjzY9M2DWdDiZNUvXi3nxTLfixaZOqI7dpE2j6XeXaNfj7b5m4JISwLTExqqtxypS0204aMBjg7bdhyZK0fW+9BQsWgJN1340XGpAZGUJT/furW/UpY5uPHVNLhJ49q2FQo0bBsmWShAohbIuTk/pm36SJZiFERamez/uT0Jkz1QqjkoSKzEgiKjTXurVaTSNltMeVK9C4sZrQpImUBPTXX1WWLIQQtsDVVa2nXKGCJm9/7Ro0bQo7dqhtFxf44Qd4/335Xi+yJomosAo1aqgB7HXqqO2oKGjXDr77TsOgvvhCDWgSQghrZjLBCy+oOzkaOXECGjSA06fVdvHiKiHt2VOzkISNkERUWA0/P9i7Fzp1UtsGA7zyihrulLNlF3Jp1SpYulSDNxZCCDMYDGpgvUb1tn/7DZo1g9BQtf3YY6pUn4YjBIQNkURUWBUPD1i/HgYOTNs3cSK89hro9RoEo9OpgauXL+fzmwshRDYVKqRKjzz3XL6/9VdfqbeNiVHbDRuqJLRKlXwPRdgoSUSF1XFygoULYdastH3Llqme0qiofA7GYIAePWD+/Hx+YyGEyIZZs9RAzHxmNMLo0Wp2fEoN6B49YNcuKFky38MRNkzmsAmrpNPBiBFQrpyqQpKYqMYbBQaqOUQPlLHNO87OsH27KnoqhBDWxGRSqyjFxeXr2yYkqIon9+e/w4fDxx/L6sjCfJKICqv2wgvg7w+dO0NkJJw5o279/PJL2sSmPJdyj+niRTUOS/OK+0IIgfrGvmKF6p7MJ5GR0LUrHDigth0c1A2jwYPzLQRhZ+S7i7B6jRurGfWVKqnt0FA1MP6/ZZTzR0wM1K0Lc+fm45sKIUQWtmxJq5OUT92QFy9Co0ZpSWjhwmpMvyShIjckERU2oVIlNQC+cWO1HRurekm/+CKfAvDwgJ9/hnffzac3FEKIh1i+XC1VlE+CgtTdqL/+Utu+vrBvnybzo4SdkURU2IwSJWDnTnW7HtTdqEGDVLHkfLkz1aqV6gKIjMzXW2FCCJHBypX5Vmj555+hZUuIiFDb1aurxPTpp/Pl7YWdk0RU2BRXV1Vj/v330/bNmgW9ekF8fD4EcPMmVK6scaV9IUSBdeYMnDypxocWKZKnb2UywezZ6st/QoLa17KlWgmvXLk8fWtRgEgiKmyOg0Pa2sUpQ6N++gnatEn7xp5nSpVSb96xYx6/kRBCZOLjj9WU9Txe5SM5GYYOVbPhU97q5ZfV0NSiRfP0rUUBI4mosFlvvw2bNqVNYj90SA2kTxnDlGfefFMVyjMY8viNhBDiAYsWwYYNebp4e2wsdOuWfgjqxIlqWGqhQnn2tqKAkkRU2LSOHWH/frU8KMDff6tk9NChPH7jS5fULfqDB/P4jYQQAlUu5PJllQnm4X3xsDBo3lx9yQe1wMjSpTBpUp7mvqIAk0RU2LzatdXA+Ro11HZkpJpX9NNPefimAQFqGZHSpfPwTYQQ4j+TJqnxRynLGOWBs2fVzPjgYLXt6aluxffvn2dvKYQkosI+BASo2nZt2qjtxETo2VNNZMqToVSOjvDpp1ChQp6P1RJCCD75RC1l5OiYJ6ffvVuVx7tyRW2XLatu+LRunSdvJ0QqSUSF3fDyUst/3v/t/f33VYmnpKQ8etNLl6BBAwgJyaM3EEIUaPfuqds8RYrkWb2k776Ddu0gKkptP3iXSYi8JImosCvOzrBkCUyZkrbvyy+hSxe1OJLF+fqq8Vp5lukKIQq0qVPVqm56vcVPbTKpv5WvvJI297JjR1WoXkYdifwiiaiwOzodjB+vvuU7O6t9v/6qlgX9918Lv5mbmxqM+sQTFj6xEEIA772npq9beLq6Xg+vvaZmw6d4+201Id/Dw6JvJcRDSSIq7FbfvrBtW1rNuxMn1ED8P//Mgze7fl2tOXr9eh6cXAhR4BgMqo6Sry906mTRU0dFqVMuW5a27+OP4fPP1Sx5IfKTJKLCrrVokX4VkGvXIDAQduyw8Bt5eMDt2xAebuETCyEKpFmzoE6dtCWNLOTq1fR/A11c1ByokSOlPJPQhiSiwu6lrItct67ajo6GDh3S9wbkWtGiatq+LL4shLCEF16AUaPUusYWcvy4uit05oza9vaGnTtVhREhtJKjRPTatWvUr18fnU5H0kMmaUyaNAkPDw98fX3TPRITE3McsBA54esLe/aou+eg5ha9+qoaH2XR6ks3b8KAAap3VAghzJWcrP5AVa6sBnFaSMo4+dBQtf3YY3D4sOodFUJLZieiR44coXHjxtSqVStbx48YMYKwsLB0DxcXF3PfVohcc3eHtWthyJC0fVOmQL9+FpyQmpysMt7z5y10QiFEgTJnjiroacFZ8l9/7cBzz6khp6BWnzt8WOW6QmjN7ES0UqVKnDt3jj59+uRFPELkKUdHmD9f/a1PGQ/13XfQqZMjMTHOuX8DPz84d059kAghhLmaN4c+fSwyS95ohOXLq/POO44YjWrf88+r2/ElS+b69EJYhNmJqLe3Nx5S20HYuP/9D9asSRt+tXevA6NHN+XyZQuc3NFRFS0dOzatQrQQQjyM0ajGCdWrp/5A5VJCAvTt68i6dWndniNHqolJbm65Pr0QFpPnhRrmzZvHV199hbOzMzVr1mTkyJG0aNEi02MTExPTjR+Njo4GwGAwYEiptiseKaWtpM0e7rnnYMcOHd26OXLrlo7r14vQpImJDRuSePrpXA4cDQ/HaelSkps1w2THa+TJtWae+9tJ/q6Zx96vNYcFC9Bt3UryunW5rqEUEQHPP+/IoUOqr8nBwcTcuUbefttIcnKeLldvF+z9WssLuWkrncmUs6kae/bsoWXLlhgMBpyy+KW5c+cOhQsXxsXFhdu3b7N48WImTJjAzp07CcxkhPSkSZOYPHlyhv0rV66kcOHCOQlTiEcKCyvM1KkNuXGjCAAuLkkMH36M+vVzV4rJQa/HaOEi1MK2JSQk0KtXLwBWr16NqwVnRAvbVur4cYr+9RcXXnwxV+cJDXVn6tSG/PuvunPp4pLEiBHHqFdPSsuJvBMXF0efPn2IiorC09PTrNfmaSKamVatWlGtWjUWLlyY4bnMekTLli1LaGgo3t7eOQmzQDIYDGzfvp22bdvi7GyBcY8FQHi4gXbt4jh7tgSgehBmzzYyaJAxdyc2GHCYNw/ja69B8eIWiNS6yLVmntjYWIoVKwbAzZs3KZqy2oJ4JLu91kwmixXwDArS0b27IxER6nw+PiZGjtzLwIH17avN8pjdXmt5KDIyEj8/vxwlovm+hkJCQgLFs/hAdnFxyXRGvbOzs1wMOSDtln0+PjB58mHWrOnEDz84YDTq+N//HLlyxZFPPgGHnFbcvXULZs/GsUoV6NHDojFbE7nWsuf+NpI2yxm7a7eFC+HoUViyRI0vz6E1a9Rqcil9OU88AevXJ3HmTJT9tVk+kXbLvty0k0UL2vfu3ZsRI0akbo8dO5YrV64Aqrfzo48+IiQkhLffftuSbyuERTg7G1m+PJmxY9P2zZmj6krHxeXwpKVLw8WLdp2ECiFyoVgxVW0jh0moyQSffqqK0qckoa1aqfU1UlaUE8KaWTQRvXDhAv/880/qtpeXFx06dKBUqVL4+flx6NAh9u/fj7+/vyXfVgiLcXCADz+ERYvSPhfWrlV/2G/ezOFJPT3Vp8WSJWoWgRBCpOjTBz76KEcvTUqCd96BESPSFubo1w9++00t9iaELcjxrfkWLVrw4PDS4ODgdNujRo1i1KhROX0LITTzxhtQtqyquRcTA0eOqCLQv/4KVavm4ISRkap2iskEr79u8XiFEDZm+nS4dw9mzMjRy2NioHdv+OWXtH2TJsGECbJmvLAtsta8EFlo107d3krpwP/nH5WM7t+fg5OVKKFWW5IkVAgBqphnDqvBhIaquvcpSaiTEyxbppYsliRU2BpJRIV4iKeegqAgePJJtX3nDrRpA6tX5+BkKUuZrF8P169bKkQhhC16910YP97sl505Aw0bwvHjatvLC7ZsUbfkhbBFkogK8Qhlyqhe0Hbt1LZer26JffRR2risbIuLU4O6li2zdJhCCFswdqxaZzgHdu2CwEC4elVtBwTAwYNgx2tmiAJAElEhssHTEzZtUmNHU4wZA2+9BWYtKFG4sBpw+sEHFo9RCGHlTCb1ByMHSxt9+y20b5+2anCdOupuzRNPWDhGIfKZJKJCZJOzM3z9tZpjkGLRIrVU6L17ZpzI318N5DpwAE6ftnicQggrpdPBrFnqtnw2mUwwebK69Z7ypbdTJ9i7V1V9EsLWSSIqhBl0OtUTunIlpKzeuXUrNG1q5rBPoxGGDoW5c/MiTCGEtRk0CH780ayX6PXw2mtqNnyKgQPVMHMPD4tGJ4RmJBEVIgd694YdO1QtaoBTp9QEgj/+yOYJHBxg82b46qs8i1EIYSUMBnVP3YxxPFFR0LFj+uHks2aphZjMWFVbCKsniagQOdS0KRw+DBUqqO0bN6BJE9VDmi1+fuoT5ezZHNaEEkLYBGdn+P57eOmlbB1+9aqalLRzp9p2cVGdqSNGSHkmYX8kERUiF6pWVRMGGjRQ2/fuqfFbixebcZKxY9PfexNC2Ae9Hrp0gX37sv2S48fV35MzZ9S2t7eaLf/CC3kUoxAak0RUiFwqVUp9UHTrpraTk+HNN9XE+GyVd1q8WE3JF0LYl5gYdTs+m4Xrf/kFmjWDsDC1XamS+qLbuHEexiiExiQRFcICCheGn35KPxl2+nTo2xcSEx/x4hIl1Alu3ICff87TOIUQ+ah4cbUucN26jzz0iy9U52lsrNpu3FgN/alUKY9jFEJjkogKYSGOjjB7tqpV7fDfb9bKlfDMM3D7djZO8MUXaj36R2auQgirdvMmtGwJ58498lCjUf3aDxqkfgZ1G37nTvUdVQh7J4moEBY2ZAisW6eWkgY1PKxxY7VW/UNNmABHj6qZCUII2xUTo+q7pZTVyEJ8PLz4InzySdq+kSPVEsKurnkcoxBWQhJRIfJA586q4LSPj9oOCVHlnY4ceciLChVSXSBRUTBvXg7WDxVCaM5kgooVVfkMX98sD4uIgDZtYM0ate3gAJ9/Dh9/nHZHRYiCQC53IfJIvXpqokG1amr71i11t27duke8cO9emDgRLl3K8xiFEBYUFAQtWqhf9of46y9o1AgOHVLb7u5qvuLAgXkfohDWRhJRIfJQ+fJw8KD6bAJ1K65Hj0csqNS5s7qPX7Fi3gcohLAcR0fVC1q0aJaHHDyoktC//1bbfn5q+E7HjvkTohDWRhJRIfJYsWKwZYuaQQ/qzt2778KwYarUU6aKF1eTlqZOVePNhBDWy2hUv9j16sEPP6gC9pn46Sdo3RoiI9X2E0+oTtQ6dfIxViGsjCSiQuQDFxf49lsYPz5t3/z50L17WrmWDK5fV2NFDx/OlxiFEDk0Y4b6ZU6Z9v4Ak0ktz9mzZ1pRjNatVe9oQEA+ximEFZJEVIh8otPBlCmwZEnaWtEbN6pxo+HhmbzgscfUONG2bfM1TiGEmZ58Ut1vz2SWUVKSKs30/vtp+/r3V+VFvbzyL0QhrJUkokLks1dfhd9+A09Ptf3772pGfaYlB4sUUb0sM2ZIz6gQ1iYhQf33uefSZ5r/iYlRReq//DJtX8qX0UKF8ilGIaycJKJCaKBNGzhwAMqWVduXL6tao3v2ZHJwcrLqPgkKyscIhRAPlZysVquYMCHTp//9Vy3X+euvatvZOW14jk6Xj3EKYeUkERVCIzVrqtyydm21ffeu+lxbseKBA52d1WL2968fKoTQloODmoH4zDMZnvrzT3WX48QJte3lpSYsvvxyPscohA3IUSJ67do16tevj06nIykp6aHHzp8/nwoVKlCqVCkCAwM5efJkTt5SCLtUunT60i0Gg/qwmjr1gXr2KbNwV6yAcePyPU4hxH1u3VLdmgMGQJMm6Z7auRMCA+HaNbVdrpyqF9qqlQZxCmEDzE5Ejxw5QuPGjalVq9Yjj121ahXTp09n69at3Lx5k549e9KuXTuioqJyEqsQdsnDAzZsgLffTts3YQK88YZKTNOJiFD3/LKYnSuEyGN//aUmEm7alOGpZcugfXuIjlbbTz+t7npUr56/IQphS8xORCtVqsS5c+fo06fPI4+dM2cOQ4cOpUqVKgAMGzYMT09PVq5caX6kQtgxJ6e05f1SLFmiekrTfW8bNgy++UbdFpQlQIXIf489BtOnq/pL/zGZ1GJor76qZsmDmr+0d+9DV/kUQpCDRNTb2xsPD49HHqfX6zlx4gSBgYHp9jdu3JggmXQhRAY6HYwcqephu7iofTt2QNOmabf50OnUY/9+9YTcXRAifyQlqdIWDg7wzjtQuDAAej3066dmw6d45x21lK+7u0axCmFDnPLqxJGRkSQlJeHj45Nuv4+PD3/88Uemr0lMTCQxpdovEP3f/Q2DwYAhwz1KkZWUtpI2yz5rarNu3WDrVh09ejgSGanj9Glo0MDE+vVJqROb8PbG0ceHZL0+k/v3+cea2s0W3N9O8nfNPFpfaw7z5uEwZQpJFy6AtzegJhj27OnInj2qT0enM/Hxx0aGDjViNGo/gkbrNrNV0m7my01b5VkiavzvN1D3QJ0KBweH1OceNGPGDCZPnpxh/+7duyn837dPkX3bt2/XOgSbY01tNnWqO1OnNiQ01IPQUB3Nm+sYMeJ36ta9qQ545RU4dAjHxESSU7pQNWJN7WbNElLqTgK7du3C1dVVw2hsk1bXmkNAAN7Dh3PryBEAwsPdmDatIdeuqYLAhQol87//BVO5cii//aZJiFmS38+ckXbLvri4uBy/Vmcy5Wyg2Z49e2jZsiUGgwEnp4z5bHx8PO7u7hw8eJBGjRql7h8wYAAxMTGZjhPNrEe0bNmyhIaG4v3fN1DxaAaDge3bt9O2bVucs1jzWKRnrW0WEQHduzsSFKR6XBwdTcyfb+TNN//7MhcZiVPjxiSPH48pZTH7fGSt7WatYmNjKVasGAA3b96kaNGi2gZkQzS71k6dUvWXypdP3RUcrKNrV0fCw1VHS4kSJtatS6ZBA+saty2/nzkj7Wa+yMhI/Pz8iIqKwjNltZZsyrMeUTc3N6pXr05wcHC6RPTo0aP0zeID08XFBZdMenacnZ3lYsgBaTfzWVub+fmpEqL9+sFPP0Fyso7Bgx25csWRGTPAwccH3ngDp5Yt00o8acDa2s1a3d9G0mY5k+/t9v774OYGmzcDarJ8r16Q0gFUuTL89puOxx7Ls4/TXJNrLWek3bIvN+1k0YL2vXv3ZsSIEanbgwcPZubMmYSEhGA0Glm4cCGXLl3KMhEVQmTk5garV6uJTCk+/hh694aERB2MHat6a+LjVWknIYTl/PgjLFoEwMKF0LVrWhLapIlaefexx7QLTwhbZ9GvcBcuXEh3a33gwIFERETQqlUrYmNjqVq1Klu3bsVX6lkIYRYHB5V8li8PQ4aoSRA//gg3bsD69VCiBKp2zOXL6pNR1hAUIueMRlWiafBgKFECoxFGDofZs9MOefFFVTdUhvkKkTs5TkRbtGjBg8NLg4ODMxw3fvx4xo8fn9O3EULcZ9AgtVLLiy9CbCwcPKjWqP/1V6g0fjwkJEgSKkRu/fMPzJsH9eoR36wdL78MP/+c9vTo0fDhh+oLohAid+TXSAgb06mTWhY05cbCX39Bo0ZwOPoJtZSL0Qi//KJtkELYskqV4J9/uFWnHa1apSWhjo7w1Veo8dny6SmERcivkhA2qE4dOHIEnnhCbUdEQMuWsGYN8Ntv0KULnD6taYxC2JydO9Xauno9F0KL0LChWqIT1FK8mzap5eWFEJYjiagQNiogQN2aT1lpMDERXngBPjnXCdPJU1CzprYBCmFrIiMhIoIDhxxo1EjdoQcoXVotZtahg7bhCWGPJBEVwoZ5eanxof37p+0bORLe+bKGWvP6yy/TunSEEJlLmWTbsyc/9FpH63ZO3L6tdtWsqX6FatXSLDoh7JokokLYuEKFYMmS9Gtdf/45dO+cRPK336tMVQiRufh4aNIE09x5zJwJvXrr0OvVU23bqp7QsmW1DVEIeyaJqBB2QKeD8ePh22/T6tpv+s2JJvHbCR34X4aas0XUhLBvrq4kd+vBhweaM3p02u7XXlM17L28tAtNiIJAElEh7MjLL8OWLWkfnkEnXWnYEC5/vQ3atFE1n4QQqrrEqVPci9Hx7P7RjP+5VupTU6fC4sWaLlYmRIEhiagQdqZVKzh0SNUbBbh6Ffq9502YqZS2gQlhTRYtwtSgId0bhbJli9rl7AzffQfjxkk5XiHyiySiQtih6tXVBIunn1bb+2Kfpuz+VSxf4w6hoZCcrG2AQmjsdL3X6F3kF3ac8QOgaFHYtg1kBWoh8pckokLYKV9f2LsXnntObSclwdv947lbrSGmyVMe/mIh7NXKlRz65hyBLZz5IULVPitfXt1FaNFC08iEKJAkERXCjrm7w7p18M47ajsBN16LmsOwc2+nzgwWosAwGIgc/TEH31zGvXtqV7166u5BtWrahiZEQZXjteaFELbB0RHmz4eKFWH4cFhn6g5r4J/waH7ouwn3AS9pHaIQec5kNDFhijPzr+0lBg9ALUD2/ffqC5sQQhvSIypEAaDTwbvvqiVAXV3VvtL7V5M08B2uHQvXNjgh8pj+t51c8G3Ggml3iMYLI44MHarWkJckVAht2WWPqMFgILkAT8YwGAw4OTmRkJBQoNvBHPe3GYCzndZt6d4ddu9W40YXRbzJr8aOJD/nw6ZNULeu1tEJYXl37sDI8d60uFWOOAqj08GcOTBsmNaRCSHAzhLR6OhoIiIiSExZrq2AMplM+Pr6cu3aNXRSgyRbHmwzFxcXSpQogaenp9ahWVzDhmpMXMeOOi5cKIMuzMgfDd9G915rnv74Ra3DE8Jiru8Mod3gSpwNqcU3rMDNDVZ/D926aR2ZECKF3SSi0dHR3LhxAw8PD0qUKIGzs3OBTcKMRiMxMTF4eHjg4CCjL7Ijpc3c3d1JTk4mKiqKGzduANhlMvrYY2qWcNeucOiACZKTmPeJgQblYPBgraMTIveCd0dTvm0g3Uz/4yzjKFkSNm2CBg20jkwIcT+7SUQjIiLw8PCgTJkyBTYBTWE0GtHr9bi6ukoimk0pbebm5oaDgwNFihTh+vXrRERE2GUiCuDtDdu3w6uvOvL66iVggu/egejjfzNqUSXk0hG2asMG6N3bk0DTKo7QgKpV4ddf1YQ9IYR1sYuPGoPBQGJiIl5eXgU+CRWWodPp8PLyIjExEYPBoHU4ecbVVc0aHjNGbbdmByOXPM7otsHEx2sbmxA5sfr94+zqOp/4eNhBW2o38+TQIUlChbBWdpGIpkzIsdcJJkIbKdeTvU/4cnCA6dPhq69gr0Mr+rCSWbvq0KoV3LqldXRCZE9ysqoM8cesLfThe5zR07u3Wi2peHGtoxNCZMUuEtEU0hsqLKmgXU8DBsCmzQ785tET0OEatJsZT6zgwgWtIxPi4eLioF/XKObOhRmMoQV7GDm2ECtWgIuL1tEJIR7GrhJRIUTutG8PBw6Avz90Zy1tbq2kUUMTBw5oHZkQmbt5EwbWD+azX8rTgCAcHXV8tsiNDz9ExjkLYQPsZrKSEMIynnpKlXd6tuM8Qk4nknBHxwutIpn7bXFe7FWweomFdQsJgQ4d4PqlmpRhBH+712Lzz9CundaRCSGyS74vCiEyKFMG9h1woNkzbnhwjyBDHY70nsPMmWAyaR2dELBvH3z49FqcL4VgoBDL/T9g50FXSUKFsDFmJ6IJCQkMHDgQPz8/fHx86NWrF5GRkZkeO2nSJDw8PPD19U33KOgF54WwBZ6e8Msv0OuNIkxhAj/wIqNHw9tvQ1KS1tGJgmzVKmjfJonhsZMZwNc8+aTqxX/qKa0jE0KYy+xEdNiwYZw9e5aQkBCuXr0KQJ8+fbI8fsSIEYSFhaV7uMjocYubNGkSOp2Oy5cv29S5zTFgwACaNGmCyYwuucTERKpVq8bUqVPzMDL75ewMX38Nj334Ov/ijzsxlP16HN07JnDvntbRiYLGZIKZH+kY0Oce8QYnWrOTHc/MYv9+1YsvhLA9ZiWiUVFRLF26lBkzZuDp6YmLiwuffPIJ27Zt49y5c3kVoxAsXryY5cuXs2jRIrNms7u4uPD1118zefJkfv311zyM0H7pdDB2rKo3WtfpFG+yiIvbL9KsGfy3+JQQec5ggM8/fwq3CaPYxjM4kEy3N0qw8RcH7HTNCSEKBLMS0eDgYEwmE/Xr10/dV6ZMGQICAggKCrJ4cEIA3Lp1i/fee4+hQ4dSrVo1s1/ftGlTevXqxVtvvUW8VGnPsT59YMrOQJ4u+g9neYLrJ2/yz+NdiDwhbSryVvS/MQQ91o/Q7fdYSR/mM5Rp0x35+mvVay+EsF1mzZoPDw/H29sbJ6f0L/Px8SE8PDzT18ybN4+vvvoKZ2dnatasyciRI2nRokWmxyYmJqYbPxodHQ2olZMetrqNwWDAZDJhNBoxGo3m/JPsRsrtaqPRmPpzSptY8txatO/kyZNJSEhg6NChOX7/ESNG8P333zN79mzGpCwjdJ/M2iylLQ0GA46Ojjn/B9iRRo3gt32FeL5zIlsu16dC4hXCJwcT5O5Hw/811jo8q3f/37FH/V0TSujRa+hbd6ZV4hk2cYimzkEMW/wUvXsbZKzyI6RcX3KdmUfazXy5aSuzElGj0ZjpbVEHB4dME4Rhw4YxZswYXFxcuH37NosXL6Z9+/bs3LmTwMDADMfPmDGDyZMnZ9i/e/duChcunPU/wskJX19fYmJi0Ov15vyT7EZKAh8TE8O9/wbv3bPQIL77z53y5SC/REVFsWjRIrp06UKRIkUyvH+jRo04f/58lq8fPXo0o0aNonz58jRt2pTZs2fz1ltvZfgyleL+NtPr9cTHx7Nv3z6S5BMvnWlDokh6P5nyyQA3Of7+M3y/YxLFBlXXOjSrlpCQkPrzrl27cHV11TAa6xe9I5S2CydTwhRGecDEVca//ANeXuWRkTbZt337dq1DsEnSbtkXFxeX49ealYh6e3tz9+5dTCZTuoT09u3blChRIsPxxYoVS/25ePHivP/++2zZsoWVK1dmmoiOGTOG9957L3U7OjqasmXL0rJlS7y9vbOMKyEhgWvXruHh4VFg/7CnTABzd3dnxYoVLFy4kEuXLlG8eHGef/55Zs6cmXpMq1atuHz5Mv/880+G80yePJkpU6Zw8eJFypcvn+7chQsXZvHixXz99ddcvXqVkiVL0r17dyZOnEjRokVTz6HX61m4cCHffvstFy5cwMXFhYYNGzJx4kQaNGiQ4b127drF6tWr+fnnn4mMjOSbb76hf//+AKxduxa9Xk+3bt3wzGQg2EsvvZThm5her2fevHnEx8fTpk2b1Nd17dqV/fv3ExwcTNu2bdO9xmQyce/ePYoUKZJ6bSckJODm5kazZs0K7HX1MPHdnmFjvV48GbkX0NN/21h2xo6j6Y5x6BylMlxmYmNjU39u1apVut8bkd7BYWto98X/cEV9Ed7rVJEDo8fRbcyLspxzNhkMBrZv307btm2lzcwg7Wa+rKonZYdZiWjt2rXR6/WcOXOGGjVqACoJvXjxInXq1MnWORISEiiexcK/Li4umc6od3Z2fujFkJycjE6nw8HBAYcCupRGSvL0/vvvs337doYNG0b16tXZtWsXn332GUajkQULFqR7TWZtlXKe+9syZd+IESPYs2cPY8aM4bHHHmPLli3Mnz+fAwcOcPjwYQoVKoRer6dTp07s2rWLLl26MHbsWMLCwpg7dy4tWrRgy5YttGzZMt1533jjDYoUKcKcOXNSZ7mnvPfGjRvR6XQ888wzmcY7bty4dNuJiYl0796dhIQEvvjiC1q3bp36XLt27Rg+fDibNm2i3QPFBlN69FOuo5Q20Ol0j7z+CirnCr5U+WczW2u+RrurPwLQ+uA0gqpe4KnjS3HzzvouRkF1/3Uk11XmkpNM7Gw5jWcOTEjdd9KzGT4Hf8Dj4hFptxyQNssZabfsy007mZWI+vj48Pzzz/Puu++yZs0aChUqxNChQ6lbty5169ald+/e+Pv788knnwAwduxY3nrrLcqVK0diYiJz5swhJCSEn376KccB50TduhAWlq9vaRZfXzh2zDLn2rFjB0ePHsXX1xdPT0969+7NlStXWLZsGZ9++mmuSmft2rWLY8eOUaVKFQCef/55nJ2d+fLLL1m6dClvvfUWH330Ebt27WL48OGp1wGoEl+VK1dm8ODBnD17Nt15XV1dOXToUKbDL06cOIG3t3emPe4Pio+Pp2vXruzYsYNFixbx+uuvp3u+SpUq6HQ6goODc/LPF5lwdCtEwrze7F1aj6Yb38cBEw2v/sj5shcovGMTAY2lpo7Ivugb9zhdtz/PhK1N3Xewcn/qHf8KnYsOLmoYnBAiT5i9xOeiRYt45513qFixIkajkZYtW7J+/XoALly4kG6ykZeXFx06dCAiIoKkpCSaNGnC/v378ff3t9g/IDvCwgpOmZkPPviAKlWqpBtL2apVK7Zv387ff//NE088keNzDx8+PDUJTTFixAi+/PJLNm7cyJtvvsmCBQsoUaJEhrqdJUuWpHfv3nz55ZecPXuW6tXTxhKOHj06yzHAYWFhGd4zM3FxcXTu3Jndu3ezdOlSXnnllQzHODs74+3tTWhoaHb+ueIR4uPjadq0KVFRUQQHB3P048d54sPeFCGGx+NPcqtJHX7/6Afqvd9S61CFDbiyLQTjc50J1F8AwIiOw10+ovHakSQkJtC0kbrWWrZsKb1UQtgRsxNRT09Pvv3220yfe7CnadSoUYwaNSpnkVmQr6/WETycJePr0KFDhn0lS5YE1DCK3GjWrFmGfY899hhubm5cuHCBCxcucOvWLXr06EFycjIxMTHpji1btiwAISEh6RLRWrVqPfR9H1XAPjY2lk6dOnHgwAG+++67hy6wkNWEO2E+o9GY+jtvNBppOO1Z/qpziJgXO+GXdI2SplsUH9WGnbs+ouXmETg4SruLzB0Zu4FqM17BE/UFOgYP/p62isAPngUyXmtCCPthdiJqiyx129sWFClSJMO+lNJDycnJuTp3SkL7IE9PTxISElIHK//888/8/PPPWZ7nwdn8mcWcwtfXl7CHjKu4d+8eHTt2JCgoiFWrVvHCCy9keaxer+fOnTvZ6mEVOVO5e02iL5zgeIOXqHNrK44Yab31fQ6XDaL6kaV4lZXK4yKNPi6JvS0n0vbo9NR9f7k8gcvmddRqXVnDyIQQ+aVAJKIivUKFCmWZlD6sBENmJYySk5OJiIigZs2aeHl5AdCjRw/+97//ZXkecxLB2rVrs3HjRiIiIjKME42KiqJ9+/YEBwfz008/0bVr14ee68KFC5hMJp5++ulsv78wn2cFb2rd2My+Z6bQbM8UABqFruWfx/4k7NvVVO1VW+MIhTW4dvg6Ee360Pbe/tR9h8r0pMaRb/As7aFhZEKI/FQwp5gXcP7+/ty6dSvTW9579+7N8nUhISEZ9gUHB5OcnEydOnWoWrUqHh4eREdH06RJkywfpUqVynas3bp1w2QyZajndufOHdq0acOJEydYu3btI5NQgG3btgFk61iROw7OjjTbPZngSZu4qysKQEXDBSr0bsCebvMwJj98uIWwb0fG/4Jn4xrU/i8JTcKRg11n0ejKaklChShgJBEtgGrUqEFiYiK7d+9Ot/+rr77i6NGjWb7u008/zVCz8+OPPwbg5ZdfxtnZmTfeeIOdO3eyc+fOTM9x8OBBs2Lt2rUrLi4ubNy4Md3+vn37cuzYMRo2bMixY8eYNGlSusfnn3+e4VwbN26kRIkSWa7sJSzv6YnPEr3zGCFuTwFQCAMt1v+P332fI/zPWxpHJ/Jb4j09e+oOp8G05/AiCoB/Hcvy9+K9BK4bgc5BxhELUdDIrfkCqF+/fsyYMYM+ffowevRo/P392bx5M6tWraJhw4YEBQVl+rrLly/TsGFD3njjDYoVK8aPP/7IunXreP3111OTu2nTpnHo0CHat2/PK6+8QmBgIEWLFuWvv/5izZo1xMTEcO7cuWzH6uXlxYABA/jyyy+5fv06ZcqUwWg0sm/fPkD14GbWi9ulSxcGDRqUun3y5En27t3LjBkzslxVSeSNgJaPkRh+lL0tx9A8eDYADSI2E/HkEwRNWUnDcW00jlDkh/Nr/sSxb29aJP6Zuu93v+eocnAZpStkXltaCGH/pEe0ACpRogR79uyhdu3aTJgwgVdffZUbN26wd+/edCsfPei7776jQYMGTJw4kf79+3PhwgXmzp3L119/nXqMu7s7e/fuZerUqRw7dozBgwfTv39/vvnmGypXrsz3339vdrzjx4/H1dWVOXPmAKrQ/L179zCZTFk+UkqKpZg1axZlypRh2LBhZr+/yFqJEiUyXfHqQS5FCtH82Kccn/Yrt3VqlbQSpls0HN+WfTUHE38r5hFnELYqKTGZne1nUeGFp6n8XxKaSCH29phP3esb8MpmEprda00IYWNMViwqKsoEmCIiIh56XHx8vOns2bOm+Pj4fIrMuiUnJ5vu3LljSk5O1joUi1m0aJGpUKFCprNnz5r92n379pkcHR1NmzdvzvKYzNpMrqtH0+v1pvXr15v0en22XxNx+l/TiRJtTCZIfVx3CjAdn7cvDyO1DjExMSbABJju3LmjdTh57u9tF01/uDdM9//6kktVU8jq42afKyfXWkEnbZYz0m7mi4iIMAGmqKgos18rPaLCJrzxxhv069ePN99885F1Re+XmJjIgAEDmDhxIh07dszDCEV2edfw46mwrex/cQFxuAHgn3SVp4Y1Z/uTw4n6N/YRZxDWzmhIZk/3+fg88yQ1Y9VQHyM69jcYgV/4Saq8KJUThBCKDJYTNuP+IQDZ5eLiYtaYVJE/dI4ONF09mIuvtiPuhVeoee8wDphoe3o2oWV/4Ny4RTScnHFxBmH9zv9wCl57jRZxx1P3/esUwJ3PVtD07aYaRiaEsEbSIyqEyJH4+HjatGnDBx98QHx8fI7O8Vi7SjwRuZ8DXT8hARcA/Iw3aDilI4cDXuTWH7Icq62IDotjR93RVOr1NI/fl4QeqPk2Ra//yRO5SEItca0JIayTJKJCiBxJqV5w5syZXC276ODsSJN1w4nc9Qd/FGueur/RtR9xe6oyu7ovQB+XcTEFYR1MRhMHhq8lxr8KbYJn4oRaLOOyc2WOzz9Akz++oLBP1qunZYelrjUhhPWRRFQIYRX8W1ahZsRuDr61nMj/ZtZ7EEurdUO4XKwWR6Ztx4zhwSIfXNp4mpMl2tBkdg9KG28Aakb8vjZTKB15mjpDAjWOUAhh7SQRFUJYDZ2DjsAvX8F45jwHK/dP3V9Ff4YG45/hiE9nLv52QbsABQBhZyLZ88RgArrUovadXan7T3m35NaOP2i2fTyFirhoGKEQwlZIIiqEsDolq5Ug8MJSQpYe4qxH/dT9DW9tomzHGux98h3CT4VpGGHBFP1vDDuaT8WtRkVanP0cR9Rt8quOFTgyeh1P3dpJmdZVNY5SCGFL7CoRNaesjxCPIteT9qr2b0S1u4cJGvQtYY6lAbVMaPPTCylSqyK76o3i5rlIjaO0f4nRiezpPp+kMuVos28CXkQDEIcbu9t8iHf4WRrM6Ao6WaJTCGEeu0hEHR0dATKsgy5EbqRcTynXl9CGztGBhgtfxjP0AnubTyD+v9qjhYmn1bGPKVy9HNsDJxH59x2NI7U/cbcT2N3zC24Wr0qLdcMobroNQBKOHHriTeJP/UXL7WNx93bVOFIhhK2yi0TU2dkZFxcXoqKipBdLWITJZCIqKgoXFxecnZ21DsdqFS5cGBeX/BkLWLikO833TObe6cvsrfO/1HJPHsTS9tBkClUOYF/9EYQF38iXeOzZnav32NF+FvdKVKDlT4Mom3wl9bmgsi8QuuMsjf/8Gu8n/fMtpvy81oQQ+cduCtqXKFGCGzducP36dby8vHB2dkZXQG8TGY1G9Ho9CQkJODjYxXeNPJfSZvHx8SQnJxMVFUVMTAz+/vn3QWtr3N3duXv3Lr/++ivu7u759r6lapSiVPAcwo+9R3D/qTQ4swQnkilCDM1+/xRD3XnseewVSswYQY0XquVbXPYg7Nh1/nznCxocmU8bYtI993vJDrjPmUbDl+rke1xaXWtCiLxnN4mop6cnABEREdy4UbB7REwmE/Hx8bi5uRXYZNxcD7aZi4sL/v7+qdeVsD4+dcvi8+fXhB0azYW3PqXen0twIwFnkmhxcQn0XMIJz2YkvDmUelM74+QmPduZMSabOD5vP4bZn1Hvxjra/FcHFNSynL+X7UHxWWOo92L+J6BCCPtnN4koqGTU09MTg8FAcnLyo19gpwwGA/v27aNZs2ZyWzmbUtqsefPmcjvexvg2rojv6YWEnpzAkcHzqX14IV6mKABqR++DT/cRNrc0ZwMHUGFyfyq0KKdxxNbh1vlITo5eRZnfFlFX/0e655Jw4miVvpRdMIoGbR/XKEIhREFgV4loCmdn5wKdSDg6OpKUlISrq2uBbgdzpLSZJKHZl5CQQPfu3bl58yatWrXSvN38avngd/BD4sNHcXjYEsqs/Yyyhn8A8E3+F999k6DlJI57Nie688s8NfV5ipX30jTm/BYfbeDUjF9x+G45tW9soi3pV6y6pSvFH43fpsb8ATSuYz3DUqztWhNCWI5dJqJCiLyXnJzMb7/9lvqztXDz8aTR6v9hSh7K8U93kTh7IfXDN6bWvKwTvRdW7CVxxSAO+XfB1K071Ud0oFg5+0xK4+8mcmLWDhJXraX2pZ9pSFSGY856NSS63xDqTOtBayssRG+t15oQIvckERVC2CWdowN13m8D77ch7Pdr/D1+Of67v6OCXq3M5IKexjd+ggU/YVjgxPGizbjdsgflB3fisVblbLokZujJcP7+cgdOv26kxrXfaMy9DMfc1Plwru7LVJjUj+oda2gQpRBC5KB8U0JCAgMHDsTPzw8fHx969epFZGTWBaXnz59PhQoVKFWqFIGBgZw8eTI38QohhNl865WlyZZxlI8/z4XvjrC/1pDU9ewBnEmizt1dtFk3mEptyhNaKIAD1d5k/6BVXA4Ks/o17u9cusvBD35lZ633+NulOn61fWn6VV8aXfuRIvclofG4cbR8T36f/CtFY67T/OgsAiQJFUJoyOwe0WHDhnH+/HlCQkJwcXGhX79+9OnTh61bt2Y4dtWqVUyfPp19+/ZRpUoV5s2bR7t27bhw4QJeXvZ5G0wIYb10Djqq9K1Plb71SYr/lOOfHyDm+w1U+WMNvslp1TZKJ12j9PnFcH4xfAE3HMpyxbcBhlr18Gpdl9LP1qFk5aKa9JreuRbD35vOEbXjd1xOHqHc9QMEGP4hMIvjo/HkdKVuuL7UnZrvtqW+l1u+xiuEEA9jViIaFRXF0qVL2bNnT2pZm08++YSyZcty7tw5qlVLX7Nvzpw5DB06lCpVqgAqiV2wYAErV65k4MCBFvonCCGE+ZzcnKkzvCUMbwmmOVze9Ac3vtiIx+HtVIsKohBpK7X5G6/h/+81+HcN/AoMh3CdD6EelYnyq4axclXcqlegcCU/PKuWpnh1X4qUcMlRompMMnLrzE1uHr9O1JnrxP91HYfLFyn67zl875zFP/ka9R72enSEuNfhVq1n8OrRhupvNSWwsEzuEUJYJ7MS0eDgYEwmE/Xr10/dV6ZMGQICAggKCkqXiOr1ek6cOMGsWbPSnaNx48YEBQWZlYjGxsbi6ipLyGWXwWAgISGB2NhYmV2aTdJm5ouNjU33s623W8nWlSjZ+j3gPe7ciufiyiBif91L0T/3Uy7qNO7EpTvewxRO5XvhcO8AXAA2pz/fDbxIdHAnxrkY8c4eGBxciXZMa6MjlXrganLESR+Pu/42HklRFDLG40kUHiTjkUWcsQ9sG3DkousTRFZuROFnmlLp9WYEBBQn4L/n9SY9+lh9LlpGe/Z2reUX+buWM9Ju5rv/d9RcZiWi4eHheHt74+SU/mU+Pj6Eh4en2xcZGUlSUhI+Pj4Zjv3jj/Q161IkJiaSmJiYuh0VpWZ3lisndf+EsGZlypTROgQrFAXGKEj8FxIzPts+cpeF3icZEv6A03/A6a/gUwud1krJtSaE9crJMutmTVYyGo2ZrtTj4OCA0WjMcCyQ4fjMjk0xY8YMvLy8Uh8BAQGZHieEEEIIIazLwyavZ8WsHlFvb2/u3r2LyWRKl2Devn2bEiVKpDu2ePHi6HQ6bt++nW5/ZsemGDNmDO+9917q9t27dylXrhxXr16VyU1miI6OpmzZsly7dk2WqMwmabOckXYzn7RZzki7mU/aLGek3cwXFRVFQEAAxYsXN/u1ZiWitWvXRq/Xc+bMGWrUUCU/bt++zcWLF6lTJ/06xG5ublSvXp3g4GAaNWqUuv/o0aP07ds30/O7uLjg4pKxmLKXl5dcDDmQsuSpyD5ps5yRdjOftFnOSLuZT9osZ6TdzOfgYHZVUPNuzfv4+PD888/z7rvvEhUVRXx8PEOHDqVu3brUrVuX3r17M2LEiNTjBw8ezMyZMwkJCcFoNLJw4UIuXbqUZSIqhBBCCCEKDrPriC5atIh33nmHihUrYjQaadmyJevXrwfgwoUL6SYbDRw4kIiICFq1akVsbCxVq1Zl69at+Pr6WuwfIIQQQgghbJPZiainpyfffvttps8FBwdn2Dd+/HjGjx9vfmSoW/UTJ07M9Ha9yJq0m/mkzXJG2s180mY5I+1mPmmznJF2M19u2kxnyslceyGEEEIIIXLJ/FGlQgghhBBCWIAkokIIIYQQQhOSiAohhBBCCE3YVCL6ww8/oNPpWLZsmdahWL0dO3bQtm1bfHx8KF26NM2bN2fPnj1ah2XVQkJCePHFF/Hx8cHX15eGDRuya5ellmG0b3fu3KFjx47odDr+/vtvrcOxWgkJCQwcOBA/Pz98fHzo1atXjlYiKWiuXbtG/fr10el0JCUlaR2OTTh69CgdO3akVKlS+Pn50apVK06ePKl1WFbtxx9/JDAwEB8fH/z9/enYsSOnTp3SOiybMXPmTHQ6ndm5hs0kouHh4YwfP54nn3xS61BswsaNGxk5ciT//vsvN27coFevXnTs2FE+9B5izJgxdO7cmatXrxIWFka/fv3o2rUrUVFRWodm1a5cucKTTz5JpUqVtA7F6g0bNoyzZ88SEhLC1atXAejTp4/GUVm3I0eO0LhxY2rVqqV1KDbl/fffZ+DAgYSGhnLjxg0aNGhAly5dtA7Lqm3evJlPPvmEsLAwrly5Qu3atWnfvn2Wy5KLNGfOnGHFihX4+/ub/VqbmTXfpUsXunfvztKlS+nfvz/9+/fXOiSbEhUVRdGiRTly5Aj169fXOhyrlJSUhJNTWkWzuLg43N3dCQoKokGDBhpGZt1iYmJITEzk3r17VKhQgb/++kuS0kxERUVRsmRJ9uzZQ+PGjQG4fv06ZcuW5ezZs1SrVk3jCK1TZGQkLi4uHDt2jJYtW2IwGNL9norMPfj37OzZszzxxBOEhYXh4+OjYWS249SpU9SqVYvw8HBKlSqldThWKykpiUaNGjFr1iz69+/PsmXLaNGiRbZfbxM9osuXLycpKYl+/fppHYpNun37NlOmTKFGjRrSq/AQD364HT58GFdXVx577DGNIrINHh4eeHt7ax2G1QsODsZkMqX7IlimTBkCAgIICgrSMDLr5u3tjYeHh9Zh2JzM/p75+PhQokQJjSKyLaGhocycOZNnnnlGktBH+PDDD6lfv75Zyef9rP5r5fXr15k4cSIHDx7UOhSb9PLLL7NixQqqVavGpk2bKFSokNYh2YTIyEhef/11xo0bJ3+4hUWEh4fj7e2dIUHw8fEhPDxco6hEQfD3338zYsQIPvvsMxwdHbUOx+o1bdqUAwcO0LhxYzZs2KB1OFbt+PHjrFixghMnTuT4HFbRI1qvXj18fX0zPABef/11xo8fn6NxB/buYe2W4rvvvuPu3bv06NGDJk2aFPgPvOy0WWJiIj169KBu3bqMGTNGo0itS3baTTyc0WhEp9Nl2O/g4CBj0ESeuXPnDp07d+bVV1+lb9++WodjE/bv3094eDjVqlWjWbNmJCQkaB2SVdLr9fTv358vv/wyV3ctrKJH9Pfff890/+LFiwGVjIqMsmq3B3l5eTF16lRWrFjBmjVrGDx4cB5HZr0e1WZJSUn07NkTR0dHVqxYgYODVXxX01x2rzWRNW9vb+7evYvJZEqXkN6+fVt63UWeiImJoUOHDjz99NN8+umnWodjU0qVKsXnn39OkSJF2LlzJ506ddI6JKszZcoUGjVqROvWrXN1Hqv+lD1y5AiHDh2iaNGiqY8DBw4waNAgypUrp3V4VstkMmXaw+Lk5CS3ZR7CaDTy8ssvc+vWLTZs2ICrq6vWIQk7Urt2bfR6PWfOnEndd/v2bS5evEidOnU0jEzYo/j4eJ599llKly7N0qVLM+2NF2mSk5Mz7HNwcMDR0VE+N7Nw9OhRVq1alS5Hu3r1Ks8++yxNmzbN9nmsOhFdtGgR9+7d4+7du6mPJk2a8Pnnn3PlyhWtw7NaUVFRNGvWjMOHD6cmpbNmzeLWrVt07txZ6/Cskslk4s033+T8+fP8+uuvMjlCWJyPjw/PP/887777LlFRUcTHxzN06FDq1q1L3bp1tQ5P2BG9Xk+3bt1wcXFh9erVUmUgG06fPk3nzp05ffo0oNpw+PDh+Pj4mJVUFSTbtm0jOjo6XY4WEBDAL7/8wv79+7N9HqtOREXOFC1alIEDBzJy5Eh8fX0pU6YMW7ZsYefOnZQuXVrr8KzS1atXWbJkCZcuXeLxxx9PNw5y6dKlWocn7MSiRYvw8/OjYsWKlC5dmri4ONavX691WMLOHD58mK1bt3Ls2DECAgLS/T3bsWOH1uFZpZo1a9K2bVv69++Pj48PFSpU4MqVK2zfvh13d3etw7NrNlNHVAghhBBC2BfpERVCCCGEEJqQRFQIIYQQQmhCElEhhBBCCKEJSUSFEEIIIYQmJBEVQgghhBCakERUCCGEEEJoQhJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEyAfTp09Hp9NleMydO1fr0IQQQjOy1rwQQuSDe/fuERsbm7o9YcIEtm3bxoEDByhTpoyGkQkhhHactA5ACCEKgiJFilCkSBEAxo8fz7Zt29izZ48koUKIAk1uzQshRD6aMGEC3333HXv27KF8+fJahyOEEJqSRFQIIfLJxIkT+fbbbyUJFUKI/0giKoQQ+WDixIksX75cklAhhLiPjBEVQog8Nm3aNL744gs2btyIq6srYWFhABQrVgwXFxeNoxNCCO3IrHkhhMhDJpOJokWLEh0dneG5o0ePUq9ePQ2iEkII6yCJqBBCCCGE0ISMERVCCCGEEJqQRFQIIYQQQmhCElEhhBBCCKEJSUSFEEIIIYQmJBEVQgghhBCakERUCCGEEEJoQhJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEEEIIoQlJRIUQQgghhCYkERVCCCGEEJr4P6MHSnznfycIAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "# 추가 코드 - 후버 손실의 모습을 보여줍니다.\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(8, 3.5))\n",
        "z = np.linspace(-4, 4, 200)\n",
        "z_center = np.linspace(-1, 1, 200)\n",
        "plt.plot(z, huber_fn(0, z), \"b-\", linewidth=2, label=\"huber($z$)\")\n",
        "plt.plot(z, z ** 2 / 2, \"r:\", linewidth=1)\n",
        "plt.plot(z_center, z_center ** 2 / 2, \"r\", linewidth=2)\n",
        "plt.plot([-1, -1], [0, huber_fn(0., -1.)], \"k--\")\n",
        "plt.plot([1, 1], [0, huber_fn(0., 1.)], \"k--\")\n",
        "plt.gca().axhline(y=0, color='k')\n",
        "plt.gca().axvline(x=0, color='k')\n",
        "plt.text(2.1, 3.5, r\"$\\frac{1}{2}z^2$\", color=\"r\", fontsize=15)\n",
        "plt.text(3.0, 2.2, r\"$|z| - \\frac{1}{2}$\", color=\"b\", fontsize=15)\n",
        "plt.axis([-4, 4, 0, 4])\n",
        "plt.grid(True)\n",
        "plt.xlabel(\"$z$\")\n",
        "plt.legend(fontsize=14)\n",
        "plt.title(\"후버 손실\", fontsize=14)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L2CATH3GAxrc"
      },
      "source": [
        "사용자 정의 손실 함수를 테스트하기 위해 기본 케라스 모델을 만들고 캘리포니아 주택 데이터셋으로 훈련해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 73,
      "metadata": {
        "tags": [],
        "id": "H1VF7XsaAxrc"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 캘리포니아 주택 데이터셋을 로드, 분할 및 스케일을 조정한 다음 간단한 케라스 모델을 생성합니다.\n",
        "\n",
        "from sklearn.datasets import fetch_california_housing\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "housing = fetch_california_housing()\n",
        "X_train_full, X_test, y_train_full, y_test = train_test_split(\n",
        "    housing.data, housing.target.reshape(-1, 1), random_state=42)\n",
        "X_train, X_valid, y_train, y_valid = train_test_split(\n",
        "    X_train_full, y_train_full, random_state=42)\n",
        "\n",
        "scaler = StandardScaler()\n",
        "X_train_scaled = scaler.fit_transform(X_train)\n",
        "X_valid_scaled = scaler.transform(X_valid)\n",
        "X_test_scaled = scaler.transform(X_test)\n",
        "\n",
        "input_shape = X_train.shape[1:]\n",
        "\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 74,
      "metadata": {
        "id": "48JLXr2TAxrc"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=huber_fn, optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 75,
      "metadata": {
        "scrolled": true,
        "id": "tjwQXkydAxrc",
        "outputId": "4c1d078d-3649-4bd4-8aac-2ef6651af963",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 9s 7ms/step - loss: 0.6227 - mae: 0.9913 - val_loss: 0.3209 - val_mae: 0.6431\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 6ms/step - loss: 0.2743 - mae: 0.5869 - val_loss: 0.2338 - val_mae: 0.5314\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f454126dca0>"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4-07_kd7Axrd"
      },
      "source": [
        "## 사용자 정의 요소를 가진 모델을 저장하고 로드하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 76,
      "metadata": {
        "id": "Q7tph7CeAxrd"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss\")  # 추가 코드 - 정상적으로 저장됩니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 77,
      "metadata": {
        "id": "AB2rQdS1Axrd"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss\",\n",
        "                                   custom_objects={\"huber_fn\": huber_fn})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 78,
      "metadata": {
        "id": "NDkbf9O8Axrd",
        "outputId": "51b0fb25-c485-4d75-cf0c-c1c9ebc157d5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 6s 9ms/step - loss: 0.2162 - mae: 0.5078 - val_loss: 0.2125 - val_mae: 0.4929\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 6ms/step - loss: 0.1917 - mae: 0.4733 - val_loss: 0.1858 - val_mae: 0.4636\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f4580677eb0>"
            ]
          },
          "metadata": {},
          "execution_count": 78
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 79,
      "metadata": {
        "id": "mJH4gpIDAxrd"
      },
      "outputs": [],
      "source": [
        "def create_huber(threshold=1.0):\n",
        "    def huber_fn(y_true, y_pred):\n",
        "        error = y_true - y_pred\n",
        "        is_small_error = tf.abs(error) < threshold\n",
        "        squared_loss = tf.square(error) / 2\n",
        "        linear_loss  = threshold * tf.abs(error) - threshold ** 2 / 2\n",
        "        return tf.where(is_small_error, squared_loss, linear_loss)\n",
        "    return huber_fn"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 80,
      "metadata": {
        "id": "T3d5H-VFAxrd"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=create_huber(2.0), optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 81,
      "metadata": {
        "scrolled": true,
        "id": "BFdizsdgAxrd",
        "outputId": "7053b7f7-9943-4e9c-d0a8-f413b732c037",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 5ms/step - loss: 0.2043 - mae: 0.4619 - val_loss: 0.1949 - val_mae: 0.4446\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.1967 - mae: 0.4545 - val_loss: 0.1957 - val_mae: 0.4490\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f4603a6d580>"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 82,
      "metadata": {
        "id": "59g4fFXUAxrd"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss_threshold_2\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 83,
      "metadata": {
        "id": "_Kq6XfV1Axrd"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss_threshold_2\",\n",
        "                                   custom_objects={\"huber_fn\": create_huber(2.0)})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 84,
      "metadata": {
        "id": "VZM9Q3aSAxre",
        "outputId": "593a5f02-6a30-45d1-a86d-604f87c18f83",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.1933 - mae: 0.4486 - val_loss: 0.1918 - val_mae: 0.4359\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.1898 - mae: 0.4451 - val_loss: 0.1917 - val_mae: 0.4418\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f4540cf1ca0>"
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 85,
      "metadata": {
        "id": "Geic7wZ9Axre"
      },
      "outputs": [],
      "source": [
        "class HuberLoss(tf.keras.losses.Loss):\n",
        "    def __init__(self, threshold=1.0, **kwargs):\n",
        "        self.threshold = threshold\n",
        "        super().__init__(**kwargs)\n",
        "\n",
        "    def call(self, y_true, y_pred):\n",
        "        error = y_true - y_pred\n",
        "        is_small_error = tf.abs(error) < self.threshold\n",
        "        squared_loss = tf.square(error) / 2\n",
        "        linear_loss  = self.threshold * tf.abs(error) - self.threshold**2 / 2\n",
        "        return tf.where(is_small_error, squared_loss, linear_loss)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 86,
      "metadata": {
        "id": "dq0XagVZAxre"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 간단한 케라스 모델을 또 하나 생성합니다.\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 87,
      "metadata": {
        "id": "YU2e8QsUAxre"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=HuberLoss(2.), optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 88,
      "metadata": {
        "id": "1sav9-HsAxre",
        "outputId": "70e603e7-3be6-4d1d-f9ea-afec67ec5ba9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 1.2170 - mae: 1.2163 - val_loss: 0.4254 - val_mae: 0.6675\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.3430 - mae: 0.6103 - val_loss: 0.2829 - val_mae: 0.5537\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f45317830d0>"
            ]
          },
          "metadata": {},
          "execution_count": 88
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 89,
      "metadata": {
        "id": "hWAYNxvXAxre"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss_class\")  # 추가 코드 - 정상 저장"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 90,
      "metadata": {
        "id": "r7u-ejM6Axre"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss_class\",\n",
        "                                   custom_objects={\"HuberLoss\": HuberLoss})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 91,
      "metadata": {
        "id": "Dj4Ng3SlAxre",
        "outputId": "8667837b-9283-4a68-f4b4-ef9a7093d4b1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.2639 - mae: 0.5326 - val_loss: 0.2434 - val_mae: 0.4888\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.2206 - mae: 0.4853 - val_loss: 0.2072 - val_mae: 0.4626\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f454064aa60>"
            ]
          },
          "metadata": {},
          "execution_count": 91
        }
      ],
      "source": [
        "# 추가 코드 - 정상적으로 로드되고 모델을 정상적으로 사용할 수 있음을 보여줍니다.\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 92,
      "metadata": {
        "id": "YSN4u6loAxre",
        "outputId": "3e485e4d-b65d-4c0f-a2ab-a8ea0ba87e91",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 92
        }
      ],
      "source": [
        "model.loss.threshold  # 추가 코드 - threshold가 올바르게 로드되었습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJYIUz1bAxrf"
      },
      "source": [
        "## 기타 사용자 정의 기능"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 93,
      "metadata": {
        "id": "KgqccYnMAxrf"
      },
      "outputs": [],
      "source": [
        "def my_softplus(z):\n",
        "    return tf.math.log(1.0 + tf.exp(z))\n",
        "\n",
        "def my_glorot_initializer(shape, dtype=tf.float32):\n",
        "    stddev = tf.sqrt(2. / (shape[0] + shape[1]))\n",
        "    return tf.random.normal(shape, stddev=stddev, dtype=dtype)\n",
        "\n",
        "def my_l1_regularizer(weights):\n",
        "    return tf.reduce_sum(tf.abs(0.01 * weights))\n",
        "\n",
        "def my_positive_weights(weights):  # 반환 값은 tf.nn.relu(weights)입니다.\n",
        "    return tf.where(weights < 0., tf.zeros_like(weights), weights)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 94,
      "metadata": {
        "id": "AA-eG-A6Axrf"
      },
      "outputs": [],
      "source": [
        "layer = tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                              kernel_initializer=my_glorot_initializer,\n",
        "                              kernel_regularizer=my_l1_regularizer,\n",
        "                              kernel_constraint=my_positive_weights)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 95,
      "metadata": {
        "id": "4tcdzl_bAxrf",
        "outputId": "92924b37-b58b-48bc-d719-11f01a907502",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 5ms/step - loss: 1.6232 - mae: 0.8312 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.9223 - mae: 0.6060 - val_loss: inf - val_mae: inf\n",
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.6089 - mae: 0.5286 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.5374 - mae: 0.4972 - val_loss: inf - val_mae: inf\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f45413199a0>"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 요소가 많이 포함된 모델을 빌드, 훈련, 저장, 로드, 훈련할 수 있습니다.\n",
        "\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                          kernel_initializer=my_glorot_initializer,\n",
        "                          kernel_regularizer=my_l1_regularizer,\n",
        "                          kernel_constraint=my_positive_weights)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[\"mae\"])\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.save(\"my_model_with_many_custom_parts\")\n",
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_many_custom_parts\",\n",
        "    custom_objects={\n",
        "       \"my_l1_regularizer\": my_l1_regularizer,\n",
        "       \"my_positive_weights\": my_positive_weights,\n",
        "       \"my_glorot_initializer\": my_glorot_initializer,\n",
        "       \"my_softplus\": my_softplus,\n",
        "    }\n",
        ")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 96,
      "metadata": {
        "id": "Tm1vXkgqAxrf"
      },
      "outputs": [],
      "source": [
        "class MyL1Regularizer(tf.keras.regularizers.Regularizer):\n",
        "    def __init__(self, factor):\n",
        "        self.factor = factor\n",
        "\n",
        "    def __call__(self, weights):\n",
        "        return tf.reduce_sum(tf.abs(self.factor * weights))\n",
        "\n",
        "    def get_config(self):\n",
        "        return {\"factor\": self.factor}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 97,
      "metadata": {
        "id": "ou_qdhJKAxrf",
        "outputId": "36e736a5-ce6e-4c91-b605-8f22c00f1a22",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 1.5960 - mae: 0.8338 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.8550 - mae: 0.6055 - val_loss: inf - val_mae: inf\n",
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 5ms/step - loss: 0.5982 - mae: 0.5289 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.5138 - mae: 0.4949 - val_loss: 2.2021 - val_mae: 0.5112\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453fb89370>"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ],
      "source": [
        "# 추가 코드 - 이번에는 사용자 정의 규제 클래스를 사용하는 경우 정상적으로 작동하는지 보여줍니다.\n",
        "\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                          kernel_regularizer=MyL1Regularizer(0.01),\n",
        "                          kernel_constraint=my_positive_weights,\n",
        "                          kernel_initializer=my_glorot_initializer),\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[\"mae\"])\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.save(\"my_model_with_many_custom_parts\")\n",
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_many_custom_parts\",\n",
        "    custom_objects={\n",
        "       \"MyL1Regularizer\": MyL1Regularizer,\n",
        "       \"my_positive_weights\": my_positive_weights,\n",
        "       \"my_glorot_initializer\": my_glorot_initializer,\n",
        "       \"my_softplus\": my_softplus,\n",
        "    }\n",
        ")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RXmm0A9rAxrf"
      },
      "source": [
        "## 사용자 정의 지표"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 98,
      "metadata": {
        "id": "xnKzilGhAxrf"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 다시 한 번 기본 케라스 모델을 만들어 보겠습니다.\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 99,
      "metadata": {
        "id": "DJPKkv0FAxrg"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[create_huber(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 100,
      "metadata": {
        "id": "s5sLLfxVAxrg",
        "outputId": "f2c8749d-6935-46cf-ef6b-6b3fd180921e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 3.8670 - huber_fn: 1.5925\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.8146 - huber_fn: 0.3913\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453fa04ac0>"
            ]
          },
          "metadata": {},
          "execution_count": 100
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 지표로 모델 훈련하기\n",
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Vh4j8hFAxrg"
      },
      "source": [
        "**노트**: 동일한 함수를 손실과 지표로 사용하면 약간 다른 결과에 놀랄 수 있습니다. 이는 부분적으로 연산이 정확히 같은 순서로 계산되지 않아 미세한 부동 소수점 오류가 발생할 수 있기 때문입니다. 더 중요한 것은 샘플 가중치 또는 클래스 가중치를 사용하는 경우 방정식이 약간 달라진다는 것입니다:\n",
        "* `fit()` 메서드는 에포크 시작 이후 지금까지 본 모든 배치 손실의 평균을 추적합니다. 각 배치 손실은 가중치가 적용된 샘플 손실의 합을 _배치 크기_로 나눈 값입니다. (가중치의 합으로 나눈 것이 아니므로 배치 손실은 손실의 가중 평균이 _아니다_.).\n",
        "* 에포크 시작 이후의 지표는 가중치가 적용된 샘플 손실의 합을 지금까지 본 모든 가중치의 합으로 나눈 값과 같습니다. 즉, 모든 샘플 손실의 가중 평균입니다. 위와 같지 않습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z8_GhFUHAxrg"
      },
      "source": [
        "### 스트리밍 지표"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 101,
      "metadata": {
        "id": "fzk8uz9qAxrg",
        "outputId": "3a0e89eb-983b-4ae8-d3d3-800e53d5aa3c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.8>"
            ]
          },
          "metadata": {},
          "execution_count": 101
        }
      ],
      "source": [
        "precision = tf.keras.metrics.Precision()\n",
        "precision([0, 1, 1, 1, 0, 1, 0, 1], [1, 1, 0, 1, 0, 1, 0, 1])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 102,
      "metadata": {
        "id": "uyltKAhEAxrg",
        "outputId": "1df76147-66d9-49f4-c3f3-18c163b1f572",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.5>"
            ]
          },
          "metadata": {},
          "execution_count": 102
        }
      ],
      "source": [
        "precision([0, 1, 0, 0, 1, 0, 1, 1], [1, 0, 1, 1, 0, 0, 0, 0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 103,
      "metadata": {
        "id": "EzyuXv7wAxrg",
        "outputId": "014048e8-4267-4bdb-8c97-382cec614a8b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.5>"
            ]
          },
          "metadata": {},
          "execution_count": 103
        }
      ],
      "source": [
        "precision.result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 104,
      "metadata": {
        "id": "9CzGj48DAxrg",
        "outputId": "3bd80d3d-25fa-430e-8950-ebf305661465",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'true_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>,\n",
              " <tf.Variable 'false_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>]"
            ]
          },
          "metadata": {},
          "execution_count": 104
        }
      ],
      "source": [
        "precision.variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 105,
      "metadata": {
        "id": "XGk8OUh5Axrg"
      },
      "outputs": [],
      "source": [
        "precision.reset_states()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HnU7bMCqAxrg"
      },
      "source": [
        "Creating a streaming metric:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 106,
      "metadata": {
        "id": "wfr8lxBUAxrh"
      },
      "outputs": [],
      "source": [
        "class HuberMetric(tf.keras.metrics.Metric):\n",
        "    def __init__(self, threshold=1.0, **kwargs):\n",
        "        super().__init__(**kwargs)  # 기본 매개변수 처리 (예를 들어, dtype)\n",
        "        self.threshold = threshold\n",
        "        self.huber_fn = create_huber(threshold)\n",
        "        self.total = self.add_weight(\"total\", initializer=\"zeros\")\n",
        "        self.count = self.add_weight(\"count\", initializer=\"zeros\")\n",
        "\n",
        "    def update_state(self, y_true, y_pred, sample_weight=None):\n",
        "        sample_metrics = self.huber_fn(y_true, y_pred)\n",
        "        self.total.assign_add(tf.reduce_sum(sample_metrics))\n",
        "        self.count.assign_add(tf.cast(tf.size(y_true), tf.float32))\n",
        "\n",
        "    def result(self):\n",
        "        return self.total / self.count\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JJUFrZyaAxrh"
      },
      "source": [
        "**추가 자료** – 이 섹션의 나머지 부분에서는 `HuberMetric` 클래스를 테스트하고 `tf.keras.metrics.Mean`를 서브클래싱하는 또 다른 예를 보여줍니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 107,
      "metadata": {
        "id": "1CXamD7gAxrh",
        "outputId": "a1461a36-1b97-49fc-8522-bf2807c4d3ef",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=14.0>"
            ]
          },
          "metadata": {},
          "execution_count": 107
        }
      ],
      "source": [
        "m = HuberMetric(2.)\n",
        "\n",
        "# total = 2 * |10 - 2| - 2²/2 = 14\n",
        "# count = 1\n",
        "# result = 14 / 1 = 14\n",
        "m(tf.constant([[2.]]), tf.constant([[10.]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 108,
      "metadata": {
        "id": "m2gmSzaEAxrh",
        "outputId": "dd96714f-015e-4809-853d-064aa6deff14",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=7.0>"
            ]
          },
          "metadata": {},
          "execution_count": 108
        }
      ],
      "source": [
        "# total = total + (|1 - 0|² / 2) + (2 * |9.25 - 5| - 2² / 2) = 14 + 7 = 21\n",
        "# count = count + 2 = 3\n",
        "# result = total / count = 21 / 3 = 7\n",
        "m(tf.constant([[0.], [5.]]), tf.constant([[1.], [9.25]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 109,
      "metadata": {
        "id": "YtwdTpYDAxrh",
        "outputId": "83745964-7d11-4447-a371-13272c5ba350",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=7.0>"
            ]
          },
          "metadata": {},
          "execution_count": 109
        }
      ],
      "source": [
        "m.result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 110,
      "metadata": {
        "id": "Ae19HKP3Axrh",
        "outputId": "9075240d-208f-4b14-adbf-792993b8e648",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'total:0' shape=() dtype=float32, numpy=21.0>,\n",
              " <tf.Variable 'count:0' shape=() dtype=float32, numpy=3.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 110
        }
      ],
      "source": [
        "m.variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 111,
      "metadata": {
        "id": "2yBflIIAAxrh",
        "outputId": "c231d5fe-b090-452d-c578-13690144794f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'total:0' shape=() dtype=float32, numpy=0.0>,\n",
              " <tf.Variable 'count:0' shape=() dtype=float32, numpy=0.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 111
        }
      ],
      "source": [
        "m.reset_states()\n",
        "m.variables"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IHirmthOAxrh"
      },
      "source": [
        "`HuberMetric` 클래스가 제대로 작동하는지 확인해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 112,
      "metadata": {
        "id": "3AH5XvvXAxrh"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 113,
      "metadata": {
        "id": "m5ExIJ5oAxri"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=create_huber(2.0), optimizer=\"nadam\",\n",
        "              metrics=[HuberMetric(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 114,
      "metadata": {
        "id": "wU0qMrn2Axri",
        "outputId": "adf3e67e-3a56-4b00-b3d8-75f643bbe112",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.5445 - huber_metric_1: 0.5445\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2850 - huber_metric_1: 0.2850\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453f827640>"
            ]
          },
          "metadata": {},
          "execution_count": 114
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 115,
      "metadata": {
        "id": "d38H2hPkAxri"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_metric\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 116,
      "metadata": {
        "id": "Fm1d2JgSAxri"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_a_custom_metric\",\n",
        "    custom_objects={\n",
        "        \"huber_fn\": create_huber(2.0),\n",
        "        \"HuberMetric\": HuberMetric\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 117,
      "metadata": {
        "id": "4hsey5XSAxri",
        "outputId": "e60a3b94-e9f1-452b-9dae-dd2babb3f9a4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.2275 - huber_metric_1: 0.2275\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2054 - huber_metric_1: 0.2054\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453ecba4f0>"
            ]
          },
          "metadata": {},
          "execution_count": 117
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dAZg4651Axri"
      },
      "source": [
        "`model.metrics`은 모델 손실 다음에 모델의 지표를 담고 있으므로 `HuberMetric`은 `model.metrics[-1]`입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 118,
      "metadata": {
        "id": "7bIynYbPAxri",
        "outputId": "841f386c-9098-4f00-8f39-af548361f622",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 118
        }
      ],
      "source": [
        "model.metrics[-1].threshold"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UZZVBn1lAxri"
      },
      "source": [
        "잘 작동하는 것 같습니다! 더 간단하게는 다음과 같이 클래스를 만들 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 119,
      "metadata": {
        "id": "uB8h98UCAxri"
      },
      "outputs": [],
      "source": [
        "class HuberMetric(tf.keras.metrics.Mean):\n",
        "    def __init__(self, threshold=1.0, name='HuberMetric', dtype=None):\n",
        "        self.threshold = threshold\n",
        "        self.huber_fn = create_huber(threshold)\n",
        "        super().__init__(name=name, dtype=dtype)\n",
        "\n",
        "    def update_state(self, y_true, y_pred, sample_weight=None):\n",
        "        metric = self.huber_fn(y_true, y_pred)\n",
        "        super(HuberMetric, self).update_state(metric, sample_weight)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}        "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tbYo0q7gAxrj"
      },
      "source": [
        "이 클래스는 크기를 더 잘 처리하며 샘플 가중치도 지원합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 120,
      "metadata": {
        "id": "KwwfYIUzAxrj"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 121,
      "metadata": {
        "id": "mQFwmPxvAxrj"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=tf.keras.losses.Huber(2.0), optimizer=\"nadam\",\n",
        "              weighted_metrics=[HuberMetric(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 122,
      "metadata": {
        "scrolled": true,
        "id": "1Z760c4eAxrj",
        "outputId": "21f909bf-9e5b-42a3-882d-44217ba187d6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.5566 - HuberMetric: 1.1217\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.1664 - HuberMetric: 0.3353\n"
          ]
        }
      ],
      "source": [
        "np.random.seed(42)\n",
        "sample_weight = np.random.rand(len(y_train))\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2,\n",
        "                    sample_weight=sample_weight)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 123,
      "metadata": {
        "id": "tiohRlrmAxrj",
        "outputId": "cadcb59a-88c7-44b1-db49-03634c5dc464",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0.5566383600234985, 0.5566384291922625)"
            ]
          },
          "metadata": {},
          "execution_count": 123
        }
      ],
      "source": [
        "(history.history[\"loss\"][0],\n",
        " history.history[\"HuberMetric\"][0] * sample_weight.mean())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 124,
      "metadata": {
        "id": "ApXwk937Axrj"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_metric_v2\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 125,
      "metadata": {
        "id": "BIK-TcXEAxrj"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_metric_v2\",\n",
        "                                   custom_objects={\"HuberMetric\": HuberMetric})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 126,
      "metadata": {
        "id": "DA9CXZgTAxrj",
        "outputId": "27ba9b0e-2b06-41e4-844b-cd0607952251",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.2683 - HuberMetric: 0.2683\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2313 - HuberMetric: 0.2313\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453fd0dd90>"
            ]
          },
          "metadata": {},
          "execution_count": 126
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 127,
      "metadata": {
        "scrolled": true,
        "id": "YGc3D2KbAxrk",
        "outputId": "c92f615e-ce31-4d3c-a7af-6dd43cd0e992",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 127
        }
      ],
      "source": [
        "model.metrics[-1].threshold"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHEBZvKPAxrk"
      },
      "source": [
        "## 사용자 정의 층"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 128,
      "metadata": {
        "id": "z1_acb9zAxrk"
      },
      "outputs": [],
      "source": [
        "exponential_layer = tf.keras.layers.Lambda(lambda x: tf.exp(x))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 129,
      "metadata": {
        "id": "brtmfdE-Axrk",
        "outputId": "f42ae1d3-4eda-4c08-d7bf-ddc26226565b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float32, numpy=array([0.36787948, 1.        , 2.7182817 ], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 129
        }
      ],
      "source": [
        "# 추가 코드 - 다른 층과 마찬가지로 함수로 사용할 수 있습니다:\n",
        "exponential_layer([-1., 0., 1.])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xn6MwQRAAxrk"
      },
      "source": [
        "회귀 모델의 출력에 지수 층을 추가하면 예측할 값이 양수이고 스케일이 매우 다른 경우(예: 0.001, 10., 10000) 유용할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 130,
      "metadata": {
        "id": "4YY1BOGbAxrk",
        "outputId": "28c2667b-5329-4d5a-e313-23cfd7d52518",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.7319 - val_loss: 0.4391\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.4257 - val_loss: 0.3694\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.4032 - val_loss: 0.3514\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.4107 - val_loss: 0.4558\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.4232 - val_loss: 0.3643\n",
            "162/162 [==============================] - 0s 2ms/step - loss: 0.3755\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.37553852796554565"
            ]
          },
          "metadata": {},
          "execution_count": 130
        }
      ],
      "source": [
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "    exponential_layer\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"sgd\")\n",
        "model.fit(X_train_scaled, y_train, epochs=5,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gSgHQIJ6Axrk"
      },
      "source": [
        "또는 타깃을 타깃의 로그로 대체하고 출력 층에서 활성화 함수를 사용하지 않는 것이 더 나을 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 131,
      "metadata": {
        "id": "vVR13Y31Axrk"
      },
      "outputs": [],
      "source": [
        "class MyDense(tf.keras.layers.Layer):\n",
        "    def __init__(self, units, activation=None, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.units = units\n",
        "        self.activation = tf.keras.activations.get(activation)\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        self.kernel = self.add_weight(\n",
        "            name=\"kernel\", shape=[batch_input_shape[-1], self.units],\n",
        "            initializer=\"he_normal\")\n",
        "        self.bias = self.add_weight(\n",
        "            name=\"bias\", shape=[self.units], initializer=\"zeros\")\n",
        "        super().build(batch_input_shape)  # 마지막에 호출해야 합니다.\n",
        "\n",
        "    def call(self, X):\n",
        "        return self.activation(X @ self.kernel + self.bias)\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        return tf.TensorShape(batch_input_shape.as_list()[:-1] + [self.units])\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"units\": self.units,\n",
        "                \"activation\": tf.keras.activations.serialize(self.activation)}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 132,
      "metadata": {
        "id": "_Lh96KyuAxrk",
        "outputId": "e8e414df-875e-443d-f9be-6b9465128b80",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 1.7176 - val_loss: 0.7603\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.6328 - val_loss: 0.5426\n",
            "162/162 [==============================] - 1s 9ms/step - loss: 0.5437\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 층을 보통처럼 사용할 수 있음을 보여줍니다.\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    MyDense(30, activation=\"relu\", input_shape=input_shape),\n",
        "    MyDense(1)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)\n",
        "model.save(\"my_model_with_a_custom_layer\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 133,
      "metadata": {
        "id": "OMJn9UA7Axrk",
        "outputId": "97062da7-7f43-4183-9eac-6d476edc42fb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.4904 - val_loss: 0.5000\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.4324 - val_loss: 0.4676\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f453f245880>"
            ]
          },
          "metadata": {},
          "execution_count": 133
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 층을 가진 모델을 로드하는 방법을 보여줍니다.\n",
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_layer\",\n",
        "                                   custom_objects={\"MyDense\": MyDense})\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 134,
      "metadata": {
        "id": "8I7ct6m1Axrl"
      },
      "outputs": [],
      "source": [
        "class MyMultiLayer(tf.keras.layers.Layer):\n",
        "    def call(self, X):\n",
        "        X1, X2 = X\n",
        "        print(\"X1.shape: \", X1.shape ,\" X2.shape: \", X2.shape)  # extra code\n",
        "        return X1 + X2, X1 * X2, X1 / X2\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        batch_input_shape1, batch_input_shape2 = batch_input_shape\n",
        "        return [batch_input_shape1, batch_input_shape1, batch_input_shape1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wuI7HeD9Axrl"
      },
      "source": [
        "사용자 정의 층은 다음과 같이 함수형 API를 사용하여 호출할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 135,
      "metadata": {
        "id": "H6X91b3DAxrl",
        "outputId": "099ee2ac-71f6-4961-ae0d-8e588be68c7a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "X1.shape:  (None, 2)  X2.shape:  (None, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,\n",
              " <KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,\n",
              " <KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>)"
            ]
          },
          "metadata": {},
          "execution_count": 135
        }
      ],
      "source": [
        "# 추가 코드 - 심볼릭 입력으로 MyMultiLayer를 테스트합니다.\n",
        "inputs1 = tf.keras.layers.Input(shape=[2])\n",
        "inputs2 = tf.keras.layers.Input(shape=[2])\n",
        "MyMultiLayer()((inputs1, inputs2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e_tIIUkAAxrl"
      },
      "source": [
        "`call()` 메서드는 심볼릭 입력을 받고 심볼릭 출력을 반환한다는 점에 유의하세요. 이 단계에서는 부분적인 크기만 지정합니다. 배치 크기를 알 수 없기 때문에 첫 번째 차원이 `None`입니다.\n",
        "\n",
        "실제 데이터를 사용자 정의 층에 전달할 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 136,
      "metadata": {
        "id": "kvY8ufHVAxrl",
        "outputId": "232bc4b9-3f39-43fc-ae9a-0fe79defddf5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "X1.shape:  (2, 2)  X2.shape:  (2, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[ 9., 18.],\n",
              "        [ 6., 10.]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[18., 72.],\n",
              "        [ 8., 21.]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[0.5      , 0.5      ],\n",
              "        [0.5      , 2.3333333]], dtype=float32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 136
        }
      ],
      "source": [
        "# 추가 코드 - 실제 데이터로 MyMultiLayer를 테스트합니다.\n",
        "X1, X2 = np.array([[3., 6.], [2., 7.]]), np.array([[6., 12.], [4., 3.]]) \n",
        "MyMultiLayer()((X1, X2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MofR-d5TAxrl"
      },
      "source": [
        "이제 훈련과 테스트 중에 다르게 동작하는 층을 만들어 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 137,
      "metadata": {
        "id": "YJkH97AEAxrl"
      },
      "outputs": [],
      "source": [
        "class MyGaussianNoise(tf.keras.layers.Layer):\n",
        "    def __init__(self, stddev, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.stddev = stddev\n",
        "\n",
        "    def call(self, X, training=None):\n",
        "        if training:\n",
        "            noise = tf.random.normal(tf.shape(X), stddev=self.stddev)\n",
        "            return X + noise\n",
        "        else:\n",
        "            return X\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        return batch_input_shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6IPN9u64Axrl"
      },
      "source": [
        "다음은 이 사용자 정의 층을 사용하는 간단한 모델입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 138,
      "metadata": {
        "id": "j96OkK2AAxrl",
        "outputId": "df35ca94-0148-42e9-f7ca-15d8ce1ac300",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 2.3023 - val_loss: 24.5856\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 1.4380 - val_loss: 16.8901\n",
            "162/162 [==============================] - 1s 2ms/step - loss: 1.2175\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.2175121307373047"
            ]
          },
          "metadata": {},
          "execution_count": 138
        }
      ],
      "source": [
        "# 추가 코드 - MyGaussianNoise 테스트\n",
        "tf.random.set_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    MyGaussianNoise(stddev=1.0, input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                          kernel_initializer=\"he_normal\"),\n",
        "    tf.keras.layers.Dense(1)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNNjOu3IAxrm"
      },
      "source": [
        "## 사용자 정의 모델"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 139,
      "metadata": {
        "id": "Tdb4LT1DAxrm"
      },
      "outputs": [],
      "source": [
        "class ResidualBlock(tf.keras.layers.Layer):\n",
        "    def __init__(self, n_layers, n_neurons, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden = [tf.keras.layers.Dense(n_neurons, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "                       for _ in range(n_layers)]\n",
        "\n",
        "    def call(self, inputs):\n",
        "        Z = inputs\n",
        "        for layer in self.hidden:\n",
        "            Z = layer(Z)\n",
        "        return inputs + Z"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 140,
      "metadata": {
        "id": "9j7Am2wXAxrm"
      },
      "outputs": [],
      "source": [
        "class ResidualRegressor(tf.keras.Model):\n",
        "    def __init__(self, output_dim, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden1 = tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "        self.block1 = ResidualBlock(2, 30)\n",
        "        self.block2 = ResidualBlock(2, 30)\n",
        "        self.out = tf.keras.layers.Dense(output_dim)\n",
        "\n",
        "    def call(self, inputs):\n",
        "        Z = self.hidden1(inputs)\n",
        "        for _ in range(1 + 3):\n",
        "            Z = self.block1(Z)\n",
        "        Z = self.block2(Z)\n",
        "        return self.out(Z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 141,
      "metadata": {
        "id": "eogB59RYAxrm",
        "outputId": "3ec7f38e-cfe2-4088-a56c-21c44718577e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 7s 5ms/step - loss: 82.1872\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 1.7681\n",
            "162/162 [==============================] - 1s 3ms/step - loss: 1.3183\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:absl:Found untraced functions such as _update_step_xla, dense_20_layer_call_fn, dense_20_layer_call_and_return_conditional_losses, dense_21_layer_call_fn, dense_21_layer_call_and_return_conditional_losses while saving (showing 5 of 9). These functions will not be directly callable after loading.\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 모델을 정상적으로 사용할 수 있음을 보여줍니다.\n",
        "tf.random.set_seed(42)\n",
        "model = ResidualRegressor(1)\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2)\n",
        "score = model.evaluate(X_test_scaled, y_test)\n",
        "model.save(\"my_custom_model\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 142,
      "metadata": {
        "id": "vJWP5N0eAxrm",
        "outputId": "41c1cca3-c54a-4f1c-c7fe-3d9c8928477f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 5s 5ms/step - loss: 1.1792\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.7320\n",
            "1/1 [==============================] - 0s 95ms/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.8881801],\n",
              "       [1.5947373],\n",
              "       [2.8808699]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 142
        }
      ],
      "source": [
        "# 추가 코드 - 모델을 로드하고 훈련을 계속하거나 이를 사용하여 예측을 수행할 수 있습니다.\n",
        "model = tf.keras.models.load_model(\"my_custom_model\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2)\n",
        "model.predict(X_test_scaled[:3])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ftCLc4maAxrm"
      },
      "source": [
        "대신 시퀀셜 API를 사용하여 모델을 정의할 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 143,
      "metadata": {
        "id": "g7DwZgzgAxrm"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)\n",
        "block1 = ResidualBlock(2, 30)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                          kernel_initializer=\"he_normal\"),\n",
        "    block1, block1, block1, block1,\n",
        "    ResidualBlock(2, 30),\n",
        "    tf.keras.layers.Dense(1)\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dtgyQ107Axrm"
      },
      "source": [
        "## 모델 구성 요소에 기반한 손실과 지표"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e4xJVV_hAxrm"
      },
      "source": [
        "**경고**: TF 2.2의 문제([#46858](https://github.com/tensorflow/tensorflow/issues/46858))로 인해 `super().build()`가 실패합니다. 대신 `self.built = True`를 설정하여 이 문제를 해결할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 144,
      "metadata": {
        "id": "hw5LlKwGAxrm"
      },
      "outputs": [],
      "source": [
        "class ReconstructingRegressor(tf.keras.Model):\n",
        "    def __init__(self, output_dim, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden = [tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "                       for _ in range(5)]\n",
        "        self.out = tf.keras.layers.Dense(output_dim)\n",
        "        self.reconstruction_mean = tf.keras.metrics.Mean(\n",
        "            name=\"reconstruction_error\")\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        n_inputs = batch_input_shape[-1]\n",
        "        self.reconstruct = tf.keras.layers.Dense(n_inputs)\n",
        "        self.built = True  # super().build(batch_input_shape)에 대한 해결 방법\n",
        "\n",
        "    def call(self, inputs, training=None):\n",
        "        Z = inputs\n",
        "        for layer in self.hidden:\n",
        "            Z = layer(Z)\n",
        "        reconstruction = self.reconstruct(Z)\n",
        "        recon_loss = tf.reduce_mean(tf.square(reconstruction - inputs))\n",
        "        self.add_loss(0.05 * recon_loss)\n",
        "        if training:\n",
        "            result = self.reconstruction_mean(recon_loss)\n",
        "            self.add_metric(result)\n",
        "        return self.out(Z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 145,
      "metadata": {
        "id": "oVJF57b8Axrn",
        "outputId": "ab9e8c8c-b36d-47e8-a071-fad5643918e3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 6s 6ms/step - loss: 1.1287 - reconstruction_error: 1.5699\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4535 - reconstruction_error: 0.7050\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4225 - reconstruction_error: 0.5210\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.3715 - reconstruction_error: 0.4137\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.3517 - reconstruction_error: 0.3610\n",
            "162/162 [==============================] - 0s 2ms/step\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드\n",
        "tf.random.set_seed(42)\n",
        "model = ReconstructingRegressor(1)\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=5)\n",
        "y_pred = model.predict(X_test_scaled)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gP9ugK0OAxrn"
      },
      "source": [
        "## 자동 미분을 사용해 그레이디언트 계산하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 146,
      "metadata": {
        "id": "E0es0JCqAxrn"
      },
      "outputs": [],
      "source": [
        "def f(w1, w2):\n",
        "    return 3 * w1 ** 2 + 2 * w1 * w2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 147,
      "metadata": {
        "id": "UWAjkQuUAxrn",
        "outputId": "41987b1b-7d0a-424a-f2e6-6cd14b8fdcfb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "36.000003007075065"
            ]
          },
          "metadata": {},
          "execution_count": 147
        }
      ],
      "source": [
        "w1, w2 = 5, 3\n",
        "eps = 1e-6\n",
        "(f(w1 + eps, w2) - f(w1, w2)) / eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 148,
      "metadata": {
        "id": "lj0wtEByAxrn",
        "outputId": "317ea90d-d9d7-4450-8280-17eba5f3e862",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10.000000003174137"
            ]
          },
          "metadata": {},
          "execution_count": 148
        }
      ],
      "source": [
        "(f(w1, w2 + eps) - f(w1, w2)) / eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 149,
      "metadata": {
        "id": "-ByM5o2uAxrn"
      },
      "outputs": [],
      "source": [
        "w1, w2 = tf.Variable(5.), tf.Variable(3.)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "gradients = tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 150,
      "metadata": {
        "id": "CnXg-CRNAxrn",
        "outputId": "bb0a7776-7ab8-4c4d-a73f-bbee641afbdc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 150
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 151,
      "metadata": {
        "id": "3u9cxDqiAxrn",
        "outputId": "e2bbcac0-8f5e-4b2d-9621-f48a3da1046e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "A non-persistent GradientTape can only be used to compute one set of gradients (or jacobians)\n"
          ]
        }
      ],
      "source": [
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "dz_dw1 = tape.gradient(z, w1)  # 텐서 36.0를 반환합니다.\n",
        "try:\n",
        "    dz_dw2 = tape.gradient(z, w2)  # RuntimeError가 발생합니다!\n",
        "except RuntimeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 152,
      "metadata": {
        "id": "7ZnWfZUGAxrn"
      },
      "outputs": [],
      "source": [
        "with tf.GradientTape(persistent=True) as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "dz_dw1 = tape.gradient(z, w1)  # 텐서 36.0를 반환합니다.\n",
        "dz_dw2 = tape.gradient(z, w2)  # 텐서 10.0을 반환하고 이제 정상적으로 작동합니다!\n",
        "del tape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 153,
      "metadata": {
        "id": "pGo4P-XXAxrn",
        "outputId": "2aab53bf-8e92-454d-80a4-912b05570a0a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>)"
            ]
          },
          "metadata": {},
          "execution_count": 153
        }
      ],
      "source": [
        "dz_dw1, dz_dw2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 154,
      "metadata": {
        "id": "6miw3KASAxrn"
      },
      "outputs": [],
      "source": [
        "c1, c2 = tf.constant(5.), tf.constant(3.)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(c1, c2)\n",
        "\n",
        "gradients = tape.gradient(z, [c1, c2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 155,
      "metadata": {
        "id": "0ecoqYDZAxrn",
        "outputId": "1b9462e5-2de2-435a-f8e5-027c33f3b230",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[None, None]"
            ]
          },
          "metadata": {},
          "execution_count": 155
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 156,
      "metadata": {
        "id": "d1W-u8IvAxro"
      },
      "outputs": [],
      "source": [
        "with tf.GradientTape() as tape:\n",
        "    tape.watch(c1)\n",
        "    tape.watch(c2)\n",
        "    z = f(c1, c2)\n",
        "\n",
        "gradients = tape.gradient(z, [c1, c2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 157,
      "metadata": {
        "id": "I3vK-4zvAxro",
        "outputId": "7f357ab6-9d9e-4bd7-e138-132cc10e209c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 157
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 158,
      "metadata": {
        "id": "mzqVAdkIAxro",
        "outputId": "93747eb3-7fe7-445c-e7f1-2abd03658d75",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=136.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=30.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 158
        }
      ],
      "source": [
        "# 추가 코드 - 벡터가 주어지면 tape.gradient()를 사용해 벡터의 합의 그레이디언트을 계산합니다.\n",
        "with tf.GradientTape() as tape:\n",
        "    z1 = f(w1, w2 + 2.)\n",
        "    z2 = f(w1, w2 + 5.)\n",
        "    z3 = f(w1, w2 + 7.)\n",
        "\n",
        "tape.gradient([z1, z2, z3], [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 159,
      "metadata": {
        "id": "7MhsExJNAxro",
        "outputId": "95227409-aff3-4298-ed80-600ce09de9b6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=136.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=30.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 159
        }
      ],
      "source": [
        "# 추가 코드 - 이전 셀과 동일한 결과를 얻었음을 보여줍니다.\n",
        "with tf.GradientTape() as tape:\n",
        "    z1 = f(w1, w2 + 2.)\n",
        "    z2 = f(w1, w2 + 5.)\n",
        "    z3 = f(w1, w2 + 7.)\n",
        "    z = z1 + z2 + z3\n",
        "\n",
        "tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 160,
      "metadata": {
        "id": "tkA4n1OJAxro"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 야코비안과 헤시안을 계산하는 방법을 보여줍니다.\n",
        "with tf.GradientTape(persistent=True) as hessian_tape:\n",
        "    with tf.GradientTape() as jacobian_tape:\n",
        "        z = f(w1, w2)\n",
        "    jacobians = jacobian_tape.gradient(z, [w1, w2])\n",
        "hessians = [hessian_tape.gradient(jacobian, [w1, w2])\n",
        "            for jacobian in jacobians]\n",
        "del hessian_tape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 161,
      "metadata": {
        "id": "N4BFsV00Axro",
        "outputId": "27a37eb3-6146-4984-c5e8-aca22e784d5e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 161
        }
      ],
      "source": [
        "jacobians"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 162,
      "metadata": {
        "id": "1YXKHV9kAxro",
        "outputId": "15ce456f-c149-438a-c1f0-073793700747",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[<tf.Tensor: shape=(), dtype=float32, numpy=6.0>,\n",
              "  <tf.Tensor: shape=(), dtype=float32, numpy=2.0>],\n",
              " [<tf.Tensor: shape=(), dtype=float32, numpy=2.0>, None]]"
            ]
          },
          "metadata": {},
          "execution_count": 162
        }
      ],
      "source": [
        "hessians"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 163,
      "metadata": {
        "id": "nT-SMLBDAxro"
      },
      "outputs": [],
      "source": [
        "def f(w1, w2):\n",
        "    return 3 * w1 ** 2 + tf.stop_gradient(2 * w1 * w2)\n",
        "\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)  # stop_gradient()를 사용하지 않은 경우와 동일한 결과\n",
        "\n",
        "gradients = tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 164,
      "metadata": {
        "id": "Aq8qy9puAxro",
        "outputId": "649b3092-0258-4701-a128-b1a9dcfe9915",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=30.0>, None]"
            ]
          },
          "metadata": {},
          "execution_count": 164
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 165,
      "metadata": {
        "id": "fI4KbdJBAxro",
        "outputId": "187f1829-e7f5-4709-fcbd-3e348a5fa3d9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=inf>]"
            ]
          },
          "metadata": {},
          "execution_count": 165
        }
      ],
      "source": [
        "x = tf.Variable(1e-50)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = tf.sqrt(x)\n",
        "\n",
        "tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 166,
      "metadata": {
        "id": "ISzNiEoOAxro",
        "outputId": "1b575946-7cbf-4516-a049-8d0b13c7a37e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=30.0>"
            ]
          },
          "metadata": {},
          "execution_count": 166
        }
      ],
      "source": [
        "tf.math.log(tf.exp(tf.constant(30., dtype=tf.float32)) + 1.)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 167,
      "metadata": {
        "id": "qhjNdoqlAxrp",
        "outputId": "6c600b3b-fafe-473b-dd52-21fde42cdec1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(1,), dtype=float32, numpy=array([nan], dtype=float32)>]"
            ]
          },
          "metadata": {},
          "execution_count": 167
        }
      ],
      "source": [
        "x = tf.Variable([1.0e30])\n",
        "with tf.GradientTape() as tape:\n",
        "    z = my_softplus(x)\n",
        "\n",
        "tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 168,
      "metadata": {
        "id": "EX9a8h0EAxrp"
      },
      "outputs": [],
      "source": [
        "def my_softplus(z):\n",
        "    return tf.math.log(1 + tf.exp(-tf.abs(z))) + tf.maximum(0., z)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rzYK3EMAAxrp"
      },
      "source": [
        "다음은 이 방정식이 log(1 + exp(_z_))와 같다는 것을 증명하는 예입니다:\n",
        "* softplus(_z_) = log(1 + exp(_z_))\n",
        "* softplus(_z_) = log(1 + exp(_z_)) - log(exp(_z_)) + log(exp(_z_)) ; **같은 값을 더하고 뺍니다**\n",
        "* softplus(_z_) = log\\[(1 + exp(_z_)) / exp(_z_)\\] + log(exp(_z_)) ; **since log(_a_) - log(_b_) = log(_a_ / _b_)**\n",
        "* softplus(_z_) = log\\[(1 + exp(_z_)) / exp(_z_)\\] + _z_ ; **since log(exp(_z_)) = _z_**\n",
        "* softplus(_z_) = log\\[1 / exp(_z_) + exp(_z_) / exp(_z_)\\] + _z_ ; **since (1 + _a_) / _b_ = 1 / _b_ + _a_ / _b_**\n",
        "* softplus(_z_) = log\\[exp(–_z_) + 1\\] + _z_ ; **1 / exp(_z_) = exp(–z)이고 exp(_z_) / exp(_z_) = 1 이므로**\n",
        "* softplus(_z_) = softplus(–_z_) + _z_ ; **맨 위의 정의를 사용합니다. 다만 –_z_ 입니다**\n",
        "* softplus(_z_) = softplus(–|_z_|) + max(0, _z_) ; **_z_ < 0 나 _z_ ≥ 0 인 경우를 고려하면**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 169,
      "metadata": {
        "id": "xDkt0wV5Axrp"
      },
      "outputs": [],
      "source": [
        "@tf.custom_gradient\n",
        "def my_softplus(z):\n",
        "    def my_softplus_gradients(grads):  # grads = 상위 층에서 역전파된 그레이디언트\n",
        "        return grads * (1 - 1 / (1 + tf.exp(z)))  # 안정적인 소프트플러스 그레이디언트\n",
        "\n",
        "    result = tf.math.log(1 + tf.exp(-tf.abs(z))) + tf.maximum(0., z)\n",
        "    return result, my_softplus_gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 170,
      "metadata": {
        "id": "HX9qyqsrAxrp",
        "outputId": "b7c10d45-4059-4dce-be5c-13e5cb8b1eeb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(1,), dtype=float32, numpy=array([1000.], dtype=float32)>,\n",
              " [<tf.Tensor: shape=(1,), dtype=float32, numpy=array([1.], dtype=float32)>])"
            ]
          },
          "metadata": {},
          "execution_count": 170
        }
      ],
      "source": [
        "# 추가 코드 - 함수와 이 함수의 그레이디언트가 안정적임을 보여줍니다.\n",
        "x = tf.Variable([1000.])\n",
        "with tf.GradientTape() as tape:\n",
        "    z = my_softplus(x)\n",
        "\n",
        "z, tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lAS7G_AdAxrp"
      },
      "source": [
        "## 사용자 정의 훈련"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 171,
      "metadata": {
        "id": "ANc8FNOXAxrp"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)  # 추가 코드 - 재현성 보장\n",
        "l2_reg = tf.keras.regularizers.l2(0.05)\n",
        "model = tf.keras.models.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          kernel_regularizer=l2_reg),\n",
        "    tf.keras.layers.Dense(1, kernel_regularizer=l2_reg)\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 172,
      "metadata": {
        "id": "_zmZ0EA9Axrp"
      },
      "outputs": [],
      "source": [
        "def random_batch(X, y, batch_size=32):\n",
        "    idx = np.random.randint(len(X), size=batch_size)\n",
        "    return X[idx], y[idx]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 173,
      "metadata": {
        "id": "tWl1XI8dAxrp"
      },
      "outputs": [],
      "source": [
        "def print_status_bar(step, total, loss, metrics=None):\n",
        "    metrics = \" - \".join([f\"{m.name}: {m.result():.4f}\"\n",
        "                          for m in [loss] + (metrics or [])])\n",
        "    end = \"\" if step < total else \"\\n\"\n",
        "    print(f\"\\r{step}/{total} - \" + metrics, end=end)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 174,
      "metadata": {
        "id": "ki9ew4DnAxrp"
      },
      "outputs": [],
      "source": [
        "np.random.seed(42)\n",
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 175,
      "metadata": {
        "id": "mR-EpV9IAxrp"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
        "loss_fn = tf.keras.losses.mean_squared_error\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.MeanAbsoluteError()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 176,
      "metadata": {
        "scrolled": true,
        "id": "gIAbJpB-Axrp",
        "outputId": "a26758ad-02c0-4d0f-c6e8-6c80c2a185d4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "362/362 - mean: 3.0069 - mean_absolute_error: 0.5982\n",
            "Epoch 2/5\n",
            "362/362 - mean: 1.6644 - mean_absolute_error: 0.5106\n",
            "Epoch 3/5\n",
            "362/362 - mean: 1.0784 - mean_absolute_error: 0.4921\n",
            "Epoch 4/5\n",
            "362/362 - mean: 0.8206 - mean_absolute_error: 0.4929\n",
            "Epoch 5/5\n",
            "362/362 - mean: 0.7142 - mean_absolute_error: 0.4979\n"
          ]
        }
      ],
      "source": [
        "for epoch in range(1, n_epochs + 1):\n",
        "    print(f\"Epoch {epoch}/{n_epochs}\")\n",
        "    for step in range(1, n_steps + 1):\n",
        "        X_batch, y_batch = random_batch(X_train_scaled, y_train)\n",
        "        with tf.GradientTape() as tape:\n",
        "            y_pred = model(X_batch, training=True)\n",
        "            main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "            loss = tf.add_n([main_loss] + model.losses)\n",
        "\n",
        "        gradients = tape.gradient(loss, model.trainable_variables)\n",
        "        optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "\n",
        "        # 추가 코드 - 모델 변수에 제약이 있는 경우\n",
        "        for variable in model.variables:\n",
        "            if variable.constraint is not None:\n",
        "                variable.assign(variable.constraint(variable))\n",
        "\n",
        "        mean_loss(loss)\n",
        "        for metric in metrics:\n",
        "            metric(y_batch, y_pred)\n",
        "\n",
        "        print_status_bar(step, n_steps, mean_loss, metrics)\n",
        "\n",
        "    for metric in [mean_loss] + metrics:\n",
        "        metric.reset_states()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 177,
      "metadata": {
        "id": "bsXR8eRhAxrq",
        "outputId": "22002a88-f0f0-43ee-c2a7-aacd1a1ae23d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "50f911c7adce4fb0a71fb22fce8c1530",
            "51f805bb4e5d44b4a3599caf24986c93",
            "725687c401de439a963b47188d940120",
            "40d755adc4664465b98826facf5f592c",
            "2cdda034442b43b9bb7a88f59ef229bb",
            "4725414ca5e64d8aaa93937703f32a74",
            "17a94b30d22b4dc68f204b243ad0d8ba",
            "3843bc9287a3467c86933cb46222301d",
            "55f99d6a4804474eba7f2c86db6b45a7",
            "9ad6122c91a342c2adafa8bc4c011d41",
            "2c2161e048d7452eb505ef5adbe9c967",
            "108bac51174e462bbf2eff6ffc567d42",
            "aae98f057c7f4aee8de18db7e2d478ed",
            "ff37e558c6554e9f89aa67a61e61305c",
            "f068d44442aa45cda0b7fda143179983",
            "dc5abaf1e7014919be2208d67dcd9f57",
            "db0b0cad0e53407eb9f9f33c8e8112c2",
            "c714959ef9194564b2a899c5ba859fa0",
            "9d30ba4dd41e47d6bc47005de3ceee51",
            "fcf02eb696cc42a7961cb5b7512fd3f1",
            "1343c3caecbb4bd2bf778d176e95ab07",
            "fdd103a3def44dd8b227477b4f4d00f1",
            "5aaf16ba926e4694864a91d58d44767f",
            "82ba9ce9f5cb4615ac520433e0820e5a",
            "5eb2de2b8fe24a72a830a838553e052e",
            "bb90d55fa36e41639c30ee1975b55442",
            "21f98a332efa4c2c88898648cd3fade4",
            "779777f1f2dd437fb87947186724c26f",
            "a81c7b081bf24822977971bd5a8a544f",
            "e9ea76ef42c04a6b93e6b249c4ecb8af",
            "3cd72b41e4cf40e0942797ee86b00cb9",
            "2f64673ccf364ea99d935fcf54b31ecd",
            "4d1125dcc9e0407b9e25914a84617a99",
            "ff573d3e0bbd4c5a9c95f4ef1643f6b3",
            "9d17f7b4f0164cd7944993b7da20f2b2",
            "c6c9406ba83f4a79a5100e12109a8e40",
            "aa7610377675451cb63971067dd08557",
            "10f9ea212fed4d8097ebe31375b5ca50",
            "713a3d65b5af4c9497da45486522b54b",
            "39bff00010e04734ac62e630d7c1f504",
            "ee8ae85a8446491c8678d5611afba486",
            "37fc2b3928a9448b9855d98bd77a1c9e",
            "034cefb19cd94f29959c75dbcc44d37a",
            "61edb78c77584b5791df43a2aae6babe",
            "2f7ef837faf0438e9e6bbe3a3e72406d",
            "091c93f5b90f48b3b68fc289184cbf78",
            "95df3c913d434c6cafd35ef7d548af81",
            "34a670611ad84d2387fd173be864bd46",
            "003fdb8957b34ea2a02410585c4ccdfc",
            "c2ddd16882d441afba5bbe843a4757cd",
            "2a90abffd4c34967b63f41f70a42bae5",
            "42d5de9229464700b5a2e5937b966bf3",
            "20f07dabefbf4fb79015c4ee1dfd0706",
            "536b38949b6744fc96db6b34a9caa204",
            "32ac84927da54192b94862fbd5a58e7e",
            "b1d41d7fa3d34742aedc75bea54a10f2",
            "05cd309292bb43749ed88b996700513d",
            "467f2cc2e28e46118d3820ae60807831",
            "c91757a32f82457b806abcedddd8ff47",
            "d5295ceab9ca4ab3bd22de5483920be7",
            "7a9654650b1b403d917f517fa7f02e24",
            "165a168dbe5e4080bf1c5c287451da2a",
            "3268a69c1ba34237bcca7cd886477e0a",
            "3fdbb02e66ad4858be62b08b9a6818f2",
            "e09d363111d3400aaa901338188ee23d",
            "f26674a577514e86ad881ae477fa0e75"
          ]
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "50f911c7adce4fb0a71fb22fce8c1530"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "108bac51174e462bbf2eff6ffc567d42"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "5aaf16ba926e4694864a91d58d44767f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ff573d3e0bbd4c5a9c95f4ef1643f6b3"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2f7ef837faf0438e9e6bbe3a3e72406d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b1d41d7fa3d34742aedc75bea54a10f2"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "# 추가 코드 - tqdm 패키지를 사용하여 멋진 진행률 표시줄을 표시하는 방법을 보여줍니다.\n",
        "\n",
        "from tqdm.notebook import trange\n",
        "from collections import OrderedDict\n",
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train_scaled, y_train)\n",
        "                with tf.GradientTape() as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "\n",
        "                gradients = tape.gradient(loss, model.trainable_variables)\n",
        "                optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))\n",
        "\n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "\n",
        "                steps.set_postfix(status)\n",
        "\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nufrN7ggAxrq"
      },
      "source": [
        "## 텐서플로 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 178,
      "metadata": {
        "id": "00O0nYkAAxrq"
      },
      "outputs": [],
      "source": [
        "def cube(x):\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 179,
      "metadata": {
        "id": "uEg23hqxAxrq",
        "outputId": "4808c245-8778-4d3a-8683-b9eeb1e4d273",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8"
            ]
          },
          "metadata": {},
          "execution_count": 179
        }
      ],
      "source": [
        "cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 180,
      "metadata": {
        "id": "lrj1qhFyAxrq",
        "outputId": "623df092-76e8-484d-96db-bbe7f96d7277",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 180
        }
      ],
      "source": [
        "cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 181,
      "metadata": {
        "id": "guBTABFmAxrq",
        "outputId": "3bd9ab38-1299-4341-bd95-08c2af29e626",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.eager.polymorphic_function.polymorphic_function.Function at 0x7f453e8634c0>"
            ]
          },
          "metadata": {},
          "execution_count": 181
        }
      ],
      "source": [
        "tf_cube = tf.function(cube)\n",
        "tf_cube"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 182,
      "metadata": {
        "id": "IfvIk4gFAxrq",
        "outputId": "f985c7f3-b8b0-44b1-dede-b73c2fc9589e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=8>"
            ]
          },
          "metadata": {},
          "execution_count": 182
        }
      ],
      "source": [
        "tf_cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 183,
      "metadata": {
        "id": "zFa3QVFYAxrq",
        "outputId": "33298385-9bac-424c-fa9b-4b676577ec00",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 183
        }
      ],
      "source": [
        "tf_cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 184,
      "metadata": {
        "id": "IWMZGSPEAxrq"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def tf_cube(x):\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GyTfJbwrAxrq"
      },
      "source": [
        "**노트:** 이 섹션의 나머지 코드는 부록 D의 내용입니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ksujIMzAxrr"
      },
      "source": [
        "### TF 함수와 콘크리트 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 185,
      "metadata": {
        "id": "ejIT2i2pAxrr",
        "outputId": "e1556eef-8d7e-4912-dc80-d5fe15d5f779",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<ConcreteFunction tf_cube(x) at 0x7F4540C9E0D0>"
            ]
          },
          "metadata": {},
          "execution_count": 185
        }
      ],
      "source": [
        "concrete_function = tf_cube.get_concrete_function(tf.constant(2.0))\n",
        "concrete_function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 186,
      "metadata": {
        "id": "fTl77eYAAxrr",
        "outputId": "dbcc67df-f393-4a3a-cac0-39e688eb2937",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 186
        }
      ],
      "source": [
        "concrete_function(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 187,
      "metadata": {
        "id": "pb0a8LkdAxrr",
        "outputId": "9c6d3ebf-7154-484b-948f-ad6a4bd44a22",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 187
        }
      ],
      "source": [
        "concrete_function is tf_cube.get_concrete_function(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iJH3KGaPAxrr"
      },
      "source": [
        "### 함수 정의와 그래프 탐험하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 188,
      "metadata": {
        "id": "EPPO52zAAxrr",
        "outputId": "a050a9e7-00da-417c-d257-96361f153dc8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.framework.func_graph.FuncGraph at 0x7f45404101f0>"
            ]
          },
          "metadata": {},
          "execution_count": 188
        }
      ],
      "source": [
        "concrete_function.graph"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 189,
      "metadata": {
        "id": "rRyMNm6ZAxrr",
        "outputId": "5d53401a-8b6f-4d79-be7b-c43251c30fc3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'pow/y' type=Const>,\n",
              " <tf.Operation 'pow' type=Pow>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 189
        }
      ],
      "source": [
        "ops = concrete_function.graph.get_operations()\n",
        "ops"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 190,
      "metadata": {
        "id": "wghSPcJuAxrr",
        "outputId": "69dacc49-f0d4-4c93-a353-49c7e6fb145a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor 'x:0' shape=() dtype=float32>,\n",
              " <tf.Tensor 'pow/y:0' shape=() dtype=float32>]"
            ]
          },
          "metadata": {},
          "execution_count": 190
        }
      ],
      "source": [
        "pow_op = ops[2]\n",
        "list(pow_op.inputs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 191,
      "metadata": {
        "id": "8CAcuouzAxrr",
        "outputId": "a5cfa59d-4e8d-42f2-aa2a-771619e7e4b3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor 'pow:0' shape=() dtype=float32>]"
            ]
          },
          "metadata": {},
          "execution_count": 191
        }
      ],
      "source": [
        "pow_op.outputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 192,
      "metadata": {
        "id": "ki6l6QlIAxrr",
        "outputId": "1623f807-d2f2-4556-c0d2-2fdb5bbfcf73",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Operation 'x' type=Placeholder>"
            ]
          },
          "metadata": {},
          "execution_count": 192
        }
      ],
      "source": [
        "concrete_function.graph.get_operation_by_name('x')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 193,
      "metadata": {
        "id": "LJe6V79wAxrr",
        "outputId": "2031d802-ef8d-4096-bbce-20ff5c6b5578",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor 'Identity:0' shape=() dtype=float32>"
            ]
          },
          "metadata": {},
          "execution_count": 193
        }
      ],
      "source": [
        "concrete_function.graph.get_tensor_by_name('Identity:0')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 194,
      "metadata": {
        "id": "eMau0rWJAxrr",
        "outputId": "3a03afee-b3a5-47fb-f83c-79c4a65bfada",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"__inference_tf_cube_519251\"\n",
              "input_arg {\n",
              "  name: \"x\"\n",
              "  type: DT_FLOAT\n",
              "}\n",
              "output_arg {\n",
              "  name: \"identity\"\n",
              "  type: DT_FLOAT\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 194
        }
      ],
      "source": [
        "concrete_function.function_def.signature"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m3YK9ebhAxrs"
      },
      "source": [
        "### TF 함수가 파이썬 함수를 트레이싱하여 계산 그래프를 추출하는 방법"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 195,
      "metadata": {
        "id": "aQ2EBGBOAxrs"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def tf_cube(x):\n",
        "    print(f\"x = {x}\")\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 196,
      "metadata": {
        "id": "yqAJdSL-Axrs",
        "outputId": "04e6df58-b158-4526-e0d9-3bcce0c4f023",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 197,
      "metadata": {
        "id": "rHNroERdAxrs",
        "outputId": "ef29087a-e456-4231-fe3a-e83da8f7159f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 197
        }
      ],
      "source": [
        "result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 198,
      "metadata": {
        "id": "Du-arMTdAxrs",
        "outputId": "4def561e-0b15-4dd6-e4d1-d25e60bffddd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = 2\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 199,
      "metadata": {
        "id": "-0FJXsF2Axrs",
        "outputId": "627b1949-a71a-42ff-b684-06943ef5f4f9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = 3\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 200,
      "metadata": {
        "id": "EQiq6YLJAxrs",
        "outputId": "807894e6-4d48-4715-cebe-0775a0860993",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(1, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant([[1., 2.]]))  # 새로운 크기: 트레이싱!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 201,
      "metadata": {
        "id": "J6Wec1VzAxrs",
        "outputId": "ae497deb-b6d9-47c2-ef7a-4c6c71f0cf14",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tensorflow:5 out of the last 5 calls to <function tf_cube at 0x7f4540ec09d0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(2, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant([[3., 4.], [5., 6.]]))  # 새로운 크기: 트레이싱!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 202,
      "metadata": {
        "id": "giQiBq6CAxrs"
      },
      "outputs": [],
      "source": [
        "result = tf_cube(tf.constant([[7., 8.], [9., 10.]]))  # 같은 크기: 트레이싱 없음"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5CIVAatSAxrs"
      },
      "source": [
        "It is also possible to specify a particular input signature:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 203,
      "metadata": {
        "id": "XK8U-vrNAxrs"
      },
      "outputs": [],
      "source": [
        "@tf.function(input_signature=[tf.TensorSpec([None, 28, 28], tf.float32)])\n",
        "def shrink(images):\n",
        "    print(\"Tracing\", images)  # 트레이싱을 출력하는 추가 코드\n",
        "    return images[:, ::2, ::2] # 행과 열의 절반을 삭제합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 204,
      "metadata": {
        "id": "TXOCIeVhAxrt"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 205,
      "metadata": {
        "id": "MBjUObyWAxrt",
        "outputId": "657b5765-f12b-4423-e818-eebc0f49d5f4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tracing Tensor(\"images:0\", shape=(None, 28, 28), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "img_batch_1 = tf.random.uniform(shape=[100, 28, 28])\n",
        "img_batch_2 = tf.random.uniform(shape=[50, 28, 28])\n",
        "preprocessed_images = shrink(img_batch_1)  # 정상 작동, 함수 트레이싱\n",
        "preprocessed_images = shrink(img_batch_2)  # 정상 작동, 동일한 콘크리트 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 206,
      "metadata": {
        "id": "ootcVC_nAxrt",
        "outputId": "e893b669-7993-4db8-8975-fb72f4934df3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Python inputs incompatible with input_signature:\n",
            "  inputs: (\n",
            "    tf.Tensor(\n",
            "[[[0.7413678  0.62854624]\n",
            "  [0.01738465 0.3431449 ]]\n",
            "\n",
            " [[0.51063764 0.3777541 ]\n",
            "  [0.07321596 0.02137029]]], shape=(2, 2, 2), dtype=float32))\n",
            "  input_signature: (\n",
            "    TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name=None)).\n"
          ]
        }
      ],
      "source": [
        "img_batch_3 = tf.random.uniform(shape=[2, 2, 2])\n",
        "try:\n",
        "    preprocessed_images = shrink(img_batch_3)  # ValueError! 호환되지 않는 입력\n",
        "except ValueError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4OjBRI8BAxrt"
      },
      "source": [
        "### Autograph를 사용해 제어 흐름 표현하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xi63dzBGAxrt"
      },
      "source": [
        "`range()`를 사용하는 \"정적\" `for` 루프입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 207,
      "metadata": {
        "id": "FgAkT4DFAxrt"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in range(10):\n",
        "        x += 1\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 208,
      "metadata": {
        "id": "0DaLQaE0Axrt",
        "outputId": "dd8d5fa3-467a-4a36-fbf6-730f10578bfe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=15>"
            ]
          },
          "metadata": {},
          "execution_count": 208
        }
      ],
      "source": [
        "add_10(tf.constant(5))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 209,
      "metadata": {
        "id": "y4CNvOlwAxrt",
        "outputId": "b1d998b9-4eac-407b-9f54-9094f1a3f0ca",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'add/y' type=Const>,\n",
              " <tf.Operation 'add' type=AddV2>,\n",
              " <tf.Operation 'add_1/y' type=Const>,\n",
              " <tf.Operation 'add_1' type=AddV2>,\n",
              " <tf.Operation 'add_2/y' type=Const>,\n",
              " <tf.Operation 'add_2' type=AddV2>,\n",
              " <tf.Operation 'add_3/y' type=Const>,\n",
              " <tf.Operation 'add_3' type=AddV2>,\n",
              " <tf.Operation 'add_4/y' type=Const>,\n",
              " <tf.Operation 'add_4' type=AddV2>,\n",
              " <tf.Operation 'add_5/y' type=Const>,\n",
              " <tf.Operation 'add_5' type=AddV2>,\n",
              " <tf.Operation 'add_6/y' type=Const>,\n",
              " <tf.Operation 'add_6' type=AddV2>,\n",
              " <tf.Operation 'add_7/y' type=Const>,\n",
              " <tf.Operation 'add_7' type=AddV2>,\n",
              " <tf.Operation 'add_8/y' type=Const>,\n",
              " <tf.Operation 'add_8' type=AddV2>,\n",
              " <tf.Operation 'add_9/y' type=Const>,\n",
              " <tf.Operation 'add_9' type=AddV2>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 209
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(5)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ih2sdaI1Axrt"
      },
      "source": [
        "`tf.while_loop()`를 사용하는 \"동적\" 루프입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 210,
      "metadata": {
        "id": "ZSytfbE7Axrv"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - tf.while_loop를 사용하는 방법을 보여줍니다(일반적으로 @tf.function이 더 간단합니다).\n",
        "@tf.function\n",
        "def add_10(x):\n",
        "    condition = lambda i, x: tf.less(i, 10)\n",
        "    body = lambda i, x: (tf.add(i, 1), tf.add(x, 1))\n",
        "    final_i, final_x = tf.while_loop(condition, body, [tf.constant(0), x])\n",
        "    return final_x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 211,
      "metadata": {
        "id": "rdfAbdJIAxrv",
        "outputId": "5ce332b8-48d6-4bfa-a42d-73d497ca1aa1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=15>"
            ]
          },
          "metadata": {},
          "execution_count": 211
        }
      ],
      "source": [
        "add_10(tf.constant(5))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 212,
      "metadata": {
        "id": "ofZlMzAsAxrv",
        "outputId": "49f26827-9391-4ad5-f451-308f5fcbe423",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'Const' type=Const>,\n",
              " <tf.Operation 'while/maximum_iterations' type=Const>,\n",
              " <tf.Operation 'while/loop_counter' type=Const>,\n",
              " <tf.Operation 'while' type=StatelessWhile>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 212
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(5)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "To_1E-ULAxrv"
      },
      "source": [
        "(autograph가 감지한) `tf.range()`를 사용하는 \"동적\" `for` 루프:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 213,
      "metadata": {
        "id": "Ik3ff2wUAxrv"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in tf.range(10):\n",
        "        x = x + 1\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 214,
      "metadata": {
        "id": "LQGPCnrRAxrw",
        "outputId": "328f49de-d94d-44dc-a094-78eb6e46671a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'range/start' type=Const>,\n",
              " <tf.Operation 'range/limit' type=Const>,\n",
              " <tf.Operation 'range/delta' type=Const>,\n",
              " <tf.Operation 'range' type=Range>,\n",
              " <tf.Operation 'sub' type=Sub>,\n",
              " <tf.Operation 'floordiv' type=FloorDiv>,\n",
              " <tf.Operation 'mod' type=FloorMod>,\n",
              " <tf.Operation 'zeros_like' type=Const>,\n",
              " <tf.Operation 'NotEqual' type=NotEqual>,\n",
              " <tf.Operation 'Cast' type=Cast>,\n",
              " <tf.Operation 'add' type=AddV2>,\n",
              " <tf.Operation 'zeros_like_1' type=Const>,\n",
              " <tf.Operation 'Maximum' type=Maximum>,\n",
              " <tf.Operation 'while/maximum_iterations' type=Const>,\n",
              " <tf.Operation 'while/loop_counter' type=Const>,\n",
              " <tf.Operation 'while' type=StatelessWhile>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 214
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(0)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h5cAtawIAxrw"
      },
      "source": [
        "### TF 함수에서 변수와 다른 리소스 다루기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 215,
      "metadata": {
        "id": "PRVaBEmGAxrw",
        "outputId": "d0c44c26-86b6-400e-9eb9-177e1c985a6b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 215
        }
      ],
      "source": [
        "counter = tf.Variable(0)\n",
        "\n",
        "@tf.function\n",
        "def increment(counter, c=1):\n",
        "    return counter.assign_add(c)\n",
        "\n",
        "increment(counter)  # counter는 이제 1과 같습니다.\n",
        "increment(counter)  # counter는 이제 2와 같습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 216,
      "metadata": {
        "id": "pMIyf8CjAxrw",
        "outputId": "024c723e-b5f7-45da-c2ca-1f7153e49bd2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"counter\"\n",
              "type: DT_RESOURCE"
            ]
          },
          "metadata": {},
          "execution_count": 216
        }
      ],
      "source": [
        "function_def = increment.get_concrete_function(counter).function_def\n",
        "function_def.signature.input_arg[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 217,
      "metadata": {
        "id": "9HG88LCFAxrw"
      },
      "outputs": [],
      "source": [
        "counter = tf.Variable(0)\n",
        "\n",
        "@tf.function\n",
        "def increment(c=1):\n",
        "    return counter.assign_add(c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 218,
      "metadata": {
        "id": "XPosV6MJAxrw",
        "outputId": "3fadfca8-7980-4205-820c-96801916933d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 218
        }
      ],
      "source": [
        "increment()\n",
        "increment()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 219,
      "metadata": {
        "id": "0i3CSugcAxrw",
        "outputId": "3ff5c0c0-4ab5-4288-c1f4-99a2f38006d9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"assignaddvariableop_resource\"\n",
              "type: DT_RESOURCE"
            ]
          },
          "metadata": {},
          "execution_count": 219
        }
      ],
      "source": [
        "function_def = increment.get_concrete_function().function_def\n",
        "function_def.signature.input_arg[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 220,
      "metadata": {
        "id": "LlXsSHykAxrw"
      },
      "outputs": [],
      "source": [
        "class Counter:\n",
        "    def __init__(self):\n",
        "        self.counter = tf.Variable(0)\n",
        "\n",
        "    @tf.function\n",
        "    def increment(self, c=1):\n",
        "        return self.counter.assign_add(c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 221,
      "metadata": {
        "id": "PmsokL4UAxrw",
        "outputId": "8ce775f5-d663-4df9-d4c2-f213100d2aa0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 221
        }
      ],
      "source": [
        "c = Counter()\n",
        "c.increment()\n",
        "c.increment()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 222,
      "metadata": {
        "scrolled": true,
        "id": "uKTfW75IAxrw",
        "outputId": "93d325c3-f954-4315-da59-ac05d69680dc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "def tf__add(x):\n",
            "    with ag__.FunctionScope('add_10', 'fscope', ag__.ConversionOptions(recursive=True, user_requested=True, optional_features=(), internal_convert_user_code=True)) as fscope:\n",
            "        do_return = False\n",
            "        retval_ = ag__.UndefinedReturnValue()\n",
            "\n",
            "        def get_state():\n",
            "            return (x,)\n",
            "\n",
            "        def set_state(vars_):\n",
            "            nonlocal x\n",
            "            (x,) = vars_\n",
            "\n",
            "        def loop_body(itr):\n",
            "            nonlocal x\n",
            "            i = itr\n",
            "            x = ag__.ld(x)\n",
            "            x += 1\n",
            "        i = ag__.Undefined('i')\n",
            "        ag__.for_stmt(ag__.converted_call(ag__.ld(tf).range, (10,), None, fscope), None, loop_body, get_state, set_state, ('x',), {'iterate_names': 'i'})\n",
            "        try:\n",
            "            do_return = True\n",
            "            retval_ = ag__.ld(x)\n",
            "        except:\n",
            "            do_return = False\n",
            "            raise\n",
            "        return fscope.ret(retval_, do_return)\n",
            "\n"
          ]
        }
      ],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in tf.range(10):\n",
        "        x += 1\n",
        "    return x\n",
        "\n",
        "print(tf.autograph.to_code(add_10.python_function))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 223,
      "metadata": {
        "id": "W_I7ORibAxrx"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 구문 강조 표시로 autograph 코드를 표시하는 방법을 보여줍니다.\n",
        "def display_tf_code(func):\n",
        "    from IPython.display import display, Markdown\n",
        "    if hasattr(func, \"python_function\"):\n",
        "        func = func.python_function\n",
        "    code = tf.autograph.to_code(func)\n",
        "    display(Markdown(f'```python\\n{code}\\n```'))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 224,
      "metadata": {
        "id": "iFIBmaSWAxrx",
        "outputId": "af60c962-8456-43ac-fdab-3443219fe0c5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 488
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "```python\ndef tf__add(x):\n    with ag__.FunctionScope('add_10', 'fscope', ag__.ConversionOptions(recursive=True, user_requested=True, optional_features=(), internal_convert_user_code=True)) as fscope:\n        do_return = False\n        retval_ = ag__.UndefinedReturnValue()\n\n        def get_state():\n            return (x,)\n\n        def set_state(vars_):\n            nonlocal x\n            (x,) = vars_\n\n        def loop_body(itr):\n            nonlocal x\n            i = itr\n            x = ag__.ld(x)\n            x += 1\n        i = ag__.Undefined('i')\n        ag__.for_stmt(ag__.converted_call(ag__.ld(tf).range, (10,), None, fscope), None, loop_body, get_state, set_state, ('x',), {'iterate_names': 'i'})\n        try:\n            do_return = True\n            retval_ = ag__.ld(x)\n        except:\n            do_return = False\n            raise\n        return fscope.ret(retval_, do_return)\n\n```"
          },
          "metadata": {}
        }
      ],
      "source": [
        "display_tf_code(add_10)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PB0Sa4q9Axrx"
      },
      "source": [
        "## 케라스에 TF 함수 사용하기(또는 사용하지 않기)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SpjPf0nJAxrx"
      },
      "source": [
        "기본적으로 tf.keras는 사용자 지정 코드를 TF 함수로 자동 변환하므로 `tf.function()`을 사용할 필요가 없습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 225,
      "metadata": {
        "id": "7-1v1OwYAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 손실 함수\n",
        "def my_mse(y_true, y_pred):\n",
        "    print(\"손실 my_mse() 트레이싱\")\n",
        "    return tf.reduce_mean(tf.square(y_pred - y_true))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 226,
      "metadata": {
        "id": "p1skutQgAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 지표 함수\n",
        "def my_mae(y_true, y_pred):\n",
        "    print(\"지표 my_mae() 트레이싱\")\n",
        "    return tf.reduce_mean(tf.abs(y_pred - y_true))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 227,
      "metadata": {
        "id": "PpTUWOw7Axrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 층\n",
        "class MyDense(tf.keras.layers.Layer):\n",
        "    def __init__(self, units, activation=None, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.units = units\n",
        "        self.activation = tf.keras.activations.get(activation)\n",
        "\n",
        "    def build(self, input_shape):\n",
        "        self.kernel = self.add_weight(name='kernel', \n",
        "                                      shape=(input_shape[1], self.units),\n",
        "                                      initializer='uniform',\n",
        "                                      trainable=True)\n",
        "        self.biases = self.add_weight(name='bias', \n",
        "                                      shape=(self.units,),\n",
        "                                      initializer='zeros',\n",
        "                                      trainable=True)\n",
        "        super().build(input_shape)\n",
        "\n",
        "    def call(self, X):\n",
        "        print(\"MyDense.call() 트레이싱\")\n",
        "        return self.activation(X @ self.kernel + self.biases)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 228,
      "metadata": {
        "id": "YxycmeS9Axrx"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 229,
      "metadata": {
        "id": "PS8DpbmLAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 모델\n",
        "class MyModel(tf.keras.Model):\n",
        "    def __init__(self, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden1 = MyDense(30, activation=\"relu\")\n",
        "        self.hidden2 = MyDense(30, activation=\"relu\")\n",
        "        self.output_ = MyDense(1)\n",
        "\n",
        "    def call(self, input):\n",
        "        print(\"MyModel.call() 트레이싱\")\n",
        "        hidden1 = self.hidden1(input)\n",
        "        hidden2 = self.hidden2(hidden1)\n",
        "        concat = tf.keras.layers.concatenate([input, hidden2])\n",
        "        output = self.output_(concat)\n",
        "        return output\n",
        "\n",
        "model = MyModel()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 230,
      "metadata": {
        "id": "93rXOAQQAxrx"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 231,
      "metadata": {
        "id": "P5NZCM16Axry",
        "outputId": "1f8587ad-1428-48bd-be86-7d7298a342a4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "363/363 [==============================] - ETA: 0s - loss: 1.5066 - my_mae: 0.8456MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "363/363 [==============================] - 4s 4ms/step - loss: 1.5066 - my_mae: 0.8456 - val_loss: 0.4713 - val_my_mae: 0.4838\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.4442 - my_mae: 0.4783 - val_loss: 0.6293 - val_my_mae: 0.4699\n",
            "162/162 [==============================] - 0s 2ms/step - loss: 0.4209 - my_mae: 0.4687\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.4209076166152954, 0.4687177836894989]"
            ]
          },
          "metadata": {},
          "execution_count": 231
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-d4A6OrzAxry"
      },
      "source": [
        "이 기능은 `dynamic=True`로 모델을 생성하거나 모델 생성자에서 `super().__init__(dynamic=True, **kwargs)`를 호출하여 해제할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 232,
      "metadata": {
        "id": "31LkRTp5Axry"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 233,
      "metadata": {
        "id": "IAfSvsoIAxry"
      },
      "outputs": [],
      "source": [
        "model = MyModel(dynamic=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 234,
      "metadata": {
        "id": "OjqYAiKdAxry"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WC2KyPdXAxry"
      },
      "source": [
        "이제 사용자 정의 코드가 각 반복마다 호출됩니다. 너무 많은 출력을 피하기 위해 작은 데이터셋으로 검증과 평가를 수행해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 235,
      "metadata": {
        "id": "T8s4Ap8cAxry",
        "outputId": "67056948-aed6-4f44-b554-f2ebe209d445",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tensorflow:5 out of the last 14481 calls to <function _BaseOptimizer._update_step_xla at 0x7f453e002ee0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n",
            "WARNING:tensorflow:6 out of the last 14482 calls to <function _BaseOptimizer._update_step_xla at 0x7f453e002ee0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[5.633415222167969, 2.066986083984375]"
            ]
          },
          "metadata": {},
          "execution_count": 235
        }
      ],
      "source": [
        "model.fit(X_train_scaled[:64], y_train[:64], epochs=1,\n",
        "          validation_data=(X_valid_scaled[:64], y_valid[:64]), verbose=0)\n",
        "model.evaluate(X_test_scaled[:64], y_test[:64], verbose=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fahebWVbAxry"
      },
      "source": [
        "또는 `run_eagerly=True`로 모델을 컴파일할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 236,
      "metadata": {
        "id": "GQeEapv4Axry"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 237,
      "metadata": {
        "id": "UCIW1vLWAxry"
      },
      "outputs": [],
      "source": [
        "model = MyModel()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 238,
      "metadata": {
        "id": "UUKjwuBtAxry"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae], run_eagerly=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 239,
      "metadata": {
        "id": "5JxmvAB_Axrz",
        "outputId": "d02cf753-ae7c-4a92-a950-359f3cc438b4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[5.677860736846924, 2.0716867446899414]"
            ]
          },
          "metadata": {},
          "execution_count": 239
        }
      ],
      "source": [
        "model.fit(X_train_scaled[:64], y_train[:64], epochs=1,\n",
        "          validation_data=(X_valid_scaled[:64], y_valid[:64]), verbose=0)\n",
        "model.evaluate(X_test_scaled[:64], y_test[:64], verbose=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LpAV_M0WAxrz"
      },
      "source": [
        "## 추가 자료 - 사용자 정의 옵티마이저"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rDvHsV5UAxrz"
      },
      "source": [
        "사용자 정의 옵티마이저를 정의하는 것은 그리 일반적이지 않지만, 만약 여러분이 옵티마이저를 작성해야 한다면 다음 예시를 참고하세요:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 266,
      "metadata": {
        "id": "eg-Ls3HoAxrz"
      },
      "outputs": [],
      "source": [
        "class MyMomentumOptimizer(tf.keras.optimizers.Optimizer):\n",
        "    def __init__(self, learning_rate=0.001, momentum=0.0, name=\"MyMomentumOptimizer\", **kwargs):\n",
        "        super().__init__(name, **kwargs)\n",
        "        self._learning_rate = self._build_learning_rate(learning_rate)\n",
        "        self.momentum = momentum\n",
        "\n",
        "    def build(self, var_list):\n",
        "        \"\"\"옵티마이저 변수 초기화\n",
        "        매개변수:\n",
        "          var_list: 모델 변수 목록\n",
        "        \"\"\"\n",
        "        super().build(var_list)\n",
        "        if hasattr(self, \"_built\") and self._built:\n",
        "            return\n",
        "        self.momentums = []\n",
        "        for var in var_list:\n",
        "            self.momentums.append(\n",
        "                self.add_variable_from_reference(\n",
        "                    model_variable=var, variable_name=\"m\"\n",
        "                )\n",
        "            )\n",
        "        self._built = True\n",
        "\n",
        "    def update_step(self, gradient, variable):\n",
        "        \"\"\"주어진 그레이디언트와 관련된 모델 변수로 스텝을 업데이트합니다.\"\"\"\n",
        "        lr = tf.cast(self.learning_rate, variable.dtype)\n",
        "        m = None\n",
        "        var_key = self._var_key(variable)\n",
        "        momentum = tf.cast(self.momentum, variable.dtype)\n",
        "        m = self.momentums[self._index_dict[var_key]]\n",
        "\n",
        "        if isinstance(gradient, tf.IndexedSlices):\n",
        "            # 희소한 그레이디언트\n",
        "            add_value = tf.IndexedSlices(\n",
        "                -gradient.values * lr, gradient.indices\n",
        "            )\n",
        "            if m is not None:\n",
        "                m.assign(m * momentum)\n",
        "                m.scatter_add(add_value)\n",
        "                variable.assign_add(m)\n",
        "            else:\n",
        "                variable.scatter_add(add_value)\n",
        "        else:\n",
        "            # 밀집 그레이디언트\n",
        "            if m is not None:\n",
        "                m.assign(-gradient * lr + m * momentum)\n",
        "                variable.assign_add(m)\n",
        "            else:\n",
        "                variable.assign_add(-gradient * lr)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {\n",
        "            **base_config,\n",
        "            \"learning_rate\": self._serialize_hyperparameter(self._learning_rate),\n",
        "            \"momentum\": self._serialize_hyperparameter(self.momentum),\n",
        "        }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 267,
      "metadata": {
        "id": "_G2N34TzAxrz"
      },
      "outputs": [],
      "source": [
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 268,
      "metadata": {
        "id": "37SeM0eAAxrz",
        "outputId": "4207e954-6b31-4bc4-d561-c1ead209ab35",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 4.3086\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 1.4749\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.8609\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.7112\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.6669\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7f452c4bf190>"
            ]
          },
          "metadata": {},
          "execution_count": 268
        }
      ],
      "source": [
        "model = tf.keras.Sequential([tf.keras.layers.Dense(1, input_shape=[8])])\n",
        "model.compile(loss=\"mse\", optimizer=MyMomentumOptimizer(learning_rate=0.001))\n",
        "model.fit(X_train_scaled, y_train, epochs=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fVo-JwwwAxrz"
      },
      "source": [
        "# 연습문제 해답"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zcyVmMsAAxrz"
      },
      "source": [
        "## 1. to 11."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aviEodMhAxrz"
      },
      "source": [
        "부록 A 참조"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kAgNhFzmAxrz"
      },
      "source": [
        "## 12. _층 정규화_ 를 수행하는 사용자 정의 층을 구현하세요.\n",
        "\n",
        "_15장에서 순환 신경망을 사용할 때 이런 종류의 층을 사용합니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8FwneyKMAxr0"
      },
      "source": [
        "### a.\n",
        "_문제: `build()` 메서드에서 두 개의 훈련 가능한 가중치 *α*와 *β*를 정의합니다. 두 가중치 모두 크기가 `input_shape[-1:]`이고 데이터 타입은 `tf.float32`입니다. *α*는 1로 초기화되고 *β*는 0으로 초기화되어야 합니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sspawhAfAxr0"
      },
      "source": [
        "솔루션: 아래 참조."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gValdr8aAxr0"
      },
      "source": [
        "### b.\n",
        "_문제: `call()` 메서드는 샘플의 특성마다 평균 μ와 표준편차 σ를 계산해야 합니다. 이를 위해 전체 샘플의 평균 μ와 분산 σ<sup>2</sup>을 반환하는 `tf.nn.moments(inputs, axes=-1, keepdims=True)`을 사용할 수 있습니다(분산의 제곱근으로 표준편차를 계산합니다). 그다음 *α*⊗(*X* - μ)/(σ + ε) + *β*를 계산하여 반환합니다. 여기에서 ⊗는 원소별\n",
        "곱셈(`*`)을 나타냅니다. ε은 안전을 위한 항입니다(0으로 나누어지는 것을 막기 위한 작은 상수. 예를 들면 0.001)._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 269,
      "metadata": {
        "id": "Zqsjv7PpAxr0"
      },
      "outputs": [],
      "source": [
        "class LayerNormalization(tf.keras.layers.Layer):\n",
        "    def __init__(self, eps=0.001, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.eps = eps\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        self.alpha = self.add_weight(\n",
        "            name=\"alpha\", shape=batch_input_shape[-1:],\n",
        "            initializer=\"ones\")\n",
        "        self.beta = self.add_weight(\n",
        "            name=\"beta\", shape=batch_input_shape[-1:],\n",
        "            initializer=\"zeros\")\n",
        "        super().build(batch_input_shape) # 반드시 끝에 와야 합니다\n",
        "\n",
        "    def call(self, X):\n",
        "        mean, variance = tf.nn.moments(X, axes=-1, keepdims=True)\n",
        "        return self.alpha * (X - mean) / (tf.sqrt(variance + self.eps)) + self.beta\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        return batch_input_shape\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"eps\": self.eps}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ozD6O4PdAxr0"
      },
      "source": [
        "_ε_ 하이퍼파라미터(`eps`)는 필수가 아닙니다. 또한 `tf.sqrt(variance) + self.eps` 보다 `tf.sqrt(variance + self.eps)`를 계산하는 것이 좋습니다. sqrt(z)의 도함수는 z=0에서 정의되지 않기 때문에 분산 벡터의 한 원소가 0에 가까우면 훈련이 이리저리 널뜁니다. 제곱근 안에 _ε_를 넣으면 이런 현상을 방지할 수 있습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7QH6eoWmAxr0"
      },
      "source": [
        "### c.\n",
        "_문제: 사용자 정의 층이 `tf.keras.layers.LayerNormalization` 층과 동일한(또는 거의 동일한) 출력을 만드는지 확인하세요._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m0jDFJFKAxr0"
      },
      "source": [
        "각 클래스의 객체를 만들고 데이터(예를 들면, 훈련 세트)를 적용해 보죠. 차이는 무시할 수 있는 수준입니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 270,
      "metadata": {
        "id": "clVKIuKYAxr0",
        "outputId": "89de40ef-1ebd-45d4-e977-00743e193ddd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=3.894674e-08>"
            ]
          },
          "metadata": {},
          "execution_count": 270
        }
      ],
      "source": [
        "X = X_train.astype(np.float32)\n",
        "\n",
        "custom_layer_norm = LayerNormalization()\n",
        "keras_layer_norm = tf.keras.layers.LayerNormalization()\n",
        "\n",
        "tf.reduce_mean(tf.keras.losses.mean_absolute_error(\n",
        "    keras_layer_norm(X), custom_layer_norm(X)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLcPCLA1Axr0"
      },
      "source": [
        "네 충분히 가깝네요. 조금 더 확실하게 알파와 베타를 완전히 랜덤하게 지정하고 다시 비교해 보죠:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 271,
      "metadata": {
        "id": "WSpbT-C4Axr0",
        "outputId": "31811197-a4e8-4a2a-9cb4-a8799355c949",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=1.7680229e-08>"
            ]
          },
          "metadata": {},
          "execution_count": 271
        }
      ],
      "source": [
        "random_alpha = np.random.rand(X.shape[-1])\n",
        "random_beta = np.random.rand(X.shape[-1])\n",
        "\n",
        "custom_layer_norm.set_weights([random_alpha, random_beta])\n",
        "keras_layer_norm.set_weights([random_alpha, random_beta])\n",
        "\n",
        "tf.reduce_mean(tf.keras.losses.mean_absolute_error(\n",
        "    keras_layer_norm(X), custom_layer_norm(X)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "etMAN21uAxr0"
      },
      "source": [
        "여전히 무시할 수 있는 수준입니다! 사용자 정의 층이 잘 동작합니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tczap3MOAxr0"
      },
      "source": [
        "## 13. 사용자 정의 훈련 반복을 사용해 패션 MNIST 데이터셋으로 모델을 훈련해보세요.\n",
        "\n",
        "_패션 MNIST 데이터셋은 10장에서 소개했습니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2tOj7JKKAxr1"
      },
      "source": [
        "### a.\n",
        "_문제: 에포크, 반복, 평균 훈련 손실, (반복마다 업데이트되는) 에포크의 평균 정확도는 물론 에포크 끝에서 검증 손실과 정확도를 출력하세요._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 272,
      "metadata": {
        "id": "HxmdPj7gAxr1",
        "outputId": "f0e6b1a6-5c35-4998-be58-fc4835bdfa4c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "29515/29515 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26421880/26421880 [==============================] - 1s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "5148/5148 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4422102/4422102 [==============================] - 1s 0us/step\n"
          ]
        }
      ],
      "source": [
        "(X_train_full, y_train_full), (X_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()\n",
        "X_train_full = X_train_full.astype(np.float32) / 255.\n",
        "X_valid, X_train = X_train_full[:5000], X_train_full[5000:]\n",
        "y_valid, y_train = y_train_full[:5000], y_train_full[5000:]\n",
        "X_test = X_test.astype(np.float32) / 255."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 273,
      "metadata": {
        "id": "gf78zFM7Axr1"
      },
      "outputs": [],
      "source": [
        "tf.keras.backend.clear_session()\n",
        "np.random.seed(42)\n",
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 274,
      "metadata": {
        "id": "iKpfDKCsAxr1"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
        "    tf.keras.layers.Dense(100, activation=\"relu\"),\n",
        "    tf.keras.layers.Dense(10, activation=\"softmax\"),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 275,
      "metadata": {
        "id": "3qJ_XgmaAxr1"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "optimizer = tf.keras.optimizers.Nadam(learning_rate=0.01)\n",
        "loss_fn = tf.keras.losses.sparse_categorical_crossentropy\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.SparseCategoricalAccuracy()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 276,
      "metadata": {
        "id": "pSytRZyVAxr1",
        "outputId": "d7db8c02-eeaa-406f-b568-d773da5d914b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "3096ab840037443b80a9818b7e4989a6",
            "d63c59d24dda429fa1333c4e23ca6344",
            "5f36734c480b433daec58e42ed1284e2",
            "5cec4d5adda04ee4a4a5fa5283ddc5c1",
            "573d23990e9640e49929ca9a746309cf",
            "d1de157e593f45339ac5c43527b8b504",
            "132b1568f5754699b4c3bc3c43082740",
            "2dd29e769ea54e9fbd04e14a98db8738",
            "1e1d8934f97442f7897a4c8367a13959",
            "8fc4e8f7bbc845d9ad1271fd18865e6f",
            "2ad51fc7b81b46d596586b2a1159ef72",
            "7958895ed36745aca7979f1e5798f6dc",
            "d7bf879a9b9b4430927b737817159dae",
            "cc0cf0b0c08c40558f920ebc0c809368",
            "8543dcca83834a1d82dcab8fabc0ae31",
            "5d1897b92ee04301949ac3a88084ff0f",
            "827a8b19a68f4dbfa852f33b86e12e33",
            "b14ebc0138c943ca80fbcf8b8ff5c2db",
            "cb9f3e2b1a8146c38ac7f477914bffbe",
            "41c3a097176f42389782b6b8164c4e85",
            "38b534f2ea4a461280f0dd82bde28671",
            "2aae78cc44894e3d99bb2f59ef018bf9",
            "f99a47b1cb6449e1a775ec7324521163",
            "a250b418d2f84246b57d0d438e67d21a",
            "8898fabf12c64f1ba3a92307a40e1190",
            "ab72b6a92fd6451699418c9a6f880cde",
            "71f6e47e8057441e9ead3dd98a321677",
            "48567e5a3477404b877b640a424f6b2a",
            "784a75cb5ff044debb6aebbe02be9601",
            "192abc61449043a88e3ea540903bacbe",
            "a394b3f2f6bf48cf9d0b5425a425f75c",
            "faf22d55735344878352c77991f334f7",
            "84d2668e088340598e8d4a0e4398b7d8",
            "b958209ef86f4cd59821713429bdddb7",
            "861e1cdc0e1e48ceb3598371744c1759",
            "5c148efbaa8c4c0ea8f4a8553be9c756",
            "ab23de79e6924777916b0aa023a736a6",
            "9282ef074f5d48a1a57cdf3b606f38ba",
            "3a27289f6d59406495862e320268a78c",
            "b5a99e9d4d0d49938d6e7adc3e2a45fe",
            "ac58f4655417430c991f40927201a72b",
            "01bbfd587cc6401b84115c202da49101",
            "3411443c7cdc4f88965a8f52bb2c4d2e",
            "ed58564ff7d846f7a5b76e689530aa90",
            "2ba530a5dc1f4e828af2d257c0052980",
            "4ccaafbce5e94f9989dd191c4a4ccfc6",
            "8b730bac117541c3bb5c0d9935817357",
            "4c910c7305764b1d95db16d44d66fc39",
            "a071653231944cd4a3b221079a3ee36e",
            "5a19a539337c417098d4a86f9b966218",
            "0c2c034d925b442398de3f2566b7b087",
            "4adb57eda2694c72aa0ab5c523fad02f",
            "7542909be713457f95964a5ea9475d7c",
            "8757851aa0fe4c2f88e5c2cd4d34876a",
            "59c912eb175b40b1ae64641267dffca9",
            "45fb703b2c6540bea49129c3401990f7",
            "b89a96f72a504b10b30e5f39657bbc9e",
            "dcc7fa32166b427282ff29134633816f",
            "0ba34c47cfc2423f9c1dbee1d5f5de40",
            "9fb75e798a0c434bb5be15813db9e1bd",
            "ab85381aaa26420f9fa15c7c11a3a60d",
            "71e03f5c68d6403eae1edb74b2ae931c",
            "9af5b5e9a37a4c16b7747323383b3f4b",
            "958c9e5a15814715a248e042d5728440",
            "887a665f4b694dba8ce9c8edb3a2fd92",
            "e093dbc96edc4335a8e34b2c2a01e59b"
          ]
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3096ab840037443b80a9818b7e4989a6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "7958895ed36745aca7979f1e5798f6dc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f99a47b1cb6449e1a775ec7324521163"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b958209ef86f4cd59821713429bdddb7"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2ba530a5dc1f4e828af2d257c0052980"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "45fb703b2c6540bea49129c3401990f7"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train, y_train)\n",
        "                with tf.GradientTape() as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "                gradients = tape.gradient(loss, model.trainable_variables)\n",
        "                optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))                    \n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "                steps.set_postfix(status)\n",
        "            y_pred = model(X_valid)\n",
        "            status[\"val_loss\"] = np.mean(loss_fn(y_valid, y_pred))\n",
        "            status[\"val_accuracy\"] = np.mean(tf.keras.metrics.sparse_categorical_accuracy(\n",
        "                tf.constant(y_valid, dtype=np.float32), y_pred))\n",
        "            steps.set_postfix(status)\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qt9pd6eJAxr1"
      },
      "source": [
        "### b.\n",
        "_문제: 상위 층과 하위 층에 학습률이 다른 옵티마이저를 따로 사용해보세요._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 277,
      "metadata": {
        "id": "7W2-nkwWAxr1"
      },
      "outputs": [],
      "source": [
        "tf.keras.backend.clear_session()\n",
        "np.random.seed(42)\n",
        "tf.random.set_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 278,
      "metadata": {
        "id": "ZESiHnZtAxr1"
      },
      "outputs": [],
      "source": [
        "lower_layers = tf.keras.Sequential([\n",
        "    tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
        "    tf.keras.layers.Dense(100, activation=\"relu\"),\n",
        "])\n",
        "upper_layers = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(10, activation=\"softmax\"),\n",
        "])\n",
        "model = tf.keras.Sequential([\n",
        "    lower_layers, upper_layers\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 279,
      "metadata": {
        "id": "lQZwhSFPAxr1"
      },
      "outputs": [],
      "source": [
        "lower_optimizer = tf.keras.optimizers.SGD(learning_rate=1e-4)\n",
        "upper_optimizer = tf.keras.optimizers.Nadam(learning_rate=1e-3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 280,
      "metadata": {
        "id": "rRadcAKJAxr1"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "loss_fn = tf.keras.losses.sparse_categorical_crossentropy\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.SparseCategoricalAccuracy()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 281,
      "metadata": {
        "id": "jHSdXV7QAxr1",
        "outputId": "c5bfa7c0-33e1-4770-b90b-e62224ceef03",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "cd3d2c8899c84879b33a993da9fb467a",
            "48b452f3b4d9437aa5c9ce621d69b739",
            "1a12bcd25ab741ae88a3600b4795a3ac",
            "3b5f5798509843a8b98f96e396a74a78",
            "be42c35fa2d749e387dccf534c6f094f",
            "d85bfc8e1f4d4d33bc36b76b01b033ec",
            "4adad9caf7354ae7a685e566fcd0d1c7",
            "91bb22cbd5974a168268f94bb68bb839",
            "3323ed68ee144f4d9c49ddccdfbb93f8",
            "09ef6a00de0b44aaba6ee703c6e8f4e5",
            "628d97777bb240da91045eeae00a1b06",
            "1f444bc7372848a8937f49e21b3da248",
            "2a6e62142bd944acbed1788fb53f4dfe",
            "5ed9470933094e58af6281ebcc24f52b",
            "e1b8c6affe694c9688ff8b97c2714491",
            "fdfea471f042440994434481dc86fe65",
            "8dc2273f63654bae940044944d4724a3",
            "7fee2f50f6ba4ea4b6dc437d0a08e1cc",
            "1b3434c93c6f4f4fbe839c30a2c13cb7",
            "f6afa40bd52d4dd99fd192fbfadd41fe",
            "accd2dd993f243fb9c0f1ff1225305d3",
            "f4ed30bb6d62431fa25ac815314e1d92",
            "c1d5bf1e03374b5cbd69c4e4f2fd0728",
            "fb9e642ad7fc4e0988dda7dbc6e8fa07",
            "99dc7a67c4e54a6790df953adcfac22b",
            "af5eb15ffb994017ba2a1cf31d936c81",
            "5d88c0bc61a74849bd7bb74be8185145",
            "648308e3b11f4c76b4d8610863cfba42",
            "f540280c456d40969cee66e8f90d55c1",
            "e4bbcb2276434905a303fe69229c001a",
            "2b8d60843e6c45829af9d0a882840cb8",
            "181e77dc3cd84f9d9d2b7656c6009a59",
            "89c3415e15b34eec91e79737cce6bb95",
            "c4f6a60e30774cb8a1c3e2caa24d86c1",
            "f346083be6cd43db98e08d1aa5fef1fe",
            "6cbe72fd36184bb090781680e8ebd309",
            "6383f49a150e4df4b6829aa2caccdaf6",
            "e3a007c6d8b8438fb1db909960fbb0da",
            "03501881d18042dcb902ffe847082320",
            "6f6d4a2791c84e359e7ef92c5cf7c097",
            "fda5f487c63046868ff6c3955a97e1da",
            "fd015d783bcc48f6a4940cea400b6f6a",
            "a3b8b73bb24a43ff805d39ed80f5a7c0",
            "138aa7b2064b4bc4abbd17e912da5279",
            "627e227f64e2459c8132d7b2ec5242d2",
            "6f4b637e493348f590c26a2d12ab19a5",
            "7c411fbde94149ddb5cbd2be62751866",
            "9a2747605a174cfd8690a9f271ee773d",
            "bc29e6fa14ab4c468b545d09f9f517cc",
            "d4aab3d4ea294b3c85acf41a3937f028",
            "fed3f26d64e74ef19b0b35cb7792b41a",
            "7a368d48b8d245b39c8e64a3e9fe7c4d",
            "143b0a7f495541beb047dc115638eeaa",
            "84ad3d39b6144f3d827a3f80dc7fa764",
            "f4f5e48271b84afab183b5696e1b8413",
            "9a857ecebbf04669b7f000ba02db184c",
            "e21da7addf4c4b3f8accccea18f55223",
            "2eeaee2864584a2db48bbad93b818d79",
            "c347f0ca44b147a88895da9d3a681577",
            "cdf9ab54c43d494ba116250174aa73d6",
            "d801de870b5f490da1dcd96a9c14e32b",
            "cfe54c4cab464a38bc7c1d2547728ba0",
            "78a0dba88d7b4aa385f1af0efa2799e4",
            "f78099f2db4f454cbbc4f31f319bf431",
            "2129d2bae2a44f3692781bc50bd03629",
            "d3670fdc10b6476084e5398c5bf03c42"
          ]
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cd3d2c8899c84879b33a993da9fb467a"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1f444bc7372848a8937f49e21b3da248"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c1d5bf1e03374b5cbd69c4e4f2fd0728"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c4f6a60e30774cb8a1c3e2caa24d86c1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "627e227f64e2459c8132d7b2ec5242d2"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9a857ecebbf04669b7f000ba02db184c"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train, y_train)\n",
        "                with tf.GradientTape(persistent=True) as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "                for layers, optimizer in ((lower_layers, lower_optimizer),\n",
        "                                          (upper_layers, upper_optimizer)):\n",
        "                    gradients = tape.gradient(loss, layers.trainable_variables)\n",
        "                    optimizer.apply_gradients(zip(gradients, layers.trainable_variables))\n",
        "                del tape\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))                    \n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "                steps.set_postfix(status)\n",
        "            y_pred = model(X_valid)\n",
        "            status[\"val_loss\"] = np.mean(loss_fn(y_valid, y_pred))\n",
        "            status[\"val_accuracy\"] = np.mean(tf.keras.metrics.sparse_categorical_accuracy(\n",
        "                tf.constant(y_valid, dtype=np.float32), y_pred))\n",
        "            steps.set_postfix(status)\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    },
    "colab": {
      "provenance": []
    },
    "accelerator": "GPU",
    "gpuClass": "standard",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "50f911c7adce4fb0a71fb22fce8c1530": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_51f805bb4e5d44b4a3599caf24986c93",
              "IPY_MODEL_725687c401de439a963b47188d940120",
              "IPY_MODEL_40d755adc4664465b98826facf5f592c"
            ],
            "layout": "IPY_MODEL_2cdda034442b43b9bb7a88f59ef229bb"
          }
        },
        "51f805bb4e5d44b4a3599caf24986c93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4725414ca5e64d8aaa93937703f32a74",
            "placeholder": "​",
            "style": "IPY_MODEL_17a94b30d22b4dc68f204b243ad0d8ba",
            "value": "All epochs: 100%"
          }
        },
        "725687c401de439a963b47188d940120": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3843bc9287a3467c86933cb46222301d",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_55f99d6a4804474eba7f2c86db6b45a7",
            "value": 5
          }
        },
        "40d755adc4664465b98826facf5f592c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9ad6122c91a342c2adafa8bc4c011d41",
            "placeholder": "​",
            "style": "IPY_MODEL_2c2161e048d7452eb505ef5adbe9c967",
            "value": " 5/5 [00:33&lt;00:00,  6.65s/it]"
          }
        },
        "2cdda034442b43b9bb7a88f59ef229bb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4725414ca5e64d8aaa93937703f32a74": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "17a94b30d22b4dc68f204b243ad0d8ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3843bc9287a3467c86933cb46222301d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "55f99d6a4804474eba7f2c86db6b45a7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9ad6122c91a342c2adafa8bc4c011d41": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2c2161e048d7452eb505ef5adbe9c967": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "108bac51174e462bbf2eff6ffc567d42": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_aae98f057c7f4aee8de18db7e2d478ed",
              "IPY_MODEL_ff37e558c6554e9f89aa67a61e61305c",
              "IPY_MODEL_f068d44442aa45cda0b7fda143179983"
            ],
            "layout": "IPY_MODEL_dc5abaf1e7014919be2208d67dcd9f57"
          }
        },
        "aae98f057c7f4aee8de18db7e2d478ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_db0b0cad0e53407eb9f9f33c8e8112c2",
            "placeholder": "​",
            "style": "IPY_MODEL_c714959ef9194564b2a899c5ba859fa0",
            "value": "Epoch 1/5: 100%"
          }
        },
        "ff37e558c6554e9f89aa67a61e61305c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9d30ba4dd41e47d6bc47005de3ceee51",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fcf02eb696cc42a7961cb5b7512fd3f1",
            "value": 362
          }
        },
        "f068d44442aa45cda0b7fda143179983": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1343c3caecbb4bd2bf778d176e95ab07",
            "placeholder": "​",
            "style": "IPY_MODEL_fdd103a3def44dd8b227477b4f4d00f1",
            "value": " 362/362 [00:06&lt;00:00, 57.42it/s, loss=0.671, mean_absolute_error=0.502]"
          }
        },
        "dc5abaf1e7014919be2208d67dcd9f57": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "db0b0cad0e53407eb9f9f33c8e8112c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c714959ef9194564b2a899c5ba859fa0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9d30ba4dd41e47d6bc47005de3ceee51": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fcf02eb696cc42a7961cb5b7512fd3f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1343c3caecbb4bd2bf778d176e95ab07": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fdd103a3def44dd8b227477b4f4d00f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5aaf16ba926e4694864a91d58d44767f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_82ba9ce9f5cb4615ac520433e0820e5a",
              "IPY_MODEL_5eb2de2b8fe24a72a830a838553e052e",
              "IPY_MODEL_bb90d55fa36e41639c30ee1975b55442"
            ],
            "layout": "IPY_MODEL_21f98a332efa4c2c88898648cd3fade4"
          }
        },
        "82ba9ce9f5cb4615ac520433e0820e5a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_779777f1f2dd437fb87947186724c26f",
            "placeholder": "​",
            "style": "IPY_MODEL_a81c7b081bf24822977971bd5a8a544f",
            "value": "Epoch 2/5: 100%"
          }
        },
        "5eb2de2b8fe24a72a830a838553e052e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e9ea76ef42c04a6b93e6b249c4ecb8af",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3cd72b41e4cf40e0942797ee86b00cb9",
            "value": 362
          }
        },
        "bb90d55fa36e41639c30ee1975b55442": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2f64673ccf364ea99d935fcf54b31ecd",
            "placeholder": "​",
            "style": "IPY_MODEL_4d1125dcc9e0407b9e25914a84617a99",
            "value": " 362/362 [00:06&lt;00:00, 58.78it/s, loss=0.631, mean_absolute_error=0.495]"
          }
        },
        "21f98a332efa4c2c88898648cd3fade4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "779777f1f2dd437fb87947186724c26f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a81c7b081bf24822977971bd5a8a544f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e9ea76ef42c04a6b93e6b249c4ecb8af": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3cd72b41e4cf40e0942797ee86b00cb9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2f64673ccf364ea99d935fcf54b31ecd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4d1125dcc9e0407b9e25914a84617a99": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ff573d3e0bbd4c5a9c95f4ef1643f6b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9d17f7b4f0164cd7944993b7da20f2b2",
              "IPY_MODEL_c6c9406ba83f4a79a5100e12109a8e40",
              "IPY_MODEL_aa7610377675451cb63971067dd08557"
            ],
            "layout": "IPY_MODEL_10f9ea212fed4d8097ebe31375b5ca50"
          }
        },
        "9d17f7b4f0164cd7944993b7da20f2b2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_713a3d65b5af4c9497da45486522b54b",
            "placeholder": "​",
            "style": "IPY_MODEL_39bff00010e04734ac62e630d7c1f504",
            "value": "Epoch 3/5: 100%"
          }
        },
        "c6c9406ba83f4a79a5100e12109a8e40": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ee8ae85a8446491c8678d5611afba486",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_37fc2b3928a9448b9855d98bd77a1c9e",
            "value": 362
          }
        },
        "aa7610377675451cb63971067dd08557": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_034cefb19cd94f29959c75dbcc44d37a",
            "placeholder": "​",
            "style": "IPY_MODEL_61edb78c77584b5791df43a2aae6babe",
            "value": " 362/362 [00:06&lt;00:00, 56.11it/s, loss=0.624, mean_absolute_error=0.497]"
          }
        },
        "10f9ea212fed4d8097ebe31375b5ca50": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "713a3d65b5af4c9497da45486522b54b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "39bff00010e04734ac62e630d7c1f504": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ee8ae85a8446491c8678d5611afba486": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "37fc2b3928a9448b9855d98bd77a1c9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "034cefb19cd94f29959c75dbcc44d37a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "61edb78c77584b5791df43a2aae6babe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2f7ef837faf0438e9e6bbe3a3e72406d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_091c93f5b90f48b3b68fc289184cbf78",
              "IPY_MODEL_95df3c913d434c6cafd35ef7d548af81",
              "IPY_MODEL_34a670611ad84d2387fd173be864bd46"
            ],
            "layout": "IPY_MODEL_003fdb8957b34ea2a02410585c4ccdfc"
          }
        },
        "091c93f5b90f48b3b68fc289184cbf78": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c2ddd16882d441afba5bbe843a4757cd",
            "placeholder": "​",
            "style": "IPY_MODEL_2a90abffd4c34967b63f41f70a42bae5",
            "value": "Epoch 4/5: 100%"
          }
        },
        "95df3c913d434c6cafd35ef7d548af81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_42d5de9229464700b5a2e5937b966bf3",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_20f07dabefbf4fb79015c4ee1dfd0706",
            "value": 362
          }
        },
        "34a670611ad84d2387fd173be864bd46": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_536b38949b6744fc96db6b34a9caa204",
            "placeholder": "​",
            "style": "IPY_MODEL_32ac84927da54192b94862fbd5a58e7e",
            "value": " 362/362 [00:07&lt;00:00, 56.94it/s, loss=0.604, mean_absolute_error=0.493]"
          }
        },
        "003fdb8957b34ea2a02410585c4ccdfc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c2ddd16882d441afba5bbe843a4757cd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2a90abffd4c34967b63f41f70a42bae5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "42d5de9229464700b5a2e5937b966bf3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "20f07dabefbf4fb79015c4ee1dfd0706": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "536b38949b6744fc96db6b34a9caa204": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "32ac84927da54192b94862fbd5a58e7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b1d41d7fa3d34742aedc75bea54a10f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_05cd309292bb43749ed88b996700513d",
              "IPY_MODEL_467f2cc2e28e46118d3820ae60807831",
              "IPY_MODEL_c91757a32f82457b806abcedddd8ff47"
            ],
            "layout": "IPY_MODEL_d5295ceab9ca4ab3bd22de5483920be7"
          }
        },
        "05cd309292bb43749ed88b996700513d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7a9654650b1b403d917f517fa7f02e24",
            "placeholder": "​",
            "style": "IPY_MODEL_165a168dbe5e4080bf1c5c287451da2a",
            "value": "Epoch 5/5: 100%"
          }
        },
        "467f2cc2e28e46118d3820ae60807831": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3268a69c1ba34237bcca7cd886477e0a",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3fdbb02e66ad4858be62b08b9a6818f2",
            "value": 362
          }
        },
        "c91757a32f82457b806abcedddd8ff47": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e09d363111d3400aaa901338188ee23d",
            "placeholder": "​",
            "style": "IPY_MODEL_f26674a577514e86ad881ae477fa0e75",
            "value": " 362/362 [00:06&lt;00:00, 59.07it/s, loss=0.626, mean_absolute_error=0.502]"
          }
        },
        "d5295ceab9ca4ab3bd22de5483920be7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7a9654650b1b403d917f517fa7f02e24": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "165a168dbe5e4080bf1c5c287451da2a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3268a69c1ba34237bcca7cd886477e0a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3fdbb02e66ad4858be62b08b9a6818f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "e09d363111d3400aaa901338188ee23d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f26674a577514e86ad881ae477fa0e75": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3096ab840037443b80a9818b7e4989a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d63c59d24dda429fa1333c4e23ca6344",
              "IPY_MODEL_5f36734c480b433daec58e42ed1284e2",
              "IPY_MODEL_5cec4d5adda04ee4a4a5fa5283ddc5c1"
            ],
            "layout": "IPY_MODEL_573d23990e9640e49929ca9a746309cf"
          }
        },
        "d63c59d24dda429fa1333c4e23ca6344": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d1de157e593f45339ac5c43527b8b504",
            "placeholder": "​",
            "style": "IPY_MODEL_132b1568f5754699b4c3bc3c43082740",
            "value": "All epochs: 100%"
          }
        },
        "5f36734c480b433daec58e42ed1284e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2dd29e769ea54e9fbd04e14a98db8738",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1e1d8934f97442f7897a4c8367a13959",
            "value": 5
          }
        },
        "5cec4d5adda04ee4a4a5fa5283ddc5c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8fc4e8f7bbc845d9ad1271fd18865e6f",
            "placeholder": "​",
            "style": "IPY_MODEL_2ad51fc7b81b46d596586b2a1159ef72",
            "value": " 5/5 [02:16&lt;00:00, 27.30s/it]"
          }
        },
        "573d23990e9640e49929ca9a746309cf": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d1de157e593f45339ac5c43527b8b504": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "132b1568f5754699b4c3bc3c43082740": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2dd29e769ea54e9fbd04e14a98db8738": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1e1d8934f97442f7897a4c8367a13959": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8fc4e8f7bbc845d9ad1271fd18865e6f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2ad51fc7b81b46d596586b2a1159ef72": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7958895ed36745aca7979f1e5798f6dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d7bf879a9b9b4430927b737817159dae",
              "IPY_MODEL_cc0cf0b0c08c40558f920ebc0c809368",
              "IPY_MODEL_8543dcca83834a1d82dcab8fabc0ae31"
            ],
            "layout": "IPY_MODEL_5d1897b92ee04301949ac3a88084ff0f"
          }
        },
        "d7bf879a9b9b4430927b737817159dae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_827a8b19a68f4dbfa852f33b86e12e33",
            "placeholder": "​",
            "style": "IPY_MODEL_b14ebc0138c943ca80fbcf8b8ff5c2db",
            "value": "Epoch 1/5: 100%"
          }
        },
        "cc0cf0b0c08c40558f920ebc0c809368": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cb9f3e2b1a8146c38ac7f477914bffbe",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_41c3a097176f42389782b6b8164c4e85",
            "value": 1718
          }
        },
        "8543dcca83834a1d82dcab8fabc0ae31": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_38b534f2ea4a461280f0dd82bde28671",
            "placeholder": "​",
            "style": "IPY_MODEL_2aae78cc44894e3d99bb2f59ef018bf9",
            "value": " 1718/1718 [00:27&lt;00:00, 50.81it/s, loss=0.5, sparse_categorical_accuracy=0.822]"
          }
        },
        "5d1897b92ee04301949ac3a88084ff0f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "827a8b19a68f4dbfa852f33b86e12e33": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b14ebc0138c943ca80fbcf8b8ff5c2db": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb9f3e2b1a8146c38ac7f477914bffbe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "41c3a097176f42389782b6b8164c4e85": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "38b534f2ea4a461280f0dd82bde28671": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2aae78cc44894e3d99bb2f59ef018bf9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f99a47b1cb6449e1a775ec7324521163": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a250b418d2f84246b57d0d438e67d21a",
              "IPY_MODEL_8898fabf12c64f1ba3a92307a40e1190",
              "IPY_MODEL_ab72b6a92fd6451699418c9a6f880cde"
            ],
            "layout": "IPY_MODEL_71f6e47e8057441e9ead3dd98a321677"
          }
        },
        "a250b418d2f84246b57d0d438e67d21a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_48567e5a3477404b877b640a424f6b2a",
            "placeholder": "​",
            "style": "IPY_MODEL_784a75cb5ff044debb6aebbe02be9601",
            "value": "Epoch 2/5: 100%"
          }
        },
        "8898fabf12c64f1ba3a92307a40e1190": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_192abc61449043a88e3ea540903bacbe",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a394b3f2f6bf48cf9d0b5425a425f75c",
            "value": 1718
          }
        },
        "ab72b6a92fd6451699418c9a6f880cde": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_faf22d55735344878352c77991f334f7",
            "placeholder": "​",
            "style": "IPY_MODEL_84d2668e088340598e8d4a0e4398b7d8",
            "value": " 1718/1718 [00:27&lt;00:00, 69.74it/s, loss=0.413, sparse_categorical_accuracy=0.852]"
          }
        },
        "71f6e47e8057441e9ead3dd98a321677": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "48567e5a3477404b877b640a424f6b2a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "784a75cb5ff044debb6aebbe02be9601": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "192abc61449043a88e3ea540903bacbe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a394b3f2f6bf48cf9d0b5425a425f75c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "faf22d55735344878352c77991f334f7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "84d2668e088340598e8d4a0e4398b7d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b958209ef86f4cd59821713429bdddb7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_861e1cdc0e1e48ceb3598371744c1759",
              "IPY_MODEL_5c148efbaa8c4c0ea8f4a8553be9c756",
              "IPY_MODEL_ab23de79e6924777916b0aa023a736a6"
            ],
            "layout": "IPY_MODEL_9282ef074f5d48a1a57cdf3b606f38ba"
          }
        },
        "861e1cdc0e1e48ceb3598371744c1759": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3a27289f6d59406495862e320268a78c",
            "placeholder": "​",
            "style": "IPY_MODEL_b5a99e9d4d0d49938d6e7adc3e2a45fe",
            "value": "Epoch 3/5: 100%"
          }
        },
        "5c148efbaa8c4c0ea8f4a8553be9c756": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ac58f4655417430c991f40927201a72b",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_01bbfd587cc6401b84115c202da49101",
            "value": 1718
          }
        },
        "ab23de79e6924777916b0aa023a736a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3411443c7cdc4f88965a8f52bb2c4d2e",
            "placeholder": "​",
            "style": "IPY_MODEL_ed58564ff7d846f7a5b76e689530aa90",
            "value": " 1718/1718 [00:26&lt;00:00, 67.57it/s, loss=0.385, sparse_categorical_accuracy=0.862]"
          }
        },
        "9282ef074f5d48a1a57cdf3b606f38ba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3a27289f6d59406495862e320268a78c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b5a99e9d4d0d49938d6e7adc3e2a45fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ac58f4655417430c991f40927201a72b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01bbfd587cc6401b84115c202da49101": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3411443c7cdc4f88965a8f52bb2c4d2e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ed58564ff7d846f7a5b76e689530aa90": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2ba530a5dc1f4e828af2d257c0052980": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_4ccaafbce5e94f9989dd191c4a4ccfc6",
              "IPY_MODEL_8b730bac117541c3bb5c0d9935817357",
              "IPY_MODEL_4c910c7305764b1d95db16d44d66fc39"
            ],
            "layout": "IPY_MODEL_a071653231944cd4a3b221079a3ee36e"
          }
        },
        "4ccaafbce5e94f9989dd191c4a4ccfc6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5a19a539337c417098d4a86f9b966218",
            "placeholder": "​",
            "style": "IPY_MODEL_0c2c034d925b442398de3f2566b7b087",
            "value": "Epoch 4/5: 100%"
          }
        },
        "8b730bac117541c3bb5c0d9935817357": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4adb57eda2694c72aa0ab5c523fad02f",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7542909be713457f95964a5ea9475d7c",
            "value": 1718
          }
        },
        "4c910c7305764b1d95db16d44d66fc39": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8757851aa0fe4c2f88e5c2cd4d34876a",
            "placeholder": "​",
            "style": "IPY_MODEL_59c912eb175b40b1ae64641267dffca9",
            "value": " 1718/1718 [00:27&lt;00:00, 66.63it/s, loss=0.378, sparse_categorical_accuracy=0.866]"
          }
        },
        "a071653231944cd4a3b221079a3ee36e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5a19a539337c417098d4a86f9b966218": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0c2c034d925b442398de3f2566b7b087": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4adb57eda2694c72aa0ab5c523fad02f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7542909be713457f95964a5ea9475d7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8757851aa0fe4c2f88e5c2cd4d34876a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "59c912eb175b40b1ae64641267dffca9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "45fb703b2c6540bea49129c3401990f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b89a96f72a504b10b30e5f39657bbc9e",
              "IPY_MODEL_dcc7fa32166b427282ff29134633816f",
              "IPY_MODEL_0ba34c47cfc2423f9c1dbee1d5f5de40"
            ],
            "layout": "IPY_MODEL_9fb75e798a0c434bb5be15813db9e1bd"
          }
        },
        "b89a96f72a504b10b30e5f39657bbc9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ab85381aaa26420f9fa15c7c11a3a60d",
            "placeholder": "​",
            "style": "IPY_MODEL_71e03f5c68d6403eae1edb74b2ae931c",
            "value": "Epoch 5/5: 100%"
          }
        },
        "dcc7fa32166b427282ff29134633816f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9af5b5e9a37a4c16b7747323383b3f4b",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_958c9e5a15814715a248e042d5728440",
            "value": 1718
          }
        },
        "0ba34c47cfc2423f9c1dbee1d5f5de40": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_887a665f4b694dba8ce9c8edb3a2fd92",
            "placeholder": "​",
            "style": "IPY_MODEL_e093dbc96edc4335a8e34b2c2a01e59b",
            "value": " 1718/1718 [00:27&lt;00:00, 66.17it/s, loss=0.374, sparse_categorical_accuracy=0.867]"
          }
        },
        "9fb75e798a0c434bb5be15813db9e1bd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ab85381aaa26420f9fa15c7c11a3a60d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "71e03f5c68d6403eae1edb74b2ae931c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9af5b5e9a37a4c16b7747323383b3f4b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "958c9e5a15814715a248e042d5728440": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "887a665f4b694dba8ce9c8edb3a2fd92": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e093dbc96edc4335a8e34b2c2a01e59b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cd3d2c8899c84879b33a993da9fb467a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_48b452f3b4d9437aa5c9ce621d69b739",
              "IPY_MODEL_1a12bcd25ab741ae88a3600b4795a3ac",
              "IPY_MODEL_3b5f5798509843a8b98f96e396a74a78"
            ],
            "layout": "IPY_MODEL_be42c35fa2d749e387dccf534c6f094f"
          }
        },
        "48b452f3b4d9437aa5c9ce621d69b739": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d85bfc8e1f4d4d33bc36b76b01b033ec",
            "placeholder": "​",
            "style": "IPY_MODEL_4adad9caf7354ae7a685e566fcd0d1c7",
            "value": "All epochs: 100%"
          }
        },
        "1a12bcd25ab741ae88a3600b4795a3ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_91bb22cbd5974a168268f94bb68bb839",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3323ed68ee144f4d9c49ddccdfbb93f8",
            "value": 5
          }
        },
        "3b5f5798509843a8b98f96e396a74a78": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_09ef6a00de0b44aaba6ee703c6e8f4e5",
            "placeholder": "​",
            "style": "IPY_MODEL_628d97777bb240da91045eeae00a1b06",
            "value": " 5/5 [02:43&lt;00:00, 32.60s/it]"
          }
        },
        "be42c35fa2d749e387dccf534c6f094f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d85bfc8e1f4d4d33bc36b76b01b033ec": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4adad9caf7354ae7a685e566fcd0d1c7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "91bb22cbd5974a168268f94bb68bb839": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3323ed68ee144f4d9c49ddccdfbb93f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "09ef6a00de0b44aaba6ee703c6e8f4e5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "628d97777bb240da91045eeae00a1b06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1f444bc7372848a8937f49e21b3da248": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2a6e62142bd944acbed1788fb53f4dfe",
              "IPY_MODEL_5ed9470933094e58af6281ebcc24f52b",
              "IPY_MODEL_e1b8c6affe694c9688ff8b97c2714491"
            ],
            "layout": "IPY_MODEL_fdfea471f042440994434481dc86fe65"
          }
        },
        "2a6e62142bd944acbed1788fb53f4dfe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8dc2273f63654bae940044944d4724a3",
            "placeholder": "​",
            "style": "IPY_MODEL_7fee2f50f6ba4ea4b6dc437d0a08e1cc",
            "value": "Epoch 1/5: 100%"
          }
        },
        "5ed9470933094e58af6281ebcc24f52b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1b3434c93c6f4f4fbe839c30a2c13cb7",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f6afa40bd52d4dd99fd192fbfadd41fe",
            "value": 1718
          }
        },
        "e1b8c6affe694c9688ff8b97c2714491": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_accd2dd993f243fb9c0f1ff1225305d3",
            "placeholder": "​",
            "style": "IPY_MODEL_f4ed30bb6d62431fa25ac815314e1d92",
            "value": " 1718/1718 [00:33&lt;00:00, 57.12it/s, loss=1.06, sparse_categorical_accuracy=0.675]"
          }
        },
        "fdfea471f042440994434481dc86fe65": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8dc2273f63654bae940044944d4724a3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7fee2f50f6ba4ea4b6dc437d0a08e1cc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1b3434c93c6f4f4fbe839c30a2c13cb7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f6afa40bd52d4dd99fd192fbfadd41fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "accd2dd993f243fb9c0f1ff1225305d3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f4ed30bb6d62431fa25ac815314e1d92": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c1d5bf1e03374b5cbd69c4e4f2fd0728": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fb9e642ad7fc4e0988dda7dbc6e8fa07",
              "IPY_MODEL_99dc7a67c4e54a6790df953adcfac22b",
              "IPY_MODEL_af5eb15ffb994017ba2a1cf31d936c81"
            ],
            "layout": "IPY_MODEL_5d88c0bc61a74849bd7bb74be8185145"
          }
        },
        "fb9e642ad7fc4e0988dda7dbc6e8fa07": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_648308e3b11f4c76b4d8610863cfba42",
            "placeholder": "​",
            "style": "IPY_MODEL_f540280c456d40969cee66e8f90d55c1",
            "value": "Epoch 2/5: 100%"
          }
        },
        "99dc7a67c4e54a6790df953adcfac22b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e4bbcb2276434905a303fe69229c001a",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2b8d60843e6c45829af9d0a882840cb8",
            "value": 1718
          }
        },
        "af5eb15ffb994017ba2a1cf31d936c81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_181e77dc3cd84f9d9d2b7656c6009a59",
            "placeholder": "​",
            "style": "IPY_MODEL_89c3415e15b34eec91e79737cce6bb95",
            "value": " 1718/1718 [00:33&lt;00:00, 56.88it/s, loss=0.645, sparse_categorical_accuracy=0.783]"
          }
        },
        "5d88c0bc61a74849bd7bb74be8185145": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "648308e3b11f4c76b4d8610863cfba42": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f540280c456d40969cee66e8f90d55c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e4bbcb2276434905a303fe69229c001a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b8d60843e6c45829af9d0a882840cb8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "181e77dc3cd84f9d9d2b7656c6009a59": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "89c3415e15b34eec91e79737cce6bb95": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c4f6a60e30774cb8a1c3e2caa24d86c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f346083be6cd43db98e08d1aa5fef1fe",
              "IPY_MODEL_6cbe72fd36184bb090781680e8ebd309",
              "IPY_MODEL_6383f49a150e4df4b6829aa2caccdaf6"
            ],
            "layout": "IPY_MODEL_e3a007c6d8b8438fb1db909960fbb0da"
          }
        },
        "f346083be6cd43db98e08d1aa5fef1fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_03501881d18042dcb902ffe847082320",
            "placeholder": "​",
            "style": "IPY_MODEL_6f6d4a2791c84e359e7ef92c5cf7c097",
            "value": "Epoch 3/5: 100%"
          }
        },
        "6cbe72fd36184bb090781680e8ebd309": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fda5f487c63046868ff6c3955a97e1da",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fd015d783bcc48f6a4940cea400b6f6a",
            "value": 1718
          }
        },
        "6383f49a150e4df4b6829aa2caccdaf6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a3b8b73bb24a43ff805d39ed80f5a7c0",
            "placeholder": "​",
            "style": "IPY_MODEL_138aa7b2064b4bc4abbd17e912da5279",
            "value": " 1718/1718 [00:31&lt;00:00, 48.61it/s, loss=0.58, sparse_categorical_accuracy=0.8]"
          }
        },
        "e3a007c6d8b8438fb1db909960fbb0da": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "03501881d18042dcb902ffe847082320": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6f6d4a2791c84e359e7ef92c5cf7c097": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fda5f487c63046868ff6c3955a97e1da": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fd015d783bcc48f6a4940cea400b6f6a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a3b8b73bb24a43ff805d39ed80f5a7c0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "138aa7b2064b4bc4abbd17e912da5279": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "627e227f64e2459c8132d7b2ec5242d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6f4b637e493348f590c26a2d12ab19a5",
              "IPY_MODEL_7c411fbde94149ddb5cbd2be62751866",
              "IPY_MODEL_9a2747605a174cfd8690a9f271ee773d"
            ],
            "layout": "IPY_MODEL_bc29e6fa14ab4c468b545d09f9f517cc"
          }
        },
        "6f4b637e493348f590c26a2d12ab19a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d4aab3d4ea294b3c85acf41a3937f028",
            "placeholder": "​",
            "style": "IPY_MODEL_fed3f26d64e74ef19b0b35cb7792b41a",
            "value": "Epoch 4/5: 100%"
          }
        },
        "7c411fbde94149ddb5cbd2be62751866": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7a368d48b8d245b39c8e64a3e9fe7c4d",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_143b0a7f495541beb047dc115638eeaa",
            "value": 1718
          }
        },
        "9a2747605a174cfd8690a9f271ee773d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_84ad3d39b6144f3d827a3f80dc7fa764",
            "placeholder": "​",
            "style": "IPY_MODEL_f4f5e48271b84afab183b5696e1b8413",
            "value": " 1718/1718 [00:31&lt;00:00, 59.45it/s, loss=0.539, sparse_categorical_accuracy=0.812]"
          }
        },
        "bc29e6fa14ab4c468b545d09f9f517cc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d4aab3d4ea294b3c85acf41a3937f028": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fed3f26d64e74ef19b0b35cb7792b41a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7a368d48b8d245b39c8e64a3e9fe7c4d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "143b0a7f495541beb047dc115638eeaa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "84ad3d39b6144f3d827a3f80dc7fa764": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f4f5e48271b84afab183b5696e1b8413": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9a857ecebbf04669b7f000ba02db184c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e21da7addf4c4b3f8accccea18f55223",
              "IPY_MODEL_2eeaee2864584a2db48bbad93b818d79",
              "IPY_MODEL_c347f0ca44b147a88895da9d3a681577"
            ],
            "layout": "IPY_MODEL_cdf9ab54c43d494ba116250174aa73d6"
          }
        },
        "e21da7addf4c4b3f8accccea18f55223": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d801de870b5f490da1dcd96a9c14e32b",
            "placeholder": "​",
            "style": "IPY_MODEL_cfe54c4cab464a38bc7c1d2547728ba0",
            "value": "Epoch 5/5: 100%"
          }
        },
        "2eeaee2864584a2db48bbad93b818d79": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_78a0dba88d7b4aa385f1af0efa2799e4",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f78099f2db4f454cbbc4f31f319bf431",
            "value": 1718
          }
        },
        "c347f0ca44b147a88895da9d3a681577": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2129d2bae2a44f3692781bc50bd03629",
            "placeholder": "​",
            "style": "IPY_MODEL_d3670fdc10b6476084e5398c5bf03c42",
            "value": " 1718/1718 [00:33&lt;00:00, 60.11it/s, loss=0.529, sparse_categorical_accuracy=0.813]"
          }
        },
        "cdf9ab54c43d494ba116250174aa73d6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d801de870b5f490da1dcd96a9c14e32b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cfe54c4cab464a38bc7c1d2547728ba0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "78a0dba88d7b4aa385f1af0efa2799e4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f78099f2db4f454cbbc4f31f319bf431": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2129d2bae2a44f3692781bc50bd03629": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d3670fdc10b6476084e5398c5bf03c42": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}