4.3. 自定义层.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "4.3. 自定义层.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/4.3.%20%E8%87%AA%E5%AE%9A%E4%B9%89%E5%B1%82.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F6t3z2NaIP0h",
        "colab_type": "text"
      },
      "source": [
        "##4.3. 自定义层\n",
        "&emsp;&emsp;深度学习的一个魅力在于神经网络中各式各样的层，例如全连接层和后面章节中将要介绍的卷积层、池化层与循环层。虽然keras提供了大量常用的层，但有时候我们依然希望自定义层。本节将介绍如何使用keras来自定义一个层，从而可以被重复调用。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6W1eBKqhT7Ah",
        "colab_type": "text"
      },
      "source": [
        "###4.3.1. 不含模型参数的自定义层\n",
        "&emsp;&emsp;我们先介绍如何定义一个不含模型参数的自定义层。这个层里不含模型参数。下面的CenteredLayer类通过继承Layer类自定义了一个将输入减掉均值后输出的层，并将层的计算定义在了call函数里。这个层里不含模型参数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2B8Oi0taUL_j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers,Sequential"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RhH5famjUi2P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uBLtFEemUlJ6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class CenteredLayer(layers.Layer):\n",
        "  def __init__(self):\n",
        "    super(CenteredLayer,self).__init__()\n",
        "  \n",
        "  def call(self,x):\n",
        "    return x-tf.reduce_mean(x)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Sf-Q329WICD",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;我们可以实例化这个层，然后做前向计算。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2l6rEG_xU5hQ",
        "colab_type": "code",
        "outputId": "0d489bad-73be-4204-d84e-d5f0fa91a2ca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "layer=CenteredLayer()\n",
        "layer(tf.constant([1,2,3,4,5]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: id=4, shape=(5,), dtype=int32, numpy=array([-2, -1,  0,  1,  2], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iSwzXBrGWM3d",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;我们也可以用它来构造更复杂的模型。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k6V4ybhZVRUp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "net=Sequential()\n",
        "net.add(layers.Dense(128))\n",
        "net.add(CenteredLayer())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WQVZfTsCWS7G",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;下面打印自定义层各个输出的均值。因为均值是浮点数，所以它的值是一个很接近0的数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X70EzPyFVdzM",
        "colab_type": "code",
        "outputId": "b5df3bf5-e6ae-45cd-b3f3-e9888c9b8d69",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "y=net(tf.random.uniform(shape=(4,8)))\n",
        "tf.reduce_mean(y).numpy()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "-3.259629e-09"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fTuGxTIxVgaB",
        "colab_type": "text"
      },
      "source": [
        "###4.3.2. 含模型参数的自定义层\n",
        "&emsp;&emsp;我们还可以自定义含模型参数的自定义层。其中的模型参数可以通过训练学出。我们尝试实现一个含权重参数和偏差参数的全连接层。它使用ReLU函数作为激活函数。其中in_units和units分别代表输入个数和输出个数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QZD_VXlnWgJ9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class MyDense(layers.Layer):\n",
        "  #units为该层的输出个数\n",
        "  def __init__(self,units):\n",
        "    super(MyDense,self).__init__()\n",
        "    self.units=units\n",
        "\n",
        "  def build(self,input_shape):\n",
        "    self.kernel=self.add_weight(name='kernel',shape=[int(input_shape[-1]),self.units])\n",
        "\n",
        "  def call(self,input):\n",
        "    return tf.matmul(input,self.kernel)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dirRGM7vamoi",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;下面，我们实例化MyDense类并访问它的模型参数。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m0zSIMOsZujJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mydense=MyDense(10)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EsDtuj7paFvz",
        "colab_type": "code",
        "outputId": "67854713-3163-4ebb-db38-3f3fd7a48824",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 252
        }
      },
      "source": [
        "mydense.trainable_weights"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'my_dense/kernel:0' shape=(256, 10) dtype=float32, numpy=\n",
              " array([[-0.06160554, -0.04592701,  0.03480589, ..., -0.10063919,\n",
              "         -0.08592829,  0.06521817],\n",
              "        [ 0.05079053,  0.00844572,  0.00859797, ..., -0.10493287,\n",
              "         -0.14006785, -0.08032777],\n",
              "        [ 0.06736386,  0.1280073 , -0.01667269, ..., -0.01657282,\n",
              "         -0.08774073,  0.14624679],\n",
              "        ...,\n",
              "        [-0.0043927 ,  0.01602679, -0.06134421, ...,  0.06288308,\n",
              "          0.01501806, -0.12082192],\n",
              "        [ 0.14401358, -0.13923575, -0.03092163, ..., -0.03969427,\n",
              "          0.08999053, -0.14942415],\n",
              "        [-0.12680429,  0.11857036, -0.04953749, ..., -0.12889509,\n",
              "         -0.01627132,  0.00323282]], dtype=float32)>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "urDfhXKtaXFm",
        "colab_type": "text"
      },
      "source": [
        "&emsp;&emsp;我们可以直接使用自定义层做前向计算"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zt8i9lFgbY1i",
        "colab_type": "code",
        "outputId": "c73e4200-86b5-4442-ee6e-5f19a584def2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "y=mydense(tf.random.normal(shape=(5,256)))\n",
        "y.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([Dimension(5), Dimension(10)])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p0H638NLbarU",
        "colab_type": "code",
        "outputId": "5fee7e54-e922-478c-8114-6177288e963f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 50
        }
      },
      "source": [
        "net=Sequential()\n",
        "net.add(MyDense(8))\n",
        "net.add(MyDense(1))\n",
        "for layer in net.layers:\n",
        "  print(layer.name)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "my_dense_1\n",
            "my_dense_2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "puDFvWNbbz6Q",
        "colab_type": "code",
        "outputId": "a722fd94-36a7-4c53-ee8c-ce986385be4e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 67
        }
      },
      "source": [
        "net(tf.random.uniform(shape=(2,64)))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: id=124, shape=(2, 1), dtype=float32, numpy=\n",
              "array([[ 0.00493592],\n",
              "       [-0.56078476]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0rAmhTi7cB8u",
        "colab_type": "text"
      },
      "source": [
        "###4.3.3. 小结\n",
        "* 可以通过Layer类自定义神经网络中的层，从而可以被重复调用"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nVdFhlLM4f2N",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}