5.3. 多输入通道和多输出通道.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "5.3. 多输入通道和多输出通道.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chongzicbo/Dive-into-Deep-Learning-tf.keras/blob/master/5.3.%20%E5%A4%9A%E8%BE%93%E5%85%A5%E9%80%9A%E9%81%93%E5%92%8C%E5%A4%9A%E8%BE%93%E5%87%BA%E9%80%9A%E9%81%93.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UXRn-OQYeeYx",
        "colab_type": "text"
      },
      "source": [
        "##5.3. 多输入通道和多输出通道\n",
        "前面两节里我们用到的输入和输出都是二维数组，但真实数据的维度经常更高。例如，彩色图像在高和宽2个维度外还有RGB（红、绿、蓝）3个颜色通道。假设彩色图像的高和宽分别是$h$和$w$（像素），那么它可以表示为一个$3 \\times h \\times w$的多维数组。我们将大小为3的这一维称为通道（channel）维。本节我们将介绍含多个输入通道或多个输出通道的卷积核。\n",
        "\n",
        "###5.3.1. 多输入通道\n",
        "当输入数据含多个通道时，我们需要构造一个输入通道数与输入数据的通道数相同的卷积核，从而能够与含多通道的输入数据做互相关运算。假设输入数据的通道数为$c_i$，那么卷积核的输入通道数同样为$c_i$。设卷积核窗口形状为$k_h \\times k_w$。当$c_i=1$时，我们知道卷积核只包含一个形状为$k_h×k_w$的二维数组。当$c_i>1$时，我们将会为每个输入通道各分配一个形状为$k_h\\times k_w$的核数组。把这$c_i$个数组在输入通道维上连结，即得到一个形状为$c_i \\times k_h \\times k_w$的卷积核。由于输入和卷积核各有$c_i$个通道，我们可以在各个通道上对输入的二维数组和卷积核的二维核数组做互相关运算，再将这$c_i$个互相关运算的二维输出按通道相加，得到一个二维数组。这就是含多个通道的输入数据与多输入通道的卷积核做二维互相关运算的输出。\n",
        "\n",
        "图5.4展示了含2个输入通道的二维互相关计算的例子。在每个通道上，二维输入数组与二维核数组做互相关运算，再按通道相加即得到输出。图5.4中阴影部分为第一个输出元素及其计算所使用的输入和核数组元素：\n",
        "$$(1\\times1+2\\times2+4\\times3+5\\times4)+(0\\times0+1\\times1+3\\times2+4\\times3)=56$$\n",
        "<center><img src=\"https://zh.gluon.ai/_images/conv_multi_in.svg\" width=\"300\"/></center>\n",
        "<center>图 5.4 含2个输入通道的互相关计算</center>\n",
        "\n",
        "接下来我们实现含多个输入通道的互相关运算。我们只需要对每个通道做互相关运算，然后通过add_n函数来进行累加。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v1PGCdEra2aa",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 64
        },
        "outputId": "7521e56a-6778-4cac-b9c8-e19ea66d08dd"
      },
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<p style=\"color: red;\">\n",
              "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
              "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
              "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
              "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zViZXN9Ia-1J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "tf.enable_eager_execution() #启用动态图计算，tf2.x不需要"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TS8mXhCKaaRo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def corr2d(X,K):\n",
        "  h,w=tf.shape(K).numpy()\n",
        "  Y=tf.Variable(tf.zeros((tf.shape(X)[0].numpy()-h+1,tf.shape(X)[1].numpy()-w+1)))\n",
        "  for i in range(tf.shape(Y)[0]):\n",
        "    for j in range(tf.shape(Y)[1]):\n",
        "      tf.assign(Y[i,j],tf.reduce_sum(X[i:i+h,j:j+w]*K))\n",
        "  return Y"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rJ3oNdsYbCfR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def corr2d_multi_in(X,K):\n",
        "  # 首先沿着X和K的第0维（通道维）遍历。然后使用*将结果列表变成add_n函数的位置参数\n",
        "  # （positional argument）来进行相加\n",
        "  return tf.add_n([corr2d(x,k) for x,k in zip(X,K)])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iYVa9BO3gd9J",
        "colab_type": "text"
      },
      "source": [
        "我们可以构造图5.4中的输入数组X、核数组K来验证互相关运算的输出。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VZB6wPJhbXNI",
        "colab_type": "code",
        "outputId": "79b772b9-5c7a-4b9f-aa85-29b88eb1eb4d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        }
      },
      "source": [
        "X=tf.constant([[[0, 1, 2], [3, 4, 5], [6, 7, 8]],\n",
        "              [[1, 2, 3], [4, 5, 6], [7, 8, 9]]],dtype=tf.float32)\n",
        "K=tf.constant([[[0, 1], [2, 3]], [[1, 2], [3, 4]]],dtype=tf.float32)\n",
        "corr2d_multi_in(X,K)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: id=193, shape=(2, 2), dtype=float32, numpy=\n",
              "array([[ 56.,  72.],\n",
              "       [104., 120.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cMlPVEn_gg8T",
        "colab_type": "text"
      },
      "source": [
        "###5.3.2. 多输出通道\n",
        "当输入通道有多个时，因为我们对各个通道的结果做了累加，所以不论输入通道数是多少，输出通道数总是为1。设卷积核输入通道数和输出通道数分别为 $c_i$ 和 $c_o$ ，高和宽分别为 $k_h$ 和 $k_w$ 。如果希望得到含多个通道的输出，我们可以为每个输出通道分别创建形状为 $c_i \\times k_h \\times k_w$ 的核数组。将它们在输出通道维上连结，卷积核的形状即 $c_o \\times c_i \\times k_h \\times k_w$ 。在做互相关运算时，每个输出通道上的结果由卷积核在该输出通道上的核数组与整个输入数组计算而来。\n",
        "\n",
        "下面我们实现一个互相关运算函数来计算多个通道的输出。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9EOri-MEcmfT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def corr2d_multi_in_out(X,K):\n",
        "  # 对K的第0维遍历，每次同输入X做互相关计算。所有结果使用stack函数合并在一起\n",
        "  return tf.stack([corr2d_multi_in(X,k) for k in K])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aGW2UJ-MhHbw",
        "colab_type": "text"
      },
      "source": [
        "我们将核数组K同K+1（K中每个元素加一）和K+2连结在一起来构造一个输出通道数为3的卷积核。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UYC36_SOeiJ2",
        "colab_type": "code",
        "outputId": "97e8cfca-2fe5-4986-e7a7-7d9ea6d5a56e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "K=tf.stack(values=[K,K+1,K+2])\n",
        "K.shape"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([Dimension(3), Dimension(2), Dimension(2), Dimension(2)])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sZzSJdeyhOh1",
        "colab_type": "text"
      },
      "source": [
        "下面我们对输入数组$X$与核数组$K$做互相关运算。此时的输出含有3个通道。其中第一个通道的结果与之前输入数组$X$与多输入通道、单输出通道核的计算结果一致。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xhucG8Aoes5g",
        "colab_type": "code",
        "outputId": "a746258b-f448-44bf-bc94-fa8fe7d7a41b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 179
        }
      },
      "source": [
        "corr2d_multi_in_out(X,K)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: id=784, shape=(3, 2, 2), dtype=float32, numpy=\n",
              "array([[[ 56.,  72.],\n",
              "        [104., 120.]],\n",
              "\n",
              "       [[ 76., 100.],\n",
              "        [148., 172.]],\n",
              "\n",
              "       [[ 96., 128.],\n",
              "        [192., 224.]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j_5Qt_1khZZQ",
        "colab_type": "text"
      },
      "source": [
        "###5.3.3.  1×1 卷积层\n",
        "最后我们讨论卷积窗口形状为 $1 \\times 1 （ k_h=k_w=1 ）$的多通道卷积层。我们通常称之为 $1\\times1$ 卷积层，并将其中的卷积运算称为 $1\\times1$ 卷积。因为使用了最小窗口， $1\\times1$ 卷积失去了卷积层可以识别高和宽维度上相邻元素构成的模式的功能。实际上， $1\\times1$ 卷积的主要计算发生在通道维上。图5.5展示了使用输入通道数为3、输出通道数为2的 $1\\times1$ 卷积核的互相关计算。值得注意的是，输入和输出具有相同的高和宽。输出中的每个元素来自输入中在高和宽上相同位置的元素在不同通道之间的按权重累加。假设我们将通道维当作特征维，将高和宽维度上的元素当成数据样本，那么 $1\\times1$ 卷积层的作用与全连接层等价。\n",
        "<center><img src=\"https://zh.gluon.ai/_images/conv_1x1.svg\" width=\"300\"/></center>\n",
        "<center>图 5.5 使用输入通道数为3、输出通道数为2的 1×1 卷积核的互相关计算。输入和输出具有相同的高和宽</center>\n",
        "\n",
        "下面我们使用全连接层中的矩阵乘法来实现 $1\\times1$ 卷积。这里需要在矩阵乘法运算前后对数据形状做一些调整。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1M-c-heNevgI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def corr2d_multi_in_out_1x1(X,K):\n",
        "  c_i,h,w=X.shape\n",
        "  c_o=K.shape[0]\n",
        "  X=tf.reshape(X,(c_i,h*w))\n",
        "  K=tf.reshape(K,(c_o,c_i))\n",
        "  Y=tf.matmul(K,X) #全连接层的矩阵乘法\n",
        "  return tf.reshape(Y,(c_o,h,w))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vt9QrUuUiaXR",
        "colab_type": "text"
      },
      "source": [
        "经验证，做 $1\\times1$ 卷积时，以上函数与之前实现的互相关运算函数corr2d_multi_in_out等价。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "es5-CiWM4-XO",
        "colab_type": "code",
        "outputId": "7a20edb5-5504-4ca6-ea8f-6c15ba53a973",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "X=tf.random.uniform(shape=(3,3,3))\n",
        "K=tf.random.uniform(shape=(2,3,1,1))\n",
        "Y1=corr2d_multi_in_out_1x1(X,K)\n",
        "Y2=corr2d_multi_in_out(X,K)\n",
        "tf.norm((Y1-Y2)).numpy()"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ef_8P-955VzF",
        "colab_type": "text"
      },
      "source": [
        "在之后的模型里我们将会看到 $1 \\times 1$ 卷积层被当作保持高和宽维度形状不变的全连接层使用。于是，我们可以通过调整网络层之间的通道数来控制模型复杂度。\n",
        "\n",
        "5.3.4. 小结\n",
        "* 使用多通道可以拓展卷积层的模型参数。\n",
        "* 假设将通道维当作特征维，将高和宽维度上的元素当成数据样本，那么 $1 \\times 1$ 卷积层的作用与全连接层等价。\n",
        "* $1 \\times 1$ 卷积层通常用来调整网络层之间的通道数，并控制模型复杂度。\n"
      ]
    }
  ]
}