introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-lst-format.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Image classification training with image format demo\n",
    "\n",
    "1. [Introduction](#Introduction)\n",
    "2. [Prerequisites and Preprocessing](#Prequisites-and-Preprocessing)\n",
    "  1. [Permissions and environment variables](#Permissions-and-environment-variables)\n",
    "  2. [Prepare the data](#Prepare-the-data)\n",
    "3. [Fine-tuning The Image Classification Model](#Fine-tuning-the-Image-classification-model)\n",
    "  1. [Training parameters](#Training-parameters)\n",
    "  2. [Training](#Training)\n",
    "4. [Set Up Hosting For The Model](#Set-up-hosting-for-the-model)\n",
    "  1. [Create model](#Create-model)\n",
    "  2. [Create endpoint configuration](#Create-endpoint-configuration)\n",
    "  3. [Create endpoint](#Create-endpoint)\n",
    "  4. [Perform inference](#Perform-inference)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "Welcome to our end-to-end example of the image classification algorithm training with image format. In this demo, we will use the Amazon sagemaker image classification algorithm in transfer learning mode to fine-tune a pre-trained model (trained on imagenet data) to learn to classify a new dataset. In particular, the pre-trained model will be fine-tuned using [caltech-256 dataset](http://www.vision.caltech.edu/Image_Datasets/Caltech256/). \n",
    "\n",
    "To get started, we need to set up the environment with a few prerequisite steps, for permissions, configurations, and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prequisites and Preprocessing\n",
    "\n",
    "### Permissions and environment variables\n",
    "\n",
    "Here we set up the linkage and authentication to AWS services. There are three parts to this:\n",
    "\n",
    "* The roles used to give learning and hosting access to your data. This will automatically be obtained from the role used to start the notebook\n",
    "* The S3 bucket that you want to use for training and model data\n",
    "* The Amazon sagemaker image classification docker image which need not be changed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import boto3\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "role = get_execution_role()\n",
    "\n",
    "bucket='<<bucket-name>>' # customize to your bucket\n",
    "containers = {'us-west-2': '433757028032.dkr.ecr.us-west-2.amazonaws.com/image-classification:latest',\n",
    "              'us-east-1': '811284229777.dkr.ecr.us-east-1.amazonaws.com/image-classification:latest',\n",
    "              'us-east-2': '825641698319.dkr.ecr.us-east-2.amazonaws.com/image-classification:latest',\n",
    "              'eu-west-1': '685385470294.dkr.ecr.eu-west-1.amazonaws.com/image-classification:latest'}\n",
    "training_image = containers[boto3.Session().region_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fine-tuning the Image classification model\n",
    "\n",
    "### Prepare the data\n",
    "The caltech 256 dataset consist of images from 257 categories (the last one being a clutter category) and has 30k images with a minimum of 80 images and a maximum of about 800 images per category. \n",
    "\n",
    "The image classification algorithm can take two types of input formats. The first is a [RecordIO format](https://mxnet.incubator.apache.org/tutorials/basic/record_io.html) (content type: application/x-recordio) and the other is a [lst format](https://mxnet.incubator.apache.org/how_to/recordio.html?highlight=im2rec) (content type: application/x-image). Files for both these formats are available at http://data.dmlc.ml/mxnet/data/caltech-256/. In this example, we will use the lst format for training and use the training/validation split [specified here](http://data.dmlc.ml/mxnet/data/caltech-256/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import urllib.request\n",
    "\n",
    "def download(url):\n",
    "    filename = url.split(\"/\")[-1]\n",
    "    if not os.path.exists(filename):\n",
    "        urllib.request.urlretrieve(url, filename)\n",
    "\n",
    "\n",
    "# Caltech-256 image files\n",
    "download('http://www.vision.caltech.edu/Image_Datasets/Caltech256/256_ObjectCategories.tar')\n",
    "!tar -xf 256_ObjectCategories.tar\n",
    "\n",
    "# Tool for creating lst file\n",
    "download('https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/im2rec.py')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "mkdir -p caltech_256_train_60\n",
    "for i in 256_ObjectCategories/*; do\n",
    "    c=`basename $i`\n",
    "    mkdir -p caltech_256_train_60/$c\n",
    "    for j in `ls $i/*.jpg | shuf | head -n 60`; do\n",
    "        mv $j caltech_256_train_60/$c/\n",
    "    done\n",
    "done\n",
    "\n",
    "python im2rec.py --list --recursive caltech-256-60-train caltech_256_train_60/\n",
    "python im2rec.py --list --recursive caltech-256-60-val 256_ObjectCategories/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A .lst file is a tab-separated file with three columns that contains a list of image files. The first column specifies the image index, the second column specifies the class label index for the image, and the third column specifies the relative path of the image file. The image index in the first column should be unique across all of the images. Here we make an image list file using the [im2rec](https://github.com/apache/incubator-mxnet/blob/master/tools/im2rec.py) tool from MXNet. You can also create the .lst file in your own way. An example of .lst file is shown as follows. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!head -n 3 ./caltech-256-60-train.lst > example.lst\n",
    "f = open('example.lst','r')\n",
    "lst_content = f.read()\n",
    "print(lst_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you are bringing your own image files to train, please ensure that the .lst file follows the same format as described above. In order to train with the lst format interface, passing the lst file for both training and validation in the appropriate format is mandatory. Once we have the data available in the correct format for training, the next step is to upload the image and .lst file to S3 bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Four channels: train, validation, train_lst, and validation_lst\n",
    "s3train = 's3://{}/train/'.format(bucket)\n",
    "s3validation = 's3://{}/validation/'.format(bucket)\n",
    "s3train_lst = 's3://{}/train_lst/'.format(bucket)\n",
    "s3validation_lst = 's3://{}/validation_lst/'.format(bucket)\n",
    "\n",
    "# upload the image files to train and validation channels\n",
    "!aws s3 cp caltech_256_train_60 $s3train --recursive --quiet\n",
    "!aws s3 cp 256_ObjectCategories $s3validation --recursive --quiet\n",
    "\n",
    "# upload the lst files to train_lst and validation_lst channels\n",
    "!aws s3 cp caltech-256-60-train.lst $s3train_lst --quiet\n",
    "!aws s3 cp caltech-256-60-val.lst $s3validation_lst --quiet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have all the data stored in S3 bucket. The image and lst files will be converted to RecordIO file internelly by the image classification algorithm. But if you want do the conversion, the following cell shows how to do it using the [im2rec](https://github.com/apache/incubator-mxnet/blob/master/tools/im2rec.py) tool. Note that this is just an example of creating RecordIO files. We are **_not_** using them for training in this notebook. More details on creating RecordIO files can be found in this [tutorial](https://mxnet.incubator.apache.org/how_to/recordio.html?highlight=im2rec)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "python im2rec.py --resize 256 --quality 90 --num-thread 16 caltech-256-60-val 256_ObjectCategories/\n",
    "python im2rec.py --resize 256 --quality 90 --num-thread 16 caltech-256-60-train caltech_256_train_60/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After you created the RecordIO files, you can upload them to the train and validation channels for training. To train with RecordIO format, you can follow \"[Image-classification-fulltraining.ipynb](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-fulltraining.ipynb)\" and \"[Image-classification-transfer-learning.ipynb](https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb)\". Again, we will **_not_** use the RecordIO file for the training. The following sections will only show you how to train a model with images and list files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before training the model, we need to setup the training parameters. The next section will explain the parameters in detail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fine-tuning the Image Classification Model\n",
    "\n",
    "### Training parameters\n",
    "There are two kinds of parameters that need to be set for training. The first one are the parameters for the training job. These include:\n",
    "\n",
    "* **Input specification**: These are the training and validation channels that specify the path where training data is present. These are specified in the \"InputDataConfig\" section. The main parameters that need to be set is the \"ContentType\" which can be set to \"application/x-recordio\" or \"application/x-image\" based on the input data format and the S3Uri which specifies the bucket and the folder where the data is present. \n",
    "* **Output specification**: This is specified in the \"OutputDataConfig\" section. We just need to specify the path where the output can be stored after training\n",
    "* **Resource config**: This section specifies the type of instance on which to run the training and the number of hosts used for training. If \"InstanceCount\" is more than 1, then training can be run in a distributed manner. \n",
    "\n",
    "Apart from the above set of parameters, there are hyperparameters that are specific to the algorithm. These are:\n",
    "\n",
    "* **num_layers**: The number of layers (depth) for the network. We use 18 in this samples but other values such as 50, 152 can be used.\n",
    "* **image_shape**: The input image dimensions,'num_channels, height, width', for the network. It should be no larger than the actual image size. The number of channels should be same as the actual image.\n",
    "* **num_training_samples**: This is the total number of training samples. It is set to 15240 for caltech dataset with the current split.\n",
    "* **num_classes**: This is the number of output classes for the new dataset. Imagenet was trained with 1000 output classes but the number of output classes can be changed for fine-tuning. For caltech, we use 257 because it has 256 object categories + 1 clutter class.\n",
    "* **mini_batch_size**: The number of training samples used for each mini batch. In distributed training, the number of training samples used per batch will be N * mini_batch_size where N is the number of hosts on which training is run.\n",
    "* **epochs**: Number of training epochs.\n",
    "* **learning_rate**: Learning rate for training.\n",
    "* **top_k**: Report the top-k accuracy during training.\n",
    "* **resize**: Resize the image before using it for training. The images are resized so that the shortest side is of this parameter. If the parameter is not set, then the training data is used as such without resizing.\n",
    "* **checkpoint_frequency**: Period to store model parameters (in number of epochs).\n",
    "* **use_pretrained_model**: Set to 1 to use pretrained model for transfer learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "isConfigCell": true
   },
   "outputs": [],
   "source": [
    "# The algorithm supports multiple network depth (number of layers). They are 18, 34, 50, 101, 152 and 200\n",
    "# For this training, we will use 18 layers\n",
    "num_layers = 18\n",
    "# we need to specify the input image shape for the training data\n",
    "image_shape = \"3,224,224\"\n",
    "# we also need to specify the number of training samples in the training set\n",
    "num_training_samples = 15240\n",
    "# specify the number of output classes\n",
    "num_classes = 257\n",
    "# batch size for training\n",
    "mini_batch_size = 128\n",
    "# number of epochs\n",
    "epochs = 6\n",
    "# learning rate\n",
    "learning_rate = 0.01\n",
    "# report top_5 accuracy\n",
    "top_k = 5\n",
    "# resize image before training\n",
    "resize = 256\n",
    "# period to store model parameters (in number of epochs), in this case, we will save parameters from epoch 2, 4, and 6\n",
    "checkpoint_frequency = 2\n",
    "# Since we are using transfer learning, we set use_pretrained_model to 1 so that weights can be \n",
    "# initialized with pre-trained weights\n",
    "use_pretrained_model = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training\n",
    "Run the training using Amazon sagemaker CreateTrainingJob API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import time\n",
    "import boto3\n",
    "from time import gmtime, strftime\n",
    "\n",
    "\n",
    "s3 = boto3.client('s3')\n",
    "# create unique job name \n",
    "job_name_prefix = 'sagemaker-imageclassification-notebook'\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "job_name = job_name_prefix + timestamp\n",
    "training_params = \\\n",
    "{\n",
    "    # specify the training docker image\n",
    "    \"AlgorithmSpecification\": {\n",
    "        \"TrainingImage\": training_image,\n",
    "        \"TrainingInputMode\": \"File\"\n",
    "    },\n",
    "    \"RoleArn\": role,\n",
    "    \"OutputDataConfig\": {\n",
    "        \"S3OutputPath\": 's3://{}/{}/output'.format(bucket, job_name_prefix)\n",
    "    },\n",
    "    \"ResourceConfig\": {\n",
    "        \"InstanceCount\": 1,\n",
    "        \"InstanceType\": \"ml.p2.xlarge\",\n",
    "        \"VolumeSizeInGB\": 50\n",
    "    },\n",
    "    \"TrainingJobName\": job_name,\n",
    "    \"HyperParameters\": {\n",
    "        \"image_shape\": image_shape,\n",
    "        \"num_layers\": str(num_layers),\n",
    "        \"num_training_samples\": str(num_training_samples),\n",
    "        \"num_classes\": str(num_classes),\n",
    "        \"mini_batch_size\": str(mini_batch_size),\n",
    "        \"epochs\": str(epochs),\n",
    "        \"learning_rate\": str(learning_rate),\n",
    "        \"top_k\": str(top_k),\n",
    "        \"resize\": str(resize),\n",
    "        \"checkpoint_frequency\": str(checkpoint_frequency),\n",
    "        \"use_pretrained_model\": str(use_pretrained_model)    \n",
    "    },\n",
    "    \"StoppingCondition\": {\n",
    "        \"MaxRuntimeInSeconds\": 360000\n",
    "    },\n",
    "#Training data should be inside a subdirectory called \"train\"\n",
    "#Validation data should be inside a subdirectory called \"validation\"\n",
    "#The algorithm currently only supports fullyreplicated model (where data is copied onto each machine)\n",
    "    \"InputDataConfig\": [\n",
    "        {\n",
    "            \"ChannelName\": \"train\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": 's3://{}/train/'.format(bucket),\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"application/x-image\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        },\n",
    "        {\n",
    "            \"ChannelName\": \"validation\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": 's3://{}/validation/'.format(bucket),\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"application/x-image\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        },\n",
    "        {\n",
    "            \"ChannelName\": \"train_lst\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": 's3://{}/train_lst/'.format(bucket),\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"application/x-image\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        },\n",
    "        {\n",
    "            \"ChannelName\": \"validation_lst\",\n",
    "            \"DataSource\": {\n",
    "                \"S3DataSource\": {\n",
    "                    \"S3DataType\": \"S3Prefix\",\n",
    "                    \"S3Uri\": 's3://{}/validation_lst/'.format(bucket),\n",
    "                    \"S3DataDistributionType\": \"FullyReplicated\"\n",
    "                }\n",
    "            },\n",
    "            \"ContentType\": \"application/x-image\",\n",
    "            \"CompressionType\": \"None\"\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "print('Training job name: {}'.format(job_name))\n",
    "print('\\nInput Data Location: {}'.format(training_params['InputDataConfig'][0]['DataSource']['S3DataSource']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create the Amazon SageMaker training job\n",
    "sagemaker = boto3.client(service_name='sagemaker')\n",
    "sagemaker.create_training_job(**training_params)\n",
    "\n",
    "# confirm that the training job has started\n",
    "status = sagemaker.describe_training_job(TrainingJobName=job_name)['TrainingJobStatus']\n",
    "print('Training job current status: {}'.format(status))\n",
    "\n",
    "try:\n",
    "    # wait for the job to finish and report the ending status\n",
    "    sagemaker.get_waiter('training_job_completed_or_stopped').wait(TrainingJobName=job_name)\n",
    "    training_info = sagemaker.describe_training_job(TrainingJobName=job_name)\n",
    "    status = training_info['TrainingJobStatus']\n",
    "    print(\"Training job ended with status: \" + status)\n",
    "except:\n",
    "    print('Training failed to start')\n",
    "     # if exception is raised, that means it has failed\n",
    "    message = sagemaker.describe_training_job(TrainingJobName=job_name)['FailureReason']\n",
    "    print('Training failed with the following error: {}'.format(message))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "training_info = sagemaker.describe_training_job(TrainingJobName=job_name)\n",
    "status = training_info['TrainingJobStatus']\n",
    "print(\"Training job ended with status: \" + status)\n",
    "print (training_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you see the message,\n",
    "\n",
    "> `Training job ended with status: Completed`\n",
    "\n",
    "then that means training sucessfully completed and the output model was stored in the output path specified by `training_params['OutputDataConfig']`.\n",
    "\n",
    "You can also view information about and the status of a training job using the AWS SageMaker console. Just click on the \"Jobs\" tab."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Up Hosting For The Model\n",
    "\n",
    "A trained model does nothing on its own. We now want to use the model to perform inference. For this example, that means predicting the class label given an input image.\n",
    "\n",
    "This section involves several steps,\n",
    "\n",
    "1. [Create model](#CreateModel) - Create model for the training output\n",
    "1. [Create endpoint configuration](#CreateEndpointConfiguration) - Create a configuration defining an endpoint.\n",
    "1. [Create endpoint](#CreateEndpoint) - Use the configuration to create an inference endpoint.\n",
    "1. [Perform inference](#Perform Inference) - Perform inference on some input data using the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create model\n",
    "\n",
    "We now create a SageMaker Model from the training output. Using the model we can create an Endpoint Configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import boto3\n",
    "from time import gmtime, strftime\n",
    "\n",
    "sage = boto3.Session().client(service_name='sagemaker') \n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "model_name=\"image-classification-model\" + timestamp\n",
    "print(model_name)\n",
    "info = sage.describe_training_job(TrainingJobName=job_name)\n",
    "model_data = info['ModelArtifacts']['S3ModelArtifacts']\n",
    "print(model_data)\n",
    "hosting_image = containers[boto3.Session().region_name]\n",
    "primary_container = {\n",
    "    'Image': hosting_image,\n",
    "    'ModelDataUrl': model_data,\n",
    "}\n",
    "\n",
    "create_model_response = sage.create_model(\n",
    "    ModelName = model_name,\n",
    "    ExecutionRoleArn = role,\n",
    "    PrimaryContainer = primary_container)\n",
    "\n",
    "print(create_model_response['ModelArn'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create endpoint configuration\n",
    "At launch, we will support configuring REST endpoints in hosting with multiple models, e.g. for A/B testing purposes. In order to support this, customers create an endpoint configuration, that describes the distribution of traffic across the models, whether split, shadowed, or sampled in some way.\n",
    "\n",
    "In addition, the endpoint configuration describes the instance type required for model deployment, and at launch will describe the autoscaling configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from time import gmtime, strftime\n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "endpoint_config_name = job_name_prefix + '-epc-' + timestamp\n",
    "endpoint_config_response = sage.create_endpoint_config(\n",
    "    EndpointConfigName = endpoint_config_name,\n",
    "    ProductionVariants=[{\n",
    "        'InstanceType':'ml.p2.xlarge',\n",
    "        'InitialInstanceCount':1,\n",
    "        'ModelName':model_name,\n",
    "        'VariantName':'AllTraffic'}])\n",
    "\n",
    "print('Endpoint configuration name: {}'.format(endpoint_config_name))\n",
    "print('Endpoint configuration arn:  {}'.format(endpoint_config_response['EndpointConfigArn']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create endpoint\n",
    "Lastly, the customer creates the endpoint that serves up the model, through specifying the name and configuration defined above. The end result is an endpoint that can be validated and incorporated into production applications. This takes 9-11 minutes to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import time\n",
    "\n",
    "timestamp = time.strftime('-%Y-%m-%d-%H-%M-%S', time.gmtime())\n",
    "endpoint_name = job_name_prefix + '-ep-' + timestamp\n",
    "print('Endpoint name: {}'.format(endpoint_name))\n",
    "\n",
    "endpoint_params = {\n",
    "    'EndpointName': endpoint_name,\n",
    "    'EndpointConfigName': endpoint_config_name,\n",
    "}\n",
    "endpoint_response = sagemaker.create_endpoint(**endpoint_params)\n",
    "print('EndpointArn = {}'.format(endpoint_response['EndpointArn']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, now the endpoint can be created. It may take sometime to create the endpoint..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# get the status of the endpoint\n",
    "response = sagemaker.describe_endpoint(EndpointName=endpoint_name)\n",
    "status = response['EndpointStatus']\n",
    "print('EndpointStatus = {}'.format(status))\n",
    "    \n",
    "try:\n",
    "    sagemaker.get_waiter('endpoint_in_service').wait(EndpointName=endpoint_name)\n",
    "finally:\n",
    "    resp = sagemaker.describe_endpoint(EndpointName=endpoint_name)\n",
    "    status = resp['EndpointStatus']\n",
    "    print(\"Arn: \" + resp['EndpointArn'])\n",
    "    print(\"Create endpoint ended with status: \" + status)\n",
    "\n",
    "    if status != 'InService':\n",
    "        message = sagemaker.describe_endpoint(EndpointName=endpoint_name)['FailureReason']\n",
    "        print('Training failed with the following error: {}'.format(message))\n",
    "        raise Exception('Endpoint creation did not succeed')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you see the message,\n",
    "\n",
    "> `Endpoint creation ended with EndpointStatus = InService`\n",
    "\n",
    "then congratulations! You now have a functioning inference endpoint. You can confirm the endpoint configuration and status by navigating to the \"Endpoints\" tab in the AWS SageMaker console.\n",
    "\n",
    "We will finally create a runtime object from which we can invoke the endpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perform inference\n",
    "Finally, the customer can now validate the model for use. They can obtain the endpoint from the client library using the result from previous operations, and generate classifications from the trained model using that endpoint.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "runtime = boto3.Session().client(service_name='runtime.sagemaker') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Download test image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "!wget -O /tmp/test.jpg http://www.vision.caltech.edu/Image_Datasets/Caltech256/images/008.bathtub/008_0007.jpg\n",
    "file_name = '/tmp/test.jpg'\n",
    "# test image\n",
    "from IPython.display import Image\n",
    "Image(file_name)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import numpy as np\n",
    "with open(file_name, 'rb') as f:\n",
    "    payload = f.read()\n",
    "    payload = bytearray(payload)\n",
    "response = runtime.invoke_endpoint(EndpointName=endpoint_name, \n",
    "                                   ContentType='application/x-image', \n",
    "                                   Body=payload)\n",
    "result = response['Body'].read()\n",
    "# result will be in json format and convert it to ndarray\n",
    "result = json.loads(result)\n",
    "# the result will output the probabilities for all classes\n",
    "# find the class with maximum probability and print the class index\n",
    "index = np.argmax(result)\n",
    "object_categories = ['ak47', 'american-flag', 'backpack', 'baseball-bat', 'baseball-glove', 'basketball-hoop', 'bat', 'bathtub', 'bear', 'beer-mug', 'billiards', 'binoculars', 'birdbath', 'blimp', 'bonsai-101', 'boom-box', 'bowling-ball', 'bowling-pin', 'boxing-glove', 'brain-101', 'breadmaker', 'buddha-101', 'bulldozer', 'butterfly', 'cactus', 'cake', 'calculator', 'camel', 'cannon', 'canoe', 'car-tire', 'cartman', 'cd', 'centipede', 'cereal-box', 'chandelier-101', 'chess-board', 'chimp', 'chopsticks', 'cockroach', 'coffee-mug', 'coffin', 'coin', 'comet', 'computer-keyboard', 'computer-monitor', 'computer-mouse', 'conch', 'cormorant', 'covered-wagon', 'cowboy-hat', 'crab-101', 'desk-globe', 'diamond-ring', 'dice', 'dog', 'dolphin-101', 'doorknob', 'drinking-straw', 'duck', 'dumb-bell', 'eiffel-tower', 'electric-guitar-101', 'elephant-101', 'elk', 'ewer-101', 'eyeglasses', 'fern', 'fighter-jet', 'fire-extinguisher', 'fire-hydrant', 'fire-truck', 'fireworks', 'flashlight', 'floppy-disk', 'football-helmet', 'french-horn', 'fried-egg', 'frisbee', 'frog', 'frying-pan', 'galaxy', 'gas-pump', 'giraffe', 'goat', 'golden-gate-bridge', 'goldfish', 'golf-ball', 'goose', 'gorilla', 'grand-piano-101', 'grapes', 'grasshopper', 'guitar-pick', 'hamburger', 'hammock', 'harmonica', 'harp', 'harpsichord', 'hawksbill-101', 'head-phones', 'helicopter-101', 'hibiscus', 'homer-simpson', 'horse', 'horseshoe-crab', 'hot-air-balloon', 'hot-dog', 'hot-tub', 'hourglass', 'house-fly', 'human-skeleton', 'hummingbird', 'ibis-101', 'ice-cream-cone', 'iguana', 'ipod', 'iris', 'jesus-christ', 'joy-stick', 'kangaroo-101', 'kayak', 'ketch-101', 'killer-whale', 'knife', 'ladder', 'laptop-101', 'lathe', 'leopards-101', 'license-plate', 'lightbulb', 'light-house', 'lightning', 'llama-101', 'mailbox', 'mandolin', 'mars', 'mattress', 'megaphone', 'menorah-101', 'microscope', 'microwave', 'minaret', 'minotaur', 'motorbikes-101', 'mountain-bike', 'mushroom', 'mussels', 'necktie', 'octopus', 'ostrich', 'owl', 'palm-pilot', 'palm-tree', 'paperclip', 'paper-shredder', 'pci-card', 'penguin', 'people', 'pez-dispenser', 'photocopier', 'picnic-table', 'playing-card', 'porcupine', 'pram', 'praying-mantis', 'pyramid', 'raccoon', 'radio-telescope', 'rainbow', 'refrigerator', 'revolver-101', 'rifle', 'rotary-phone', 'roulette-wheel', 'saddle', 'saturn', 'school-bus', 'scorpion-101', 'screwdriver', 'segway', 'self-propelled-lawn-mower', 'sextant', 'sheet-music', 'skateboard', 'skunk', 'skyscraper', 'smokestack', 'snail', 'snake', 'sneaker', 'snowmobile', 'soccer-ball', 'socks', 'soda-can', 'spaghetti', 'speed-boat', 'spider', 'spoon', 'stained-glass', 'starfish-101', 'steering-wheel', 'stirrups', 'sunflower-101', 'superman', 'sushi', 'swan', 'swiss-army-knife', 'sword', 'syringe', 'tambourine', 'teapot', 'teddy-bear', 'teepee', 'telephone-box', 'tennis-ball', 'tennis-court', 'tennis-racket', 'theodolite', 'toaster', 'tomato', 'tombstone', 'top-hat', 'touring-bike', 'tower-pisa', 'traffic-light', 'treadmill', 'triceratops', 'tricycle', 'trilobite-101', 'tripod', 't-shirt', 'tuning-fork', 'tweezer', 'umbrella-101', 'unicorn', 'vcr', 'video-projector', 'washing-machine', 'watch-101', 'waterfall', 'watermelon', 'welding-mask', 'wheelbarrow', 'windmill', 'wine-bottle', 'xylophone', 'yarmulke', 'yo-yo', 'zebra', 'airplanes-101', 'car-side-101', 'faces-easy-101', 'greyhound', 'tennis-shoes', 'toad', 'clutter']\n",
    "print(\"Result: label - \" + object_categories[index] + \", probability - \" + str(result[index]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Clean up\n",
    "\n",
    "When we're done with the endpoint, we can just delete it and the backing instances will be released. Uncomment and run the following cell to delete the endpoint and model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sage.delete_endpoint(EndpointName=endpoint_name)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_mxnet_p36",
   "language": "python",
   "name": "conda_mxnet_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 2
}