1. Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Programming Computer Vision : Basics\n",
    "___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Images\n",
    "Images can be read using Image class of Python library **PIL** [(Python Imaging Library)](http://www.pythonware.com/products/pil/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "im = Image.open('images/flower.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The return value 'im' is a PIL image object. Thus the following image would be read.\n",
    "<img src=\"images/flower.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "## Color conversions\n",
    "We can use the `convert()` method for Color conversions. An image can be converted to grayscale using the .convert('L') function where 'L' simply is a mode that defines images as 8-bit pixels of black & white. To learn about other modes, you can visit http://pillow.readthedocs.org/en/3.1.x/handbook/concepts.html.\n",
    "The library supports transformations between each supported mode and the 'L' and 'RGB' modes. To convert between other modes, you may have to use an intermediate image (typically an “RGB” image)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "gray = im.convert('L')\n",
    "gray.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/gray.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "## Enhancement\n",
    "The ImageEnhance module can be used for image enhancement. Once created from an image, an enhancement object can be used to quickly try out different settings.\n",
    "You can adjust contrast, brightness, color balance and sharpness in this way."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import ImageEnhance\n",
    "\n",
    "enh = ImageEnhance.Contrast(im)\n",
    "enh.enhance(1.4).show(\"30% more contrast\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/high_contrast.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "\n",
    "## Converting into other file format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import os, sys\n",
    "from PIL import Image\n",
    "\n",
    "def convertToJPEG():\n",
    "    for infile in sys.argv[1:]:\n",
    "        f, e = os.path.splitext(infile)\n",
    "        outfile = f + \".jpg\"\n",
    "        if infile != outfile:\n",
    "            try:\n",
    "                Image.open(infile).save(outfile)\n",
    "            except IOError:\n",
    "                print(\"cannot convert\", infile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a function that converts the images in our specified file format. The PIL function open() creates a PIL image object and the save() method saves the image to a file with the given filename.\n",
    "___\n",
    "## Creating Thumbnails"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import os, sys\n",
    "from PIL import Image\n",
    "\n",
    "size = (128, 128)\n",
    "\n",
    "def createThumbnails():\n",
    "    for infile in sys.argv[1:]:\n",
    "        outfile = os.path.splitext(infile)[0] + \".thumbnail\"\n",
    "        if infile != outfile:\n",
    "            try:\n",
    "                im = Image.open(infile)\n",
    "                im.thumbnail(size)\n",
    "                im.save(outfile, \"JPEG\")\n",
    "            except IOError:\n",
    "                print(\"cannot create thumbnail for\", infile)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The thumbnail() method takes a tuple specifying the new size and converts the image to a thumbnail image with size that fits within the tuple.\n",
    "\n",
    "___\n",
    "## Copy and paste regions\n",
    "Cropping a region from an image is done using the crop() method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "box = (100,100,400,400)\n",
    "region = im.crop(box)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The region is defined by a 4-tuple, where coordinates are (left, upper, right, lower). PIL uses a coordinate system with (0, 0) in the upper left corner. The extracted region can for example be rotated and then put back using the paste() method like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "region = region.transpose(Image.ROTATE_180)\n",
    "im.paste(region,box)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "____\n",
    "## Resize and rotate\n",
    "To resize an image, call resize() with a tuple giving the new size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "out = im.resize((128,128))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To rotate an image, use counter clockwise angles and rotate() like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "out = im.rotate(45)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "\n",
    "## Using Matplotlib to plot images, points and lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "from pylab import *\n",
    "# read image to array\n",
    "im = array(Image.open('images/flower.jpg'))\n",
    "# plot the image\n",
    "imshow(im)\n",
    "# some points\n",
    "x = [100,100,400,400]\n",
    "y = [200,500,200,500]\n",
    "# plot the points with red star-markers\n",
    "plot(x,y,'r*')\n",
    "# line plot connecting the first two points\n",
    "plot(x[:2],y[:2])\n",
    "# add title and show the plot\n",
    "title('Plotting: \"flower.jpg')\n",
    "show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"captures/mpl.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Image Contours"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.0, 400.0, 0.0, 300.0)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from PIL import Image\n",
    "from pylab import *\n",
    "# read image to array\n",
    "im = array(Image.open('images/flower.jpg').convert('L'))\n",
    "# create a new figure\n",
    "figure()\n",
    "# don’t use colors\n",
    "gray()\n",
    "# show contours with origin upper left corner\n",
    "contour(im, origin='image')\n",
    "axis('equal')\n",
    "axis('off')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/contours.png\">\n",
    "### Histograms:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "figure()\n",
    "hist(im.flatten(),128)\n",
    "show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/hist.png\">\n",
    "This shows the distribution of pixel values. A number of bins is specified for the span of values and each bin gets a count of how many pixels have values in the bin’s range. The visualization of the (graylevel) image histogram is done using the hist() function.\n",
    "The second argument specifies the number of bins to use. Note that the image needs to be flattened first, because hist() takes a one-dimensional array as input. The method flatten() converts any array to a one-dimensional array with values taken row-wise.\n",
    "___\n",
    "\n",
    "## Graylevel transforms using NumPy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "from numpy import *\n",
    "im = array(Image.open('images/flower.jpg').convert('L'))\n",
    "im2 = 255 - im #invert image\n",
    "im3 = (100.0/255) * im + 100 #clamp to interval 100...200\n",
    "im4 = 255.0 * (im/255.0)**2 #squared"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Converting these numpy arrays back into our grayscale images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "npim2 = Image.fromarray(uint8(im2))\n",
    "npim2.show()\n",
    "npim3 = Image.fromarray(uint8(im3))\n",
    "npim3.show()\n",
    "npim4 = Image.fromarray(uint8(im4))\n",
    "npim4.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus the three transformed grayscale images can be compared as follows:\n",
    "<img src=\"images/grayscale.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___\n",
    "## Image De-noising\n",
    "Image de-noising is the process of removing image noise while at the same time trying to preserve details and structures"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from numpy import *\n",
    "\n",
    "def denoise(im, U_init, tolerance=0.1, tau=0.125, tv_weight=100):\n",
    "    \"\"\" An implementation of the Rudin-Osher-Fatemi (ROF) denoising model\n",
    "        using the numerical procedure presented in Eq. (11) of A. Chambolle\n",
    "        (2005). Implemented using periodic boundary conditions \n",
    "        (essentially turning the rectangular image domain into a torus!).\n",
    "    \n",
    "        Input:\n",
    "        im - noisy input image (grayscale)\n",
    "        U_init - initial guess for U\n",
    "        tv_weight - weight of the TV-regularizing term\n",
    "        tau - steplength in the Chambolle algorithm\n",
    "        tolerance - tolerance for determining the stop criterion\n",
    "    \n",
    "        Output:\n",
    "        U - denoised and detextured image (also the primal variable)\n",
    "        T - texture residual\"\"\"\n",
    "    \n",
    "    #---Initialization\n",
    "    m,n = im.shape #size of noisy image\n",
    "\n",
    "    U = U_init\n",
    "    Px = im #x-component to the dual field\n",
    "    Py = im #y-component of the dual field\n",
    "    error = 1 \n",
    "    iteration = 0\n",
    "\n",
    "    #---Main iteration\n",
    "    while (error > tolerance):\n",
    "        Uold = U\n",
    "\n",
    "        #Gradient of primal variable\n",
    "        LyU = vstack((U[1:,:],U[0,:])) #Left translation w.r.t. the y-direction\n",
    "        LxU = hstack((U[:,1:],U.take([0],axis=1))) #Left translation w.r.t. the x-direction\n",
    "\n",
    "        GradUx = LxU-U #x-component of U's gradient\n",
    "        GradUy = LyU-U #y-component of U's gradient\n",
    "\n",
    "        #First we update the dual varible\n",
    "        PxNew = Px + (tau/tv_weight)*GradUx #Non-normalized update of x-component (dual)\n",
    "        PyNew = Py + (tau/tv_weight)*GradUy #Non-normalized update of y-component (dual)\n",
    "        NormNew = maximum(1,sqrt(PxNew**2+PyNew**2))\n",
    "\n",
    "        Px = PxNew/NormNew #Update of x-component (dual)\n",
    "        Py = PyNew/NormNew #Update of y-component (dual)\n",
    "\n",
    "        #Then we update the primal variable\n",
    "        RxPx =hstack((Px.take([-1],axis=1),Px[:,0:-1])) #Right x-translation of x-component\n",
    "        RyPy = vstack((Py[-1,:],Py[0:-1,:])) #Right y-translation of y-component\n",
    "        DivP = (Px-RxPx)+(Py-RyPy) #Divergence of the dual field.\n",
    "        U = im + tv_weight*DivP #Update of the primal variable\n",
    "\n",
    "        #Update of error-measure\n",
    "        error = linalg.norm(U-Uold)/sqrt(n*m);\n",
    "        iteration += 1;\n",
    "\n",
    "        print iteration, error\n",
    "\n",
    "    #The texture residual\n",
    "    T = im - U\n",
    "    print 'Number of ROF iterations: ', iteration\n",
    "    \n",
    "    return U,T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we used the function roll(), which as the name suggests, \"rolls\" the values of an array cyclically around an axis. This is very convenient for computing neighbor differences, in this case for derivatives. We also used linalg.norm() which measures the difference between two arrays (in this case the image matrices U and Uold)\n",
    "\n",
    "We can now use the denoise function to remove noise from a real image This is the image to be tested:\n",
    "<img src=\"images/noiseimage.jpg\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from PIL import Image\n",
    "import pylab\n",
    "\n",
    "im = array(Image.open('images/noiseimage.jpg').convert('L'))\n",
    "U,T = denoise(im,im)\n",
    "\n",
    "pylab.figure()\n",
    "pylab.gray()\n",
    "pylab.imshow(U)\n",
    "pylab.axis('equal')\n",
    "pylab.axis('off')\n",
    "pylab.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting de-noised image is:\n",
    "    <img src=\"images/denoisedimage.jpg\">\n",
    "  ___"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus we are done with the basics of Computer Vision. Next we would level up a bit by exploring the OpenCV library.\n",
    "<hr>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}