Numpy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy\n",
    "\n",
    "This notebook includes a collection of commands and functions provided in the Numpy library that are really useful and speed up the data manipulation instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Numpy\n",
    "I first begin by importing the nump library. I use the Python generic notation to import it and I rename it as `np` so I can use `np` instead of `numpy` everywhere."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy array\n",
    "I begin by first creating a numpy array and then accessing its elements using the index values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "[1 2]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define an array arr with integer elements 1,2,3 and 4\n",
    "integerArray = np.array([1,2,3,4], int)\n",
    "\n",
    "# We can access these elements by using index values\n",
    "print(integerArray[0])\n",
    "\n",
    "# We can also use ranges to access values\n",
    "print(integerArray[:2])\n",
    "\n",
    "# Find if a value exists in the array\n",
    "# Returns true if value exists else returns false\n",
    "2 in integerArray # Returns true"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Numpy also provides quick methods to define an array. Basic methods include:\n",
    "1. concatenation - combine arrays\n",
    "2. zeros - array with all values zero\n",
    "3. ones - array with all values one\n",
    "4. range - define the array as a range of values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4 5 6]\n",
      "[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      "[1 1 1 1 1 1 1 1 1 1]\n",
      "[0 1 2 3 4 5 6 7 8 9]\n"
     ]
    }
   ],
   "source": [
    "integerArray2 = np.array([5,6], int)\n",
    "\n",
    "# Concatenate two arrays\n",
    "print(np.concatenate((integerArray, integerArray2)))\n",
    "\n",
    "# Array of zeros\n",
    "print(np.zeros(10))\n",
    "\n",
    "# Array of ones with type int\n",
    "print(np.ones(10, dtype=int))\n",
    "\n",
    "# Range of numbers\n",
    "rangeArray = np.array(range(10), int)\n",
    "print(rangeArray)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multidimensional arrays\n",
    "\n",
    "Further, Numpy library also enables us to create multidimensional arrays.\n",
    "Moreover, the `reshape` function allows us to convert Numpy single dimensional arrays to multidimensional arrays and vice versa using `flatten`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 2. 3.]\n",
      " [4. 5. 6.]]\n",
      "[[0 1]\n",
      " [2 3]\n",
      " [4 5]\n",
      " [6 7]\n",
      " [8 9]]\n",
      "[0 1 2 3 4 5 6 7 8 9]\n"
     ]
    }
   ],
   "source": [
    "# Multidimensional array\n",
    "floatArray = np.array([[1,2,3], [4,5,6]], float)\n",
    "print(floatArray)\n",
    "\n",
    "# Convert one dimensional to multi-dimensional arrays\n",
    "rangeArray = rangeArray.reshape(5, 2)\n",
    "print(rangeArray)\n",
    "# and vice versa too\n",
    "rangeArray = rangeArray.flatten()\n",
    "print(rangeArray)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Concatenate` function on multidimensional arrays can be performed on any axis. For a two dimensional array (array[row][column]), to concatenate along row, we set axis as 0 (default is also 0). To concatenate along column, we set axis to 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]\n",
      " [7 8]]\n",
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]\n",
      " [7 8]]\n",
      "[[1 2 5 6]\n",
      " [3 4 7 8]]\n"
     ]
    }
   ],
   "source": [
    "# Concatenation of multi-dimensional arrays\n",
    "arr1 = np.array([[1,2], [3,4]], int)\n",
    "arr2 = np.array([[5,6], [7,8]], int)\n",
    "print(np.concatenate((arr1, arr2)))\n",
    "\n",
    "# Based on dimension 1\n",
    "print(np.concatenate((arr1, arr2), axis=0))\n",
    "\n",
    "# Based on dimension 2\n",
    "print(np.concatenate((arr1, arr2), axis=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy array operations\n",
    "Numpy arrays allow us to directly apply a range of operations such as addition, subtraction etc. The operators are overloaded in the library which makes the process possible. The operations occur on an element by element basis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Array 1\n",
      " [[1 2]\n",
      " [3 4]]\n",
      "Array 2\n",
      " [[5 6]\n",
      " [7 8]]\n",
      "Array 1 + Array 2\n",
      " [[ 6  8]\n",
      " [10 12]]\n",
      "Array 1 * Array 2\n",
      " [[ 5 12]\n",
      " [21 32]]\n",
      "Square root of Array 1\n",
      " [[1.         1.41421356]\n",
      " [1.73205081 2.        ]]\n",
      "Square root of Array 1\n",
      " [[0.         0.69314718]\n",
      " [1.09861229 1.38629436]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Array 1\\n {}\".format(arr1))\n",
    "print(\"Array 2\\n {}\".format(arr2))\n",
    "\n",
    "# Addition\n",
    "print(\"Array 1 + Array 2\\n {}\".format(arr1 + arr2))\n",
    "\n",
    "# Multiplication\n",
    "print(\"Array 1 * Array 2\\n {}\".format(arr1 * arr2))\n",
    "\n",
    "# Square root\n",
    "print(\"Square root of Array 1\\n {}\".format(np.sqrt(arr1)))\n",
    "\n",
    "# Log\n",
    "print(\"Log of Array 1\\n {}\".format(np.log(arr1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, these opertations will throw errors if the dimensions of the arrays are not same. There are many other operations that are available to us such as `exp`, `floor`, `abs` and many trignometric funcitons.\n",
    "\n",
    "Operations can be specific to a given array too such as sum of all elements of the array and others. If we define the axis, then we can calculate the operations across the respective row or column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sum of elements of Array 1: 10\n",
      "Mean of elements of Array 1: 2.5\n",
      "Minimum of elements of Array 1: 1\n",
      "Index of minimum of elements of Array 1: 3\n"
     ]
    }
   ],
   "source": [
    "# Sum of array elements\n",
    "print(\"Sum of elements of Array 1: {}\".format(arr1.sum()))\n",
    "\n",
    "# Mean of array elements\n",
    "print(\"Mean of elements of Array 1: {}\".format(arr1.mean()))\n",
    "# We can also calulate variance using var() and standard deviation using std()\n",
    "\n",
    "# Minimum of array elements\n",
    "print(\"Minimum of elements of Array 1: {}\".format(arr1.min()))\n",
    "# We can also calculate maximum value using max()\n",
    "\n",
    "# Index of maximum of array elements can be found using arg before the funciton name\n",
    "print(\"Index of minimum of elements of Array 1: {}\".format(arr1.argmax()))\n",
    "# We can also find index of minimum value using argmin()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample Array\n",
      " [[5 2 3]\n",
      " [3 4 5]\n",
      " [1 1 1]]\n",
      "Unique values: [1 2 3 4 5]\n",
      "Diagonal\n",
      " [5 4 1]\n",
      "Sorted\n",
      " [[2 3 5]\n",
      " [3 4 5]\n",
      " [1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "sampleArray = np.array([[5,2,3], [3,4,5], [1,1,1]], int)\n",
    "print(\"Sample Array\\n {}\".format(sampleArray))\n",
    "\n",
    "# Get unqiue values\n",
    "print(\"Unique values: {}\".format(np.unique(sampleArray)))\n",
    "\n",
    "# Get diagonal values\n",
    "print(\"Diagonal\\n {}\".format(sampleArray.diagonal()))\n",
    "\n",
    "# Sort values in the multidimensional array\n",
    "print(\"Sorted\\n {}\".format(np.sort(sampleArray)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compare arrays\n",
    "We can also use comparison operators to compare complete arrays with one another"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Array 1 > Array 2\n",
      "[[False False]\n",
      " [False False]]\n",
      "Array 1 == 2\n",
      " [[False  True]\n",
      " [False False]]\n"
     ]
    }
   ],
   "source": [
    "# We can compare complete arrays of equal size element wise\n",
    "print(\"Array 1 > Array 2\\n{}\".format(arr1 > arr2))\n",
    "\n",
    "# We can compare elements of an array with a given value\n",
    "print(\"Array 1 == 2\\n {}\".format(arr1 == 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vector and Matrix operations\n",
    "Numpy is specifically famous for its efficiency while working with vectors and matrices. Some of the useful functions are described here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dot of Matrix 1 and Matrix 2\n",
      " [[ 4  4  4]\n",
      " [10 10 10]\n",
      " [16 16 16]]\n",
      "Cross of Matrix 1 and Matrix 2\n",
      " [[-1  2 -1]\n",
      " [ 0  0  0]\n",
      " [-1  2 -1]]\n",
      "Outer of Matrix 1 and Matrix 2\n",
      " [[1 1 1 0 0 0 1 1 1]\n",
      " [2 2 2 0 0 0 2 2 2]\n",
      " [3 3 3 0 0 0 3 3 3]\n",
      " [4 4 4 0 0 0 4 4 4]\n",
      " [5 5 5 0 0 0 5 5 5]\n",
      " [6 6 6 0 0 0 6 6 6]\n",
      " [7 7 7 0 0 0 7 7 7]\n",
      " [8 8 8 0 0 0 8 8 8]\n",
      " [9 9 9 0 0 0 9 9 9]]\n",
      "Inner of Matrix 1 and Matrix 2\n",
      " [[ 6  0  6]\n",
      " [15  0 15]\n",
      " [24  0 24]]\n"
     ]
    }
   ],
   "source": [
    "vector = np.array([1,2,3,4], int)\n",
    "matrix1 = np.array([[1,2,3], [4,5,6], [7,8,9]], int)\n",
    "matrix2 = np.array([[1,1,1], [0,0,0], [1,1,1]], int)\n",
    "\n",
    "# Dot operator\n",
    "print(\"Dot of Matrix 1 and Matrix 2\\n {}\".format(np.dot(matrix1, matrix2)))\n",
    "\n",
    "# Cross operator\n",
    "print(\"Cross of Matrix 1 and Matrix 2\\n {}\".format(np.cross(matrix1, matrix2)))\n",
    "\n",
    "# Outer operator\n",
    "print(\"Outer of Matrix 1 and Matrix 2\\n {}\".format(np.outer(matrix1, matrix2)))\n",
    "\n",
    "# Inner operator\n",
    "print(\"Inner of Matrix 1 and Matrix 2\\n {}\".format(np.inner(matrix1, matrix2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Random array\n",
    "We can use the random function to get arrays with random values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random array: [0.76073446 0.99345894 0.46903477 0.84603991 0.38853774]\n",
      "Random matrix:\n",
      " [[0.16632713 0.58049315 0.34176713 0.94322875]\n",
      " [0.22748443 0.82143576 0.81514027 0.29211289]\n",
      " [0.96835549 0.89565579 0.48977925 0.83669168]\n",
      " [0.41939199 0.76978422 0.85281824 0.26429364]\n",
      " [0.40808517 0.33635079 0.34038655 0.9892858 ]]\n",
      "Random integer array: [9. 3. 5. 7. 4. 7. 8. 8. 9. 4.]\n",
      "Random matrix:\n",
      "[[5 3 0 2 9]\n",
      " [0 4 4 7 8]]\n"
     ]
    }
   ],
   "source": [
    "# Random array\n",
    "print(\"Random array: {}\".format(np.random.rand(5)))\n",
    "\n",
    "# Random matrix\n",
    "print(\"Random matrix:\\n {}\".format(np.random.rand(5,4)))\n",
    "\n",
    "# Random array of integers in a range (say 0-9)\n",
    "randomArray = np.floor(np.random.rand(10) * 10)\n",
    "print(\"Random integer array: {}\".format(randomArray))\n",
    "\n",
    "# Futher simplification\n",
    "print(\"Random matrix:\\n{}\".format(np.random.randint(0, 10, (2,5))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy Strategies\n",
    "There are a total of 4 startegies to consider in the Numpy library that can help us work better with loops.\n",
    "These include:\n",
    "1. Universal Functions\n",
    "2. Aggregations\n",
    "3. Broadcasting\n",
    "4. Slicing, masking and fancy indexing\n",
    "\n",
    "I've already adressed them in the examples above but here I would create the exact distinctions.\n",
    "These functions are essential as they are very quick in their operation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Universal Functions\n",
    "These functions operate element-wise on the elements of an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Array: [0 1 2 3 4 5 6 7 8 9]\n",
      "New array: [ 0 10 20 30 40 50 60 70 80 90]\n"
     ]
    }
   ],
   "source": [
    "# Multiply all elements of an array by 10\n",
    "arr = np.array(range(10), int)\n",
    "print(\"Array: {}\".format(arr))\n",
    "print(\"New array: {}\".format(arr * 10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aggregation Functions\n",
    "These functions summarize the information contained in an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Standard deviation of array: 2.8722813232690143\n"
     ]
    }
   ],
   "source": [
    "# Get standard deviation of all array values\n",
    "print(\"Standard deviation of array: {}\".format(arr.std()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Broadcasting\n",
    "This defines the rules of how universal functions operate on arrays. This determines how a number might be added to a matrix, how a vector gets added to a matrix etc. \n",
    "It broadcasts the smaller of the two to match the bigger so the operation may be performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Final sum after broadcasting:\n",
      " [[ 2  3  4]\n",
      " [10  9  8]]\n"
     ]
    }
   ],
   "source": [
    "# Add a row to a matrix\n",
    "mat = np.array([[1,2,3], [9,8,7]], int)\n",
    "arr = np.array([1,1,1], int)\n",
    "# On adding the two, arr will get broadcasted and will get added as a matrix with both rows same as [1,1,1]\n",
    "print(\"Final sum after broadcasting:\\n {}\".format(mat + arr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slicing, masking and fancy indexing\n",
    "We can slice numpy arrays as and where needed or maybe get values based on index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Array: [0 1 2 3 4 5 6 7 8 9]\n",
      "Array value from index 0 to 4: [0 1 2 3 4]\n",
      "Array values with mask as true: [0 1 2 7 8 9]\n",
      "Array values with index in list: [2 4 6]\n",
      "Array values with index in list: [7]\n"
     ]
    }
   ],
   "source": [
    "bigArray = np.array(range(10))\n",
    "print(\"Array: {}\".format(bigArray))\n",
    "\n",
    "# Slice array from index 0 to 4\n",
    "print(\"Array value from index 0 to 4: {}\".format(bigArray[:5]))\n",
    "\n",
    "# Masking using boolean values and operators\n",
    "mask = (bigArray > 6) | (bigArray < 3)\n",
    "print(\"Array values with mask as true: {}\".format(bigArray[mask]))\n",
    "\n",
    "# Fancy indexing\n",
    "ind = [2,4,6]\n",
    "print(\"Array values with index in list: {}\".format(bigArray[ind]))\n",
    "\n",
    "# Combine all three\n",
    "print(\"Array values with index in list: {}\".format(bigArray[bigArray > 6][:1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion\n",
    "Here, I have worked with a few key functions available in Numpy that facilitate array, vector and matrix multiplication. This understanding was essential as it is the basis for many other libraries such as Sklearn which uses Numpy for its algorithms."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}