13_iterators.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "K8Gvk-KiwRZF"
   },
   "source": [
    "# Iterators & Iterables\n",
    "\n",
    "- All iterators are iterable, but not all iterables are iterators.\n",
    "- Iterators have state. They remember where they left off and can continue.\n",
    "- Lists and tuples are iterable but they are not iterators.\n",
    "- The `iter()` function can turn an iterable into an iterator.\n",
    "- Iterators have a `__next__()` method. \n",
    "- The builtin `next()` function will advance an iterator and return its 'next' value.\n",
    "- [Iterators and Iterables | YouTube.com](https://youtu.be/jTYiNjvnHZY) Corey Schafer\n",
    "\n",
    "\n",
    "## Itertools Module\n",
    "- `count()`\n",
    "- `cycle()`\n",
    "- `combinations()`\n",
    "- `combinations_with_replacement()`\n",
    "- `permutation()`\n",
    "- `islice()`\n",
    "- `accumulate()`\n",
    "- `starmap()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9i7NDP96-iti"
   },
   "source": [
    "## Iterable vs Iterator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vAxHrVq3y_O9"
   },
   "outputs": [],
   "source": [
    "def is_iterable(obj):\n",
    "    \"\"\" Returns True if the object is iterable. \"\"\"\n",
    "    return '__iter__' in dir(obj)\n",
    "\n",
    "\n",
    "def is_iterator(obj):\n",
    "    \"\"\" Returns True if the object is an iterator. \"\"\"\n",
    "    return '__next__' in dir(obj)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "2V6htu-K8De7",
    "outputId": "0bcbac9f-738f-4e4e-8583-a4467179a0b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iterable: True\n",
      "Iterator: False\n"
     ]
    }
   ],
   "source": [
    "print(\"Iterable:\", is_iterable(range(10)))\n",
    "print(\"Iterator:\", is_iterator(range(10)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_Kseyz_G-b8l"
   },
   "source": [
    "Turn an iterable into an iterator with the `iter()` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "id": "S5ReqnCw749D",
    "outputId": "311b97db-6d94-4fe3-fd47-ead758be1a08"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iterable: True\n",
      "Iterator: True\n"
     ]
    }
   ],
   "source": [
    "print(\"Iterable:\", is_iterable(iter(range(10))))\n",
    "print(\"Iterator:\", is_iterator(iter(range(10))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mGTQ2hWL9gwI"
   },
   "source": [
    "Above `iter(range(start, stop, step))` is similar to `itertools.count(start, step)` below. Both can take a starting point and a step parameter and both are iterable. However the range function requires a stopping point and count does not. More importantly, count is an iterator, and range is not - unless we make it so."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TRjQPr8qO8Zg"
   },
   "source": [
    "### Iterators Can Continue - this is the key to iterators!\n",
    "\n",
    "Plain iterables can not do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dpCTIbYKeNrT"
   },
   "outputs": [],
   "source": [
    "import itertools"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "id": "EL8neUQa6Dmt",
    "outputId": "91276d21-6927-4a54-e10e-7b2effbbc1b7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n",
      "10\n"
     ]
    }
   ],
   "source": [
    "counter = itertools.count(1)\n",
    "for _ in range(10):\n",
    "    print(next(counter))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "id": "cs0XaY1_6Xz7",
    "outputId": "3b73d25a-b97b-4465-a864-1df0631b9e7e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11\n",
      "12\n",
      "13\n",
      "14\n",
      "15\n",
      "16\n",
      "17\n",
      "18\n",
      "19\n",
      "20\n"
     ]
    }
   ],
   "source": [
    "# The counter knows where it stopped, and can continue\n",
    "# This is the special sause of being an iterator.\n",
    "for _ in range(10):\n",
    "    print(next(counter))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QtCdpPX5Nk-R"
   },
   "source": [
    "Cycle will continue rotating through an iterable indefinately. Since cycle is an iterator it will remember where it stops and can continue on from there at a later time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "gOpIwlC9NO_Y",
    "outputId": "388d8d4c-991b-4d2e-ea6b-e7e546f67ad8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Alpha\n",
      "Beta\n",
      "Gamma\n",
      "Alpha\n"
     ]
    }
   ],
   "source": [
    "it = itertools.cycle((\"Alpha\", \"Beta\", \"Gamma\"))\n",
    "\n",
    "for _ in range(4):\n",
    "    print(next(it))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Rq-sIJKeOgtq"
   },
   "source": [
    "Notice below how the second for loop will begin with \"Beta\" since we last saw \"Alpha\" and \"Beta\" is next in the sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 85
    },
    "colab_type": "code",
    "id": "owQC2ahON-71",
    "outputId": "7ce5c36e-5dd2-49d4-f48d-1d2b529ccf36"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beta\n",
      "Gamma\n",
      "Alpha\n",
      "Beta\n"
     ]
    }
   ],
   "source": [
    "for _ in range(4):\n",
    "    print(next(it))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LHeNcgnTPc_J"
   },
   "source": [
    "If we did this again, what would it start with?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "id": "e-kqSmphPutM",
    "outputId": "5eecd853-57e3-460b-ea26-aa4aa4b18632"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "my_iter_range = iter(range(10))\n",
    "\n",
    "for itm in my_iter_range:\n",
    "    print(itm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LLYOlRGaL38K"
   },
   "outputs": [],
   "source": [
    "for itm in my_iter_range:\n",
    "    print(itm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "aQQe7m38L67k"
   },
   "source": [
    "Nothing is printed because `my_iter_range` is exhausted!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UN1Atp9PGsFd"
   },
   "source": [
    "### Unpacking with Star\n",
    "\n",
    "Like sequences, iterators can be unpacked with the `*`. Be careful not to unpack an infinite iterator! `*itertools.count()` would like to go on and on forever. If this happens use the `control-c` command, this will stop Python. If you let it continue, eventually your computer will run out of memory and crash. This will not hurt your computer, but it might be rather embarrassing. In some cases `control-c` won't work, and you may need to force quit Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 119
    },
    "colab_type": "code",
    "id": "Mo8Vb0oF0AIw",
    "outputId": "d0350dac-accd-4006-b5de-bc159325f993"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n",
      "(1, 3)\n",
      "(1, 4)\n",
      "(2, 3)\n",
      "(2, 4)\n",
      "(3, 4)\n"
     ]
    }
   ],
   "source": [
    "it = itertools.combinations((1,2,3,4), 2)\n",
    "print(*it, sep='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 187
    },
    "colab_type": "code",
    "id": "EESGV64HzVlo",
    "outputId": "c2c396bb-a2af-48fa-ccfb-f6bc2b007ac7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 1)\n",
      "(1, 2)\n",
      "(1, 3)\n",
      "(1, 4)\n",
      "(2, 2)\n",
      "(2, 3)\n",
      "(2, 4)\n",
      "(3, 3)\n",
      "(3, 4)\n",
      "(4, 4)\n"
     ]
    }
   ],
   "source": [
    "it = itertools.combinations_with_replacement((1,2,3,4), 2)\n",
    "print(*it, sep='\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 221
    },
    "colab_type": "code",
    "id": "e06EZaWn0IZK",
    "outputId": "549b7290-2d36-44a5-a580-d85e19783a70"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n",
      "(1, 3)\n",
      "(1, 4)\n",
      "(2, 1)\n",
      "(2, 3)\n",
      "(2, 4)\n",
      "(3, 1)\n",
      "(3, 2)\n",
      "(3, 4)\n",
      "(4, 1)\n",
      "(4, 2)\n",
      "(4, 3)\n"
     ]
    }
   ],
   "source": [
    "it = itertools.permutations((1,2,3,4), 2)\n",
    "print(*it, sep='\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0pNN2vlfLG5B"
   },
   "source": [
    "### Slice An Iterator\n",
    "\n",
    "`itertools.islice()` is used to slice an iterator. It's pronounced \"i-slice\" not \"is-lice\" - if you're curious."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "N11HEZs23W3u",
    "outputId": "100d979e-3b86-48f6-9f14-36e8d51b54de"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100\n"
     ]
    }
   ],
   "source": [
    "start, stop, step = 50, 101, 5\n",
    "it = itertools.islice(itertools.count(), start, stop, step)\n",
    "print(*it, sep=', ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-Of7BeyUTT1B"
   },
   "source": [
    "This is an amazing ability and saves us from casting the iterator to a list so we can slice it. Casting would take more time and memory - possibly a lot more. It also pretty much defeats the purpose of using an iterator as you can't call next on a list or continue where it left off. While debugging, however, it can be very useful to cast an iterator to a list just to see its guts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UvcFYXP1bmpE"
   },
   "source": [
    "### Partial Sum\n",
    "\n",
    "Partial Sum can be used to transform relative weights into cumulative weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aLfXTvUual8O"
   },
   "outputs": [],
   "source": [
    "import operator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "K3hgW-55cp7O",
    "outputId": "5ec5ce53-4bde-4944-826b-d9e747c0e17b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n"
     ]
    }
   ],
   "source": [
    "# Relative\n",
    "rel_weights = list(range(1, 11))\n",
    "print(rel_weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "qh4FSd5MKQlc",
    "outputId": "54cfd18a-a9ef-4c45-faa2-bc5d3739daba"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 3, 6, 10, 15, 21, 28, 36, 45, 55]\n"
     ]
    }
   ],
   "source": [
    "# Cumulative\n",
    "partial_sum = itertools.accumulate(rel_weights, operator.add)\n",
    "cum_weights = list(partial_sum)\n",
    "print(cum_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JWDXQuP2btcv"
   },
   "source": [
    "### Adjacent Difference\n",
    "Adjacent Difference can be used to transform cumulative weights back into relative weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "id": "YmsxsvM8UKDr",
    "outputId": "4917afb3-026f-42fd-c1b0-227bc63e1321"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n"
     ]
    }
   ],
   "source": [
    "# Back to Relative\n",
    "adjacent_diff = itertools.starmap(\n",
    "    operator.sub, zip(cum_weights, [0] + cum_weights))\n",
    "rel_weights = list(adjacent_diff)\n",
    "print(rel_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rDYCup6hwAoj"
   },
   "source": [
    "## Groupby Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JGuyuHCgwDBA"
   },
   "outputs": [],
   "source": [
    "party_of_casters = [\n",
    "    {\n",
    "        \"name\": \"Gandolf\",\n",
    "        \"class\": \"Wizard\",\n",
    "        \"alignment\": \"Good\",\n",
    "    }, {\n",
    "        \"name\": \"Habius\",\n",
    "        \"class\": \"Cleric\",\n",
    "        \"alignment\": \"Evil\",\n",
    "    }, {\n",
    "        \"name\": \"Merlin\",\n",
    "        \"class\": \"Wizard\",\n",
    "        \"alignment\": \"Good\",\n",
    "    }, {\n",
    "        \"name\": \"Raven\",\n",
    "        \"class\": \"Druid\",\n",
    "        \"alignment\": \"Neutral\",\n",
    "    }, {\n",
    "        \"name\": \"Morgause\",\n",
    "        \"class\": \"Sorceress\",\n",
    "        \"alignment\": \"Evil\",\n",
    "    }, {\n",
    "        \"name\": \"Jinx\",\n",
    "        \"class\": \"Warlock\",\n",
    "        \"alignment\": \"Evil\",\n",
    "    }, {\n",
    "        \"name\": \"Zoaster\",\n",
    "        \"class\": \"Cleric\",\n",
    "        \"alignment\": \"Good\",\n",
    "    },\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "id": "oVjrs6XNxyfe",
    "outputId": "1dce4fdc-00bb-4651-fcad-5b40959444f9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cleric: Habius, Zoaster\n",
      "Druid: Raven\n",
      "Sorceress: Morgause\n",
      "Warlock: Jinx\n",
      "Wizard: Gandolf, Merlin\n"
     ]
    }
   ],
   "source": [
    "party_of_casters = sorted(\n",
    "    party_of_casters, key=lambda p: (p['class'], p['name']))\n",
    "\n",
    "class_groups = itertools.groupby(\n",
    "    party_of_casters, lambda p: p['class'])\n",
    "\n",
    "for class_name, players in class_groups:\n",
    "    print(f\"{class_name}:\", end=' ')\n",
    "    print(*(p['name'] for p in players), sep=', ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 68
    },
    "colab_type": "code",
    "id": "9ZwhRiUQyHTT",
    "outputId": "86d00ca3-1c2d-4217-9b4c-c872dc146286"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evil: Habius, Jinx, Morgause\n",
      "Good: Gandolf, Merlin, Zoaster\n",
      "Neutral: Raven\n"
     ]
    }
   ],
   "source": [
    "party_of_casters = sorted(\n",
    "    party_of_casters, key=lambda p: (p['alignment'], p['name']))\n",
    "\n",
    "align_groups = itertools.groupby(\n",
    "    party_of_casters, lambda p: p['alignment'])\n",
    "\n",
    "for align, players in align_groups:\n",
    "    print(f\"{align}:\", end=' ')\n",
    "    print(*(p['name'] for p in players), sep=', ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n1NmDgto1Ysy"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "13_iterators.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}