lec9-visualizing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CS231n Winter 2016: Lecture 9\n",
    "## Topics\n",
    "Visualization, Deep Dream, Neural Style, Adversarial Examples\n",
    "**TODO:** add topics. hm maybe there is some plugin to do this quick?\n",
    "\n",
    "## Sources\n",
    "- video: https://www.youtube.com/watch?v=ta5fdaqDT3M"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/ta5fdaqDT3M?rel=0&amp;controls=1&amp;showinfo=0\" frameborder=\"0\" allowfullscreen></iframe>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "def play_youtube(video_id):\n",
    "    return HTML(f'<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/{video_id}?rel=0&amp;controls=1&amp;showinfo=0\" frameborder=\"0\" allowfullscreen></iframe>')\n",
    "play_youtube('ta5fdaqDT3M')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## visualize patches that maximally activate neurons\n",
    "- pick arbitrary neuron on layer N(for example pool 5)\n",
    "- pipe a lot of images and see which activates this neuron\n",
    "- then we could visualize layer by putting the image which has miximum feedback on this neuron\n",
    "\n",
    "## visualize weights\n",
    "- visualizing raw weights works only for 1st layer\n",
    "  - but here is hard to find something new and interesting. because almost always they are rotated [gabor like feature](https://en.wikipedia.org/wiki/Gabor_filter)\n",
    "  - PCA could give sin/cos like images\n",
    "  \n",
    "\n",
    "## visualize representation space (e.g. with t-SNE)\n",
    "\n",
    "[t-SNE visualization of CNN codes](https://cs.stanford.edu/people/karpathy/cnnembed/)\n",
    "- take the last CNN layer immediately before classifier. \n",
    "- it's like be n-dimention \"code\" for image - summarize content of the image\n",
    "- use t-SNE to embede it to 2d\n",
    "- pass all images and find the place in 2d for it\n",
    "- in result we will get map where nearest images have similar \"code\" in CNN and could be classified in the similar way\n",
    "\n",
    "## occusion experiments\n",
    "\n",
    "- slide patch of zeros over the image \n",
    "- look how probability of classifier change\n",
    "- we get heat map of probability\n",
    "\n",
    "## Visualizing activations\n",
    "\n",
    "- tool: http://yosinski.com/deepvis\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## deconv approaches (single backward pass)\n",
    "researches\n",
    "- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps Karen Simonyan, Andrea Vedaldi, Andrew Zisserman](https://arxiv.org/abs/1312.6034)\n",
    "- [Striving for Simplicity: The All Convolutional Net Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller](https://arxiv.org/abs/1412.6806)\n",
    "\n",
    "steps\n",
    "- get one neuron\n",
    "- set gradient for it all he rest will be 0\n",
    "- back propagate its gradient back to the image\n",
    "- we get reconstructed image\n",
    "- use guided back prob (instead)\n",
    "  - change backwarp for ReLU layer\n",
    "  \n",
    "  by default\n",
    "  $$\n",
    "  f_i^{l+1} = relu(f_i^l) = max(f_i^l, 0)\n",
    "  $$\n",
    "  $$\n",
    "  R_i^l = (f_i^l > 0) R_i^{l+1},\\ where\\ R_i^{l+1} = \\frac{\\delta f^{out}}{\\delta f_i^{l+1}}\n",
    "  $$\n",
    "  guide only possitive\n",
    "  $$\n",
    "  R_i^l = (f_i^l > 0) (R_i^{l+1} > 0) R_i^{l+1},\\ where\\ R_i^{l+1} = \\frac{\\delta f^{out}}{\\delta f_i^{l+1}}\n",
    "  $$\n",
    "  we keep attention to the positive influences.\n",
    "  \n",
    "  **ME:** interesting what we get when we show only negative influences. Btw this question was in a lecture. so we could try\n",
    "  \n",
    "### DeConvNet\n",
    "Gives much cleaner images.\n",
    "Ignore relu\n",
    "  $$\n",
    "  R_i^l = (R_i^{l+1} > 0) R_i^{l+1},\\ where\\ R_i^{l+1} = \\frac{\\delta f^{out}}{\\delta f_i^{l+1}}\n",
    "  $$\n",
    "but it works as well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## human experiment comparisons\n",
    "\n",
    "## optimization over image approaches (optimization)\n",
    "\n",
    "find an image that maximizes some class score\n",
    "- start with blank/random image \n",
    "- pass it though network\n",
    "- pick one class\n",
    "- add positive gradient \n",
    "- propagate it back to the image\n",
    "- update the image\n",
    "\n",
    "use L2 regularization\n",
    "$$\n",
    "\\arg \\max_{I} S_c(I) - \\lambda ||I||_2^2\n",
    "$$\n",
    "\n",
    "[Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps Karen Simonyan, Andrea Vedaldi, Andrew Zisserman](https://arxiv.org/abs/1312.6034)\n",
    "- forward prop image\n",
    "- find the class\n",
    "- set positive gradient (1)\n",
    "- back prob it to the image as heat map\n",
    "- we get a heat map of pixels which could increase the most the score for the class\n",
    "\n",
    "we could use `grabcut` for segmentation and crop out the most important images\n",
    "\n",
    "\n",
    "[Understanding Neural Networks Through Deep Visualization Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, Hod Lipson](https://arxiv.org/abs/1506.06579)\n",
    "differen regularization - ignore the penalty:\n",
    "$$\n",
    "\\arg \\max_{I} S_c(I)\n",
    "$$\n",
    "- blure image between updates\n",
    "- take any pixel with small norm to zero (to encourage sparsity)\n",
    "\n",
    "\n",
    "as well we could try to visualize image for different layers\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Information of the last layer\n",
    "[Understanding Deep Image Representations by Inverting Them Aravindh Mahendran, Andrea Vedaldi](https://arxiv.org/abs/1412.0035)\n",
    "\n",
    "and could we reconstruct image based on the last layer?\n",
    "$$\n",
    "x^* = \\arg \\max_{x \\in R^{H \\times W \\times C}} L(\\Phi(x), \\Phi_0) + \\lambda R(x)\n",
    "$$\n",
    "$$\n",
    "L(\\Phi(x), \\Phi_0) = \\|\\Phi(x) - \\Phi_0\\|^2\n",
    "$$\n",
    "and can do it for all layers, the higher we are the more abstruct result we get"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "continue here: https://youtu.be/ta5fdaqDT3M?t=2562"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DeepDream\n",
    "[github.com/google/deepdream](https://github.com/google/deepdream)\n",
    "\n",
    "amplify the features which activated in the network\n",
    "\n",
    "- apply jittering normalization to the destanation image\n",
    "- forward to the some layer\n",
    "- apply gradient equal to the activation (objective layer)\n",
    "- backward from that point\n",
    "- apply normalized update\n",
    "- apply jittering normalization to the destanation image\n",
    "\n",
    "for example: when network detects a little bit dog face in a image we will bust it and highlight this (dreaming) dog phase.\n",
    "it \"dreams\" dogs because it has a lot of dogs in training set: 1k classes and 100 is dogs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/DgPaCWJL7XI?rel=0&amp;controls=1&amp;showinfo=0\" frameborder=\"0\" allowfullscreen></iframe>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "play_youtube('DgPaCWJL7XI')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/oyxSerkkP4o?rel=0&amp;controls=1&amp;showinfo=0\" frameborder=\"0\" allowfullscreen></iframe>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "play_youtube('oyxSerkkP4o')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Style\n",
    "- [A Neural Algorithm of Artistic Style Leon A. Gatys, Alexander S. Ecker, Matthias Bethge](https://arxiv.org/abs/1508.06576)\n",
    "- https://github.com/jcjohnson/neural-style\n",
    "- deepart.io\n",
    "\n",
    "algorithm\n",
    "- input: content image, style image\n",
    "- content image:\n",
    "    - pass content image forward to the CNN\n",
    "    - store all activations - they are correspond to content of the image\n",
    "\n",
    "- style:\n",
    "    - pass forward to the CNN\n",
    "    - get pairwise statistics of activations in N layer (covariance matrixes)\n",
    "    - how often each pair of features come together\n",
    "        - get d dimension fiber from 3d matrix `w x h * d` \n",
    "          - we want statistic (covariance or Gram matrix) of those products - outer product and summed across spatial locations\n",
    "        - in other words: reshape layer in 3d matrix to 2d `d x w * h`\n",
    "        - use $G = V^TV$\n",
    "\n",
    "- loss function\n",
    "    - content loss (activations match content) (usually 1 layer)\n",
    "    - style loss (gram matrices of activations match style) (many layers)\n",
    "continue here: https://youtu.be/ta5fdaqDT3M?t=3218\n",
    "\n",
    "- because we don't have a lot of data here it is excellent case for [Limited-memory BFGS ](https://en.wikipedia.org/wiki/Limited-memory_BFGS).\n",
    "  - so we could use 2nd derivatives here\n",
    "\n",
    "**ME:**\n",
    "1. what if we do for content simetry modification as we have for style activation?\n",
    "  - for style we try to match pairwise neurons and ignore spatial distribution (Gram matrix)\n",
    "  - so could we ignore d dimension and just focus on distribution? \n",
    "    - maybe it will extract raw structure and ignore feature statistics which we have in the original image?\n",
    "2. could we generate raw image - by removing any style from it? And how would it look like?\n",
    "  - I could get few images with single style and try to extract:\n",
    "    - style of each of image \n",
    "    - target image itself\n",
    "    - or maybe summary style\n",
    "3. Maybe it's the same as Adversarial examples?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## \"Fool\" ConvNet / Adversarial Examples\n",
    "research\n",
    "- [Intriguing properties of neural networks Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus](https://arxiv.org/abs/1312.6199) - small distorshen change class of the image (ostrich class)\n",
    "- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images Anh Nguyen, Jason Yosinski, Jeff Clune](https://arxiv.org/abs/1412.1897) - some pattern could be class (but it doesn't look for us as this class\n",
    "- [Exploring the representation capabilities of the HOG descriptor](http://ieeexplore.ieee.org/document/6130416/?reload=true) we have this problem before, for example for HOG\n",
    "- [Explaining and Harnessing Adversarial Examples Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy](https://arxiv.org/abs/1412.6572) - \"primary cause of neural network\" vulnerability to adveserial perburbation is their linear nature (which we use in the forward path). we train only on linear manifolds and ignore huge amount of others (they are in the \"shadows\" of classified manifolds)\n",
    "- [Breaking Linear Classifiers on ImageNet by Andrej Karpathy](http://karpathy.github.io/2015/03/30/breaking-convnets/) - so it ever works for basic linear classier because we know where gradient leads us and we can change weights to any class we want\n",
    "\n",
    "how to solve it\n",
    "- get few crops of image\n",
    "- train on adversarial examples\n",
    "- replace linear function - but it will break generalization of the net\n",
    "- but there is no final solution yet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}