Trump (#60)

Author: janhohenheim

main.tex

@@ -117,7 +117,7 @@
 		\subsection{Definition}
 			\subfile{sections/"Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks/definition"}
 		\subsection{Implementation}
-			.
+			\subfile{sections/"Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks/implementation"}
 	\newpage
 
 

sections/Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks/definition.tex

@@ -4,14 +4,24 @@
 	\item{Weights of first dimension pooler}
 	\item{Weights of convolution pooler}
 	\item{Weights of second dimension pooler}
+	\item{Weights of shortcut projection (if needed)}
 	\item{Downscaled number of dimensions in each residual block}
 	\item{Upscaled number of dimensions in each residual block}
 	\item{Number of convolutions in each residual block}
 	\item{Total number of residual blocks}
 \end{enumerate}  
-Through traditional means we can adjust the parameters 1 to 3. \\
-Numbers 4 to 7 are predefined in ResNet. 
-Their exact values are defined empirically and experimentally. This is of course suboptimal, as we already asserted in chapter two.\\
+Through traditional means we can adjust the parameters 1 to 4. \\
+Numbers 5 to 8 are predefined in ResNet. 
+Their exact values are defined empirically and experimentally. This is of course suboptimal, as we already asserted in chapter two.
+
 We think NEAT can optimize these by encoding them as genes in the genome. \\
 However, because of the nature of our smallest building blocks, it doesn't make sense to store these genomes in a per-connection basis.\\
-All parameters can be described as state of a residual block. For the last one, we just abstract it as a link to the next block. If the algorithm decides to add a new residual block, it can be inserted in a random existing link.\\
+All parameters can be described as state of a residual block. For the last one, we just abstract it as a link to the next block. If the algorithm decides to add a new residual block, it can be inserted in a random existing link.
+
+For the parameter tuning, we treat numbers 1 to 4 as a big vector of weights inside the genome of the residual block and apply the same chances and rules of change to them as in standard NEAT , which are:
+\begin{itemize}
+	\item{Chance of selecting this genome to change weights: 80\%}
+	\item{Chance for each weight to be uniformly perturbed: 90\%}
+	\item{Chance for each weight to be set to a random new value: 10\%}
+\end{itemize}
+\cite{Stanley2002}

sections/Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks/implementation.tex

@@ -0,0 +1,4 @@
+While programming according to our algorithm, we continuously tested our code.\\
+We unfortunately found out during one of those tests, that our implementation of matrix multiplication while applying a filter is not nearly fast enough to process high quality scans of mammographies.
+
+A simple test with 32 by 32 images confirmed our fear: Deep networks with a dimensionality higher than 10 are not realistically computable in a given time. By contrast, the deepest ResNet uses more than 1000 dimensioniona in its lowest layers. Given that genetic algorithms go even further by not training one network, but 100 at a time, and considering our limited time, we had to halt further research.