GAN example

[dmccloskey]

Description

GAN example

Objectives

• add Leaky ReLU activation function
• BinaryUnitScale (-1 to 1 instead of 0 to 1)
• normalization from -1 to 1 for MNIST
• OneSideLabelSmoother (One sided label smoothing)
• example using 2-3 time steps for generator and discriminator networks with specific network part gradient updates(?)
• implementation of gradient breaks during back propogation

References

• https://medium.com/ai-society/gans-from-scratch-1-a-deep-introduction-with-code-in-pytorch-and-tensorflow-cb03cdcdba0f
• https://github.com/diegoalejogm/gans/blob/master/1.%20Vanilla%20GAN%20PyTorch.ipynb
• https://pytorch.org/docs/master/nn.html#torch.nn.LeakyReLU
• C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the Inception Architecture for
  Computer Vision. ArXiv e-prints, December 2015.
• convolution example: https://becominghuman.ai/understanding-and-building-generative-adversarial-networks-gans-8de7c1dc0e25

[dmccloskey]

Training: https://github.com/soumith/ganhacks

[dmccloskey]

Current training scheme will make it extremely difficult to implement a GAN.