DeepNorm (labmlai#114)

Author: vpj

.gitignore

@@ -13,4 +13,5 @@ labml_samples
 data
 logs
 html/
-diagrams/
+diagrams/
+.comet.config

.labml.yaml

@@ -0,0 +1,21 @@
+indicators:
+  - class_name: Scalar
+    is_print: false
+    name: param.*
+    options:
+      comet: false
+  - class_name: Scalar
+    is_print: false
+    name: grad.*
+    options:
+      comet: false
+  - class_name: Scalar
+    is_print: false
+    name: module.*
+    options:
+      comet: false
+  - class_name: Scalar
+    is_print: false
+    name: optim.*
+    options:
+      comet: false

docs/index.html

@@ -69,7 +69,7 @@
             </div>
             <h1><a href="index.html">labml.ai Annotated PyTorch Paper Implementations</a></h1>
 <p>This is a collection of simple PyTorch implementations of neural networks and related algorithms. <a href="https://github.com/labmlai/annotated_deep_learning_paper_implementations">These implementations</a> are documented with explanations, and the <a href="index.html">website</a> renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.</p>
-<p><img alt="Screenshot" src="https://nn.labml.ai/dqn-light.png"></p>
+<p><img alt="Screenshot" src="dqn-light.png"></p>
 <p>We are actively maintaining this repo and adding new implementations. <a href="https://twitter.com/labmlai"><img alt="Twitter" src="https://img.shields.io/twitter/follow/labmlai?style=social"></a> for updates.</p>
 <h2>Modules</h2>
 <h4>✨ <a href="transformers/index.html">Transformers</a></h4>
@@ -126,13 +126,14 @@ <h4>✨ <a href="optimizers/index.html">Optimizers</a></h4>
 <li><a href="optimizers/noam.html">Noam Optimizer</a> </li>
 <li><a href="optimizers/radam.html">Rectified Adam Optimizer</a> </li>
 <li><a href="optimizers/ada_belief.html">AdaBelief Optimizer</a></li></ul>
-<h4>✨ <a href="https://nn.labml.ai/normalization/index.html">Normalization Layers</a></h4>
-<ul><li><a href="https://nn.labml.ai/normalization/batch_norm/index.html">Batch Normalization</a> </li>
-<li><a href="https://nn.labml.ai/normalization/layer_norm/index.html">Layer Normalization</a> </li>
-<li><a href="https://nn.labml.ai/normalization/instance_norm/index.html">Instance Normalization</a> </li>
-<li><a href="https://nn.labml.ai/normalization/group_norm/index.html">Group Normalization</a> </li>
-<li><a href="https://nn.labml.ai/normalization/weight_standardization/index.html">Weight Standardization</a> </li>
-<li><a href="https://nn.labml.ai/normalization/batch_channel_norm/index.html">Batch-Channel Normalization</a></li></ul>
+<h4>✨ <a href="normalization/index.html">Normalization Layers</a></h4>
+<ul><li><a href="normalization/batch_norm/index.html">Batch Normalization</a> </li>
+<li><a href="normalization/layer_norm/index.html">Layer Normalization</a> </li>
+<li><a href="normalization/instance_norm/index.html">Instance Normalization</a> </li>
+<li><a href="normalization/group_norm/index.html">Group Normalization</a> </li>
+<li><a href="normalization/weight_standardization/index.html">Weight Standardization</a> </li>
+<li><a href="normalization/batch_channel_norm/index.html">Batch-Channel Normalization</a> </li>
+<li><a href="normalization/deep_norm/index.html">DeepNorm</a></li></ul>
 <h4>✨ <a href="distillation/index.html">Distillation</a></h4>
 <h4>✨ <a href="adaptive_computation/index.html">Adaptive Computation</a></h4>
 <ul><li><a href="adaptive_computation/ponder_net/index.html">PonderNet</a></li></ul>
@@ -146,7 +147,7 @@ <h3>Citing LabML</h3>
 <span class="w"> </span><span class="na">author</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">{Varuna Jayasiri, Nipun Wijerathne}</span><span class="p">,</span><span class="w"></span>
 <span class="w"> </span><span class="na">title</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">{labml.ai Annotated Paper Implementations}</span><span class="p">,</span><span class="w"></span>
 <span class="w"> </span><span class="na">year</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">{2020}</span><span class="p">,</span><span class="w"></span>
-<span class="w"> </span><span class="na">url</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">{https://nn.labml.ai/}</span><span class="p">,</span><span class="w"></span>
+<span class="w"> </span><span class="na">url</span><span class="w"> </span><span class="p">=</span><span class="w"> </span><span class="s">{}</span><span class="p">,</span><span class="w"></span>
 <span class="p">}</span><span class="w"></span></code></pre>
 
         </div>