Added robots.txt

CrypTools · Mar 11, 2018 · 4ca9e5b · 4ca9e5b
1 parent 26a5bf6
commit 4ca9e5b
Showing 1 changed file with 7 additions and 0 deletions.
diff --git a/robots.txt b/robots.txt
@@ -0,0 +1,7 @@
+# www.robotstxt.org/
+
+# Allow crawling of all content
+User-agent: *
+Disallow:
+
+Sitemap: https://cryptools.github.io/sitemap.txt