Some error correction and improvements

README.md

@@ -49,10 +49,14 @@ Based on Python Scrapy framework.
 
 ## Installation
 
-Clone the project and build docker images involved in docker-compose.
+Clone the project and change the ```yourdomain:yourport``` of the web-general to your ```host:port``` in ```web-general\templates\layout_footer.html```.
 
-    docker-compose build
-    docker-compose up -d
+Update the list of onion addresses you want to set as the crawler seed in ```onions_list\onions.txt```. Current addresses are taken from [Ahmia's list](https://ahmia.fi/onions/)
+
+Build docker images involved in docker-compose.
+
+    docker compose build
+    docker compose up -d
 
 Build and run the scraper.
 

docker-compose.yml

@@ -32,7 +32,7 @@ services:
         soft: -1
         hard: -1
     volumes:
-      - .data/elasticsearch1:/usr/share/elasticsearch/data
+      - .data/elasticsearch1:/var/lib/elasticsearch/data
       - ./elk-certs:$ELK_CERTS_DIR
     ports:
       - "127.0.0.1:9200:9200"
@@ -67,7 +67,7 @@ services:
         soft: -1
         hard: -1
     volumes:
-      - .data/elasticsearch2:/usr/share/elasticsearch/data
+      - .data/elasticsearch2:/var/lib/elasticsearch/data
       - ./elk-certs:$ELK_CERTS_DIR
     # ports:
     #   - "127.0.0.1:9201:9200"

lib/tor_elasticsearch.py

@@ -29,7 +29,7 @@
         # 'nl': spacy.load('nl_core_news_md'),
         'en': spacy.load('en_core_web_lg'),
         # 'fr': spacy.load('fr_core_news_md'),
-        'de': spacy.load('de_core_news_md'),
+        # 'de': spacy.load('de_core_news_md'),
         # 'el': spacy.load('el_core_news_md'),
         # 'it': spacy.load('it_core_news_md'),
         # 'ja': spacy.load('ja_core_news_md'),
@@ -39,7 +39,7 @@
         # 'pl': spacy.load('pl_core_news_md'),
         # 'pt': spacy.load('pt_core_news_md'),
         # 'ro': spacy.load('ro_core_news_md'),
-        'ru': spacy.load('ru_core_news_md'),
+        # 'ru': spacy.load('ru_core_news_md'),
         # 'es': spacy.load('es_core_news_md'),
     }
 except Exception as ex: