I installed Hippocampe on both RedHat 7 and Ubuntu 18.04 but it is not working only the GUI works

[samsowa]

Work Environment

Question	Answer
OS version (server)	Ubuntu 18.04, RedHat 7,
Hippocampe version / git hash
Package Type	From source
Browser type & version	If applicable

Steps to Reproduce

1. step 1: I did fresh installed Ubuntu 18.04 and RedHat 7. I follows the documentations and installed Hippocampe again
2. step 2: I run the python app.py command on both ubuntu and the redhat and accessed the GUI successfully
3. step 3: I run the command curl -XGET 192.168.153.10:5000/hippocampe/api/v1.0/shadowbook on both ubuntu and the redhat and still got below error message:
   When I run thei command curl -XGET 192.168.153.10:5000/hippocampe/api/v1.0/shadowbook, I am getting below error:
   **

<title>500 Internal Server Error</title>

Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

**

These the information from my RedHat 7 Server
First running Hippocampe
[iranalyst@hippo core]$ sudo python app.py

• Serving Flask app "app" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

Initial hippo.conf file
hippo.conf
[api]
debug : False
host : 0.0.0.0
port : 5000
threaded : True

[elasticsearch]
ip : 127.0.0.1
port : 9200
#indexNameES MUST BE LOWERCASE
indexNameES : hippocampe
typeNameESSource : source
typeNameESNew : new
typeNameESJobs: jobs

[shadowbook]
nbThreadPerCPU : 2

[freshness]
#in days
threshold : 1

[schedReport]
#in hours
threshold: 12

After added IP Address
hippo.conf
[api]
debug : False
host : 192.168.153.10
port : 5000
threaded : True

[elasticsearch]
ip : 127.0.0.1
port : 9200
#indexNameES MUST BE LOWERCASE
indexNameES : hippocampe
typeNameESSource : source
typeNameESNew : new
typeNameESJobs: jobs

[shadowbook]
nbThreadPerCPU : 2

[freshness]
#in days
threshold : 1

[schedReport]
#in hours
threshold: 12

I ran the python app.py command again and below is the is result
[iranalyst@hippo core]$ sudo python app.py

• Serving Flask app "app" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://192.168.153.10:5000/ (Press CTRL+C to quit)

Same error message shown below when I run curl -XGET 192.168.153.10:5000/hippocampe/api/v1.0/shadowbook

<title>500 Internal Server Error</title>

Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Also, below are the logs I get when I run curl -XGET 192.168.153.10:5000/hippocampe/api/v1.0/shadowbook
[iranalyst@hippo core]$ sudo python app.py

• Serving Flask app "app" (lazy loading)
• Environment: production
  WARNING: This is a development server. Do not use it in a production deployment.
  Use a production WSGI server instead.
• Debug mode: off
• Running on http://192.168.153.10:5000/ (Press CTRL+C to quit)
  [2019-10-31 14:24:26,492] ERROR in app: Exception on /hippocampe/api/v1.0/shadowbook [GET]
  Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/flask/app.py", line 2446, in wsgi_app
  response = self.full_dispatch_request()
  File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
  rv = self.handle_user_exception(e)
  File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1820, in handle_user_exception
  reraise(exc_type, exc_value, tb)
  File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
  rv = self.dispatch_request()
  File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1935, in dispatch_request
  return self.view_functionsrule.endpoint
  File "app.py", line 168, in shadowbookService
  if 'error' in reportJob:
  TypeError: argument of type 'NoneType' is not iterable
  192.168.153.10 - - [31/Oct/2019 14:24:26] "GET /hippocampe/api/v1.0/shadowbook HTTP/1.1" 500 -

Below are the Hippocampe logs
2019-10-31 14:20:30,591 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:30,921 :: services :: INFO :: type service requested
2019-10-31 14:20:30,921 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:30,925 :: services :: INFO :: sizeByType service requested
2019-10-31 14:20:30,926 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:30,927 :: services :: INFO :: sizeBySources service requested
2019-10-31 14:20:30,927 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:30,936 :: services :: INFO :: monitorSources service requested
2019-10-31 14:20:30,936 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:31,180 :: services.modules.common.ES :: INFO :: ES.checkData launched
2019-10-31 14:20:31,181 :: services.modules.common.ES :: INFO :: []
2019-10-31 14:20:31,238 :: services.modules.common.ES :: INFO :: ES.checkData launched
2019-10-31 14:20:31,238 :: services.modules.common.ES :: INFO :: ['sourceType']
2019-10-31 14:20:31,239 :: services.modules.common.ES :: INFO :: ES.checkData launched
2019-10-31 14:20:31,240 :: services.modules.common.ES :: INFO :: ['sourceType']
2019-10-31 14:20:31,240 :: services.modules.common.ES :: INFO :: ES.checkData launched
2019-10-31 14:20:31,244 :: services.modules.common.ES :: INFO :: ['sourceType']
2019-10-31 14:20:31,500 :: services.modules.common.ES :: INFO :: index hippocampe does not exist
2019-10-31 14:20:31,500 :: services :: ERROR :: no data
2019-10-31 14:20:31,501 :: services.modules.common.ES :: INFO :: index hippocampe does not exist
2019-10-31 14:20:31,503 :: services :: ERROR :: no data
2019-10-31 14:20:31,505 :: services :: INFO :: type service requested
2019-10-31 14:20:31,505 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:20:31,511 :: services.modules.common.ES :: INFO :: index hippocampe does not exist
2019-10-31 14:20:31,512 :: services :: ERROR :: no data
2019-10-31 14:20:31,514 :: services.modules.common.ES :: INFO :: ES.checkData launched
2019-10-31 14:20:31,514 :: services.modules.common.ES :: INFO :: []
2019-10-31 14:20:31,703 :: services.modules.common.ES :: INFO :: index hippocampe does not exist
2019-10-31 14:20:31,703 :: services :: ERROR :: no data
2019-10-31 14:20:31,705 :: services.modules.common.ES :: INFO :: index hippocampe does not exist
2019-10-31 14:20:31,706 :: services :: ERROR :: no data
2019-10-31 14:24:26,090 :: services :: INFO :: shadowbook service requested
2019-10-31 14:24:26,091 :: services.modules.common.ES :: INFO :: ES.checkES launched
2019-10-31 14:24:26,099 :: services.shadowbook :: INFO :: shadowbook.initJob launched
2019-10-31 14:24:26,490 :: services.shadowbook :: ERROR :: search() got an unexpected keyword argument 'doc_type'
Traceback (most recent call last):
File "/opt/Hippocampe/core/services/shadowbook.py", line 80, in initJob
nbOngoingJob = job.searchOngoingJob()
File "/opt/Hippocampe/core/services/modules/shadowbook/objects/Job.py", line 76, in searchOngoingJob
nbOngoingJob = self.search()
File "/opt/Hippocampe/core/services/modules/shadowbook/objects/ObjToIndex.py", line 67, in search
self.resSearch = self.es.search(index = self.indexNameES, doc_type = self.typeNameES, body = self.docSearch)
File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
return func(*args, params=params, **kwargs)
TypeError: search() got an unexpected keyword argument 'doc_type'

[phishandsheeps]

You have to use a version that match Elasticsearch for the python module "elasticsearch".

Try :
pip install elasticsearch==5.3.0

Here is a fix Dockerfile :

# Lightweight Hippocampe container
###################################################

# Build the base from J8 Alpine
FROM openjdk:8-jre-alpine

RUN apk add --update --no-cache python \
    python-dev \
    py-pip \
    git \
    curl \
    nodejs \
    nodejs-npm

RUN npm install -g bower
RUN pip install --upgrade pip && \
    pip install apscheduler \
        Configparser \
        elasticsearch==5.3.0 \
        flask \
        python-dateutil \
        requests \
        urllib3==1.23

COPY ./core /opt/Hippocampe/core
COPY docker-entrypoint.sh /

RUN adduser hippo -D
RUN chown -R hippo:hippo /opt/Hippocampe /docker-entrypoint.sh

USER hippo

RUN cd /opt/Hippocampe/core/static && bower install

ENTRYPOINT /docker-entrypoint.sh