Questions about deployment

[johnseekins]

Hello, Argus sounds like a really promising product to add on top of OpenTSDB. But I'm having some trouble deploying it. So...a series of questions around deployment, if you don't mind:

• When pushing the WebServices war to Tomcat
  • Does it matter if we use tomcat7 or tomcat8? Your internal stuff seems to imply tomcat7, but the docs don't indicate.
  • How do we actually set argus.config.public.location or argus.config.private.location in tomcat? I assume something in CATALINA_OPTS?
  • When running the exec.war, it appears that Arugs isn't seeing the argus.properties file I've set. Is this expected behavior?:

#:~/Argus-2.21.3# java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 -Dargus.config.public.location=/root/Argus-2.21.3/conf/argus.properties -jar ArgusWebServices/target/argus-webservices-2.21.3-war-exec.jar -httpPort 8047
Listening for transport dt_socket at address: 5005
Feb 27, 2019 3:52:27 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8047"]
Feb 27, 2019 3:52:27 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Tomcat
Feb 27, 2019 3:52:27 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.37
Feb 27, 2019 3:52:28 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/root/Argus-2.21.3/.extract/webapps/argusws/WEB-INF/lib/javax.servlet-3.1.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Feb 27, 2019 3:52:28 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/root/Argus-2.21.3/.extract/webapps/argusws/WEB-INF/lib/jsp-api-2.1-6.1.14.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/el/Expression.class
Feb 27, 2019 3:52:28 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/root/Argus-2.21.3/.extract/webapps/argusws/WEB-INF/lib/servlet-api-2.5-6.1.14.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
15:52:40.298 [localhost-startStop-1] INFO com.salesforce.dva.argus.ws.listeners.ArgusWebServletListener - Initializing Argus web services.
15:52:40.306 [localhost-startStop-1] WARN com.salesforce.dva.argus - Unable to load properties file "/root/Argus-2.21.3/conf/argus.properties". Reason: /root/Argus-2.21.3/conf/argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.605 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.607 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.607 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.608 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.608 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.608 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.609 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.609 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.609 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.610 | ost-startStop-1 |  WARN ] Unable to load properties file "notifier.properties". Reason: notifier.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.610 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.610 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:40.611 | ost-startStop-1 |  WARN ] Unable to load properties file "argus.properties". Reason: argus.properties (No such file or directory)
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:41.232 | ost-startStop-1 |  INFO ] Starting SystemMain
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:41.233 | ost-startStop-1 |  INFO ] Argus version 2.21.3 build 2019-02-26 23:25.
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:41.233 | ost-startStop-1 |  INFO ] Copyright Salesforce.com, 2019.
[ ARGUS | *NULLUSER* | *NULLTXID* | 2019-02-27 15:52:41.234 | ost-startStop-1 |  INFO ] SystemMain started.

• Web UI
  • There is no documentation I can find around the web UI at all.
  • Based on reading through some of the docker config files, I've determined it is a Node project.
  • I'm not seeing anything about how to properly run the web UI in production, only npm start, which loudly declares the job to be a "development" server. Is this expected behavior?
  • There is no indication of what a good configuration for the web UI is.
  • Currently, it appears the web UI and the web services war default to 8080 for connections. Is there a way to change this?
• Service Works/Clients
  • Again, there's no real documentation around these. What is the base expectation of service workers for a functioning system?
  • There's no list I could find (without delving into source code) of what functional worker "types" might be.
• General
  • Your documentation around base properties does not clarify what settings belong in files like cache.properties, mq.properties, or argus.properties.
  • Again, by reading through some examples from your docker builds, I've determined some settings that belong in argus.properties, but I'm unclear on what other settings might be there. Any guides on what nobs we can actually turn?
  • Redis is mentioned frequently as recommended as a cache, and I'd love to use it, but our Redis cluster has AUTH on it, and there doesn't appear to be a way to add a password to the config for Redis. Any solution for this?

I hope I'm not coming off as too complain-y. I'm just generally lost on trying to get this system set up. Which is frustrating, because it sounds like a great tool.

[johnseekins]

Things I've answered for myself by digging in to code and searching old (closed) tickets:

When running the exec.war, it appears that Arugs isn't seeing the argus.properties file I've set. Is this expected behavior?
Yes. Based on #476, there are multiple config settings we have to add to keep those warnings from firing. This is not documented. (#476 also has some basic config file examples that the rest of the documentation is sorely lacking.)

Does it matter if we use tomcat7 or tomcat8? Your internal stuff seems to imply tomcat7, but the docs don't indicate.
Everything I've found indicates we must use tomcat7. This is not documented.

How do we actually set argus.config.public.location or argus.config.private.location in tomcat? I assume something in CATALINA_OPTS?
In Debian/Ubuntu, we can edit /etc/default/tomcat7 and add something akin to JAVA_OPTS="-Djava.awt.headless=true -XX:+UseConcMarkSweepGC -Dargus.config.public.location=/etc/argus/argus.properties -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n" to point to a file in /etc/argus/argus.properties and also configure other (apparently) recommended arguments. While this is also not documented, I understand configs would differ based on deployment patterns.

Web UI

In the case of WebServices (the tomcat app) we can set tomcat configs to change the port. In the case of the web UI, we can use apache/nginx to host the compiled dist folder and set whatever port we'd like. An example apache2 config:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
<VirtualHost *:8090> 
  #ProxyPreserveHost Off
  #ProxyRequests Off
  ServerName $(hostname)
  #ServerAlias example.com
  DocumentRoot /var/www/argusweb
  #Alias "/argusws" 
  Header set Access-Control-Allow-Origin "*"
  ProxyPass /argusws  http://localhost:8080/argusws
  ProxyPassReverse /argusws http://localhost:8080/argusws
</VirtualHost>

In oder for this to work completely, we have to also proxy data for the API through apache/nginx.

[johnseekins]

New question list:

• Service Workers/Clients

  • Again, there's no real documentation around these. What is the base expectation of service workers for a functioning system?
  • There's no list I could find (without delving into source code) of what functional worker "types" might be.

• General

  • Your documentation around base properties does not clarify what settings belong in files like cache.properties, mq.properties, or argus.properties. In particular, what config options actually exist and can be set?
  • Again, by reading through some examples from your docker builds, I've determined some settings that belong in argus.properties, but I'm unclear on what other settings might be there. Any guides on what nobs we can actually turn?
  • Redis is mentioned frequently as recommended as a cache, and I'd love to use it, but our Redis cluster has AUTH on it, and there doesn't appear to be a way to add a password to the config for Redis. Any solution for this?

• Kafka connection

  • There's no documentation around what topics are needed for Kafka usage to function. As we don't like to enable auto topic creation, what topics are actually needed?

• Web UI

  • [This commit] (d797bf1) completely breaks access to the alerts page in the UI. Can this be reverted? Or at least can the change be explained?