Docker Swarm Implementation

[yosserO]

Description

How to use it:

• Initialize a swarm by running: docker swarm init
• Add at least one worker to the swarm.
• Deploy a new Stack by running: docker stack deploy -c <path-to-docker-compose-file> STACK_NAME.
  • We added an example docker-compose file under zalenium/docs/docker/docker-compose-swarm.yml.
  • An attachable overlay network must be declared in the docker-compose file. This overlay network must be passed to zalenium as the configuration parameter "--swarmOverlayNetwork". In our example we are declaring the overlay network "zalenium". We named our stack "tau". So the corresponding value for the configuration parameter "--swarmOverlayNetwork" is "tau_zalenium".
• We have two different constraints for the hub and its nodes:
  • The hub has the constraint node.role == manager and will only be deployed on the manager. (Take a look at the example docker-compose file).
  • The nodes have the constraint node.role==worker. This constraint is currently hard coded in the SwarmContainerClient.java Class.
• Logs and Videos are currently not supported.
• In order to execute commands on workers we are using this image datagridsys/skopos-plugin-swarm-exec:latest.
• We tested the solution in our own environment but we will be thankful if you test it for your own and give us feedback.
• We update the documention later after positive feedback.

Motivation and Context

Adapt zalenium to be able to run in a docker swarm.

How Has This Been Tested?

We tested it manually. For more details please check the Description.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[codecov-io]

Codecov Report

Merging #907 into pr-907-swarm will decrease coverage by 4.03%.
The diff coverage is 3.95%.

@@                Coverage Diff                 @@
##             pr-907-swarm     #907      +/-   ##
==================================================
- Coverage           58.51%   54.47%   -4.04%     
- Complexity            642      646       +4     
==================================================
  Files                  54       56       +2     
  Lines                4127     4455     +328     
  Branches              370      413      +43     
==================================================
+ Hits                 2415     2427      +12     
- Misses               1493     1805     +312     
- Partials              219      223       +4

[tstern]

During testing our solution, we've discovered a bug which keeps the Thread "AutoStartProxyPoolPoller" forever in the state "WAITING". We do use many docker operations to run Zalenium in a docker swarm. When docker operations run simultaneous the above described error occurs.

This problem is also described in the following issues:

• Multithreaded, concurrent and intensive use often cause stale containers to remain up forever #808
• Threads stuck when jersey client with apache connector is used concurrently eclipse-ee4j/jersey#3772

As a solution we run docker operations in synchronized blocks to prevent simultaneous access to docker.
See commit: #69ae6ab3

[diemol]

Hi @yosserO,

I am sorry I have not had time to review this properly, would it be possible to update this PR with the latest changes from master? I will have time this weekend to review and if everything works out we could release in 1-2 weeks as a beta.

[tstern]

I've merged the master into our implementation branch. On our side everything seems to be ok.

[diemol]

I will merge this to a protected branch so we can run all the CI tests, looks good so far.

[diemol]

Let's follow up in #982