Deployment via ECS

[ppedziwiatr]

Current configuration:

1. main instance: EC2 on t3.medium: https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#InstanceDetails:instanceId=i-0eda8ef302509bb14

Runs in node cluster mode (https://nodejs.org/api/cluster.html#cluster), started via PM2 (https://pm2.keymetrics.io/docs/usage/cluster-mode/) with -i max.
In case of t3.medium this effectively means 2 nodes.

Requires:

• .secrets/warp-wallet-jwk.json - an Arweave wallet that is used to connect with Bundlr (which is used for bundling contracts and contracts' interactions)
• .secrets/vrf-priv-key.txt - secp256k1 private key - required by the VRF feature
• vrf-pub-key.txt - secp256k1 public key - required by the VRF feature

Additional notes:
Only one worker in cluster runs the gateway tasks (e.g. transactions syncing, transactions verification)
All workers in cluster run the http server and handle both the POST and GET requests.
This is handled via gateway.lock file.

2. replica instance: EC2 on t3.micro: https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#InstanceDetails:instanceId=i-0d80ce350fba0431c
   Also runs in node cluster mode, with the --replica param. This effectively means, that node only serves data - i.e. serves only "GET" requests and does not run any tasks/jobs.

3. ALB - https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#LoadBalancers:sort=loadBalancerName sw-gw-alb with sw-gw-target-group https://eu-north-1.console.aws.amazon.com/ec2/home?region=eu-north-1#TargetGroup:targetGroupArn=arn:aws:elasticloadbalancing:eu-north-1:731675056359:targetgroup/sw-gw-target-group/d11c3b67c5addb3a
   This group has both the above EC2 instances connected.

4. Cloudfront - https://us-east-1.console.aws.amazon.com/cloudfront/v3/home?region=eu-north-1#/distributions/E2UWDPG250O74Q - https://d1o5nlqr4okus2.cloudfront.net
   with the alb and main instance node origins connected.
   All the 'post' requests are sent to the main node.
   The rest is sent to the ALB.
   Some caching policies are enabled for certain endpoints - https://us-east-1.console.aws.amazon.com/cloudfront/v3/home?region=eu-north-1#/distributions/E2UWDPG250O74Q/behaviors
   This cloudfront distribution is connected to the gateway.redstone.finance domain.

[Max-Levitskiy]

Planned changes:

1. We don't need pm2 anymore. ECS will manage horizontal scale across cluster nodes.
2. We try to read all secrets and configurations from environment variables. If an environment variable doesn't exist, use current behavior. In the future, we should use some configuration library, which gives files/environment configuration from the box.
3. gateway.lock file - we should definitely change this approach. It doesn't work in multi-node/zone/region environments. The easiest way for now - we can run two different ECS services. The first one is for the "master" node, which will have 1 instance, and an environment variable says it should do additional work. And the second one is for "worker" nodes with horizontal autoscaling.

[ppedziwiatr]

So in current env. there are in fact 3 possibilites:

1. main node that runs all the tasks AND serves both the POST and GET requests
2. main node that serves both the POST and GET requests (when gateway.lock file is present)
3. replica node, that serves only GET requests (when run with --replica param)

But yeah, in ECS env this must be handled by some kind of flag/env. variable.

If we remove the sequencer from the equation (i.e. from the gateway node) - #89 - the above could be reduced to:

1. main node that runs all the tasks AND serves both the POST and GET requests
2. replica nodes, that serves both the POST and GET requests

The sequencer-related endpoint is currently the only one, that cannot be scaled horizontally "easily".

[Max-Levitskiy]

Ok, so for compete this story, we need to complete #97 and #89 before.