Configure ECS container to be bind to an existing LB Listener, responding to a specific URL

[flaviostutz]

Having one LB per container in ECS is too expensive, so we have a separate Cloudformation stack that defines a load balancer that receives requests from a bunch of different domain names and forwards requests to specific ECS container groups depending on the domain name by using lb listener rules (forwarding by host header).

For example, https://test1.example.com and https://test43.example.com both arrives at the same AWS LB, on the same :443 listener that uses a previously provisioned wildcard certificate. In this listener we have two rules, one that forwards requests to a service container group "test1" if "Host" header is "test1.example.com" and another that forwards requests to group "test43" depending on the "Host" header request.

This was the best way we managed to cut some costs but still having separate domains for each container (we have a lot of differente containers running).

How could we do something like this using compose-cli?

[ndeloof]

Even I understand some scenarios might require a dedicated Load Balancer, I actually don't get why AWS infra doesn't provide a default one with VPCs, or even a global one for general purpose uses like Platform as a Service do.

That being said, there's probably many ways to reduce costs by sharing resources with distinct stacks deployment, but this is definitively out of the scope of the Docker ECS integration: as we translate into a CloudFormation stack, either resources are managed as part of the deployment, or they are user managed and we "just" rely on those. Users who want to implement such a "shared with distinct subdomain" Load Balancer will have to configure ALB on their own and get compose up to use it.

[flaviostutz]

I agree this kind of LB customization is out of docker-compose scope.

So for better integration of ECS to custom ALB arrangements, we could indicate "route rules" and the "listener arn" inside "ports" (as being discussed on compose-spec). So when compose is deployed, it would add rules directly to an existing listener and forward requests to the container depending on "Host" header and "path".

So, following the style being discussed in compose-spec/compose-spec#111, we could use something like:

services:
  mytest:
    image: mytest:1.0.0
    ports:
      - target: 5000
        x-aws-route: (or simply "route" after spec is adopted)
          # full struct syntax
          - procotol: http
            host: mysite.org
            path: /test
            port: 2000
            x-aws-listener: "arn:aws:ecs:region:account:xxxxxx"

By doing this, we could remove lots of things related to LB from our current implementation (ALB creation and Listener creation) and avoid some discussions (because they would be addressed outside compose scope) like:

• How to indicate/provision a certificate for HTTPS? HTTPS Support for ECS Load Balancers #775 ECS: Config option to turn off creation of load balancer resources #1089 Cannot configure ECS load balancer to use HTTP instead of HTTPS #703
• How to redirect HTTP calls to HTTPS? What about HSTS?
• How to use a single LB for multiple containers forwarded by domain/url? Add support for configuring LB rule to route certain URLs to the service #777 Support path-based with Application Load Balancers #691
• How to use an internal LB? Support internal load balancers #694
• How to use an existing Listener? Docker Ecs Integration disable creating new listener and target group support #1315
• How to use a wildcard certificate for a bunch of container services?
• How to publish a specific port for exposing a container service? ECS Load Balancer options #821 Configuring Target Group to Forward 443 to 80 #693
• How to use security services (like AWS Cognito) for authentication on container services?

Those are some issues I've been struggling for our production systems when using compose-cli and I agree they wouldn't be addressed by compose itself. I know you don't want to add more "x-aws" extensions, but in this case we could even deprecate the "x-aws-loadbalancer" extension and remove lots of code from the codebase.

The custom LB resources itself could then be created using the new "x-aws-cloudformation" extension with anything supported by Cloudformation and compose would just "plug" onto the existing LB Listener.

I think this feature would allow compose-cli to be used on more advanced scenarios.

What are your thoughts?

[ndeloof]

We don't want to remove LoadBalancer management as we want compose up to work for newcomers without any prerequisites managing AWS resources, which should be reserved to advanced users / operator people (see #1411 as a proposal to distinguish this role)

[chipzzz]

@ndeloof , what if you don't want to use a load balancer at all, can we set x-aws-loadbalancer to none somehow? I doubt we can set it to a ECS service that holds a haproxy loadbalancer right? (i'm using my own load balancer within ecs service container)

By deafault, exposing a port creates a load balancer, as you know. In my case it just sits idle.

[ndeloof]

@chipzzz nope, we don't support this scenario.
The more it goes, the more we receive request to fine tune the network resources ... ECS integration primary goal is to hide this complexity with "sane defaults", we hardly can cover boths :'(

[chipzzz]

@ndeloof, understandable. I'm wondering in general people use docker or Kubernetes. If however, they want to quickly develop locally then deploy into ecs, docker ecs cli seems to most reasonable option. Otherwise, you have to spend time creating all the AWS resources separately manually, cloud formation templates, or terraform, ecs cli etc.

I guess there's also aws ecs cli, haven't played around much with it but it doesn't really incorporate docker files. So for some one who has built on dokcer and wants to deploy to ecs, this docker ecs cli still is the best option. I think it's worth expanding functionality further in this space, i think it meets a lot of demand.

Using docker compose convert helps but expanding this further like in #1411 would be awesome.

[robogeek]

I believe it's better to attach a container to the Target Group side of an ALB, rather than to the Listener.

I believe it's necessary to inform compose CLI the ALB to use, and to tell it to not populate the ALB with listeners and target groups. That way we can -- if we need -- create the ALB we want, and connect our containers into that ALB.

[chipzzz]

@robogeek , yea i dropped by haproxy for AWS ALB works great.

[nicolas-b12]

I'm currently using the ecs-cli to manage multiple productions workload.

In the ecs-cli we can specify a target-groups argument. Those target groups (created by hand or by script earlier) will be updated when a task upgrade is ready.

Regarding the argument of keeping things simple, can't we just have a x-aws-targetgroups argument ? This way, users wanting to keep existing infrastructure are satisfied and newcomers just don't pass it. This single argument would help a lot of people while avoiding adding too much complexity

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it had not recent activity during the stale period.