[Proposal] define AWS infrastructure in a separate config file

[ndeloof]

Description
to support AWS deployment we define custom extensions x-aws-vpc, x-aws-loadbalancer, x-aws-roles ... This comes with a few drawbacks:

1. x-aws-* attributes are not validated, and users might missuse them without being warned (see Check foo implementation do support all x-foo-* extension fields compose-spec/compose-go#66)
2. AWS infrastructure is way more complex than the compose one, especially considering networking components. Many user request a way to define subnets assignment to services, which would require extra extensions. Those extensions actually define a full model on top of the compose one, which is entangled in the yaml file
3. Last but not least, custom extensions intent by compose-spec is to allow experimentation with new attributes to be proposed to the compose-spec for normalization. But obviously those aws-specific ones won't become standard attributes.

I suggest we adopt the same approach proposed by aws copilot to define the AWS infrastructure within a dedicated file (with schema to validate content). By default, ECS intregration would behave like it does today in absence of x-aws-* extensions (i.e, use AWS account's default VPC) to make newcomer experience easy. Passing an AWS infrastructure file with explicit resources ARN will allow to tweak resource assignment to services. This approach can help keep the comopse file a developer-centric one while the AWS-specific one would be more operator-centric

[ndeloof]

cc @CarmenAPuccio

[flaviostutz]

Great. So in this proposal if I want to use an existing LB listener, for example, you would pass its ARN by using this dedicated file, right? Could you give more details on the format of this file, please? I looked at CoPilot but could not get it...

[blampe]

Could these attributes be inferred from ecs-params.yaml? That would give you a well-defined schema as a starting point.

I stumbled on this issue by way of #921, and while I would prefer to use docker compose it seems I'll need to use ecs-cli for the time being. If docker compose understood how to consume ecs-cli params, that would give me a smooth onramp to come back to docker. I could understand not wanting to couple yourself to that schema, though!

[mikelhamer]

Yes please!

[EricBorland]

What's the status of this? ETA for custom subnets?

[komatom]

I am waiting also, load balancers houls load on public subnets, and normal containers or services can be on public or private subnets what is specified, second ones are used normally with EFS shares, which should also use the specified subnets per service so it matches the containers setup.