v1.65.0

Add `providers` section to Atmos manifests. Update docs @aknysh (#555)

what

• Add providers section to Atmos manifests
• Auto-generate the prefix attribute for Atmos components for Terraform backend gcs for GCP
• Update docs (https://atmos.tools/core-concepts/components/terraform-providers/)

why

Terraform utilizes plugins known as providers for communication with cloud providers, SaaS providers, and various APIs.

In order for Terraform to install these providers, the corresponding Terraform configurations need to explicitly state what providers are required. Furthermore, certain providers require additional configuration, such as specifying endpoint URLs or cloud regions, before they can be used.

Provider Configuration in Terraform

When working with Terraform, you specify provider configurations in your Terraform code. This involves declaring which providers your infrastructure requires and providing any necessary configuration parameters. These parameters may include endpoint URLs, cloud regions, access credentials, or any other provider-specific configuration parameters.

To declare a provider in Terraform, use a provider block within your Terraform configuration files, usually in a providers.tf file in the component (a.k.a. root module) directory. The provider block specifies the provider type and all the necessary configuration parameters.

Here's an AWS provider configuration example for a vpc component. The provider config is defined in the components/terraform/vpc/providers.tf file:

  provider "aws" {
    region = "us-east-2"
    assume_role = "IAM Role ARN"
  }

In this example, the aws provider block includes the region and IAM role required for Terraform to communicate with the AWS services.

By correctly defining provider configurations in your Terraform code, you ensure that Terraform can seamlessly install, configure, and use the necessary plugins to manage your infrastructure across various cloud and services.

Provider Configuration and Overrides in Atmos Manifests

Atmos allows you to define and override provider configurations using the providers section in Atmos stack manifests.
The section can be defined globally for the entire organization, OU/tenant, account, region, or per component.

For example, the providers section at the global scope can look like this:

terraform:
  providers:
    aws:
      region: "us-east-2"
      assume_role: "IAM Role ARN"

Similarly, it can be defined (or overridden) at the OU/tenant, account and region scopes in the corresponding _defaults.yaml stack manifests.

If you want to override a provider configuration for a specific component, use the component.terraform.<component>.providers section. For example, the following config can be used to override the assume_role parameter just for the vpc component:

components:
  terraform:
    vpc:
      providers:
        aws:
         assume_role: "IAM Role ARN for VPC"

You can include the providers sections in any Atmos stack manifest at any level of inheritance. Atmos will process, deep-merge and override all the providers configurations for a component in the following order:

• Global scopes (terraform.providers sections for the Org, OUs, accounts and regions)
• Base component scope (component.terraform.<base_component>.providers section)
• Current component scope (component.terraform.<component>.providers section)

Refer to Atmos Component Inheritance for more information on all types of component inheritance supported by Atmos.

When you define the providers sections, Atmos processes the inheritance chain for a component and generates a
file providers_override.tf.json in the component's folder with the final values for all the defined providers.

For example:

> atmos terraform plan vpc -s plat-ue2-prod --logs-level=Trace

Variables for the component 'vpc' in the stack 'plat-ue2-prod':
  environment: ue2
  max_subnet_count: 3
  name: common
  namespace: cp
  region: us-east-2
  stage: prod
  tenant: plat

Writing the variables to file:
components/terraform/vpc/plat-ue2-prod.terraform.tfvars.json

Writing the provider overrides to file:
components/terraform/vpc/providers_override.tf.json

The generated providers_override.tf.json file would look like this:

{
    "provider": {
      "aws": {
        "assume_role": "IAM Role ARN for VPC"
      }
    }
}

Terraform then uses the values in the generated providers_override.tf.json to override the parameters for all the providers in the file.

References

• Terraform Providers
• Terraform Override Files