Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reservation Affinity for workbench instances #13328

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
60 changes: 60 additions & 0 deletions mmv1/products/workbench/Instance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ examples:
region_override: 'us-west1-a'
vars:
instance_name: 'workbench-instance'
reservation_name: 'wbi-reservation'
ignore_read_extra:
- 'gce_setup.0.vm_image'
- name: 'workbench_instance_labels_stopped'
Expand All @@ -95,6 +96,7 @@ examples:
vars:
instance_name: 'workbench-instance'
network_name: 'wbi-test-default'
reservation_name: 'wbi-reservation'
key_name: 'my-crypto-key'
test_env_vars:
project_id: 'PROJECT_NAME'
Expand All @@ -105,6 +107,12 @@ examples:
- 'gce_setup.0.vm_image'
- 'gce_setup.0.boot_disk.0.disk_type'
- 'gce_setup.0.data_disks.0.disk_type'
- name: 'workbench_instance_confidential_compute'
primary_resource_id: 'instance'
primary_resource_name: 'fmt.Sprintf("tf-test-workbench-instance%s", context["random_suffix"])'
region_override: 'us-west1-a'
vars:
instance_name: 'workbench-instance'
virtual_fields:
- name: 'desired_state'
description: |
Expand Down Expand Up @@ -441,6 +449,58 @@ properties:
Optional. Flag to enable ip forwarding or not, default false/off.
https://cloud.google.com/vpc/docs/using-routes#canipforward
immutable: true
- name: 'confidentialInstanceConfig'
type: NestedObject
immutable: true
description: |
Confidential instance configuration.
properties:
- name: 'enableConfidentialCompute'
type: Boolean
description: |
Defines whether the instance should have confidential compute enabled.
- name: 'confidentialInstanceType'
type: Enum
description: |
Defines the type of technology used by the confidential instance.
enum_values:
- 'SEV'
- 'SEV_SNP'
- 'TDX'
- name: 'reservationAffinity'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like the tests are causing an unwanted force replacement on this.

                  ~ network_interfaces (known after apply)
                  - network_interfaces {
                      - network  = "https://www.googleapis.com/compute/v1/projects/ci-test-project-188019/global/networks/default" -> null
                      - subnet   = "https://www.googleapis.com/compute/v1/projects/ci-test-project-188019/regions/us-central1/subnetworks/default" -> null
                        # (1 unchanged attribute hidden)
        
                      - access_configs {
                          - external_ip = "34.121.56.226" -> null
                        }
                    }
        
                  + reservation_affinity { # forces replacement
                      + consume_reservation_type = "RESERVATION_ANY" # forces replacement
                    }
        
                  ~ service_accounts (known after apply)
                  - service_accounts {
                      - email  = "[email protected]" -> null
                      - scopes = [
                          - "https://www.googleapis.com/auth/cloud-platform",
                          - "https://www.googleapis.com/auth/userinfo.email",
                        ] -> null
                    }

It doesn't seem to be get added into state since a second apply shows us that it's attempting to add it. Can you take a look as to what could be causing this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API rollout for these fields is not yet done which is causing this issue. I didn't face this on my local since I have the required visibility labels for these fields. We can revisit this after the rollout is done.

type: NestedObject
immutable: true
default_from_api: true
description: |
Reservations that this instance can consume from.
properties:
- name: 'consumeReservationType'
type: Enum
immutable: true
default_from_api: true
description: |
Specifies the type of reservation from which this instance can consume resources:
RESERVATION_ANY (default), RESERVATION_SPECIFIC, or RESERVATION_NONE.
enum_values:
- 'RESERVATION_NONE'
- 'RESERVATION_ANY'
- 'RESERVATION_SPECIFIC'
- name: 'key'
immutable: true
description: |
Corresponds to the label key of a reservation resource. To target a
RESERVATION_SPECIFIC by name, use compute.googleapis.com/reservation-name
as the key and specify the name of your reservation as its value.
- name: 'values'
type: Array
immutable: true
item_type:
type: String
description: |
Corresponds to the label values of a reservation resource. This can be
either a name to a reservation in the same project or
"projects/different-project/reservations/some-reservation-name"
to target a shared reservation in the same zone but in a different project.
- name: 'proxyUri'
type: String
description: |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,23 @@
resource "google_compute_reservation" "gpu_reservation" {
name = "{{index $.Vars "reservation_name"}}"
zone = "us-central1-a"

specific_reservation {
count = 1

instance_properties {
machine_type = "n1-standard-1"

guest_accelerators {
accelerator_type = "nvidia-tesla-t4"
accelerator_count = 1
}
}
}

specific_reservation_required = false
}

resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {
name = "{{index $.Vars "instance_name"}}"
location = "us-central1-a"
Expand All @@ -11,5 +31,13 @@ resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {
project = "cloud-notebooks-managed"
family = "workbench-instances"
}
reservation_affinity {
consume_reservation_type = "RESERVATION_ANY"
}
}

depends_on = [
google_compute_reservation.gpu_reservation
]

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {
name = "{{index $.Vars "instance_name"}}"
location = "us-central1-a"

gce_setup {
machine_type = "n2d-standard-2" // cant be e2 because of accelerator

shielded_instance_config {
enable_secure_boot = true
enable_vtpm = true
enable_integrity_monitoring = true
}

metadata = {
terraform = "true"
}

confidential_instance_config {
enable_confidential_compute = true
confidential_instance_type = "SEV"
}

}
}
34 changes: 34 additions & 0 deletions mmv1/templates/terraform/examples/workbench_instance_full.tf.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,26 @@ resource "google_service_account_iam_binding" "act_as_permission" {
]
}

resource "google_compute_reservation" "gpu_reservation" {
name = "{{index $.Vars "reservation_name"}}"
zone = "us-central1-a"

specific_reservation {
count = 1

instance_properties {
machine_type = "n1-standard-4"

guest_accelerators {
accelerator_type = "nvidia-tesla-t4"
accelerator_count = 1
}
}
}

specific_reservation_required = true
}

resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {
name = "{{index $.Vars "instance_name"}}"
location = "us-central1-a"
Expand Down Expand Up @@ -72,6 +92,12 @@ resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {
terraform = "true"
}

reservation_affinity {
consume_reservation_type = "RESERVATION_SPECIFIC"
key = "compute.googleapis.com/reservation-name"
values = [google_compute_reservation.gpu_reservation.name]
}

enable_ip_forwarding = true

tags = ["abc", "def"]
Expand All @@ -90,4 +116,12 @@ resource "google_workbench_instance" "{{$.PrimaryResourceId}}" {

enable_third_party_identity = "true"

depends_on = [
google_compute_network.my_network,
google_compute_subnetwork.my_subnetwork,
google_compute_address.static,
google_service_account_iam_binding.act_as_permission,
google_compute_reservation.gpu_reservation
]

}
Loading