[Compute] Fix #20174: `az vm create`: Determine plan information when using image alias #21028

jepio · 2022-01-18T08:53:01Z

Description

Some image aliases require "plan information" when used. The "Flatcar" alias is one of those. When usingan image URN, the validator fetches plan information and appends to the request, but this was missing when using an alias. Add the same handling to alias validation, so that aliases can be used as is without extra work from the user.

Testing Guide

g=group$RANDOM
az group create -g $g -l westeurope
az vm create -g $g -n vm$RANDOM --image Flatcar

Before this PR this results in an error ("missing plan information"), after this PR the result is success.

History Notes

[Compute] Fix #20174: az vm create: Determine plan information when using image alias

This checklist is used to make sure that common guidelines for a pull request are followed.

The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.

yonzhan · 2022-01-18T08:56:56Z

Compute

wangzelin007 · 2022-01-24T08:55:15Z

Hi jepio,
Could you merge the latest code from remote dev branch first. After that, we will help you solve the remaining CI issue later.

jepio · 2022-01-24T10:21:06Z

I've rebased onto the remote dev branch. The test failures look like they test for a hard-coded sequence of remote operations, and would need updating.

wangzelin007 · 2022-01-24T12:49:51Z

/azp run

azure-pipelines · 2022-01-24T12:50:07Z

Azure Pipelines successfully started running 2 pipeline(s).

Some marketplace images require plan information to be provided when creating a virtual machine. Missing plan information results in the error message: "Creating a virtual machine from Marketplace image or a custom image sourced from a Marketplace image requires Plan information in the request." The "Flatcar" alias is one such case where plan information is needed. When an image is provided as a URN, the validator code takes care of determining the plan information and filling it in, but when using an alias no such action is taken. Perform the same lookup when using an alias, so that users can use the alias as intended and don't have to manually supply plan information. Signed-off-by: Jeremi Piotrowski <[email protected]>

jepio · 2022-01-24T15:12:00Z

@wangzelin007 OK, now I understand. I updated the dev branch now.

wangzelin007 · 2022-01-25T02:09:08Z

/azp run

azure-pipelines · 2022-01-25T02:09:24Z

Azure Pipelines successfully started running 2 pipeline(s).

wangzelin007 · 2022-01-25T02:56:14Z

Hi @jepio,
The real reason is that add _get_image_plan_info_if_exists will add one more api call.
All involved tests need to be re-recorded and re-upload

jepio · 2022-01-25T06:56:54Z

OK, how do I re-record the tests?

wangzelin007 · 2022-01-25T09:54:39Z

OK, how do I re-record the tests?
I am trying to re-record these tests, but many of them still failed in re-run.

get all failed test case in profile 2018-03-01-hybrid
azdev test vm --repo=./ --src=HEAD --tgt=origin/dev --no-exitfirst --profile 2018-03-01-hybrid --verbose --series
re-run error cases to generate recording files:
azdev test vm --no-exitfirst --live --lf --profile 2018-03-01-hybrid

get all failed test case in profile 2019-03-01-hybrid
azdev test vm --repo=./ --src=HEAD --tgt=origin/dev --no-exitfirst --profile 2019-03-01-hybrid --verbose --series
re-run error cases to generate recording files:
azdev test vm --no-exitfirst --live --lf --profile 2019-03-01-hybrid

get all failed test case in profile 2020-09-01-hybrid
azdev test vm --repo=./ --src=HEAD --tgt=origin/dev --no-exitfirst --profile 2020-09-01-hybrid --verbose --series
re-run error cases to generate recording files:
azdev test vm --no-exitfirst --live --lf --profile 2020-09-01-hybrid

get all failed test case in profile latest
azdev test vm --repo=./ --src=HEAD --tgt=origin/dev --no-exitfirst --profile latest --verbose --series
re-run error cases to generate recording files:
azdev test vm --no-exitfirst --live --lf --profile latest

wangzelin007 · 2022-01-27T09:07:04Z

src/azure-cli/azure/cli/command_modules/vm/_template_builder.py

-    if enable_hibernation is not None:
-        vm_properties['additionalCapabilities']['hibernationEnabled'] = enable_hibernation
+        if enable_hibernation is not None:
+            vm_properties['additionalCapabilities']['hibernationEnabled'] = enable_hibernation


wangzelin007 · 2022-01-27T09:07:45Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py

        self.cmd('vm get-instance-view -n {vm} -g {rg}', checks=[
            self.check('*.extensions[0].name', ['VMAccessForLinux']),
-            self.check('*.extensions[0].typeHandlerVersion', ['1.4.7.1'])
+            self.check('*.extensions[0].typeHandlerVersion', ['1.5.11'])


wangzelin007 · 2022-01-27T09:08:14Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py


 class VMBootDiagnostics(ScenarioTest):

+    @AllowLargeResponse()


wangzelin007 · 2022-01-27T09:09:45Z

src/azure-cli/azure/cli/command_modules/vm/_template_builder.py

-    vm_properties = {'hardwareProfile': {'vmSize': size, 'vmSizeProperties': vm_size_properties},
-                     'networkProfile': {'networkInterfaces': nics}, 'storageProfile': _build_storage_profile()}
+    if vm_size_properties:
+        vm_properties['hardwareProfile']['vmSizeProperties'] = vm_size_properties


fix

{ "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.", "details": [ { "code": "BadRequest", "message": { "error": { "details": [ { "message": "Could not find member vmSizeProperties on object of type VMHardwareProfile. Path properties.hardwareProfile.vmSizeProperties, line 1, position 110.", "target": "vm.properties.hardwareProfile.vmSizeProperties" }, { "message": "Could not find member additionalCapabilities on object of type Properties. Path properties.additionalCapabilities, line 1, position 1230.", "target": "vm.properties.additionalCapabilities" } ], "code": "BadRequest", "message": "The request message is invalid." } } } ] } }

wangzelin007 · 2022-01-27T09:10:56Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py

        })

-        self.cmd('vm create -n {vm} -g {rg} --image openSUSE-Leap --admin-username user11 --private-ip-address 10.0.0.5 --public-ip-sku {public_ip_sku} --public-ip-address-dns-name {dns} --generate-ssh-keys')
+        self.cmd('vm create -n {vm} -g {rg} --image UbuntuLTS --admin-username user11 --private-ip-address 10.0.0.5 --public-ip-sku {public_ip_sku} --public-ip-address-dns-name {dns} --generate-ssh-keys')


wangzelin007 · 2022-01-27T09:11:05Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py


        # verify the default should be "Basic" sku with "Dynamic" allocation method
-        self.cmd('vm create -n {vm2} -g {rg} --image openSUSE-Leap --admin-username user11 --generate-ssh-keys')
+        self.cmd('vm create -n {vm2} -g {rg} --image UbuntuLTS --admin-username user11 --generate-ssh-keys')


same as above

wangzelin007 · 2022-01-27T09:11:19Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py


 class VMSSCreateAndModify(ScenarioTest):

+    @AllowLargeResponse()


same as above

wangzelin007 · 2022-01-27T09:12:58Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2018_03_01/test_vm_commands.py

            'vmss': 'vmss1'
        })
-        self.cmd("vmss create -n {vmss} -g {rg} --vm-sku Standard_DS4_v2 --image Win2022Datacenter --admin-username clittester --admin-password Test12345678!!! --accelerated-networking --instance-count 1")
+        self.cmd("vmss create -n {vmss} -g {rg} --vm-sku Standard_DS4_v2 --image Win2012R2Datacenter --admin-username clittester --admin-password Test12345678!!! --accelerated-networking --instance-count 1")


Using image Win2022Datacenter is by design for TLS 1.2 compliant, PR link: #20701, I think we need to investigate the reason why this image can no longer be found

wangzelin007 · 2022-01-27T09:13:18Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2019_03_01/test_vm_commands.py

@@ -1077,6 +1077,7 @@ def test_vm_create_none_options(self, resource_group):

 class VMBootDiagnostics(ScenarioTest):

+    @AllowLargeResponse()


same as above

wangzelin007 · 2022-01-27T09:13:21Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2019_03_01/test_vm_commands.py

@@ -1343,7 +1344,7 @@ def test_vm_create_custom_ip(self, resource_group):
            'public_ip_sku': 'Standard'
        })

-        self.cmd('vm create -n {vm} -g {rg} --image openSUSE-Leap --admin-username user11 --private-ip-address 10.0.0.5 --public-ip-sku {public_ip_sku} --public-ip-address-dns-name {dns} --generate-ssh-keys')
+        self.cmd('vm create -n {vm} -g {rg} --image UbuntuLTS --admin-username user11 --private-ip-address 10.0.0.5 --public-ip-sku {public_ip_sku} --public-ip-address-dns-name {dns} --generate-ssh-keys')


same as above

wangzelin007 · 2022-01-27T09:13:26Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2019_03_01/test_vm_commands.py

@@ -1354,7 +1355,7 @@ def test_vm_create_custom_ip(self, resource_group):
                 checks=self.check('ipConfigurations[0].privateIpAllocationMethod', 'Static'))

        # verify the default should be "Basic" sku with "Dynamic" allocation method
-        self.cmd('vm create -n {vm2} -g {rg} --image openSUSE-Leap --admin-username user11 --generate-ssh-keys')
+        self.cmd('vm create -n {vm2} -g {rg} --image UbuntuLTS --admin-username user11 --generate-ssh-keys')


same as above

wangzelin007 · 2022-01-27T09:13:31Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2019_03_01/test_vm_commands.py

@@ -1490,6 +1491,7 @@ def test_vm_create_custom_data(self, resource_group):

 class VMSSCreateAndModify(ScenarioTest):

+    @AllowLargeResponse()


same as above

wangzelin007 · 2022-01-27T09:13:54Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2019_03_01/test_vm_commands.py

@@ -2130,7 +2132,7 @@ def test_vm_error_on_zone_unavailable(self, resource_group, resource_group_locat
        try:
            self.cmd('vm create -g {rg} -n vm1 --admin-username clitester --admin-password PasswordPassword1! --image debian --zone 1')
        except Exception as ex:
-            self.assertTrue('availability zone is not yet supported' in str(ex))
+            self.assertTrue('does not support availability zones at location' in str(ex))


wangzelin007 · 2022-01-27T09:14:16Z

src/azure-cli/azure/cli/command_modules/vm/tests/hybrid_2020_09_01/test_vm_commands.py

@@ -905,6 +905,7 @@ def test_vm_create_existing_ids_options(self, resource_group, storage_account):

 class VMSSCreateAndModify(ScenarioTest):

+    @AllowLargeResponse()


same as above

wangzelin007 · 2022-01-27T09:14:28Z

src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py

@@ -3992,7 +3992,7 @@ def test_vm_error_on_zone_unavailable(self, resource_group, resource_group_locat
        try:
            self.cmd('vm create -g {rg} -n vm1 --admin-username clitester --admin-password PasswordPassword1! --image debian --zone 1')
        except Exception as ex:
-            self.assertTrue('availability zone is not yet supported' in str(ex))
+            self.assertTrue('does not support availability zones at location' in str(ex))


same as above

wangzelin007 · 2022-01-27T09:14:58Z

src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py

@@ -6728,7 +6730,7 @@ def test_vmss_windows_patch_mode(self, resource_group):
            'rg': resource_group
        })

-        self.cmd('vmss create -g {rg} -n {vmss} --image Win2022Datacenter --enable-agent --enable-auto-update false --patch-mode Manual --orchestration-mode Flexible --admin-username azureuser --admin-password testPassword0')
+        self.cmd('vmss create -g {rg} -n {vmss} --image Win2012Datacenter --enable-agent --enable-auto-update false --patch-mode Manual --orchestration-mode Flexible --admin-username azureuser --admin-password testPassword0')


same as above

wangzelin007 · 2022-01-27T09:17:04Z

src/azure-cli/azure/cli/command_modules/vm/tests/latest/test_vm_commands.py

@@ -5367,31 +5367,33 @@ def test_dedicated_host_e2e(self, resource_group, resource_group_location):
        self.cmd('vm delete --name vm2 -g {rg2} --yes')
        self.cmd('vm host delete --name {host-name} --host-group {host-group} -g {rg2} --yes')

-    @ResourceGroupPreparer(name_prefix='cli_test_dedicated_host_', location='westus2')
+    @ResourceGroupPreparer(name_prefix='cli_test_dedicated_host3_', location='eastus2')


zhoxing-ms · 2022-03-18T06:19:33Z

Replaced by #21666

jepio requested review from jsntcy, wangzelin007 and zhoxing-ms as code owners January 18, 2022 08:53

jepio mentioned this pull request Jan 18, 2022

az vm create fails for Flatcar distro alias because plan information is unset #20174

Closed

yonzhan assigned zhoxing-ms Jan 18, 2022

yonzhan added this to the Jan 2022 (2022-02-08) milestone Jan 18, 2022

wangzelin007 changed the title ~~[Compute] az vm create: Determine plan information when using image alias (#20174)~~ [Compute] Fix #20174: az vm create: Determine plan information when using image alias Jan 18, 2022

jepio force-pushed the jepio/alias-fix-plan branch from bbbb4e2 to aa5dcf3 Compare January 24, 2022 10:20

ghost added the Compute az vm/vmss/image/disk/snapshot label Jan 24, 2022

ghost requested a review from yonzhan January 24, 2022 10:20

jepio force-pushed the jepio/alias-fix-plan branch from aa5dcf3 to ffbee38 Compare January 24, 2022 15:09

fix azdev test error

a2055e1

ghost added the Auto-Assign Auto assign by bot label Jan 27, 2022

wangzelin007 reviewed Jan 27, 2022

View reviewed changes

wangzelin007 added the do-not-merge label Jan 27, 2022

yonzhan modified the milestones: Jan 2022 (2022-02-08), Feb 2022 (2022-03-01) Jan 27, 2022

yonzhan modified the milestones: Feb 2022 (2022-03-01), Mar 2022 (2022-04-05) Feb 26, 2022

Azure deleted a comment from azure-pipelines bot Mar 3, 2022

cxznmhdcxz mentioned this pull request Mar 4, 2022

[Compute] Fix #20174: az vm create: Determine plan information when using image alias #21513

Closed

3 tasks

zhoxing-ms mentioned this pull request Mar 18, 2022

[Compute] Fix #20174: az vm create: Determine plan information when using image alias #21666

Merged

3 tasks

zhoxing-ms closed this Mar 18, 2022


		class VMSSCreateAndModify(ScenarioTest):

		@AllowLargeResponse()

		@@ -1077,6 +1077,7 @@ def test_vm_create_none_options(self, resource_group):

		class VMBootDiagnostics(ScenarioTest):

		@AllowLargeResponse()

		@@ -1490,6 +1491,7 @@ def test_vm_create_custom_data(self, resource_group):

		class VMSSCreateAndModify(ScenarioTest):

		@AllowLargeResponse()

		@@ -905,6 +905,7 @@ def test_vm_create_existing_ids_options(self, resource_group, storage_account):

		class VMSSCreateAndModify(ScenarioTest):

		@AllowLargeResponse()

[Compute] Fix #20174: az vm create: Determine plan information when using image alias #21028

[Compute] Fix #20174: az vm create: Determine plan information when using image alias #21028

Uh oh!

Conversation

jepio commented Jan 18, 2022 • edited by wangzelin007 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yonzhan commented Jan 18, 2022

Uh oh!

wangzelin007 commented Jan 24, 2022

Uh oh!

jepio commented Jan 24, 2022

Uh oh!

wangzelin007 commented Jan 24, 2022

Uh oh!

azure-pipelines bot commented Jan 24, 2022

Uh oh!

jepio commented Jan 24, 2022

Uh oh!

wangzelin007 commented Jan 25, 2022

Uh oh!

azure-pipelines bot commented Jan 25, 2022

Uh oh!

wangzelin007 commented Jan 25, 2022

Uh oh!

jepio commented Jan 25, 2022

Uh oh!

wangzelin007 commented Jan 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhoxing-ms Jan 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wangzelin007 Jan 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

[Compute] Fix #20174: `az vm create`: Determine plan information when using image alias #21028

[Compute] Fix #20174: `az vm create`: Determine plan information when using image alias #21028

jepio commented Jan 18, 2022 •

edited by wangzelin007

Loading

wangzelin007 commented Jan 25, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

zhoxing-ms Jan 28, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading

wangzelin007 Jan 27, 2022 •

edited

Loading