add another secondary storage

[sbrueseke]

we added a second secondary storage to our ACS installation via UI. This was working without any issues, but we did get some alerts I want to discuss here.

Failed to register template: a582748a-bcb5-4842-8f94-3de4c1f2c9e2 with error: HTTP Server returned 404 (expected 200 OK)

Failed to register template: e4b2a024-d2df-466d-9db8-c166d9bf0739 with error: Failed post download script: checksum "{SHA-512}13b5d4f9ef187f1212ea56c61a6f204576d5976ba0b29abf3c11b72e76a4020132c22cfd7c0c3765bc0794b3a03008bbf18930a3027ef29ebe54949f4de2a08d" didn't match the given value, "{SHA-512}24e0f3d9c25835f2633d2823dade95aae4884aed95a5142d87efeb4aaed48dbc04a82a34ac5310676853008e0de271dfa0747b4c619ed20fa72ffe2e93fadcf3"

It looks like the new secondary storage wants to download ISO from source again, but source is not available anymore. Is this how it was designed? I would expect that the new secondary storage will download the ISO/templates from the first secondary storage as long as this storage is available and only this is not the case the new secondary storage would try to donwload from source.

If this is by design, this means that you need to make sure that all downloaded ISOs still being available for download, even after years.

We now ended up in the following situation

As far as I see there a 2 api calls:
migrateResourceToAnotherSecondaryStorage
This is only working with templates, not with ISOs.

migrateSecondaryStorageData
This only can be used to migrate all from one secondary storage to another or to balance.

How do I get all ISO to newly added secondary storages, even when the download url is not accessible anymore?

[winterhazel]

Hey @sbrueseke

It looks like the new secondary storage wants to download ISO from source again, but source is not available anymore. Is this how it was designed? I would expect that the new secondary storage will download the ISO/templates from the first secondary storage as long as this storage is available and only this is not the case the new secondary storage would try to donwload from source.

Yes, that's the current behavior. ACS will always try to download templates/ISOs from the source again, even if it does not exist anymore (unable to download) or has been updated (checksum will not match).

As far as I know, If you want to move/copy the template to a new storage, you will need to do it manually in the storage and adjust the database entry for cloud.template_store_ref.

This problem was mentioned in another thread before:

[1] https://lists.apache.org/thread/g97o3w8g8dtgfl9y4ddo2jcyrg11olpg
[2] https://lists.apache.org/thread/d8lhoqck8b145y610v9q00cwgvpt1pml

I already developed both the extensions mentioned at [2]. The 2nd one (prioritize copying templates from existing image
stores instead of downloading them) would solve your issue, I just haven't taken the time to open the PR yet (sorry!). Hopefully I can open it this week.

[sbrueseke]

Thx winterhazel for that info and already taken care of an improvement.

I took a look at out db and found this:

mysql> select * from template_store_ref where template_id = 215\G
*************************** 1. row ***************************
id: 21
store_id: 1
template_id: 215
created: 2023-12-11 12:42:08
last_updated: 2025-02-04 15:37:01
job_id: b682663c-8c87-4880-93dc-30f88c8d91db
download_pct: 100
size: 627519488
store_role: Image
physical_size: 627519488
download_state: DOWNLOADED
error_str: Install completed successfully at 12/11/23, 12:43 PM
local_path: /mnt/SecStorage/83724db5-6c19-3921-bd0e-ed0186a07aa9/template/tmpl/2/215/dnld12878222805655771854tmp_
install_path: template/tmpl/2/215/215-2-5e0ae2b2-db82-3da9-a723-94cd3736f5e1.iso
url: NULL
state: Ready
destroyed: 0
is_copy: 0
update_count: 2
ref_cnt: 0
updated: 2023-12-11 12:43:42
download_url_created: 1970-01-01 00:00:00
download_url: NULL
*************************** 2. row ***************************
id: 119
store_id: 3
template_id: 215
created: 2025-02-04 16:37:57
last_updated: 2025-02-04 16:41:14
job_id: 8321f9ac-27fa-4742-a727-73cd4b578c49
download_pct: 100
size: 0
store_role: Image
physical_size: 0
download_state: DOWNLOAD_ERROR
error_str: Failed post download script: checksum "{SHA-512}13b5d4f9ef187f1212ea56c61a6f204576d5976ba0b29abf3c11b72e76a4020132c22cfd7c0c3765bc0794b3a03008bbf18930a3027ef29ebe54949f4de2a08d" didn't match the given value, "{SHA-512}24e0f3d9c25835f2633d2823dade95aae4884
local_path: /mnt/SecStorage/175a1e5a-f8fa-3449-811d-759d35773245/template/tmpl/2/215/dnld10691952881837833383tmp_
install_path: template/tmpl/2/215
url: NULL
state: Allocated
destroyed: 0
is_copy: 0
update_count: 2
ref_cnt: 0
updated: 2025-02-04 16:41:14
download_url_created: NULL
download_url: NULL

Any idea was reference template_id is? Which table is responsible for this id?
What do I need to do to fix this? As far as I understand I need to change size, physical_size, download_state, error_str and state after copying the file manually to the other secondary storage. Is this correct?

[winterhazel]

Any idea was reference template_id is? Which table is responsible for this id?

It references the table cloud.vm_template

What do I need to do to fix this? As far as I understand I need to change size, physical_size, download_state, error_str and state after copying the file manually to the other secondary storage. Is this correct?

Yes, first you need to copy template/tmpl/2/215/215-2-5e0ae2b2-db82-3da9-a723-94cd3736f5e1.iso and template/tmpl/2/215/template.properties to the same folder at the new storage.

Then, you can either restart the SSVM's service so that the templates available at the storage get re-synchronized, or change these columns. If you go with the second approach, you'll also need to change install_path to point to the ISO (template/tmpl/2/215/215-2-5e0ae2b2-db82-3da9-a723-94cd3736f5e1.iso).

[sbrueseke]

Thank you very much! Your workflow did the job. Looking forward for your code to solve this issue.