diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index 576dfe63d..9c423c352 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -14,7 +14,6 @@ the various features provided. cephadm monitoring openbao - vault wazuh walled-garden security-hardening diff --git a/doc/source/configuration/openbao.rst b/doc/source/configuration/openbao.rst index 87d1c1435..2be67338b 100644 --- a/doc/source/configuration/openbao.rst +++ b/doc/source/configuration/openbao.rst @@ -1,14 +1,14 @@ .. _openbao: -======================== -OpenBao for internal PKI -======================== +======================================== +OpenBao/Hashicorp Vault for internal PKI +======================================== -This document describes how to deploy OpenBao for +This document describes how to deploy OpenBao or Hashicorp Vault for internal PKI purposes using the `StackHPC Hashicorp collection `_ -OpenBao may be used as a Certificate Authority to generate certificates for: +OpenBao/Hashicorp Vault may be used as a Certificate Authority to generate certificates for: * OpenStack internal API * OpenStack backend APIs @@ -18,7 +18,19 @@ TLS support is described in the :kolla-ansible-doc:`Kolla Ansible documentation ` and the :kayobe-doc:`Kayobe documentation `. -OpenBao may also be used as the secret store for Barbican. +OpenBao/Hashicorp Vault may also be used as the secret store for Barbican. + +.. note:: + + StackHPC Kayobe Config's default is OpenBao and this documentation is more + focused on OpenBao. + As Hashicorp Vault support will be discontinued from stackhpc/2026.1, + we highly recommend new deployments use OpenBao. + + To configure Hashicorp Vault instead set ``stackhpc_ca_secret_store: vault``, + then follow the instruction. + + Currently the migration path from Hashicorp Vault to OpenBao is in development Background ========== @@ -30,12 +42,12 @@ creation and issuance of certificates. - The first OpenBao instance is located on the seed host. It handles infrastructure-level certificates, generating the root Certificate Authority (CA) and intermediate CA for the second OpenBao. - The ``openbao-deploy-seed.yml`` playbook sets up this instance. + The ``secret-store-deploy-seed.yml`` playbook sets up this instance. - The second OpenBao instance is within the OpenStack overcloud, located on the controller nodes. This instance uses the intermediate CA from the seed OpenBao to issue application-specific - certificates. The ``vault-openbao-overcloud.yml`` playbook is used + certificates. The ``secret-store-deploy-overcloud.yml`` playbook is used for its setup. It ensures that all controller nodes trust the intermediate CA from the root OpenBao. @@ -64,49 +76,65 @@ Deployment Setup OpenBao on the seed node ------------------------------ -1. Run openbao-deploy-seed.yml custom playbook +1. Run secret-store-deploy-seed.yml custom playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-deploy-seed.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-deploy-seed.yml 2. Encrypt generated certs/keys with ansible-vault (use proper location of vault password file) - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/openbao/OS-TLS-INT.pem ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/openbao/seed-openbao-keys.json ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/openbao/overcloud.key + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/OS-TLS-INT.pem + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/seed-vault-keys.json + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud.key + Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/OS-TLS-INT.pem ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/seed-openbao-keys.json ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/overcloud.key + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/OS-TLS-INT.pem + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/seed-vault-keys.json + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.key + Setup OpenBao HA on the overcloud hosts --------------------------------------- -1. Run openbao-deploy-overcloud.yml custom playbook +1. Run secret-store-deploy-overcloud.yml custom playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-deploy-overcloud.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-deploy-overcloud.yml 2. Encrypt overcloud openbao keys (use proper location of vault password file) - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/openbao/overcloud-openbao-keys.json + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud-vault-keys.json + Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/overcloud-openbao-keys.json + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud-vault-keys.json + Rotating OpenBao certificate on the overcloud hosts --------------------------------------------------- @@ -117,52 +145,67 @@ cannot be unsealed with an expired certificate. 1. Delete the old certificate: - .. code-block:: + .. code-block::bash rm $KAYOBE_CONFIG_PATH/openbao/overcloud.crt + # For Hashicorp Vault + rm $KAYOBE_CONFIG_PATH/vault/overcloud.crt + Or if environments are being used - .. code-block:: + .. code-block::bash rm $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/overcloud.crt + # For Hashicorp Vault + rm $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.crt + 2. Generate a new certificate (and key): - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-deploy-seed.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-deploy-seed.yml 3. Encrypt generated key with ansible-vault (use proper location of vault password file) - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/openbao/overcloud.key + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud.key + Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/openbao/overcloud.key + # For Hashicorp Vault + ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.key + 4. Copy the new certificate to the overcloud hosts. Note, if the old certificate has expired this will fail on the unseal step. - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-deploy-overcloud.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-deploy-overcloud.yml 5. Restart the containers to use the new certificate: - .. code-block:: + .. code-block::bash kayobe overcloud host command run --command "docker restart openbao" -l controllers + # For Hashicorp Vault + kayobe overcloud host command run --command "docker restart vault" -l controllers + 6. If sealed, unseal OpenBao: - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-unseal-overcloud.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-unseal-overcloud.yml Certificates generation ======================= @@ -174,7 +217,7 @@ Certificates generation .. code-block:: - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-unseal-overcloud.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-unseal-overcloud.yml Create the external TLS certificates (testing only) --------------------------------------------------- @@ -187,19 +230,19 @@ For test and development purposes it is possible to use OpenBao as a CA for the 1. Run the playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-generate-test-external-tls.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-generate-test-external-tls.yml 2. Use ansible-vault to encrypt the PEM bundle in $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy.pem. Commit the PEM bundle to the kayobe configuration. - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy.pem Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/haproxy.pem @@ -208,19 +251,19 @@ Create the internal TLS certificates 1. Run the playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-generate-internal-tls.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-generate-internal-tls.yml 2. Use ansible-vault to encrypt the PEM bundle in $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy-internal.pem. Commit the PEM bundle and root CA to the kayobe configuration. - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy-internal.pem Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/haproxy-internal.pem @@ -229,19 +272,19 @@ Create the backend TLS and RabbitMQ TLS certificates 1. Run the playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-generate-backend-tls.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-generate-backend-tls.yml 2. Use ansible-vault to encrypt the keys in $KAYOBE_CONFIG_PATH/kolla/certificates/-key.pem. Commit the certificates and keys to the kayobe configuration. - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/-key.pem Or if environments are being used - .. code-block:: + .. code-block::bash ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/-key.pem @@ -256,9 +299,11 @@ HAProxy integration is no longer required for generating OpenStack control plane 1. Create the HAProxy config to reverse proxy the OpenBao HA container - Set the openbao_front to the external VIP address or internal VIP address depending on the installation. Set the openbao_back to the IPs of the control nodes. + Set the openbao_front to the external VIP address or internal VIP address depending on the installation. + Set the openbao_back to the IPs of the control nodes. - Set the following in etc/kayobe/kolla/config/haproxy/services.d/openbao.cfg or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/config/haproxy/services.d/openbao.cfg + Set the following in etc/kayobe/kolla/config/haproxy/services.d/openbao.cfg + or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/config/haproxy/services.d/openbao.cfg .. code-block:: @@ -285,6 +330,34 @@ HAProxy integration is no longer required for generating OpenStack control plane {% endfor %} {% endraw %} + For Hashicorp Vault, set the following in etc/kayobe/kolla/config/haproxy/services.d/vault.cfg + or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/config/haproxy/services.d/vault.cfg + + .. code-block:: + + # Delete "verify none" if not using self-signed/unknown issuer + {% raw %} + frontend vault_front + mode tcp + option tcplog + bind {{ kolla_internal_vip_address }}:8200 + default_backend vault_back + + backend vault_back + mode tcp + option httpchk GET /v1/sys/health + # https://www.vaultproject.io/api-docs/system/health + # 200: initialized, unsealed, and active + # 429: standby + http-check expect rstatus (200|429) + + {% for host in groups['control'] %} + {% set host_name = hostvars[host].ansible_facts.hostname %} + {% set host_ip = 'api' | kolla_address(host) %} + server {{ host_name }} {{ host_ip }}:8200 check check-ssl verify none inter 2000 rise 2 fall 5 + {% endfor %} + {% endraw %} + 2. If HAProxy has not yet been deployed, continue to :ref:`certificates deployment `. If HAProxy has been deployed, it may be redeployed with the new OpenBao service configuration: @@ -299,12 +372,16 @@ Certificates deployment .. warning:: - The switch from HTTP to HTTPS during the deployment of internal/backend TLS certificates can temporarily disrupt service availability and necessitates a restart of all services. During this transition, endpoints may become unreachable following the HAProxy restart, persisting until the endpoint catalogue and client have been reconfigured to use HTTPS. + The switch from HTTP to HTTPS during the deployment of internal/backend TLS + certificates can temporarily disrupt service availability and necessitates a + restart of all services. During this transition, endpoints may become unreachable + following the HAProxy restart, persisting until the endpoint catalogue and client + have been reconfigured to use HTTPS. Enable the required TLS variables in kayobe and kolla ----------------------------------------------------- -1. If using OpenBao as a CA for the external API, set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml +1. If using OpenBao (or Vault) as a CA for the external API, set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml .. code-block:: @@ -348,12 +425,12 @@ Enable the required TLS variables in kayobe and kolla It is important that you are only using admin endpoints for keystone. If any admin endpoints exist for other services, they must be deleted e.g. - .. code-block:: + .. code-block::bash openstack endpoint list --interface admin -f value | \ awk '!/keystone/ {print $1}' | xargs openstack endpoint delete - .. code-block:: + .. code-block::bash kayobe overcloud service deploy @@ -365,7 +442,7 @@ Enable the required TLS variables in kayobe and kolla Restart the nova-compute container on all hypervisors: - .. code-block:: + .. code-block::bash kayobe overcloud host command run --command "systemctl restart kolla-nova_compute-container.service" --become --show-output -l compute @@ -379,7 +456,7 @@ Enable Barbican in kayobe 1. Set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml - .. code-block:: + .. code-block::yml kolla_enable_barbican: yes @@ -389,25 +466,25 @@ Generate secrets_barbican_approle_secret_id 1. Run ``uuidgen`` to generate secret id 2. Insert into secrets.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/secrets.yml - .. code-block:: + .. code-block::yml secrets_barbican_approle_secret_id: "YOUR-SECRET-GOES-HERE" Create required configuration in OpenBao ---------------------------------------- -1. Run openbao-deploy-barbican.yml custom playbook +1. Run secret-store-deploy-barbican.yml custom playbook - .. code-block:: + .. code-block::bash - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/openbao-deploy-barbican.yml + kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store-deploy-barbican.yml Add secrets_barbican_approle_id to secrets ------------------------------------------ 1. Note the role id from playbook output and insert into secrets.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/secrets.yml - .. code-block:: + .. code-block::yml secrets_barbican_approle_role_id: "YOUR-APPROLE-ID-GOES-HERE" @@ -436,6 +513,6 @@ Configure Barbican Deploy Barbican --------------- - .. code-block:: + .. code-block::bash kayobe overcloud service deploy -kt barbican diff --git a/doc/source/configuration/vault.rst b/doc/source/configuration/vault.rst deleted file mode 100644 index 126f5adc8..000000000 --- a/doc/source/configuration/vault.rst +++ /dev/null @@ -1,448 +0,0 @@ -.. _hashicorp-vault: - -================================ -Hashicorp Vault for internal PKI -================================ - -.. warning:: - - Deployment of Hashicorp Vault is deprecated and will be removed in a future release. - New deployments should use OpenBao and existing deployments will be migrated once an - method for migration is available. - See :ref:`here ` for more information. - -This document describes how to deploy Hashicorp Vault for -internal PKI purposes using the -`StackHPC Hashicorp collection `_ - -Vault may be used as a Certificate Authority to generate certificates for: - -* OpenStack internal API -* OpenStack backend APIs -* RabbitMQ - -TLS support is described in the :kolla-ansible-doc:`Kolla Ansible documentation -` and the :kayobe-doc:`Kayobe documentation -`. - -Vault may also be used as the secret store for Barbican. - -Background -========== - -Our OpenStack environment employs two separate HashiCorp Vault instances. -These instances manage the Public Key Infrastructure (PKI) by handling the -creation and issuance of certificates. - -- The first HashiCorp Vault instance is located on the seed host. - It handles infrastructure-level certificates, generating the root - Certificate Authority (CA) and intermediate CA for the second Vault. - The ``vault-deploy-seed.yml`` playbook sets up this instance. - -- The second HashiCorp Vault instance is within the OpenStack - overcloud, located on the controller nodes. This instance uses the - intermediate CA from the seed Vault to issue application-specific - certificates. The ``vault-deploy-overcloud.yml`` playbook is used - for its setup. It ensures that all controller nodes trust the - intermediate CA from the root Vault. - -The dual Vault setup enhances security by protecting the root CA's key. The more -exposed overcloud vault only possesses the intermediate key, ensuring that -the root key remains secure even if the overcloud Vault instance is compromised. - -Prerequisites -============= - -Before beginning the deployment of vault for openstack internal TLS and backend TLS you should ensure that you have the following. - - * Seed Node or a host to run the vault container on - * Overcloud controller hosts to install second vault on - * Ansible Galaxy dependencies installed: ``kayobe control host bootstrap`` - * Python dependencies installed: ``pip install -r kayobe-config/requirements.txt`` - -By default, Consul and Vault images are not synced from Docker Hub to the local -Pulp. To sync these images, set ``stackhpc_sync_hashicorp_images`` to ``true``. -The Vault deployment configuration will be automatically updated to pull images -from Pulp. - -Deployment -========== - -Setup Vault on the seed node ----------------------------- - -1. Run vault-deploy-seed.yml custom playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-deploy-seed.yml - -2. Encrypt generated certs/keys with ansible-vault (use proper location of vault password file) - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/OS-TLS-INT.pem - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/seed-vault-keys.json - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud.key - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/OS-TLS-INT.pem - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/seed-vault-keys.json - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.key - -Setup Vault HA on the overcloud hosts -------------------------------------- - -1. Run vault-deploy-overcloud.yml custom playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-deploy-overcloud.yml - -2. Encrypt overcloud vault keys (use proper location of vault password file) - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud-vault-keys.json - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud-vault-keys.json - -Rotating Vault certificate on the overcloud hosts -------------------------------------------------- - -The certificate for the overcloud vaults has an expiry time of one year. While -the cloud won't break if this expires, it will need rotating before new -certificates can be generated for internal PKI. If a vault becomes sealed, it -cannot be unsealed with an expired certificate. - -1. Delete the old certificate: - - .. code-block:: - - rm $KAYOBE_CONFIG_PATH/vault/overcloud.crt - - Or if environments are being used - - .. code-block:: - - rm $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.crt - -2. Generate a new certificate (and key): - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-deploy-seed.yml - -3. Encrypt generated key with ansible-vault (use proper location of vault password file) - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/vault/overcloud.key - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/vault/overcloud.key - -4. Copy the new certificate to the overcloud hosts. Note, if the old - certificate has expired this will fail on the unseal step. - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-deploy-overcloud.yml - -5. Restart the containers to use the new certificate: - - .. code-block:: - - kayobe overcloud host command run --command "docker restart vault" -l controllers - -6. If sealed, unseal the vault: - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-unseal-overcloud.yml - -Certificates generation -======================= - -.. note:: - - Generating certificates will fail if the Vault on the overcloud is sealed. This will happen whenever the vault containers are restarted. To unseal the - overcloud Vault, run: - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-unseal-overcloud.yml - -Create the external TLS certificates (testing only) ---------------------------------------------------- - -This method should only be used for testing. For external TLS on production systems, -See `Installing External TLS Certificates `__. - -Typically external API TLS certificates should be generated by a organisation's trusted internal or third-party CA. -For test and development purposes it is possible to use Vault as a CA for the external API. - -1. Run the playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-generate-test-external-tls.yml - -2. Use ansible-vault to encrypt the PEM bundle in $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy.pem. Commit the PEM bundle to the kayobe configuration. - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy.pem - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/haproxy.pem - -Create the internal TLS certificates ------------------------------------- - -1. Run the playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-generate-internal-tls.yml - -2. Use ansible-vault to encrypt the PEM bundle in $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy-internal.pem. Commit the PEM bundle and root CA to the kayobe configuration. - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/haproxy-internal.pem - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/haproxy-internal.pem - -Create the backend TLS and RabbitMQ TLS certificates ----------------------------------------------------- - -1. Run the playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-generate-backend-tls.yml - -2. Use ansible-vault to encrypt the keys in $KAYOBE_CONFIG_PATH/kolla/certificates/-key.pem. Commit the certificates and keys to the kayobe configuration. - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/kolla/certificates/-key.pem - - Or if environments are being used - - .. code-block:: - - ansible-vault encrypt --vault-password-file ~/vault.pass $KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/kolla/certificates/-key.pem - -.. _vault-haproxy: - -HAProxy integration -=================== - -It is possible to expose the overcloud Vault service via the Kolla Ansible HAProxy load balancer. -This provides a single highly available API endpoint, as well as monitoring of the Vault backends when combined with Prometheus. -HAProxy integration is no longer required for generating OpenStack control plane certificates, making it possible to deploy Vault and generate certificates before any containers have been deployed by Kolla Ansible. - -1. Create the HAProxy config to reverse proxy the Vault HA container - - Set the vault_front to the external VIP address or internal VIP address depending on the installation. Set the vault_back to the IPs of the control nodes. - - Set the following in etc/kayobe/kolla/config/haproxy/services.d/vault.cfg or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/config/haproxy/services.d/vault.cfg - - .. code-block:: - - # Delete "verify none" if not using self-signed/unknown issuer - {% raw %} - frontend vault_front - mode tcp - option tcplog - bind {{ kolla_internal_vip_address }}:8200 - default_backend vault_back - - backend vault_back - mode tcp - option httpchk GET /v1/sys/health - # https://www.vaultproject.io/api-docs/system/health - # 200: initialized, unsealed, and active - # 429: standby - http-check expect rstatus (200|429) - - {% for host in groups['control'] %} - {% set host_name = hostvars[host].ansible_facts.hostname %} - {% set host_ip = 'api' | kolla_address(host) %} - server {{ host_name }} {{ host_ip }}:8200 check check-ssl verify none inter 2000 rise 2 fall 5 - {% endfor %} - {% endraw %} - -2. If HAProxy has not yet been deployed, continue to :ref:`certificates deployment `. - If HAProxy has been deployed, it may be redeployed with the new Vault service configuration: - - .. code-block:: - - kayobe overcloud service deploy -kt haproxy - -.. _vault-certificates: - -Certificates deployment -======================= - -.. warning:: - - The switch from HTTP to HTTPS during the deployment of internal/backend TLS certificates can temporarily disrupt service availability and necessitates a restart of all services. During this transition, endpoints may become unreachable following the HAProxy restart, persisting until the endpoint catalogue and client have been reconfigured to use HTTPS. - -Enable the required TLS variables in kayobe and kolla ------------------------------------------------------ - -1. If using Vault as a CA for the external API, set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml - - .. code-block:: - - # Whether TLS is enabled for the external API endpoints. Default is 'no'. - kolla_enable_tls_external: yes - kolla_public_openrc_cacert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if os_distribution in ['centos', 'rocky'] else '/etc/ssl/certs/ca-certificates.crt' }}" - - See :ref:`tempest-cacert` for information on adding CA certificates to the trust store when running Tempest. - -2. Set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml - - .. code-block:: - - # Whether TLS is enabled for the internal API endpoints. Default is 'no'. - kolla_enable_tls_internal: yes - kolla_admin_openrc_cacert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if os_distribution in ['centos', 'rocky'] else '/etc/ssl/certs/ca-certificates.crt' }}" - - See :ref:`os-capacity` for information on adding CA certificates to the trust store when deploying the OpenStack Capacity exporter. - -3. Set the following in etc/kayobe/kolla/globals.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/globals.yml - - .. code-block:: - - # Internal TLS configuration - # Copy the self-signed CA into the kolla containers - kolla_copy_ca_into_containers: "yes" - # Use the following trust store within the container - openstack_cacert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if os_distribution == 'rocky' else '/etc/ssl/certs/ca-certificates.crt' }}" - - # Backend TLS config - # Enable backend TLS - kolla_enable_tls_backend: "yes" - - # If using RabbitMQ TLS: - rabbitmq_enable_tls: "yes" - -4. Deploy OpenStack - - .. warning:: - - It is important that you are only using admin endpoints for keystone. If - any admin endpoints exist for other services, they must be deleted e.g. - - .. code-block:: - - openstack endpoint list --interface admin -f value | \ - awk '!/keystone/ {print $1}' | xargs openstack endpoint delete - - .. code-block:: - - kayobe overcloud service deploy - - If VM provisioning fails with an error with this format: - - .. code-block:: - - Unable to establish connection to http://:9696/v2.0/ports/some-sort-of-uuid: Connection aborted - - Restart the nova-compute container on all hypervisors: - - .. code-block:: - - kayobe overcloud host command run --command "systemctl restart kolla-nova_compute-container.service" --become --show-output -l compute - -Barbican integration -==================== - -Barbican integration depends on :ref:`HAProxy integration `. - -Enable Barbican in kayobe -------------------------- - -1. Set the following in kayobe-config/etc/kayobe/kolla.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla.yml - - .. code-block:: - - kolla_enable_barbican: yes - -Generate secrets_barbican_approle_secret_id -------------------------------------------- - -1. Run ``uuidgen`` to generate secret id -2. Insert into secrets.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/secrets.yml - - .. code-block:: - - secrets_barbican_approle_secret_id: "YOUR-SECRET-GOES-HERE" - -Create required configuration in Vault --------------------------------------- - -1. Run vault-deploy-barbican.yml custom playbook - - .. code-block:: - - kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/vault-deploy-barbican.yml - -Add secrets_barbican_approle_id to secrets ------------------------------------------- - -1. Note the role id from playbook output and insert into secrets.yml or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/secrets.yml - - .. code-block:: - - secrets_barbican_approle_role_id: "YOUR-APPROLE-ID-GOES-HERE" - -Configure Barbican ------------------- - -1. Put required configuration in kayobe-config/etc/kayobe/kolla/config/barbican.conf or if environments are being used etc/kayobe/environments/$KAYOBE_ENVIRONMENT/kolla/config/barbican.conf - - .. code-block:: - - [secretstore] - namespace=barbican.secretstore.plugin - enable_multiple_secret_stores=false - enabled_secretstore_plugins=vault_plugin - - [vault_plugin] - vault_url = https://{{ kolla_internal_fqdn }}:8200 - use_ssl = True - {% raw %} - ssl_ca_crt_file = {{ openstack_cacert }} - {% endraw %} - approle_role_id = {{ secrets_barbican_approle_role_id }} - approle_secret_id = {{ secrets_barbican_approle_secret_id }} - kv_mountpoint = barbican - -Deploy Barbican ---------------- - - .. code-block:: - - kayobe overcloud service deploy -kt barbican diff --git a/etc/kayobe/ansible/openbao-deploy-seed.yml b/etc/kayobe/ansible/openbao-deploy-seed.yml deleted file mode 100644 index 42972b35e..000000000 --- a/etc/kayobe/ansible/openbao-deploy-seed.yml +++ /dev/null @@ -1,78 +0,0 @@ ---- -- name: Deploy OpenBao on the seed - any_errors_fatal: true - gather_facts: true - hosts: seed - vars: - openbao_bind_addr: "{{ ansible_facts['lo'].ipv4.address }}" - openbao_api_addr: "http://{{ openbao_bind_addr }}:8200" - tasks: - - name: Set a fact about the virtualenv on the remote system - ansible.builtin.set_fact: - virtualenv: "{{ ansible_python_interpreter | dirname | dirname }}" - when: - - ansible_python_interpreter is defined - - not ansible_python_interpreter.startswith('/bin/') - - not ansible_python_interpreter.startswith('/usr/bin/') - - - name: Ensure Python PyYAML and hvac modules are installed - ansible.builtin.pip: - name: - - PyYAML - - hvac - state: latest - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" - become: "{{ virtualenv is not defined }}" - - - name: Ensure OpenBao directory exists in Kayobe configuration - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/openbao/" - state: directory - delegate_to: localhost - run_once: true - - - name: Apply OpenBao role - ansible.builtin.import_role: - name: stackhpc.hashicorp.openbao - vars: - openbao_registry_url: "{{ seed_openbao_registry_url }}" - openbao_registry_username: "{{ seed_openbao_registry_username }}" - openbao_registry_password: "{{ seed_openbao_registry_password }}" - openbao_config_dir: /opt/kayobe/openbao - openbao_cluster_name: seed - openbao_docker_image: "{{ seed_openbao_docker_image }}" - openbao_docker_tag: "{{ seed_openbao_docker_tag }}" - openbao_write_keys_file: true - openbao_write_keys_file_path: "{{ kayobe_env_config_path }}/openbao/seed-openbao-keys.json" - - - name: Include Vault keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/seed-openbao-keys.json" - name: openbao_keys - - - name: Unseal OpenBao - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_unseal - vars: - vault_api_addr: "{{ openbao_api_addr }}" - vault_unseal_keys: "{{ openbao_keys.keys_base64 }}" - - - name: Apply PKI role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_pki - vars: - vault_api_addr: "{{ openbao_api_addr }}" - vault_token: "{{ openbao_keys.root_token }}" - vault_pki_root_ca_name: OS-TLS-ROOT - vault_pki_write_root_ca_to_file: true - vault_pki_intermediate_ca_name: OS-TLS-INT - vault_pki_intermediate_export: true - vault_pki_intermediate_roles: "{{ seed_openbao_pki_roles }}" - vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/openbao" - vault_pki_generate_certificates: true - vault_pki_write_certificates: true - vault_pki_certificate_subject: "{{ seed_openbao_pki_certificate_subject }}" - vault_pki_write_certificate_files: true - vault_pki_write_pem_bundle: false - vault_pki_write_int_ca_to_file: true diff --git a/etc/kayobe/ansible/openbao-generate-internal-tls.yml b/etc/kayobe/ansible/openbao-generate-internal-tls.yml deleted file mode 100644 index 2cc9e841a..000000000 --- a/etc/kayobe/ansible/openbao-generate-internal-tls.yml +++ /dev/null @@ -1,56 +0,0 @@ ---- -- name: Generate internal API certificate - hosts: controllers - run_once: true - vars: - openbao_api_addr: https://{{ internal_net_name | net_ip }}:8200 - openbao_intermediate_ca_name: OS-TLS-INT - tasks: - - name: Include OpenBao keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys - - - name: Issue a certificate for internal TLS - hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ openbao_keys.root_token }}" - mount_point: "{{ openbao_intermediate_ca_name }}" - role: "{{ overcloud_openbao_pki_internal_tls_role_name }}" - common_name: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% endif %}" - extra_params: - ip_sans: "{{ kolla_internal_vip_address }}" - register: internal_cert - environment: - https_proxy: "" - - - name: Ensure certificates directory exists - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/kolla/certificates" - state: directory - delegate_to: localhost - - - name: Ensure CA certificates directory exists - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/kolla/certificates/ca" - state: directory - delegate_to: localhost - - - name: Copy internal API PEM bundle - no_log: true - ansible.builtin.copy: - dest: "{{ kayobe_env_config_path }}/kolla/certificates/haproxy-internal.pem" - content: | - {{ internal_cert.data.certificate }} - {{ internal_cert.data.issuing_ca }} - {{ internal_cert.data.private_key }} - mode: "0600" - delegate_to: localhost - - - name: Copy root CA - ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/openbao/OS-TLS-ROOT.pem" - dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/openbao.crt" - mode: "0600" - delegate_to: localhost diff --git a/etc/kayobe/ansible/openbao-unseal-overcloud.yml b/etc/kayobe/ansible/openbao-unseal-overcloud.yml deleted file mode 100644 index 0a631c598..000000000 --- a/etc/kayobe/ansible/openbao-unseal-overcloud.yml +++ /dev/null @@ -1,37 +0,0 @@ ---- -- name: Unseal OpenBao on the overcloud - any_errors_fatal: true - gather_facts: true - hosts: controllers - tasks: - - name: Set a fact about the virtualenv on the remote system - ansible.builtin.set_fact: - virtualenv: "{{ ansible_python_interpreter | dirname | dirname }}" - when: - - ansible_python_interpreter is defined - - not ansible_python_interpreter.startswith('/bin/') - - not ansible_python_interpreter.startswith('/usr/bin/') - - - name: Ensure Python hvac module is installed - ansible.builtin.pip: - name: hvac - state: latest - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" - become: "{{ virtualenv is not defined }}" - - - name: Include OpenBao keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys - - - name: Apply OpenBao unseal role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_unseal - vars: - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_unseal_token: "{{ openbao_keys.root_token }}" - vault_unseal_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_unseal_keys: "{{ openbao_keys.keys_base64 }}" - environment: - https_proxy: "" diff --git a/etc/kayobe/ansible/openbao-unseal-seed.yml b/etc/kayobe/ansible/openbao-unseal-seed.yml deleted file mode 100644 index 82f9b8fa7..000000000 --- a/etc/kayobe/ansible/openbao-unseal-seed.yml +++ /dev/null @@ -1,34 +0,0 @@ ---- -- name: Unseal OpenBao on the seed - any_errors_fatal: true - gather_facts: true - hosts: seed - vars: - vault_api_addr: http://127.0.0.1:8200 - tasks: - - name: Set a fact about the virtualenv on the remote system - ansible.builtin.set_fact: - virtualenv: "{{ ansible_python_interpreter | dirname | dirname }}" - when: - - ansible_python_interpreter is defined - - not ansible_python_interpreter.startswith('/bin/') - - not ansible_python_interpreter.startswith('/usr/bin/') - - - name: Ensure Python hvac module is installed - ansible.builtin.pip: - name: hvac - state: latest - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" - become: "{{ virtualenv is not defined }}" - - - name: Include OpenBao keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/seed-openbao-keys.json" - name: openbao_keys - - - name: Apply OpenBao unseal role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_unseal - vars: - vault_unseal_keys: "{{ openbao_keys.keys_base64 }}" diff --git a/etc/kayobe/ansible/openbao-deploy-barbican.yml b/etc/kayobe/ansible/secret-store-deploy-barbican.yml similarity index 65% rename from etc/kayobe/ansible/openbao-deploy-barbican.yml rename to etc/kayobe/ansible/secret-store-deploy-barbican.yml index 50d55aa22..4324ff2ef 100644 --- a/etc/kayobe/ansible/openbao-deploy-barbican.yml +++ b/etc/kayobe/ansible/secret-store-deploy-barbican.yml @@ -4,8 +4,8 @@ gather_facts: true hosts: controllers[0] vars: - openbao_api_addr: https://{{ internal_net_name | net_ip }}:8200 - openbao_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" + secret_store_api_address: https://{{ internal_net_name | net_ip }}:8200 + secret_store_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" tasks: - name: Assert that secrets_barbican_approle_secret_id is defined ansible.builtin.assert: @@ -13,10 +13,10 @@ - secrets_barbican_approle_secret_id is defined fail_msg: Please define secrets_barbican_approle_secret_id in your secrets.yml - - name: Include OpenBao keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - name: Ensure hvac is installed ansible.builtin.pip: @@ -31,26 +31,26 @@ block: - name: Enable AppRole auth module hashivault_auth_method: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" method_type: approle state: enabled - name: Enable barbican kv store hashivault_secret_engine: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" name: barbican backend: kv description: Barbican kv store - name: Ensure barbican policy is defined hashivault_policy: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" name: barbican-policy state: present rules: | @@ -60,9 +60,9 @@ - name: Ensure barbican AppRole is defined hashivault_approle_role: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" bind_secret_id: true secret_id_bound_cidrs: "{{ internal_net_name | net_cidr }}" secret_id_ttl: 0 @@ -71,9 +71,9 @@ - name: Get barbican Approle ID hashivault_approle_role_id: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" name: barbican register: barbican_role_id @@ -90,18 +90,18 @@ - name: Check if barbican Approle Secret ID is defined hashivault_approle_role_secret_get: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" secret: "{{ secrets_barbican_approle_secret_id }}" name: barbican register: barbican_approle_secret_get - name: Ensure barbican AppRole Secret ID is defined hashivault_approle_role_secret: # noqa: fqcn - url: "{{ openbao_api_addr }}" - ca_cert: "{{ openbao_ca_cert }}" - token: "{{ openbao_keys.root_token }}" + url: "{{ secret_store_api_address }}" + ca_cert: "{{ secret_store_ca_cert }}" + token: "{{ secret_store_keys.root_token }}" secret: "{{ secrets_barbican_approle_secret_id }}" name: barbican when: barbican_approle_secret_get.status == "absent" diff --git a/etc/kayobe/ansible/openbao-deploy-overcloud.yml b/etc/kayobe/ansible/secret-store-deploy-overcloud.yml similarity index 64% rename from etc/kayobe/ansible/openbao-deploy-overcloud.yml rename to etc/kayobe/ansible/secret-store-deploy-overcloud.yml index 8b2a59860..aa9a200d1 100644 --- a/etc/kayobe/ansible/openbao-deploy-overcloud.yml +++ b/etc/kayobe/ansible/secret-store-deploy-overcloud.yml @@ -6,7 +6,7 @@ tasks: - name: Copy the intermediate CA ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/openbao/OS-TLS-ROOT.pem" + src: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/OS-TLS-ROOT.pem" dest: "{{ '/etc/pki/ca-trust/source/anchors/OS-TLS-ROOT.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" mode: "0644" @@ -16,12 +16,13 @@ become: true ansible.builtin.command: "{{ 'update-ca-trust' if ansible_facts.os_family == 'RedHat' else 'update-ca-certificates' }}" -- name: Deploy OpenBao on the overcloud +- name: Deploy Secret store on the overcloud any_errors_fatal: true gather_facts: true hosts: controllers vars: - openbao_bind_addr: "{{ internal_net_name | net_ip }}" + secret_store_bind_interface: "{{ internal_net_name | net_interface }}" + secret_store_bind_address: "{{ internal_net_name | net_ip }}" # This is the IP address of the first controller and therefore the leader within # OpenBao. This could be replaced with the VIP address of the internal network if # HAProxy has been configured to load balance the OpenBao API. @@ -44,16 +45,16 @@ virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" become: "{{ virtualenv is not defined }}" - - name: Ensure /opt/kayobe/openbao exists + - name: Ensure /opt/kayobe/{{ stackhpc_ca_secret_store }} exists ansible.builtin.file: - path: /opt/kayobe/openbao + path: /opt/kayobe/{{ stackhpc_ca_secret_store }} state: directory - name: Template out TLS key and cert ansible.builtin.copy: # Within the OpenBao container these uids & gids map to the openbao user - src: "{{ kayobe_env_config_path }}/openbao/{{ item }}" - dest: /opt/kayobe/openbao/{{ item }} + src: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/{{ item }}" + dest: /opt/kayobe/{{ stackhpc_ca_secret_store }}/{{ item }} owner: 100 group: 1000 mode: "0600" @@ -63,10 +64,35 @@ - "OS-TLS-INT.crt" become: true + - name: Apply vault role + ansible.builtin.import_role: + name: stackhpc.hashicorp.vault + vars: + hashicorp_registry_url: "{{ overcloud_hashicorp_registry_url }}" + hashicorp_registry_username: "{{ overcloud_hashicorp_registry_username }}" + hashicorp_registry_password: "{{ overcloud_hashicorp_registry_password }}" + consul_docker_image: "{{ overcloud_consul_docker_image }}" + consul_docker_tag: "{{ overcloud_consul_docker_tag }}" + consul_bind_interface: "{{ secret_store_bind_interface }}" + vault_bind_address: "{{ secret_store_bind_address }}" + vault_config_dir: /opt/kayobe/vault + vault_cluster_name: overcloud + vault_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" + vault_docker_image: "{{ overcloud_vault_docker_image }}" + vault_docker_tag: "{{ overcloud_vault_docker_tag }}" + vault_tls_cert: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.crt" + vault_tls_key: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.key" + copy_self_signed_ca: true + vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 + vault_write_keys_file: true + vault_write_keys_file_path: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" + when: stackhpc_ca_secret_store == "vault" + - name: Apply OpenBao role ansible.builtin.import_role: name: stackhpc.hashicorp.openbao vars: + openbao_bind_addr: "{{ secret_store_bind_address }}" openbao_registry_url: "{{ overcloud_openbao_registry_url }}" openbao_registry_username: "{{ overcloud_openbao_registry_username }}" openbao_registry_password: "{{ overcloud_openbao_registry_password }}" @@ -82,20 +108,21 @@ openbao_api_addr: https://{{ internal_net_name | net_ip }}:8200 openbao_write_keys_file: true openbao_write_keys_file_path: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" + when: stackhpc_ca_secret_store == "openbao" - - name: Include OpenBao keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - - name: Unseal first OpenBao instance + - name: Unseal first secret store instance ansible.builtin.import_role: name: stackhpc.hashicorp.vault_unseal vars: vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_unseal_token: "{{ openbao_keys.root_token }}" + vault_unseal_token: "{{ secret_store_keys.root_token }}" vault_unseal_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_unseal_keys: "{{ openbao_keys.keys_base64 }}" + vault_unseal_keys: "{{ secret_store_keys.keys_base64 }}" environment: https_proxy: "" run_once: true @@ -109,14 +136,14 @@ # Raft peers take few seconds before they report an unsealed state therefore # we must wait. - - name: Unseal all OpenBao instances + - name: Unseal all secret store instances ansible.builtin.import_role: name: stackhpc.hashicorp.vault_unseal vars: vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_unseal_token: "{{ openbao_keys.root_token }}" + vault_unseal_token: "{{ secret_store_keys.root_token }}" vault_unseal_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_unseal_keys: "{{ openbao_keys.keys_base64 }}" + vault_unseal_keys: "{{ secret_store_keys.keys_base64 }}" vault_unseal_timeout: 10 environment: https_proxy: "" @@ -126,20 +153,20 @@ gather_facts: true hosts: controllers[0] tasks: - - name: Apply OpenBao pki role + - name: Apply pki role ansible.builtin.import_role: name: stackhpc.hashicorp.vault_pki vars: - vault_token: "{{ openbao_keys.root_token }}" + vault_token: "{{ secret_store_keys.root_token }}" vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 vault_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" vault_pki_root_create: false vault_pki_intermediate_import: true vault_pki_intermediate_ca_name: OS-TLS-INT - vault_pki_intermediate_ca_bundle: "{{ lookup('file', kayobe_env_config_path + '/openbao/OS-TLS-INT.pem') }}" - vault_pki_intermediate_ca_cert: "{{ lookup('file', kayobe_env_config_path + '/openbao/OS-TLS-INT.crt') }}" - vault_pki_intermediate_roles: "{{ overcloud_openbao_pki_roles }}" + vault_pki_intermediate_ca_bundle: "{{ lookup('file', kayobe_env_config_path + '/' + stackhpc_ca_secret_store + '/OS-TLS-INT.pem') }}" + vault_pki_intermediate_ca_cert: "{{ lookup('file', kayobe_env_config_path + '/' + stackhpc_ca_secret_store + '/OS-TLS-INT.crt') }}" + vault_pki_intermediate_roles: "{{ overcloud_vault_pki_roles if stackhpc_ca_secret_store == 'vault' else overcloud_openbao_pki_roles }}" vault_pki_write_certificate_files: true - vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/openbao" + vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}" environment: https_proxy: "" diff --git a/etc/kayobe/ansible/vault-deploy-seed.yml b/etc/kayobe/ansible/secret-store-deploy-seed.yml similarity index 50% rename from etc/kayobe/ansible/vault-deploy-seed.yml rename to etc/kayobe/ansible/secret-store-deploy-seed.yml index 9617aa905..44073edf6 100644 --- a/etc/kayobe/ansible/vault-deploy-seed.yml +++ b/etc/kayobe/ansible/secret-store-deploy-seed.yml @@ -1,12 +1,12 @@ --- -- name: Deploy Hashicorp Vault on the seed +- name: Deploy CA secret store on the seed any_errors_fatal: true gather_facts: true hosts: seed vars: - consul_bind_interface: lo - vault_bind_address: "{{ ansible_facts[consul_bind_interface].ipv4.address }}" - vault_api_addr: http://{{ vault_bind_address }}:8200 + secret_store_bind_interface: lo + secret_store_bind_address: "{{ ansible_facts[secret_store_bind_interface].ipv4.address }}" + secret_store_api_address: "http://{{ secret_store_bind_address }}:8200" tasks: - name: Set a fact about the virtualenv on the remote system ansible.builtin.set_fact: @@ -26,9 +26,9 @@ virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" become: "{{ virtualenv is not defined }}" - - name: Ensure vault directory exists in Kayobe configuration + - name: Ensure secret store directory exists in Kayobe configuration ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/vault/" + path: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/" state: directory delegate_to: localhost run_once: true @@ -42,42 +42,61 @@ hashicorp_registry_password: "{{ seed_hashicorp_registry_password }}" consul_docker_image: "{{ seed_consul_docker_image }}" consul_docker_tag: "{{ seed_consul_docker_tag }}" + consul_bind_interface: "{{ secret_store_bind_interface }}" + vault_bind_address: "{{ secret_store_bind_address }}" + vault_api_addr: "{{ secret_store_api_address }}" vault_config_dir: /opt/kayobe/vault vault_cluster_name: seed vault_docker_image: "{{ seed_vault_docker_image }}" vault_docker_tag: "{{ seed_vault_docker_tag }}" vault_write_keys_file: true vault_write_keys_file_path: "{{ kayobe_env_config_path }}/vault/seed-vault-keys.json" + when: stackhpc_ca_secret_store == "vault" - - name: Include Vault keys + - name: Apply OpenBao role + ansible.builtin.import_role: + name: stackhpc.hashicorp.openbao + vars: + openbao_bind_addr: "{{ secret_store_bind_address }}" + openbao_api_addr: "{{ secret_store_api_address }}" + openbao_registry_url: "{{ seed_openbao_registry_url }}" + openbao_registry_username: "{{ seed_openbao_registry_username }}" + openbao_registry_password: "{{ seed_openbao_registry_password }}" + openbao_config_dir: /opt/kayobe/openbao + openbao_cluster_name: seed + openbao_docker_image: "{{ seed_openbao_docker_image }}" + openbao_docker_tag: "{{ seed_openbao_docker_tag }}" + openbao_write_keys_file: true + openbao_write_keys_file_path: "{{ kayobe_env_config_path }}/openbao/seed-openbao-keys.json" + when: stackhpc_ca_secret_store == "openbao" + + - name: Include {{ stackhpc_ca_secret_store }} keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/seed-vault-keys.json" - name: vault_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/seed-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - - name: Apply vault unseal role + - name: Unseal {{ stackhpc_ca_secret_store }} ansible.builtin.import_role: name: stackhpc.hashicorp.vault_unseal vars: - vault_unseal_keys: "{{ vault_keys.keys_base64 }}" + vault_api_addr: "{{ secret_store_api_address }}" + vault_unseal_keys: "{{ secret_store_keys.keys_base64 }}" - - name: Apply vault PKI role + - name: Apply PKI role ansible.builtin.import_role: name: stackhpc.hashicorp.vault_pki vars: - vault_token: "{{ vault_keys.root_token }}" + vault_api_addr: "{{ secret_store_api_address }}" + vault_token: "{{ secret_store_keys.root_token }}" vault_pki_root_ca_name: OS-TLS-ROOT vault_pki_write_root_ca_to_file: true vault_pki_intermediate_ca_name: OS-TLS-INT vault_pki_intermediate_export: true - vault_pki_intermediate_roles: "{{ seed_vault_pki_roles }}" - vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/vault" + vault_pki_intermediate_roles: "{{ seed_vault_pki_roles if stackhpc_ca_secret_store == 'vault' else seed_openbao_pki_roles }}" + vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}" vault_pki_generate_certificates: true vault_pki_write_certificates: true - vault_pki_certificate_subject: - - common_name: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}" - role: "{{ seed_vault_pki_role_name }}" - extra_params: - ip_sans: "{% for host in groups['controllers'] %}{{ internal_net_name | net_ip(host) }}{% if not loop.last %},{% endif %}{% endfor %},{{ kolla_internal_vip_address }}" + vault_pki_certificate_subject: "{{ seed_vault_pki_certificate_subject if stackhpc_ca_secret_store == 'vault' else seed_openbao_pki_certificate_subject }}" vault_pki_write_certificate_files: true vault_pki_write_pem_bundle: false vault_pki_write_int_ca_to_file: true diff --git a/etc/kayobe/ansible/openbao-generate-backend-tls.yml b/etc/kayobe/ansible/secret-store-generate-backend-tls.yml similarity index 78% rename from etc/kayobe/ansible/openbao-generate-backend-tls.yml rename to etc/kayobe/ansible/secret-store-generate-backend-tls.yml index f43513ff1..4b0eb87ee 100644 --- a/etc/kayobe/ansible/openbao-generate-backend-tls.yml +++ b/etc/kayobe/ansible/secret-store-generate-backend-tls.yml @@ -6,7 +6,7 @@ tasks: - name: Copy the intermediate CA ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/openbao/OS-TLS-ROOT.pem" + src: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/OS-TLS-ROOT.pem" dest: "{{ '/etc/pki/ca-trust/source/anchors/OS-TLS-ROOT.crt' if ansible_facts.os_family == 'RedHat' \ else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" mode: "0644" @@ -19,8 +19,8 @@ - name: Generate backend API certificates hosts: controllers:network vars: - openbao_api_addr: https://{{ internal_net_name | net_ip(groups['controllers'][0]) }}:8200 - openbao_intermediate_ca_name: OS-TLS-INT + secret_store_api_address: https://{{ internal_net_name | net_ip(groups['controllers'][0]) }}:8200 + secret_store_intermediate_ca_name: OS-TLS-INT tasks: - name: Set a fact about the virtualenv on the remote system ansible.builtin.set_fact: @@ -38,18 +38,18 @@ virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" become: "{{ virtualenv is not defined }}" - - name: Include OpenBao keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - name: Issue a certificate for backend TLS hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ openbao_api_addr }}" + url: "{{ secret_store_api_address }}" ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ openbao_keys.root_token }}" - mount_point: "{{ openbao_intermediate_ca_name }}" - role: "{{ overcloud_openbao_pki_backend_tls_role_name }}" + token: "{{ secret_store_keys.root_token }}" + mount_point: "{{ secret_store_intermediate_ca_name }}" + role: "{{ overcloud_vault_pki_backend_tls_role_name if stackhpc_ca_secret_store == 'vault' else overcloud_openbao_pki_backend_tls_role_name }}" common_name: "" extra_params: ip_sans: "{{ internal_net_name | net_ip }}" diff --git a/etc/kayobe/ansible/vault-generate-internal-tls.yml b/etc/kayobe/ansible/secret-store-generate-internal-tls.yml similarity index 67% rename from etc/kayobe/ansible/vault-generate-internal-tls.yml rename to etc/kayobe/ansible/secret-store-generate-internal-tls.yml index a585d1bc9..ea9dfd929 100644 --- a/etc/kayobe/ansible/vault-generate-internal-tls.yml +++ b/etc/kayobe/ansible/secret-store-generate-internal-tls.yml @@ -3,21 +3,21 @@ hosts: controllers run_once: true vars: - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_intermediate_ca_name: OS-TLS-INT + secret_store_api_address: https://{{ internal_net_name | net_ip }}:8200 + secret_store_intermediate_ca_name: OS-TLS-INT tasks: - - name: Include Vault keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - name: Issue a certificate for internal TLS hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ vault_api_addr }}" + url: "{{ secret_store_api_address }}" ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ vault_keys.root_token }}" - mount_point: "{{ vault_intermediate_ca_name }}" - role: "{{ overcloud_vault_pki_internal_tls_role_name }}" + token: "{{ secret_store_keys.root_token }}" + mount_point: "{{ secret_store_intermediate_ca_name }}" + role: "{{ overcloud_vault_pki_internal_tls_role_name if stackhpc_ca_secret_store == 'vault' else overcloud_openbao_pki_internal_tls_role_name }}" common_name: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% endif %}" extra_params: ip_sans: "{{ kolla_internal_vip_address }}" @@ -50,7 +50,7 @@ - name: Copy root CA ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/vault/OS-TLS-ROOT.pem" - dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/vault.crt" + src: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/OS-TLS-ROOT.pem" + dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/{{ stackhpc_ca_secret_store }}.crt" mode: "0600" delegate_to: localhost diff --git a/etc/kayobe/ansible/openbao-generate-test-external-tls.yml b/etc/kayobe/ansible/secret-store-generate-test-external-tls.yml similarity index 68% rename from etc/kayobe/ansible/openbao-generate-test-external-tls.yml rename to etc/kayobe/ansible/secret-store-generate-test-external-tls.yml index e7150fed1..206bf2b7a 100644 --- a/etc/kayobe/ansible/openbao-generate-test-external-tls.yml +++ b/etc/kayobe/ansible/secret-store-generate-test-external-tls.yml @@ -3,22 +3,22 @@ hosts: controllers run_once: true vars: - openbao_api_addr: https://{{ internal_net_name | net_ip }}:8200 + secret_store_api_address: https://{{ internal_net_name | net_ip }}:8200 # NOTE: Using the same CA as internal TLS. - openbao_intermediate_ca_name: OS-TLS-INT + secret_store_intermediate_ca_name: OS-TLS-INT tasks: - - name: Include OpenBao keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/openbao/overcloud-openbao-keys.json" - name: openbao_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - name: Issue a certificate for external TLS hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ openbao_api_addr }}" + url: "{{ secret_store_api_address }}" ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ openbao_keys.root_token }}" - mount_point: "{{ openbao_intermediate_ca_name }}" - role: "{{ overcloud_openbao_pki_external_tls_role_name }}" + token: "{{ secret_store_keys.root_token }}" + mount_point: "{{ secret_store_intermediate_ca_name }}" + role: "{{ overcloud_vault_pki_external_tls_role_name if stackhpc_ca_secret_store == 'vault' else overcloud_openbao_pki_external_tls_role_name }}" common_name: "{% if kolla_external_fqdn != kolla_external_vip_address %}{{ kolla_external_fqdn }}{% endif %}" extra_params: ip_sans: "{{ kolla_external_vip_address }}" @@ -51,7 +51,7 @@ - name: Copy root CA ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/openbao/OS-TLS-ROOT.pem" - dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/openbao.crt" + src: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/OS-TLS-ROOT.pem" + dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/{{ stackhpc_ca_secret_store }}.crt" mode: "0600" delegate_to: localhost diff --git a/etc/kayobe/ansible/vault-unseal-overcloud.yml b/etc/kayobe/ansible/secret-store-unseal-overcloud.yml similarity index 74% rename from etc/kayobe/ansible/vault-unseal-overcloud.yml rename to etc/kayobe/ansible/secret-store-unseal-overcloud.yml index d5d38678e..634e7e029 100644 --- a/etc/kayobe/ansible/vault-unseal-overcloud.yml +++ b/etc/kayobe/ansible/secret-store-unseal-overcloud.yml @@ -1,5 +1,5 @@ --- -- name: Deploy HashiCorp Vault on the overcloud +- name: Unseal secret store on the overcloud any_errors_fatal: true gather_facts: true hosts: controllers @@ -20,18 +20,18 @@ virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" become: "{{ virtualenv is not defined }}" - - name: Include Vault keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/overcloud-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - - name: Apply vault unseal role + - name: Apply secret store unseal role ansible.builtin.import_role: name: stackhpc.hashicorp.vault_unseal vars: vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_unseal_token: "{{ vault_keys.root_token }}" + vault_unseal_token: "{{ secret_store_keys.root_token }}" vault_unseal_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_unseal_keys: "{{ vault_keys.keys_base64 }}" + vault_unseal_keys: "{{ secret_store_keys.keys_base64 }}" environment: https_proxy: "" diff --git a/etc/kayobe/ansible/vault-unseal-seed.yml b/etc/kayobe/ansible/secret-store-unseal-seed.yml similarity index 74% rename from etc/kayobe/ansible/vault-unseal-seed.yml rename to etc/kayobe/ansible/secret-store-unseal-seed.yml index 3c6428b44..a404101ed 100644 --- a/etc/kayobe/ansible/vault-unseal-seed.yml +++ b/etc/kayobe/ansible/secret-store-unseal-seed.yml @@ -1,5 +1,5 @@ --- -- name: Deploy Hashicorp Vault on the seed +- name: Unseal secret store on the seed any_errors_fatal: true gather_facts: true hosts: seed @@ -22,13 +22,13 @@ virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" become: "{{ virtualenv is not defined }}" - - name: Include Vault keys + - name: Include secret store keys ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/seed-vault-keys.json" - name: vault_keys + file: "{{ kayobe_env_config_path }}/{{ stackhpc_ca_secret_store }}/seed-{{ stackhpc_ca_secret_store }}-keys.json" + name: secret_store_keys - - name: Apply vault unseal role + - name: Apply unseal role ansible.builtin.import_role: name: stackhpc.hashicorp.vault_unseal vars: - vault_unseal_keys: "{{ vault_keys.keys_base64 }}" + vault_unseal_keys: "{{ secret_store_keys.keys_base64 }}" diff --git a/etc/kayobe/ansible/vault-deploy-barbican.yml b/etc/kayobe/ansible/vault-deploy-barbican.yml deleted file mode 100644 index 54f17bb35..000000000 --- a/etc/kayobe/ansible/vault-deploy-barbican.yml +++ /dev/null @@ -1,107 +0,0 @@ ---- -- name: Configure AppRole - any_errors_fatal: true - gather_facts: true - hosts: controllers[0] - vars: - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - tasks: - - name: Assert that secrets_barbican_approle_secret_id is defined - ansible.builtin.assert: - that: - - secrets_barbican_approle_secret_id is defined - fail_msg: Please define secrets_barbican_approle_secret_id in your secrets.yml - - - name: Include Vault keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys - - - name: Ensure hvac is installed - ansible.builtin.pip: - name: hvac - state: present - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv_path }}/kayobe" - - - name: Ensure AppRole is configured - environment: - https_proxy: "" - block: - - name: Enable AppRole auth module - hashivault_auth_method: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - method_type: approle - state: enabled - - - name: Enable barbican kv store - hashivault_secret_engine: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - name: barbican - backend: kv - description: Barbican kv store - - - name: Ensure barbican policy is defined - hashivault_policy: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - name: barbican-policy - state: present - rules: | - path "barbican/*" { - capabilities = ["create", "read", "update", "delete", "list"] - } - - - name: Ensure barbican AppRole is defined - hashivault_approle_role: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - bind_secret_id: true - secret_id_bound_cidrs: "{{ internal_net_name | net_cidr }}" - secret_id_ttl: 0 - token_policies: barbican-policy - name: barbican - - - name: Get barbican Approle ID - hashivault_approle_role_id: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - name: barbican - register: barbican_role_id - - - name: Print barbican Approle ID - ansible.builtin.debug: - msg: barbican role id is {{ barbican_role_id.id }} - - - name: Write barbican Approle ID to file if requested - delegate_to: localhost - ansible.builtin.copy: - content: "{{ barbican_role_id.id }}" - dest: "{{ stackhpc_barbican_role_id_file_path | default('~/barbican-role-id') }}" - when: stackhpc_write_barbican_role_id_to_file | default(false) | bool - - - name: Check if barbican Approle Secret ID is defined - hashivault_approle_role_secret_get: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - secret: "{{ secrets_barbican_approle_secret_id }}" - name: barbican - register: barbican_approle_secret_get - - - name: Ensure barbican AppRole Secret ID is defined - hashivault_approle_role_secret: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ vault_ca_cert }}" - token: "{{ vault_keys.root_token }}" - secret: "{{ secrets_barbican_approle_secret_id }}" - name: barbican - when: barbican_approle_secret_get.status == "absent" diff --git a/etc/kayobe/ansible/vault-deploy-overcloud.yml b/etc/kayobe/ansible/vault-deploy-overcloud.yml deleted file mode 100644 index 31ccf2d2e..000000000 --- a/etc/kayobe/ansible/vault-deploy-overcloud.yml +++ /dev/null @@ -1,118 +0,0 @@ ---- -# Required for uri module to work with self-signed certificates and for systems to trust -# the self-signed CA -- name: Install CA on controllers - hosts: controllers - tasks: - - name: Copy the intermediate CA - ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/vault/OS-TLS-ROOT.pem" - dest: "{{ '/etc/pki/ca-trust/source/anchors/OS-TLS-ROOT.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' - }}" - mode: "0644" - become: true - - - name: Update system CA - become: true - ansible.builtin.command: "{{ 'update-ca-trust' if ansible_facts.os_family == 'RedHat' else 'update-ca-certificates' }}" -- name: Deploy HashiCorp Vault on the overcloud - any_errors_fatal: true - gather_facts: true - hosts: controllers - vars: - consul_bind_interface: "{{ internal_net_name | net_interface }}" - vault_bind_address: "{{ internal_net_name | net_ip }}" - tasks: - - name: Set a fact about the virtualenv on the remote system - ansible.builtin.set_fact: - virtualenv: "{{ ansible_python_interpreter | dirname | dirname }}" - when: - - ansible_python_interpreter is defined - - not ansible_python_interpreter.startswith('/bin/') - - not ansible_python_interpreter.startswith('/usr/bin/') - - - name: Ensure Python hvac module is installed - ansible.builtin.pip: - name: hvac - state: latest - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" - become: "{{ virtualenv is not defined }}" - - - name: Ensure /opt/kayobe/vault exists - ansible.builtin.file: - path: /opt/kayobe/vault - state: directory - - - name: Template out TLS key and cert - ansible.builtin.copy: - # Within the Hashicorp Vault container these uids & gids map to the vault user - src: "{{ kayobe_env_config_path }}/vault/{{ item }}" - dest: /opt/kayobe/vault/{{ item }} - owner: 100 - group: 1000 - mode: "0600" - loop: - - "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.crt" - - "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.key" - become: true - - - name: Apply vault role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault - vars: - hashicorp_registry_url: "{{ overcloud_hashicorp_registry_url }}" - hashicorp_registry_username: "{{ overcloud_hashicorp_registry_username }}" - hashicorp_registry_password: "{{ overcloud_hashicorp_registry_password }}" - consul_docker_image: "{{ overcloud_consul_docker_image }}" - consul_docker_tag: "{{ overcloud_consul_docker_tag }}" - vault_config_dir: /opt/kayobe/vault - vault_cluster_name: overcloud - vault_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_docker_image: "{{ overcloud_vault_docker_image }}" - vault_docker_tag: "{{ overcloud_vault_docker_tag }}" - vault_tls_cert: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.crt" - vault_tls_key: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}.key" - copy_self_signed_ca: true - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_write_keys_file: true - vault_write_keys_file_path: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - - - name: Include Vault keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys - - - name: Apply vault unseal role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_unseal - vars: - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_unseal_token: "{{ vault_keys.root_token }}" - vault_unseal_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_unseal_keys: "{{ vault_keys.keys_base64 }}" - environment: - https_proxy: "" - -- name: Configure PKI - any_errors_fatal: true - gather_facts: true - hosts: controllers[0] - tasks: - - name: Apply vault pki role - ansible.builtin.import_role: - name: stackhpc.hashicorp.vault_pki - vars: - vault_token: "{{ vault_keys.root_token }}" - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - vault_ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - vault_pki_root_create: false - vault_pki_intermediate_import: true - vault_pki_intermediate_ca_name: OS-TLS-INT - vault_pki_intermediate_ca_bundle: "{{ lookup('file', kayobe_env_config_path + '/vault/OS-TLS-INT.pem') }}" - vault_pki_intermediate_ca_cert: "{{ lookup('file', kayobe_env_config_path + '/vault/OS-TLS-INT.crt') }}" - vault_pki_intermediate_roles: "{{ overcloud_vault_pki_roles }}" - vault_pki_write_certificate_files: true - vault_pki_certificates_directory: "{{ kayobe_env_config_path }}/vault" - environment: - https_proxy: "" diff --git a/etc/kayobe/ansible/vault-generate-backend-tls.yml b/etc/kayobe/ansible/vault-generate-backend-tls.yml deleted file mode 100644 index 32e502fbe..000000000 --- a/etc/kayobe/ansible/vault-generate-backend-tls.yml +++ /dev/null @@ -1,82 +0,0 @@ ---- -# Required for uri module to work with self-signed certificates and for systems to trust -# the self-signed CA -- name: Install CA - hosts: controllers:network - tasks: - - name: Copy the intermediate CA - ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/vault/OS-TLS-ROOT.pem" - dest: "{{ '/etc/pki/ca-trust/source/anchors/OS-TLS-ROOT.crt' if ansible_facts.os_family == 'RedHat' \ - else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - mode: "0644" - become: true - - - name: Update system CA - become: true - ansible.builtin.command: "{{ 'update-ca-trust' if ansible_facts.os_family == 'RedHat' else 'update-ca-certificates' }}" -- name: Generate backend API certificates - hosts: controllers:network - vars: - vault_api_addr: https://{{ internal_net_name | net_ip(groups['controllers'][0]) }}:8200 - vault_intermediate_ca_name: OS-TLS-INT - tasks: - - name: Set a fact about the virtualenv on the remote system - ansible.builtin.set_fact: - virtualenv: "{{ ansible_python_interpreter | dirname | dirname }}" - when: - - ansible_python_interpreter is defined - - not ansible_python_interpreter.startswith('/bin/') - - not ansible_python_interpreter.startswith('/usr/bin/') - - - name: Ensure Python hvac module is installed - ansible.builtin.pip: - name: hvac - state: latest - extra_args: "{% if pip_upper_constraints_file %}-c {{ pip_upper_constraints_file }}{% endif %}" - virtualenv: "{{ virtualenv is defined | ternary(virtualenv, omit) }}" - become: "{{ virtualenv is not defined }}" - - - name: Include Vault keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys - - - name: Issue a certificate for backend TLS - hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ vault_keys.root_token }}" - mount_point: "{{ vault_intermediate_ca_name }}" - role: "{{ overcloud_vault_pki_backend_tls_role_name }}" - common_name: "" - extra_params: - ip_sans: "{{ internal_net_name | net_ip }}" - register: backend_cert - environment: - https_proxy: "" - - - name: Ensure certificates directory exists - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/kolla/certificates" - state: directory - delegate_to: localhost - - - name: Copy backend cert - no_log: true - ansible.builtin.copy: - dest: "{{ kayobe_env_config_path }}/kolla/certificates/{{ inventory_hostname }}-cert.pem" - content: | - {{ backend_cert.data.certificate }} - {{ backend_cert.data.issuing_ca }} - mode: "0600" - delegate_to: localhost - - - name: Copy backend key - no_log: true - ansible.builtin.copy: - dest: "{{ kayobe_env_config_path }}/kolla/certificates/{{ inventory_hostname }}-key.pem" - content: | - {{ backend_cert.data.private_key }} - mode: "0600" - delegate_to: localhost diff --git a/etc/kayobe/ansible/vault-generate-test-external-tls.yml b/etc/kayobe/ansible/vault-generate-test-external-tls.yml deleted file mode 100644 index de02ddb03..000000000 --- a/etc/kayobe/ansible/vault-generate-test-external-tls.yml +++ /dev/null @@ -1,57 +0,0 @@ ---- -- name: Generate external API certificate (for testing only) - hosts: controllers - run_once: true - vars: - vault_api_addr: https://{{ internal_net_name | net_ip }}:8200 - # NOTE: Using the same CA as internal TLS. - vault_intermediate_ca_name: OS-TLS-INT - tasks: - - name: Include Vault keys - ansible.builtin.include_vars: - file: "{{ kayobe_env_config_path }}/vault/overcloud-vault-keys.json" - name: vault_keys - - - name: Issue a certificate for external TLS - hashivault_pki_cert_issue: # noqa: fqcn - url: "{{ vault_api_addr }}" - ca_cert: "{{ '/etc/pki/tls/certs/ca-bundle.crt' if ansible_facts.os_family == 'RedHat' else '/usr/local/share/ca-certificates/OS-TLS-ROOT.crt' }}" - token: "{{ vault_keys.root_token }}" - mount_point: "{{ vault_intermediate_ca_name }}" - role: "{{ overcloud_vault_pki_external_tls_role_name }}" - common_name: "{% if kolla_external_fqdn != kolla_external_vip_address %}{{ kolla_external_fqdn }}{% endif %}" - extra_params: - ip_sans: "{{ kolla_external_vip_address }}" - register: external_cert - environment: - https_proxy: "" - - - name: Ensure certificates directory exists - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/kolla/certificates" - state: directory - delegate_to: localhost - - - name: Ensure CA certificates directory exists - ansible.builtin.file: - path: "{{ kayobe_env_config_path }}/kolla/certificates/ca" - state: directory - delegate_to: localhost - - - name: Copy external API PEM bundle - no_log: true - ansible.builtin.copy: - dest: "{{ kayobe_env_config_path }}/kolla/certificates/haproxy.pem" - content: | - {{ external_cert.data.certificate }} - {{ external_cert.data.issuing_ca }} - {{ external_cert.data.private_key }} - mode: "0600" - delegate_to: localhost - - - name: Copy root CA - ansible.builtin.copy: - src: "{{ kayobe_env_config_path }}/vault/OS-TLS-ROOT.pem" - dest: "{{ kayobe_env_config_path }}/kolla/certificates/ca/vault.crt" - mode: "0600" - delegate_to: localhost diff --git a/etc/kayobe/environments/ci-multinode/stackhpc-monitoring.yml b/etc/kayobe/environments/ci-multinode/stackhpc-monitoring.yml index 93ce650b4..d9693381e 100644 --- a/etc/kayobe/environments/ci-multinode/stackhpc-monitoring.yml +++ b/etc/kayobe/environments/ci-multinode/stackhpc-monitoring.yml @@ -1,3 +1,3 @@ --- # Path to a CA certificate file to trust in the OpenStack Capacity exporter. -stackhpc_os_capacity_openstack_cacert: "{{ kayobe_env_config_path }}/kolla/certificates/ca/vault.crt" +stackhpc_os_capacity_openstack_cacert: "{{ kayobe_env_config_path }}/kolla/certificates/ca/{{ stackhpc_ca_secret_store }}.crt" diff --git a/etc/kayobe/environments/ci-multinode/tempest.yml b/etc/kayobe/environments/ci-multinode/tempest.yml index 0657946bb..327fbef34 100644 --- a/etc/kayobe/environments/ci-multinode/tempest.yml +++ b/etc/kayobe/environments/ci-multinode/tempest.yml @@ -3,4 +3,4 @@ rally_no_sensitive_log: false # Add the Vault CA certificate to the rally container when running tempest. -tempest_cacert: "{{ kayobe_env_config_path }}/kolla/certificates/ca/vault.crt" +tempest_cacert: "{{ kayobe_env_config_path }}/kolla/certificates/ca/{{ stackhpc_ca_secret_store }}.crt" diff --git a/etc/kayobe/inventory/group_vars/all/vault b/etc/kayobe/inventory/group_vars/all/vault index 22e89a455..dcf2c0c79 100644 --- a/etc/kayobe/inventory/group_vars/all/vault +++ b/etc/kayobe/inventory/group_vars/all/vault @@ -83,3 +83,9 @@ overcloud_vault_pki_roles: locality: ["Bristol"] organization: ["StackHPC"] ou: ["OpenStack"] + +seed_vault_pki_certificate_subject: + - common_name: "{% if kolla_internal_fqdn != kolla_internal_vip_address %}{{ kolla_internal_fqdn }}{% else %}overcloud{% endif %}" + role: "{{ seed_openbao_pki_role_name }}" + extra_params: + ip_sans: "{% for host in groups['controllers'] %}{{ internal_net_name | net_ip(host) }}{% if not loop.last %},{% endif %}{% endfor %},{{ kolla_internal_vip_address }}" diff --git a/etc/kayobe/stackhpc.yml b/etc/kayobe/stackhpc.yml index ad4516c5b..d8a5b929a 100644 --- a/etc/kayobe/stackhpc.yml +++ b/etc/kayobe/stackhpc.yml @@ -183,3 +183,10 @@ download_amphora_from_ark: true # Octavia Amphora image version stackhpc_amphora_image_version: "2025.1-20250619T113933" + +################################################################################ +# Certificate Authority + +# Secret store to deploy as a Certificate Authority. +# Valid options are "vault" and "openbao". Default is "openbao". +stackhpc_ca_secret_store: openbao diff --git a/releasenotes/notes/merge-openbao-vault-playbooks-0c78364f4a86f6e0.yaml b/releasenotes/notes/merge-openbao-vault-playbooks-0c78364f4a86f6e0.yaml new file mode 100644 index 000000000..292c95710 --- /dev/null +++ b/releasenotes/notes/merge-openbao-vault-playbooks-0c78364f4a86f6e0.yaml @@ -0,0 +1,7 @@ +--- +features: + - | + Merged OpenBao playbooks and Hashicorp Vault playbooks. + They starts with the prefix "secret-store". + By default, the playbooks will deploy OpenBao. + To use Hashicorp Vault, set ``stackhpc_ca_secret_store: vault``.