Skip to content

Commit

Permalink
Split out content-provider reproducer
Browse files Browse the repository at this point in the history
It may happen people want to run the content-provider automatically, but
the actual job manually.

It may also happen the content-provider just passes, but the subsequent
"job" run fails - and we of course don't want to re-run the
content-provider.
  • Loading branch information
cjeanner authored and openshift-merge-bot[bot] committed Feb 5, 2024
1 parent 2452763 commit 69a9eb1
Show file tree
Hide file tree
Showing 6 changed files with 145 additions and 110 deletions.
1 change: 1 addition & 0 deletions docs/dictionary/en-custom.txt
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ bmc
bmh
bmo
bool
booleans
boostrap
bootmacaddress
bootmode
Expand Down
13 changes: 13 additions & 0 deletions roles/reproducer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,24 @@ None
* `cifmw_reproducer_kubecfg`: (String) Path to the CRC kubeconfig file. Defaults to the image_local_dir defined in the cifmw_libvirt_manager_configuration dict.
* `cifmw_reproducer_repositories`: (List[mapping]) List of repositories you want to synchronize from your local machine to the ansible controller.
* `cifmw_reproducer_run_job`: (Bool) Run actual CI job. Defaults to `true`.
* `cifmw_reproducer_run_content_provider`: (Bool) Run content-provider job. Defaults to `true`.
* `cifmw_reproducer_params`: (Dict) Specific parameters you want to pass to the reproducer. Defaults to `{}`.
* `cifmw_reproducer_dns_servers`: List of dns servers which should be used by the CRC VM as upstream dns servers. Defaults to 1.1.1.1, 8.8.8.8.
* `cifmw_reproducer_hp_rhos_release`: (Bool) Allows to consume rhos-release on the hypervisor. Defaults to `false`.
* `cifmw_reproducer_dnf_tweaks`: (List) Options you want to inject in dnf.conf, both on controller-0 and hypervisor. Defaults to `[]`.

### run_job and run_content_provider booleans and risks.

- For jobs with content-provider, both steps will be running by default.
- For jobs without content-provider, only the job will run by default.
- If a job with content-provider is launched with `cifmw_reproducer_run_job: false`, it will
then run the content-provider, and stop.
- If a job with content-provider is launched for a second time with `cifmw_reproducer_run_content_provider: false`,
if the first run did deploy the content-provider, it will pass.
- If a job with content-provider is launched a **first** time with `cifmw_reproducer_run_content_provider: false`,
it will NOT RUN the content-provider, **leading to a crash of the job run**.


## Warning
This role isn't intended to be called outside of the `reproducer.yml` playbook.

Expand Down
1 change: 1 addition & 0 deletions roles/reproducer/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ cifmw_reproducer_basedir: "{{ cifmw_basedir | default( ansible_user_dir ~ '/ci-f
cifmw_reproducer_kubecfg: "{{ cifmw_libvirt_manager_configuration.vms.crc.image_local_dir }}/kubeconfig"
cifmw_reproducer_params: {}
cifmw_reproducer_run_job: true
cifmw_reproducer_run_content_provider: true
cifmw_reproducer_dns_servers:
- 1.1.1.1
- 8.8.8.8
Expand Down
24 changes: 23 additions & 1 deletion roles/reproducer/tasks/ci_job.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,15 @@
dest: "/home/zuul/src/github.com/openstack-k8s-operators/ci-framework/{{ job_id }}_play.yml"
src: "play.yml.j2"

- name: Push content-provider playbook if needed
tags:
- bootstrap
when:
- use_content_provider | default(false) | bool
ansible.builtin.template:
dest: "/home/zuul/src/github.com/openstack-k8s-operators/ci-framework/{{ job_id }}_content-provider.yml"
src: "content-provider.yml.j2"

- name: Push extracted network data on controller-0
tags:
- bootstrap
Expand Down Expand Up @@ -102,11 +111,24 @@
-i ~/ci-framework-data/artifacts/zuul_inventory.yml
pre-ci-play.yml
- name: Run content-provider
when:
- use_content_provider | default(false) | bool
- cifmw_reproducer_run_content_provider | bool
environment:
ANSIBLE_LOG_PATH: "~/ansible-content-provider-{{ job_id }}.log"
ansible.builtin.command:
chdir: "/home/zuul/src/github.com/openstack-k8s-operators/ci-framework"
cmd: >-
ansible-playbook
-i ~/ci-framework-data/artifacts/zuul_inventory.yml
{{ job_id }}_content-provider.yml
- name: Run job
when:
- cifmw_reproducer_run_job | bool
environment:
ANSIBLE_LOG_PATH: "~/ansible-{{ job_id }}.log"
ANSIBLE_LOG_PATH: "~/ansible-ci-job-{{ job_id }}.log"
ansible.builtin.command:
chdir: "/home/zuul/src/github.com/openstack-k8s-operators/ci-framework"
cmd: >-
Expand Down
107 changes: 107 additions & 0 deletions roles/reproducer/templates/content-provider.yml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
{% raw %}
- name: Load parameters and expose needed data
hosts: localhost
gather_facts: true
tasks:
- name: Load interfaces-info.yml
ansible.builtin.include_vars:
file: "/etc/ci/env/networking-environment-definition.yml"
name: _net_env

- name: Set cifmw_rp_registry_ip
ansible.builtin.set_fact:
cacheable: true
cifmw_rp_registry_ip: >-
{{ _net_env.instances['controller-0'].networks.ctlplane.ip_v4 }}


- name: Configure OCPs for insecure registry
hosts: ocps
vars:
content_provider_registry_ip: "{{ hostvars['localhost']['cifmw_rp_registry_ip'] }}"
tasks:
- name: Wait for CRC to be ready
register: wait_crc
ansible.builtin.command: >-
oc login api.crc.testing:6443
-u kubeadmin
-p 12345678
--insecure-skip-tls-verify=true
retries: 60
delay: 10
until:
- wait_crc is defined
- wait_crc.rc is defined
- wait_crc.rc == 0

- name: Set insecure registry on crc node
ansible.builtin.import_tasks: ci/playbooks/tasks/set_crc_insecure_registry.yml
{% endraw %}

- name: "Content Provider"
hosts: localhost
gather_facts: false
vars:
cifmw_rp_registry_firewall: false
job_id: "{{ job_id }}"
tasks:
{% raw %}
- name: Load env variables
ansible.builtin.include_vars:
file: "{{ item }}"
loop:
- "{{ ansible_user_dir }}/{{ job_id }}-params/zuul-params.yml"
- "./scenarios/centos-9/base.yml"
- "./scenarios/centos-9/content_provider.yml"

- name: Install necessary tools
ansible.builtin.import_role:
name: install_yamls_makes
tasks_from: make_download_tools

- name: Deploy registry
ansible.builtin.import_role:
name: registry_deploy

# Imported from the ci/playbooks/content_provider/content_provider.yml play
# Ensure we get consistent name for the operator, linked to
# https://github.com/openstack-k8s-operators/ci-framework/pull/621
- name: Set var for cifmw_operator_build_operators var
when:
- zuul is defined
- "'project' in zuul"
- "'short_name' in zuul.project"
ansible.builtin.set_fact:
cifmw_operator_build_operators:
- name: "openstack-operator"
src: "~/src/github.com/{{ cifmw_operator_build_org }}/openstack-operator"
image_base: >-
{{ zuul.project.short_name | split('-') | reject('search','operator') | join('-') }}

- name: Build Operators
ansible.builtin.include_role:
name: operator_build

- name: Get the containers list from container registry
ansible.builtin.uri:
url: "http://{{ cifmw_rp_registry_ip }}:5001/v2/_catalog"
return_content: true
register: cp_imgs

- name: Add the container list to file
ansible.builtin.copy:
content: "{{ cp_imgs.content }}"
dest: "{{ ansible_user_dir }}/local_registry.log"
mode: "0644"

- name: Output needed content into consumable environment
ansible.builtin.copy:
dest: "{{ ansible_user_dir }}/{{ job_id }}-params/content-provider.yml"
content: |-
{{
{'cifmw_operator_build_output': cifmw_operator_build_output,
'content_provider_registry_ip': cifmw_rp_registry_ip
} | to_nice_yaml
}}
{% endraw %}
109 changes: 0 additions & 109 deletions roles/reproducer/templates/play.yml.j2
Original file line number Diff line number Diff line change
@@ -1,113 +1,4 @@
---
{% if use_content_provider | default(false) | bool -%}
{% raw %}
- name: Load parameters and expose needed data
hosts: localhost
gather_facts: true
tasks:
- name: Load interfaces-info.yml
ansible.builtin.include_vars:
file: "/etc/ci/env/networking-environment-definition.yml"
name: _net_env

- name: Set cifmw_rp_registry_ip
ansible.builtin.set_fact:
cacheable: true
cifmw_rp_registry_ip: >-
{{ _net_env.instances['controller-0'].networks.ctlplane.ip_v4 }}


- name: Configure OCPs for insecure registry
hosts: ocps
vars:
content_provider_registry_ip: "{{ hostvars['localhost']['cifmw_rp_registry_ip'] }}"
tasks:
- name: Wait for CRC to be ready
register: wait_crc
ansible.builtin.command: >-
oc login api.crc.testing:6443
-u kubeadmin
-p 12345678
--insecure-skip-tls-verify=true
retries: 60
delay: 10
until:
- wait_crc is defined
- wait_crc.rc is defined
- wait_crc.rc == 0

- name: Set insecure registry on crc node
ansible.builtin.import_tasks: ci/playbooks/tasks/set_crc_insecure_registry.yml
{% endraw %}

- name: "Content Provider"
hosts: localhost
gather_facts: false
vars:
cifmw_rp_registry_firewall: false
job_id: "{{ job_id }}"
tasks:
{% raw %}
- name: Load env variables
ansible.builtin.include_vars:
file: "{{ item }}"
loop:
- "{{ ansible_user_dir }}/{{ job_id }}-params/zuul-params.yml"
- "./scenarios/centos-9/base.yml"
- "./scenarios/centos-9/content_provider.yml"

- name: Install necessary tools
ansible.builtin.import_role:
name: install_yamls_makes
tasks_from: make_download_tools

- name: Deploy registry
ansible.builtin.import_role:
name: registry_deploy

# Imported from the ci/playbooks/content_provider/content_provider.yml play
# Ensure we get consistent name for the operator, linked to
# https://github.com/openstack-k8s-operators/ci-framework/pull/621
- name: Set var for cifmw_operator_build_operators var
when:
- zuul is defined
- "'project' in zuul"
- "'short_name' in zuul.project"
ansible.builtin.set_fact:
cifmw_operator_build_operators:
- name: "openstack-operator"
src: "~/src/github.com/{{ cifmw_operator_build_org }}/openstack-operator"
image_base: >-
{{ zuul.project.short_name | split('-') | reject('search','operator') | join('-') }}

- name: Build Operators
ansible.builtin.include_role:
name: operator_build

- name: Get the containers list from container registry
ansible.builtin.uri:
url: "http://{{ cifmw_rp_registry_ip }}:5001/v2/_catalog"
return_content: true
register: cp_imgs

- name: Add the container list to file
ansible.builtin.copy:
content: "{{ cp_imgs.content }}"
dest: "{{ ansible_user_dir }}/local_registry.log"
mode: "0644"

- name: Output needed content into consumable environment
ansible.builtin.copy:
dest: "{{ ansible_user_dir }}/{{ job_id }}-params/content-provider.yml"
content: |-
{{
{'cifmw_operator_build_output': cifmw_operator_build_output,
'content_provider_registry_ip': cifmw_rp_registry_ip
} | to_nice_yaml
}}
{% endraw %}
{% endif %}

{% for play in zuul_plays %}
- name: "Reproducer for {{ job_id }}: {{ play }}"
vars:
Expand Down

0 comments on commit 69a9eb1

Please sign in to comment.