Set the default subscribe catalogsource to the default redhat-operators for lvms-operator, metallb-operator, local-storage-operator

[gaol]

There is a builtin redhat-operators catalogsource installed for each version, which contains the operators:

[🎩 lgao@lins-p1 ~]$ opm alpha list packages registry.redhat.io/redhat/redhat-operator-index:v4.19|egrep "lvm|odf|metal|local|multicluster"
local-storage-operator                                        stable
lvms-operator                                                 stable-4.19
metallb-operator                                              stable
multicluster-engine                                           stable-2.9
multicluster-global-hub-operator-rh                           release-1.5
odf-csi-addons-operator                                       stable-4.19
odf-dependencies                                              stable-4.19
odf-multicluster-orchestrator                                 stable-4.19
odf-operator                                                  stable-4.19
odf-prometheus-operator                                       stable-4.19

The same for 4.16, 4.17, 4.18 and 4.20

For 4.21, it has different now because it is not GA yet:

[🎩 lgao@lins-p1 ~]$ opm alpha list packages registry.redhat.io/redhat/redhat-operator-index:v4.21|egrep "lvm|odf|metal|local|multicluster"
odf-csi-addons-operator                               stable-4.20
odf-dependencies                                      stable-4.20
odf-external-snapshotter-operator                     stable-4.20
odf-multicluster-orchestrator                         stable-4.20
odf-operator                                          stable-4.20
odf-prometheus-operator                               stable-4.20

There are some kubevirt-hypershift jobs failed on subscribing the related operators, like:

[Retry 29/30] The subscription is not yet available. Trying to get it...
[Retry 30/30] The subscription is not yet available. Trying to get it...
Error: Failed to deploy lvms-operator
csv  YAML

So IMO we should update the jobs to subscribe the correct catalogsources.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[gaol]

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14-des periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28

[openshift-ci-robot]

@gaol: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[openshift-ci-robot]

@gaol, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from base revision of release repo: could not checkout worktree: '[git checkout f37bcaedbc79c0cd6192b72d21a5b7d99eaad02b]' failed with out:  and error exec: Stdout already set

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[gaol]

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14 periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28

[openshift-ci-robot]

@gaol: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[mgencur]

Left one comment. The changes to catalog sources look good but the tests fail. So, it needs to be investigated.

[mgencur]

Why is this changed? It doesn't fit in this PR. Was there any discussion about that? Thanks

[gaol]

Sometime the machines are not stable, I am trying to lower down the memory to see it can make it better. But you are right, it shouldn't be in this PR. will revert it.

[mgencur]

I have one general question. Are these changes intentionally limited just to openshift-tests-private and "cucushift" steps from the CI registry? Asking because...

There are many more places where ODF_OPERATOR_SUB_CHANNEL is outdated, e.g. these:
config/openshift/hypershift/openshift-hypershift-release-4.18__periodics-mce.yaml
config/openshift/hypershift/openshift-hypershift-release-4.17__periodics.yaml
config/openshift/hypershift/openshift-hypershift-release-4.16.yaml

There are a lot of workflows and CI setups where METALLB_OPERATOR_SUB_SOURCE and the other catalog sources use qe-app-registry by default, for example:
./step-registry/hypershift/kubevirt/baremetalds/conformance/hypershift-kubevirt-baremetalds-conformance-workflow.yaml

We should probably try to agree on the strategy for all of these. And maybe create a JIRA to update the rest as well.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gaol
Once this PR has been reviewed and has the lgtm label, please assign amoghrd, liangquanli930 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• ci-operator/config/openshift/openshift-tests-private/OWNERS
• ci-operator/step-registry/cucushift/installer/rehearse/baremetalds/ipi/ovn/ipv4/kubevirt/hypershift/mce/OWNERS
• ci-operator/step-registry/hypershift/kubevirt/baremetalds/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gaol]

I have one general question. Are these changes intentionally limited just to openshift-tests-private and "cucushift" steps from the CI registry?

@mgencur thanks Martin for the review.

This PR tries to fix some found job failures, and not a thoroughly replacement of qe-app-registry catalogsource , but I do agree that we need to come into an agreement on the usage of the catalogsources later on tracked by a Jira and documented somewhere to unify the catalogsources usage in the testing. :)

[gaol]

/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14 periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28

[openshift-ci-robot]

@gaol: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

[openshift-ci-robot]

[REHEARSALNOTIFIER]
@gaol: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-cloud-provider-kubevirt-main-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.22-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.21-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.20-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.19-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.18-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.17-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.16-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.15-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-cloud-provider-kubevirt-release-4.14-hypershift-kubevirt-conformance-baremetalds	openshift/cloud-provider-kubevirt	presubmit	Registry content changed
pull-ci-openshift-origin-main-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.22-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.21-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.20-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.19-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.18-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.17-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.16-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.15-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.14-e2e-baremetalds-kubevirt	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-hypershift-main-e2e-kubevirt-metal-conformance	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-4.22-e2e-kubevirt-metal-conformance	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-4.21-e2e-kubevirt-metal-conformance	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-4.20-e2e-kubevirt-metal-conformance	openshift/hypershift	presubmit	Registry content changed
pull-ci-openshift-hypershift-release-4.19-e2e-kubevirt-metal-conformance	openshift/hypershift	presubmit	Registry content changed

A total of 94 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

[openshift-ci]

@gaol: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28	1166307	link	unknown	/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14-des	1166307	link	unknown	/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14-des
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28	84d0460	link	unknown	/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-n2minor-mgmt-f28
ci/rehearse/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14	84d0460	link	unknown	/pj-rehearse periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-ipi-ovn-kubevirt-hypershift-mce-guest-f14

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[asood-rh]

I have one general question. Are these changes intentionally limited just to openshift-tests-private and "cucushift" steps from the CI registry?

@mgencur thanks Martin for the review.

This PR tries to fix some found job failures, and not a thoroughly replacement of qe-app-registry catalogsource , but I do agree that we need to come into an agreement on the usage of the catalogsources later on tracked by a Jira and documented somewhere to unify the catalogsources usage in the testing. :)

[asood-rh]

@gaol why are we not fixing the jobs by ensuring qe-app-registry is created instead of using the redhat-operators? Does it not defeat the purpose of testing the operator code in CI before it is released?

[gaol]

qe-app-registry catalogsource does not have lvm-operator, while redhat-operators has all needed operators.

Does it not defeat the purpose of testing the operator code in CI before it is released?

I am not sure, but I think it is job related ? not all jobs need to test the operators, right ?

[mgencur]

For already-released versions of OpenShift, we want to test with the operators that are live/available/stable for that version. For pre-release versions, we will probably want to test with qe-app-registry which includes unreleased versions of some operators. There's a relatively new problem - operators which are built in Konflux are not being added to the qe-app-registry anymore. For example, the lvm-operator. Gradually, we will probably want to move to using specific catalog source which points to Konflux images.