Skip to content

Conversation

@arewm
Copy link
Contributor

@arewm arewm commented Jan 30, 2026

Summary

  • Add CEL rule to recognize pipelines with internal-services.appstudio.openshift.io/pipelinerun-uid label
  • Assign them konflux-release priority, matching their parent managed release pipelines
  • Includes development, staging, and kflux-ocp-p01 (production OCP) clusters
  • Adds test coverage for the new CEL rule

Background

Child pipelines created by internal-pipelinerun (such as signing pipelines) were receiving the default priority (400) instead of release priority (1000). This caused them to be blocked behind multi-arch build pipelines waiting for ppc64le/s390x capacity, even though signing pipelines don't require those resources.

🤖 Generated with Claude Code

@openshift-ci openshift-ci bot requested review from eisraeli and oswcab January 30, 2026 02:34
@github-actions
Copy link
Contributor

🤖 Gemini AI Assistant Available

Hi @arewm! I'm here to help with your pull request. You can interact with me using the following commands:

Available Commands

  • @gemini-cli /review - Request a comprehensive code review

    • Example: @gemini-cli /review Please focus on security and performance
  • @gemini-cli <your question> - Ask me anything about the codebase

    • Example: @gemini-cli How can I improve this function?
    • Example: @gemini-cli What are the best practices for error handling here?

How to Use

  1. Simply type one of the commands above in a comment on this PR
  2. I'll analyze your code and provide detailed feedback
  3. You can track my progress in the workflow logs

Permissions

Only OWNER, MEMBER, or COLLABORATOR users can trigger my responses. This ensures secure and appropriate usage.


This message was automatically added to help you get started with the Gemini AI assistant. Feel free to delete this comment if you don't need assistance.

@github-actions
Copy link
Contributor

🤖 Hi @arewm, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

@konflux-ci-qe-bot
Copy link

🤖 Pipeline Failure Analysis

Category: Test

The end-to-end tests failed due to a timeout in the konflux-demo-suite test case, caused by a missing PipelineRun that should have been automatically triggered upon component creation.

📋 Technical Details

Immediate Cause

The konflux-demo-suite end-to-end test timed out after 300 seconds because it could not find a pull-request event type PipelineRun for the component konflux-demo-component-brgz in the konflux-porf namespace. This PipelineRun was expected to be present after the component was created as part of the test's execution flow.

Contributing Factors

The context reveals that a static-admission AdmissionCheck with a 15-minute retry delay is active, managed by konflux-ci.dev/kueue-external-admission. Additionally, numerous ApplicationSets across the cluster are in an OutOfSync or Missing state, and several Kubernetes resources are flagged with Progressing health status. These environmental factors might indicate underlying cluster instability or resource contention that could delay or prevent the timely creation and recognition of PipelineRun resources. The underlying panic in the Tekton Kueue webhook also suggests potential issues within the CI/CD infrastructure that could affect resource management.

Impact

The failure to create or detect the expected PipelineRun prevented the konflux-demo-suite test from completing successfully. This directly blocked the advancement of the e2e test suite, leading to the overall failure of the Prow job.

🔍 Evidence

appstudio-e2e-tests/redhat-appstudio-e2e

Category: test
Root Cause: The e2e tests failed due to a timeout in the "konflux-demo-suite" test case. This timeout occurred because a required PipelineRun was not found within the expected timeframe, indicating a potential issue with the creation or detection of pipeline runs within the Konflux/AppStudio CI environment.

Logs:

artifacts/appstudio-e2e-tests/redhat-appstudio-e2e/test-log.txt line 1009
• [FAILED] [300.239 seconds]
[konflux-demo-suite] Maven project - Default build when Component is created [It] triggers creation of a PR in the sample repo [konflux, upstream-konflux]
/tmp/tmp.NGi4GFS4hO/tests/konflux-demo/konflux-demo.go:199

  [FAILED] Timed out after 300.001s.
  timed out when waiting for 'pull-request' event type PaC PipelineRun to be present in the user namespace "konflux-porf" for component "konflux-demo-component-brgz" with a label pointing to "konflux-demo-app"
  Expected success, but got an error:
      <*errors.errorString | 0xc00179ca80>:
      no pipelinerun found for component konflux-demo-component-brgz
      {
          s: "no pipelinerun found for component konflux-demo-component-brgz",
      }
  In [It] at: /tmp/tmp.NGi4GFS4hO/tests/konflux-demo/konflux-demo.go:222 @ 01/30/26 02:54:54.642
------------------------------
artifacts/appstudio-e2e-tests/redhat-appstudio-e2e/test-log.txt line 1043
Test Suite Failed
E0130 02:54:54.792755   17410 types.go:186] Failed to execute rule: Infra Deployments Default Test Execution: Run the Konflux-demo suite tests when an Infra-deployments PR includes changes to files outside of the specified components.
Error: error when running e2e tests: running "ginkgo --seed=1769740712 --timeout=1h30m0s --grace-period=30s --output-interceptor-mode=none --label-filter=konflux --no-color --json-report=e2e-report.json --junit-report=e2e-report.xml --procs=20 --nodes=20 --p --output-dir=/logs/artifacts ./cmd --" failed with exit code 1
make: *** [Makefile:25: ci/test/e2e] Error 1

Analysis powered by prow-failure-analysis | Build: 2017063831779414016

@arewm arewm force-pushed the kueue-internal-pipelinerun-priority branch from 4f309b8 to 7d9da59 Compare January 30, 2026 03:37
@filariow filariow requested review from filariow and gbenhaim January 30, 2026 08:08
Copy link
Member

@filariow filariow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve
/hold

Feel free to unhold when you're ready to merge
It would be nice if we could split it in two: one for dev+staging and one for production.

@gbenhaim
Copy link
Member

gbenhaim commented Jan 30, 2026

Doesn't signing pipeline run in the internal common clusters?

Copy link
Member

@gbenhaim gbenhaim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test for this change in

Tekton-Kueue Configuration Test

@openshift-ci openshift-ci bot removed the lgtm label Jan 30, 2026
@arewm arewm force-pushed the kueue-internal-pipelinerun-priority branch from 7d9da59 to c43681d Compare January 30, 2026 13:30
@arewm arewm changed the title kueue: Assign release priority to internal-pipelinerun child pipelines kueue: Assign release priority to internal-pipelinerun child pipelines (dev/staging) Jan 30, 2026
@arewm
Copy link
Contributor Author

arewm commented Jan 30, 2026

Doesn't signing pipeline run in the internal common clusters?

It does for external clusters. Internal clusters run the internal service pipelines directly on the same cluster. This is only a fix needed for the internal clusters, but having an extra kueue configured shouldn't affect external clusters.

Child pipelines created by internal-pipelinerun (such as signing pipelines)
were receiving the default priority (400) instead of release priority (1000).
This caused them to be blocked behind multi-arch build pipelines waiting for
ppc64le/s390x capacity, even though signing pipelines don't require those
resources.

Add a CEL rule to recognize pipelines with the
internal-services.appstudio.openshift.io/pipelinerun-uid label and assign
them konflux-release priority, matching their parent managed release
pipelines.

Assisted-by: Claude Code (Opus 4.5)
@arewm arewm force-pushed the kueue-internal-pipelinerun-priority branch from c43681d to cd3cb37 Compare January 30, 2026 16:05
@arewm
Copy link
Contributor Author

arewm commented Jan 30, 2026

@gbenhaim, I am not familiar with the tests, but Claude added one. If that is not correct, I can go back and modify this.

I also moved the OCP cluster to this dev/stage PR as that is the most affected cluster due to the continuous build pipelines running.

@arewm arewm changed the title kueue: Assign release priority to internal-pipelinerun child pipelines (dev/staging) kueue: Assign release priority to internal-pipelinerun child pipelines (dev/staging/ocp) Jan 30, 2026
@arewm
Copy link
Contributor Author

arewm commented Jan 30, 2026

Doesn't signing pipeline run in the internal common clusters?

Yes, the signing pipelines run on the internal clusters (kflux-ocp-p01). I've updated this PR to include kflux-ocp-p01 config alongside dev/staging, and separated the production base config into a separate PR (#10310) for the external clusters where this rule won't apply (but is harmless).

Added test coverage for the new CEL rule as well.

@openshift-ci openshift-ci bot added the lgtm label Jan 30, 2026
@openshift-ci
Copy link

openshift-ci bot commented Jan 30, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: arewm, filariow, gbenhaim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@arewm
Copy link
Contributor Author

arewm commented Jan 30, 2026

/unhold

@openshift-merge-bot openshift-merge-bot bot merged commit 8cf225c into main Jan 30, 2026
11 checks passed
@arewm arewm deleted the kueue-internal-pipelinerun-priority branch January 30, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants