🌱 Add prometheus to e2e workflow #1928
Conversation
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1928 +/- ##
==========================================
- Coverage 65.87% 65.85% -0.02%
==========================================
Files 71 71
Lines 6189 6189
==========================================
- Hits 4077 4076 -1
- Misses 1850 1851 +1
Partials 262 262
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
So, this adds a new e2e that isn't run downstream? There's no workflow to run it in upstream CI? |
Yes - it currently only adds prometheus to the environment when you run the new e2e target, so I've kept the workflow the same for now. If you'd prefer I'm happy to make that change here - it doesn't add much overhead to the e2e at all. |
Signed-off-by: dtfranz <[email protected]>
| prometheus: LATEST := $(shell curl -s https://api.github.com/repos/prometheus-operator/prometheus-operator/releases/latest | jq -cr .tag_name) | ||
| prometheus: ## Deploy Prometheus into 'prometheus' namespace | ||
| trap 'echo "Cleaning up $(TMPDIR)"; rm -rf "$(TMPDIR)"' EXIT; \ | ||
| kubectl create ns $(PROMETHEUS_NAMESPACE); \ | ||
| curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$(LATEST)/kustomization.yaml" > "$(TMPDIR)/kustomization.yaml" ; \ | ||
| curl -s "https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/refs/tags/$(LATEST)/bundle.yaml" > "$(TMPDIR)/bundle.yaml" ; \ | ||
| (cd $(TMPDIR) && $(KUSTOMIZE) edit set namespace $(PROMETHEUS_NAMESPACE)) && kubectl create -k "$(TMPDIR)" ; \ | ||
| kubectl wait --for=condition=Ready pods -n $(PROMETHEUS_NAMESPACE) -l app.kubernetes.io/name=prometheus-operator |
There was a problem hiding this comment.
Did you consider adding the prometheus setup/teardown into the e2e's test_main.go? https://github.com/openshift/operator-framework-operator-controller/blob/355dcf40d5fcb4d9e93ab312a376c4e80d46ba76/test/e2e/e2e_suite_test.go#L31-L40
That would keep the Makefile a bit cleaner, and it would be a bit easier to deal with the tmpdir cleanup.
There was a problem hiding this comment.
I think this would go against our convention of setting up and tearing down everything in the Makefile. If people want to make changes to the e2e environment, they're going to be looking for things in here, and not in the go test files IMO.
Wouldn't that also add prometheus to every test run? I was hoping not to do that. I assume there's probably ways to turn it off/on in the go test calls, but then that adds more complication, and then we'll need to add additional targets in the Makefile for that too.
If your goal is just to keep the Makefile clean, I would be happy to compromise by adding a script instead and calling it from the Makefile. It's mainly just that putting it into the go test files doesn't feel right to me.
There was a problem hiding this comment.
I think this would go against our convention of setting up and tearing down everything in the Makefile. If people want to make changes to the e2e environment, they're going to be looking for things in here, and not in the go test files IMO.
That's fair according to how everything else works. If we do want to reorganize and move setup/teardown out of Makefile, that does feel like a separate PR.
Wouldn't that also add prometheus to every test run? I was hoping not to do that. I assume there's probably ways to turn it off/on in the go test calls, but then that adds more complication, and then we'll need to add additional targets in the Makefile for that too.
That could be managed with an environment variable. However, once we get to the point where prometheus and alert manager are actually used in tests, I assume we'll want prometheus unconditionally enabled.
If your goal is just to keep the Makefile clean, I would be happy to compromise by adding a script instead and calling it from the Makefile. It's mainly just that putting it into the go test files doesn't feel right to me.
No need for a script for this PR. If folks are worried about the Makefile sprawl, let's tackle that separately.
Description
Closes #1902
Adds an additional e2e Makefile target which runs the e2e with prometheus deployed. This will allow us to check for alerts and potential performance degradation after e2e runs.
Starting out with just the minimal install as opposed to the entire
kube-prometheusstack, which has far more than we need right now.Reviewer Checklist