Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add support for validating cluster verification label in HTTP/gRPC requests #641

Draft
wants to merge 46 commits into
base: main
Choose a base branch
from

Conversation

aknuds1
Copy link
Collaborator

@aknuds1 aknuds1 commented Jan 30, 2025

What this PR does:

Which issue(s) this PR fixes:

TODO:

  • Reject with HTTP 5xx instead of 400

Checklist

  • Tests updated
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Sorry, something went wrong.

aknuds1 and others added 9 commits January 29, 2025 10:30

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
@duricanikolic duricanikolic force-pushed the arve/validate-cluster-name branch 2 times, most recently from ad74ad7 to eca96aa Compare January 31, 2025 17:59
Signed-off-by: Yuri Nikolic <[email protected]>
@duricanikolic duricanikolic force-pushed the arve/validate-cluster-name branch 2 times, most recently from 6c26bcf to 890ab6c Compare January 31, 2025 18:17
@duricanikolic duricanikolic deleted the arve/validate-cluster-name branch February 2, 2025 08:34
@aknuds1 aknuds1 restored the arve/validate-cluster-name branch February 2, 2025 10:29
@aknuds1 aknuds1 reopened this Feb 3, 2025
Signed-off-by: Yuri Nikolic <[email protected]>
@duricanikolic duricanikolic force-pushed the arve/validate-cluster-name branch 2 times, most recently from 78f96ff to 2447c47 Compare February 6, 2025 13:59
duricanikolic and others added 5 commits February 6, 2025 17:42
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
This reverts commit 2f94ffc.
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Fix test
Signed-off-by: Arve Knudsen <[email protected]>
@aknuds1 aknuds1 force-pushed the arve/validate-cluster-name branch from 84927aa to b464a96 Compare February 10, 2025 16:04
Signed-off-by: Arve Knudsen <[email protected]>
Add tests
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Signed-off-by: Arve Knudsen <[email protected]>
Comment on lines +19 to +20
// Allow for a potential path prefix being configured.
reB.WriteString(".*/(metrics|debug/pprof.*|ready")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this MUST be configurable. Each downstream will have some paths that may not need the cluster verification, for example the admin frontend for Mimir.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe that's what auxPaths is? I think this method needs more docs.

Copy link
Collaborator Author

@aknuds1 aknuds1 Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

auxPaths is for additional paths, yes. I kept a base set of hard-coded paths here since they are common (/metrics, /debug/pprof, /ready). That decision is of course up for discussion.

reB.WriteString("|" + regexp.QuoteMeta(p))
}
reB.WriteString(")")
reAuxPath := regexp.MustCompile(reB.String())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that regular expression aren't anchored to start/end of line by default, so you probably need to do that here.

Right now this matches the endpoint /api/v1/metrics/query, but I don't think that's intended.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I missed the anchoring. This is a WIP after all :)

Co-authored-by: Oleg Zaytsev <[email protected]>
"cluster_verification_label", cluster, "request_cluster_verification_label", reqCluster,
"header", clusterutil.ClusterVerificationLabelHeader, "url", r.URL, "path", r.URL.Path)
if invalidClusters != nil {
invalidClusters.WithLabelValues("http", r.URL.Path, reqCluster).Inc()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very fragile, the's no contract on which labels invalidClusters has, and if those change, this will panic when it should fail the request. I would suggest passing here a closure that would increment the correct metric (and let the caller pass an empty closure, not nil), so the responsibility of declaring labels and using them would stay on the initialization side.

Shameless plug, this is why I wrote cabify/gotoprom 5 years ago :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, will look into this when I have more time.

…rors

Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
@duricanikolic duricanikolic force-pushed the arve/validate-cluster-name branch from f339af6 to 22ac17e Compare February 17, 2025 16:44
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Signed-off-by: Yuri Nikolic <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants