To support org above team #5660
Replies: 5 comments 6 replies
-
|
Thanks for prompting this and including the motivation. 🙂 If you'll humor me for a moment, I want to tell a bit of a story: In the beginning, Concourse had a single config. It didn't have pipelines or teams - just a single configuration which provided the resources to check and the jobs to run. (Fun fact: this is why the API endpoint is called Then we added support for multiple pipelines through Then we added teams, so that pipelines could be scoped to different people and Concourse could be multi-tenant. Then we added support for logging in to multiple teams. Then we added RBAC, which is still developing into something more flexible. Concourse has always been allergic to complexity, but I think this is one area where we've tended to just add more and more hierarchy and knobs without really questioning it as much as other things (like pipeline config) - which makes me worried every time that we continue the trend. At various points I've wondered where we'd be if we stuck closer to the original design. With teams and RBAC and all these other things it feels like we're spending a lot of time and effort on "boring problems" that apply to any software project, while Concourse really needs more active growth on things like the v10 roadmap. If the only reason for adding hierarchy is to add another level of configuration, I'd like to think a bit more about the motivations and see if there are other ways to address them. One of the key things I really disliked about other CI/CD tools, which motivated Concourse's creation, was that in order to figure out where something is configured, I would have to check all possible levels going up the stack until I found my answer. (And if I didn't find it, I'd wonder if there was a level of the stack that I missed, or if I'm going crazy.) So, I'd like us to feel comfortable asking a more fundamental question: are Teams just not a good design? They've always been a little awkward, e.g. with Or, maybe there are simpler resolutions: do we need workers to be able to register with multiple teams? Or perhaps register with some generic annotation which can then be associated to teams? What was the pain point with |
Beta Was this translation helpful? Give feedback.
-
|
@vito Thanks for the story sharing and the long replying. As a software grows, more and more effort would be spent on management, statistics etc peripheral functionality, also frequently get frustrated about that, but that's something we have to face to. (technically, I never learned how to add an emoji.) This is a rough idea, what is reason why I haven't spent time on RFC yet, instead, I wanted to hear your opinion first. I have been facing the challenge for long time. You may already know that we are trying to build a big Concourse cluster to server the entire company. The current biggest project running on the cluster, is split into ~100 teams. Problems we need to address includes: performance monitoring, data analysis (e.g. build success rate, etc), charge-back, and so on ...
For data analysis, we also have a workaround. Because teams from a project are named with the same prefix, so we can use team name prefix to group teams on the data analysis dashboards. The main trigger was about worker. I considered a solution where it allows a worker to be marked with multiple teams. But I gave up the solution quickly. Because which would bring extra maintain efforts when adding/deleting teams and workers. As you suggested, "perhaps register with some generic annotation which can then be associated to teams", that could also solve my problem. But the annotation sounds similar to org, isn't it? I also wanted to keep the concept of org simple, that's reason why I proposed that an org can only be associated to teams, so that, org could be a pure concept for cluster admins, it may even not require UI changes, pipeline users would still only face to teams. |
Beta Was this translation helpful? Give feedback.
-
Maybe I missed this detail or assumed the wrong thing. If it's not too late, I think it's worth reconsidering this approach; in my experience this can lead to a huge amount of pain that may be avoidable. Concourse is designed so that pipelines can be portable and largely independent of what cluster they're on, so it really shouldn't be necessary to house every single pipeline all on one cluster. Doing so adds a lot of stability and scaling concerns and can make upgrades especially scary, since it's all-or-nothing. It also leads to growing complexity in Concourse as we have to figure out how to have a single Concourse cluster able to house all these different tenants. If instead there was a model where you have multiple Concourse clusters, perhaps one per 'org'. you will immediately remove a whole lot of maintenance headache and can more gradually roll out changes and upgrades.
To me, calling this 'org' implies a ton more than just having a way to share workers across teams. GitHub has organizations and teams, for example, and if Concourse also has organizations and teams I'll immediately assume they're meant for similar things. |
Beta Was this translation helpful? Give feedback.
-
|
Going back to the original motivation - what do you think of adding a 'worker pool' concept? So rather than worker registering with a team, a worker could register with a pool, and then teams could be associated to a pool by a cluster admin. Would that suit your needs? I think this is probably mechanically similar to 'orgs' except it's not something that's "above" a team - it's just a grouping of workers that can be associated to many teams. Something like this is probably doable: An alternative to |
Beta Was this translation helpful? Give feedback.
-
|
@vito I'm happy to hear the concept "worker pool" from you. Actually we have been using this concept for long time in our worker manager. Currently we use team+tags to identify a worker pool, but due to the small teams issue, we are hard to move forward. If Concourse can officially supports worker pool, which would perfectly map to our current model. Well, I'm not sure if
I don't get what "per-team authorized keys", we don't use that in our clusters. Can you clarify that? Maybe we miss something in our clusters? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Concourse supports only one level of teams, which is not good enough for a platform to host a lot of teams and pipelines.
In practice, people tend to use small team. Meaning that, a real project/team may have multiple teams on Concourse. I have seen at least two use cases that are hard to archive due to the restriction of one-level-team:
A project/team having multiple teams on Concourse may need to share some secrets. Due to this restriction, we have to use
var_sourcesfor sharing secrets, which adds complexities of pipeline code and OPs.Some big projects (BU) need dedicate worker pool, but their pipelines are deployed in multiple Concourse teams. We don't want to create a worker pool for a small Concourse team.
I'm thinking to add one level org above team. It could be named org, or group, or else. I'm open for naming suggestion. Let's call it "org" for now.
An org may contain [0..n] teams, but org should not directly contain team member. A team may belong to 0 or 1 org.
Changes for org includes:
fly set-orgcommand to manage orgs. Only cluster admin can manager orgs.<org>/keyto secret search paths.--org <org-name>option toconcourse worker. To find a worker to run container, consider org as well.I want to use this discussion to call for some initial comments and feedback, then I may submit a RFC.
Beta Was this translation helpful? Give feedback.
All reactions