Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclear demotion behavior #51

Closed
aguschin opened this issue Mar 11, 2022 · 4 comments · Fixed by #219
Closed

Unclear demotion behavior #51

aguschin opened this issue Mar 11, 2022 · 4 comments · Fixed by #219
Labels
product Product work is needed v2 Tasks for the second release

Comments

@aguschin
Copy link
Contributor

aguschin commented Mar 11, 2022

Initial idea to introduce gto demote $MODEL $VERSION $ENV was based on the assumption that user may want to undo the action of promotion. E.g. if he mistakenly promoted a experimental version to production.

One corner case I found out for this demote cmd is this.

  1. Let's suppose that you have a rf with v1 and v2. Then you promote to production v2, then v1, then again v2.
  2. Now you demote what's in production right now. As a result, v1 now in production.
  3. Now you demote again. As a result, v2 now in production. But you already tried to undo promotion of v2 to production, so it's confusing.

I see two options here:

  1. Keep things as they are ^. Maybe there are some user scenarios in which this makes sense.
  2. Make another assumption. If you demote v2 from production, that means that it shouldn't appear there until you promote it again. Then the example above will end up without any version of rf promoted to production.
  3. Disallow demote instruction and require to promote things instead.
  4. Use option 3 + still allow to have demote shortcut cmd which will actually execute promote $PREV_VERSION_IN_THAT_ENV, but that doesn't look very intuitive in complex scenarios also. As you can't distinguish between promote made on purpose and promote made because user called demote, if you run demote 2+ times, you'll stuck between two last versions in that env forever.

I'm leaning towards option 3 right now because it's simpler. Later we can introduce demote command as in option 1 or 2, however we decide.

WDYT?
CC @dmpetrov @mike0sv @dberenbaum @omesser @shcheklein

@aguschin aguschin added question Further information is requested product Product work is needed labels Mar 11, 2022
@aguschin
Copy link
Contributor Author

  1. Another variation of option 2 could be removing demote command and asking to deprecate things:
    • e.g. gto deprecate rf v2 will deprecate v2 completely, so it gets automatically demoted from all envs.
    • gto deprecate rf v2 production will deprecate v2 from using in production env. So it gets automatically demoted from production env. That sounds clearer to me than using gto demote rf production - because in the latter you doesn't specify which version you're demoting (but we still can print demoted version in a cmd output FYI).

How git tags will look like in this option:
[email protected] - version registration
rf@!v1.0.1 - version deprecation (should you be able to undo this action any other way except for git tag removal?)
[email protected]!production - version deprecation for particular env (should you be able to undo this action any other way except for git tag removal?)

@dberenbaum
Copy link
Contributor

3. Disallow demote instruction and require to promote things instead.

I think this seems like the most straightforward approach. I don't think the convenience is worth it here where users probably want to be very explicit about what is currently active in each environment.

By the way, can a version that's active in any environment be unregistered? How does/should that work?

@aguschin
Copy link
Contributor Author

By the way, can a version that's active in any environment be unregistered? How does/should that work?

Good question! Right now it works like this. If you unregister a version which is present in any environment, it gets demoted from them automatically. That means (among other) when you'll run gto show, you'll won't see this version promoted, or gto audit will show that it got demoted at the moment you unregistered the version.

As unregister is undoable, to promote model from the same commit once more, you'll have to register new version in the same commit (assuming that registration is a requirement for promotion, which is the default behavior).

@aguschin aguschin added v2 Tasks for the second release and removed question Further information is requested labels Mar 21, 2022
@aguschin
Copy link
Contributor Author

Considering current implementations with labels in #219, I'm going to use the 5th approach above.
If you deregister a version, all labels of that version are also considered inactive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product Product work is needed v2 Tasks for the second release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants