Better usability for non-complete cli options

[dtzar]

There are some cli options which aren't complete due to planning/time/priorities, or we know are broken. This is the case now, but I see this being the case even potentially longer term as we introduce new capabilities. So - the question is what approach should we take for these CLI options or commands to enable a better user experience?

Some ideas might include:

1. Some form of notice in the help for the new command or flag, potentially pointing to FAQ/note about what is or isn't working.
2. Default error handler in code we re-use across flags we know aren't working
3. Disabling in code the options which we know are broken/incomplete
4. Enabling opt-in feature-flag or regular separate release for "latest"/dev/alpha/beta/main-builds path with understanding you're the dev/technical person who can or wants to deal with it, go back to source and troubleshoot, ask community/slack "best effort" for help.

[FeynmanZhou]

I support idea 4. Most of the OSS projects such as Kubernetes are using this approach. I see ORAS is using reference status to showcase the CLI option status. We might need to introduce different feature flags like GA, Public preview, Experimental, etc.

But in the case the CLI option is still under early development and not available to use, it should not be included in an official release.

Maybe we can provide an automated build binary (like a weekly build) for technical users to try.

$ oras --help
Usage:
  oras [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  discover    [Preview] Discover referrers of a manifest in the remote registry
···
$ oras discover --help
[Preview] Discover referrers of a manifest in the remote registry

** This command is in preview and under development. **
···

[iamsamirzon]

I personally like idea 2 and 3 above. We need to avoid any UX that could cause confusion. Adding @rgnote and @gokarnm to get their ideas on it.

[gokarnm]

I like the suggestions by @dtzar @FeynmanZhou .

• Features which are not stable can be opted-in through something like NOTATION_EXPERIMENTAL environment variable.
• The --help can include the full list of commands, and mark ones that are experimental with instructions about how to enable them.
• Option 2 in addition to these also sounds good.

[dtzar]

Work item related to this idea: #305

[yizha1]

+1 to idea 4. How about introducing weekly-build or bi-weekly build? At current stage, daily build could be too much. With this, we could also start the work of introducing auto E2E test to assure the quality of weekly-build, at least not breaking existing beta or stable features. This will also help us to identify issues early, and users can also use the latest stable build for trying stable features and experimental feature, then provide early feedback.

[iamsamirzon]

+1 on doing weekly or bi-weekly builds of Notation.

[yizha1]

I created a new discussion focusing on the next steps of notation weekly build #261