Listing API Requirements #229
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Listing API Requirements | ||
|
|
||
| Registries provide the basic means to push, pull, discover and manage content. While push and pull are generally feature-rich and standardized, the discover and manage APIs are extremely limited. Each registry project and operator have had to implement discover (list) and manage APIs, and each of them are unique to that particular registry. | ||
|
|
||
| To foster consistency across registries, enabling the community to build tooling that works consistently across all [OCI distribution-spec conformant registries][oci-compliance], we propose a common set of listing APIs. | ||
|
|
||
| To facilitate discussions, prioritization and phased implementations, we will capture a list of scenarios and requirements. Once the requirements are felt complete and prioritized, we can transition to a design that implements the agreed prioritized items, while reserving space for the future enhancements. | ||
|
|
||
| For the purposes of examples within a scenario, we'll assume an `oci-reg` cli exists. The purpose is not to design the specific CLI, rather provide concrete examples whose specific names and parameters will evolve. | ||
|
|
||
| ## Scenarios | ||
|
|
||
| 1. A user can get a list of repositories within a given registry, within a specified org and/or namespace. | ||
| - `oci-reg list acme-rockets.io/` | ||
| - `oci-reg list acme-rockets.io/org1/` | ||
| 2. A user can get a list of tags within a given registry/namespace. | ||
| 3. A user can get a list of manifests, within a given registry/namespace. | ||
| 4. A user can get a list of artifacts that reference a specified digest. Example: (a list of Notary v2 signatures) | ||
| 5. A user can get a list of tags for a specified digest. | ||
|
|
||
| ## Requirements | ||
|
|
||
| 1. Provide paging of results | ||
| 2. Provide for registry specific extensions. | ||
| 3. Provide filtering by `artifactType` | ||
| 4. Provide filtering by annotation | ||
| 5. Provide sorting | ||
|
|
||
| ## Prioritization | ||
|
Contributor
There was a problem hiding this comment. Do these priorities relate to planned spec versions? Or are they more about required vs optional features? Are we expecting a phased delivery of features in registry implementations that are intended to move in lockstep with updates to the spec or are the phases merely a planning detail for designing the spec and we would expect most registries to implement the requirements roughly at the same time?
Contributor
Author
There was a problem hiding this comment. Great questions. I was hearing folks say, I need this now. While others might take more time to complete the design. I was just suggesting a way to breakup the design. House example: You know you want to add a 2-car garage and a deck that has that great city view. Let's get enough figured out to create the blueprint, and then we can add over time. |
||
|
|
||
| To deliver on an incremental phased delivery, the following requirements and scenarios are identified for the following phases | ||
|
|
||
| ### Phase 1 | ||
|
|
||
| ### Phase 2 | ||
|
|
||
| ### Backlog | ||
|
|
||
|
|
||
| [oci-compliance]: https://github.com/opencontainers/oci-conformance/tree/master/distribution-spec | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be useful if the list could (optionally) contain not only the tags name (as-is) but also other useful details such as the underlying manifest digest and media type, and possibly the tag creation timestamp.
It is common to need such details (e.g., showing a tag list in a UI). Looping over a name list and doing multiple requests (one for the manifest and another for the configuration, if any) requires too much time and resources.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joaodrp, this is a great topic. I've created another PR for the show/get-info requirements. I split them out to try and get a consensus on one at a time. But, your point of having to loop through a list to get details on each may be problematic as well. Open to ideas on how we split or merge a subset?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@SteveLasker i'm a little confused -- @joaodrp seems to be asking if we could include a format for registry items that allows for metadata to go along with each listed item and you point to a show/get-info requirement which in my understanding doesn't speak to his request at all since the current PR is about the listing requirements. Should we take that to mean you're not open to specifying listed item metadata in this requirements document?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@SteveLasker, thanks. I'll have a look at it and share my opinion there.
I should clarify: Some of the API features that I'm looking forward to are not feasible to implement on registries that rely on filesystem metadata (like the distribution registry does), at least not in a way that would allow them to be usable. For example, this is why the distribution registry doesn't support pagination on the tag list endpoint.
Doing a recursive walk on a remote blob storage backend and inspecting file by file is not only slow but expensive. This is especially true for large repositories, such as those with thousands of tags, where a single tag list request (with no additional details, just the tag names) can take many seconds to complete.
At some point, these limitations become a major blocker for providers. They can't ask users to wait this much time to see a list of tags in the UI (or get it through the API), nor can they fulfill most feature requests because the implementation would be extremely slow and costly.
As a result, most major registries have to adopt some database to store metadata, such as Harbor with PostgreSQL and Docker Trusted Registry with RethinkDB. This enables them to use fast and flexible query languages and attach additional metadata to each artifact, metadata which is not part of the image spec (e.g., image push date, push/pull count, repository size, etc.). And this is when they start implementing their own custom APIs.
This brings me to a key question: Is the intention to limit the distribution API spec to the information provided by the image spec and contained, e.g., in image manifests? Or do we want to look beyond that (to cover the additional data and features that each provider will otherwise implement in their own way)?