Addressing issue 89: incorporating SUP on App Registry as OCI Registry

mkdocs.yml

@@ -15,7 +15,7 @@ nav:
       - overview/edge-compute-devices.md
       - overview/workload-observability.md
   - Concepts:
-      - Workloads:
+      - Applications:
           - concepts/workloads/application-package.md
           - concepts/workloads/application-registry.md
           - concepts/workloads/local-registries.md
@@ -32,8 +32,9 @@ nav:
           - specification/margo-management-interface/device-capabilities.md
           - specification/margo-management-interface/desired-state.md
           - specification/margo-management-interface/deployment-status.md
-      - Application Package:
+      - Applications:
           - specification/application-package/application-description.md
+          - specification/application-package/application-registry.md
       - Margo Devices:
           - specification/margo-devices/device-requirements.md
       - Observability: 

src/specification/application-package/resources/index.md.jinja2

@@ -1,4 +1,9 @@
-The application description has the purpose of presenting the application, e.g., on an application catalog or marketplace from where an end user selects an application to be installed. The end user defines an `ApplicationDeployment` by specifying [configuration parameters](#defining-configurable-application-parameters) for an `ApplicationDescription`. An `ApplicationDeployment` defines the [desired state](../margo-management-interface/desired-state.md) for an application.
+# Application Description
+
+The Application Description has the purpose of presenting the application, e.g., on a WFM-internal [Application Catalog](../../personas-and-definitions/technical-lexicon.md#workload-fleet-manager)) 
+or on a public [Marketplace](../../personas-and-definitions/technical-lexicon.md#marketplace)) from where an end user selects an application to be installed. 
+The end user defines an `ApplicationDeployment` by specifying [configuration parameters](#defining-configurable-application-parameters) for an `ApplicationDescription`. 
+An `ApplicationDeployment` defines the [desired state](../margo-management-interface/desired-state.md) for an application.
 
 ### Top-level Attributes
 

system-design/concepts/workloads/application-package.md

@@ -1,27 +1,29 @@
 # Application Package
 
-The application package, which is used to [distribute an application](../../overview/workloads.md), comprises the following elements:
+The Application Package, which is used to [distribute an application](../../overview/workloads.md) that shall be deployed as [workloads](../../personas-and-definitions/technical-lexicon.md#workload) on edge devices, comprises the following elements:
 
-- The **application description**: a YAML document with the element `kind` defined as `ApplicationDescription`, which is stored in a file (for example named `margo.yaml`) and contains information about the application's [metadata](../../specification/application-package/application-description.md#metadata-attributes) (e.g., description, icon, release notes, license file, etc.), application supported [deployment configurations](../../specification/application-package/application-description.md#deploymentprofile-attributes) (e.g,  Helm charts, Compose Archive), and [configurable application parameters](../../specification/application-package/application-description.md#defining-configurable-application-parameters).  There SHALL be only one YAML file in the package root of kind `ApplicationDescription`.
+- The **Application Description**: a YAML document with the element `kind` defined as `ApplicationDescription`, which is stored in a file (for example named `margo.yaml`) and contains information about the application's [metadata](../../specification/application-package/application-description.md#metadata-attributes) (e.g., description, icon, release notes, license file, etc.), the application [components](../../personas-and-definitions/technical-lexicon.md#component) (e.g,  Helm charts, Compose Archive) which are defined in [deployment configurations](../../specification/application-package/application-description.md#deploymentprofile-attributes), and [configurable application parameters](../../specification/application-package/application-description.md#defining-configurable-application-parameters).  There SHALL be only one YAML file in the package root of kind `ApplicationDescription`.
 - The **resources**, which are additional files associated with the application (e.g., manual, icon, release notes, license file, etc.) that may be used to display more information about the application in a UI such as an [application catalog](../../personas-and-definitions/technical-lexicon.md#application-catalog) or [marketplace](../../personas-and-definitions/technical-lexicon.md#workload-marketplace) or other informative outputs.
 
-The application package has the following file/folder structure:
+The Application Package SHALL follow a folder/file structure as such:
 
 ```yaml
 /                                  # REQUIRED top-level directory 
 └── <application description>.yaml # REQUIRED a YAML document with element 'kind' as 'ApplicationDescription' stored in a file  (e.g., 'margo.yaml')
 └── resources                      # OPTIONAL folder with application files (e.g., icon, license file, release notes) that may be used for displaying additional information about the application
 ```
 
-An application aggregates one or more [OCI Containers](https://github.com/opencontainers). While the application package is made available in an [application registry](./application-registry.md), the referenced OCI artifacts are stored in a remote or [local registry](../../concepts/workloads/local-registries.md). 
+The Application Package SHALL be made available in an [Application Registry](./application-registry.md).
+
+An application aggregates one or more [components](../personas-and-definitions/software-composition.md), which each link to one or more [OCI Containers](https://github.com/opencontainers). The components  referenced in the [Application Description](../../specification/application-package/application-description.md) are stored in a [Component Registry](../../personas-and-definitions/technical-lexicon.md#component-registry), and the linked containers are provided via a [Container Registry](../../personas-and-definitions/technical-lexicon.md#container-registry). Registries can be remote (i.e., Internet-accessible) or [local](../../concepts/workloads/local-registries.md) (i.e., accessible within a local network infrastructure of the devices). 
 
 > **Note**  
-> Application catalogs or marketplaces are out of scope for Margo. The exact requirements of the marketing material shall be defined by the application marketplace beyond outlined mandatory content.
+> Application catalogs and marketplaces are out of scope of the Margo specification. The exact requirements of the marketing material shall be defined by the application marketplace beyond outlined mandatory content.
 
-The [deployment profiles](../../specification/application-package/application-description.md#deploymentprofile-attributes) specified in the application description SHALL be defined as Helm Chart components AND/OR Compose components.
+The [deployment profiles](../../specification/application-package/application-description.md#deploymentprofile-attributes) specified in the [Application Description](../../specification/application-package/application-description.md) SHALL be defined as Helm Chart components AND/OR [Compose Archive](../../personas-and-definitions/technical-lexicon.md#compose-archive) components. These components will be deployed as workloads on the edge devices:
 
-- To target devices, which run Kubernetes, applications must be packaged as Helm charts using [Helm (version 3)](https://helm.sh/docs/topics/charts/).
-- To target devices, which deploy applications using [Compose](https://www.compose-spec.io/), applications must be packaged as what Margo calls a Compose Archive, i.e., a tarball file containing the `compose.yaml` file and any additional artifacts referenced by the Compose file (e.g., configuration files, environment variable files, etc.). Margo recommends to digitally sign this package and to specify the location of the public key in the `ApplicationDescription` (see `keyLocation` [here](../../specification/application-package/application-description.md#componentproperties-attributes)). When digitally signing the package PGP encryption MUST be used.
+- To target devices, which deploy workloads using Kubernetes, components must be defined as Helm charts using [Helm (version 3)](https://helm.sh/docs/topics/charts/).
+- To target devices, which deploy workloads using [Compose](https://www.compose-spec.io/), components must be packaged as [Compose Archives](../../personas-and-definitions/technical-lexicon.md#compose-archive), i.e., a tarball file containing the `compose.yaml` file and any additional artifacts referenced by the Compose file (e.g., configuration files, environment variable files, etc.). Margo recommends to digitally sign this package and to specify the location of the public key in the `ApplicationDescription` (see `keyLocation` [here](../../specification/application-package/application-description.md#componentproperties-attributes)). When digitally signing the package PGP encryption MUST be used.
 
 > **Investigation Needed**: Question: do we need to specify the location of a SHA256 hash for the Compose Archive also (similar to the PGP key) in the ApplicationDescription? 
 > We will also discuss how we should handle secure container registries that require a username and password.

system-design/concepts/workloads/application-registry.md

@@ -1,32 +1,44 @@
 # Application Registry
 
-This section describes the Application Registry and the exchange of an [application package](./application-package.md) from an Application Developer to the Workload Fleet Manager (WFM). 
+The margo specification differentiates 4 kinds of registries: *Application Registries*, *Component Registries*, and *Container Registries* as well as *Marketplaces*.
 
-The Application Developer SHALL use a [Git repository](https://git-scm.com/) to share an [application package](./application-package.md). This Git repository is considered the Application Registry. 
+1. An **Application Registry** hosts Application Packages that define through their [Application Description](../../specification/application-package/application-description.md) the application as one or multiple [Components](../../personas-and-definitions/technical-lexicon.md#component).
+2. A **Component Registry** hosts the [Components](../../personas-and-definitions/technical-lexicon.md#component) (which are deployable as *workloads*) and are provided  as **Helm Charts** or **Compose Archives**.
+3. A **Container Registry** hosts container images referenced by those Components.
+4. A **Marketplace** lists applications to advertise them and enable purchasing for end users.
 
-The connectivity between the Workload Fleet Manager and the Application Registry SHALL be read-only. 
+Out of these 4 registries, **only the Application Registry interface is in scope** of the margo specification and its API definition can be found [here](../../specification/application-package/application-registry.md).  
 
-Upon installation request from the End User, the Workload Fleet Manager SHALL retrieve the [application package](./application-package.md) using a ``git pull`` request from the Application Registry. 
+The diagram below illustrates these functionalities and relationships of registries within margo.
 
-The Workload Fleet Manager reads in the application description file, ``margo.yaml``, and presents a user interface that allows the specification of parameters available according to ``margo.yaml``. 
+```mermaid
+flowchart
+   A[WFM, or internal Application Catalog] -- Application Descriptions link to --> B[Component Registry] 
+   C[Application Registry] -- Application Descriptions link to --> B
+   B -- hosted Components links to --> D[Container Registry]
+   A -->|pulls Application Package | C
+   F[App Developer] -->|uploads Application Package to| C
+   G["Marketplace"] -- points to Application Package --> C
+   C -->|hosts 0..*| E@{ shape: docs, label: "Application Packages"} 
+   C -->|validates token| H[Authentication Service] 
+   A -->|requests token| H
+   style H stroke-dasharray: 3 6
 
-The End User then specifies the configuration parameters for the [application package](./application-package.md). 
+   style B fill:#ABC
+   style C fill:#ABC
+   style D fill:#ABC
+   style G fill:#ABC
+```
 
-Then, the [application package](./application-package.md) is ready to be passed on to the installation process.  
+As shown in the figure above, an `Application Developer` uploads an [Application package](application-package.md) to an Application Registry. From there, it is available to a `Workload Fleet Manager` (WFM).
+The WFM acts as a client to pull an [Application Package](application-package.md) from the Application Registry. It would then list this Application Package on its UI (e.g., an internal `Application Catalog`) to enable the usage on its managed devices.
 
-> **Note** 
-> The specifics of the installation process are still under discussion: this could be for example a GitOps based approach. 
+An `Authentication Service` manages access control for the Application Registry. Hence, the WFM requests a token from there to include it into its requests to the Application Registry. The received token is then validated by the Application Registry through interaction with the Authentication Service.
 
-During this process the containers referenced in the application manifest ([Helm Chart](https://helm.sh/docs/) or [Compose](https://github.com/compose-spec/compose-spec/blob/master/03-compose-file.md)) are retrieved from container/Helm registries. 
+The Application Registry's API is compliant with the [OCI Registry API (v1.1.0)](https://github.com/opencontainers/distribution-spec/blob/v1.1.0/spec.md). A hosted Application Package is provided by listing its parts as layers in an [image manifests](https://github.com/opencontainers/image-spec/blob/v1.0.1/manifest.md)) that can be requested through the API.
 
-At a minimum, a Margo-compliant Workload Fleet Manager SHALL provide a way for an end user to manually set up a connection between the Workload Fleet Manager and an application registry. This is required so as not to prohibit an end user from being able to install any Margo-compliant application they wish, including applications developed by the end user.  
 
-The Workload Fleet Manager MAY provide enhanced user experience options such as the pre-configuring of application registries to monitor. These can include application registries from third-party application vendors or their own applications. 
+## Relevant Links
+* The technical reference of the Application Registry API is defined [here](../../specification/application-package/application-registry.md).
 
-## Secure Access to the Application Package 
-
-It is expected the connection between the Workload Fleet Manager and the Application developer’s application registry is secured using standard secure connectivity best practices. Some standard practices include the following: 
-
-- Basic authentication via HTTPS 
-- Bearer token authentication 
-- TLS certifications 
+* A reference implementation of the Application Registry is described [here](https://github.com/margo/app-package-definition-wg/blob/main/application-registry-example/app_registry_as_oci_registry.md) and includes sample applications and configuration for demonstration. It utilizes an open source OCI Registry and the [ORAS tool](https://oras.land/docs/) as the client.