add SIP - packaging components

[michelleN]

No description provided.

[itowlson]

Good stuff, thanks for getting this under way!

[itowlson]

I like this emphasis. The component cannot grant itself the permissions because we can't trust it, but we need to know what it wants from us.

(no action, just wanted to applaud)

[itowlson]

Recognises it by the manifest version declaration?

[itowlson]

PEDANTRY CORNER: spin up -f bare.wasm does work. (but your point is still right of course)

[itowlson]

Correct to say a standalone component cannot have dependencies?

[fibonacci1729]

For now at least. Not forever.

[itowlson]

I'm getting a bit confused across OCI and wkg at this point, sorry. The draft uses OCI terminology and references in some places, and wkg in others (which is what is mentioned in the preamble).

[itowlson]

spin registry pull with applications puts them in the Spin cache I believe.

[ThorstenHans]

Great stuff @michelleN

Although I am on the same page because it would streamline spin.toml and ensure the top-level application manifest remains somewhat "short", I went through it with some common middlewares and want to share an alternative approach:

What about keeping everything in spin.toml, because:

• dependencies are pre-built by component authors and consumed as .wasm binary (no-src) -> integration with spin build should be opt-in
• component source is available, but in different repository
• users may want to use variables defined on application scope (including interpolation)
• The component author is obviously in charge of documenting the needs of their components
  • IMO: Authoring dependency components at the same time / in the same context as implementing its consumer application is rare. Looking at other ecosystems that provide composition, it's way more common to consume these. If devs happen to identify good "sharable" component candidates while building an "app", it's pretty common to generalize the component, and implement/distribute it independently from the actual application lifecycle

I drafted a bit and ended up with something like this (not really different from the current approach, but adds more granular variable handling).

# ...

[variables]
app_name = { default = "My Spin App" }

[[trigger.http]]
route = "/api/..."
component = "api"

[component.api]
source = "api/target/wasm32-wasip2/release/api.wasm"
allowed_outbound_hosts = []
key_value_stores = ["default"]
dependencies = ["cors", "correlation-id", "header-injection"]

[component.cors]
source = { path = "../cors/target/wasm32-wasip2/release/cors.wasm" }

[component.cors.variables]
allowed_methods = "POST,GET,PUT,DELETE"
allowed_origings = "*"
# allowed_headers is unset (component might have smart defaults)

[component.correlation-id]
source = { path = "../cors/target/wasm32-wasip2/release/cid.wasm"} 

[component.correlation-id.variables]
header_name = "X-CORRELATION-ID"

[component.header-injection]
source = {path = "./components/hi/target/wasm32-wasip2/release/hi.wasm" }

[component.header-injection.build]
workdir = "./components/hi"
command = "cargo build --target wasm32-wasip2 --release"

[component.header-injection.variables]
header_name = "X-APP-NAME"
header_value = "{{ app_name }}"

[fibonacci1729]

Suggested change

	sql_variables = ["default"]
	sqlite_databases = ["default"]

[fibonacci1729]

Suggested change

	environment_variables = ["staging", { name = "region", default = "us" }]
	environment = ["staging", { name = "region", default = "us" }]

[fibonacci1729]

Suggested change

	| `sql_variables` | SQLite database labels the component accesses. |
	| `sqlite_databases` | SQLite database labels the component accesses. |

[fibonacci1729]

Suggested change

	| `environments_variables` | Environment variables the component needs. Each entry is a bare name, or `{ name, default }`. |
	| `environment` | Environment variables the component needs. Each entry is a bare name, or `{ name, default }`. |

[fibonacci1729]

I maybe wouldn't say these mirror the capability fields in spin.toml -- it's likely, at least some of these, will diverge from the schema associated with the name used in spin.toml (e.g. environment is divergent).

[fibonacci1729]

We should probably not allow glob patterns here; globs are relative to the consuming app's directory and the dependency component has no idea what's there. What the component actually knows is the guest path it expects to read so i'd consider restricting this to { destination }.

[itowlson]

That's an interesting thought @fibonacci1729. I like the idea of being able to say "I expect to find whatever files you want me to use at /foo, but it's up to you where you source them."

There is another use case, but I am not sure about generality: static assets. Here the component / dep / middleware doesn't want to put it on the app to supply files, because it already has specific ones it wants to use. E.g. the HTML, JavaScript and CSS for the "authorisation failed" page, or a CSV file of internal data. However, while this (I think) makes sense for a local component (a TOML file), we can't currently publish it as a standalone Wasm binary.

[fibonacci1729]

I think we'll need to enable push to embed metadata since build is optional.

[fibonacci1729]

@ThorstenHans we can't keep everything in the spin.toml because this SIP explicitly covers the producer side, i.e.: How do I, as the author of cors build it, version it, publish it, and describes what it needs -- from it's own repo, where there is no application to hang a spin.toml off. What you're describing in your alternative proposal is the consuming side.

Authoring dependency components at the same time / in the same context as implementing its consumer application is rare... it's pretty common to generalize the component, and implement/distribute it independently from the actual application lifecycle

Strongly agree and that's the argument for a standalone manifest. The moment you "generalize and distribute independently," you need an independent build/version/publish story, which is what component.toml provides.