Target environments: local polyfills for environment-specific APIs

[itowlson]

(cc @karthik2804, who I believe has been experimenting with something like this)

Consider, wearily, Bob's Discount Cloud. It may be cheap, but it does have one unique feature: a cannon that can fire evildoers into the sun. Applications fire the sun cannon via a bdc:suncannon/fire WIT interface. Bob proudly lists it as an available import in his bdc environment file.

For reasons of vendor neutrality - and, frankly, safety - the Spin CLI does not provide an implementation of bdc:suncannon/fire. Components that import this interface therefore fail to run in the CLI - the components can't even be pre-instantiated - meaning developers can't test sun-cannon based applications before deploying them.

A possible way out of this is to allow environment files to specify polyfills (or stubs or whatever) for imports not available in the CLI. E.g.

# bdc.toml
[polyfill]
"bdc:suncannon/fire@1.0.0" = { url = "https://example.com/sc.wasm", digest = "..." }

When a developer ran a component that imported an interface not available in Spin, but declared and polyfilled in the application's target environment, Spin would compose the polyfill onto the component, in exactly the same way as if the component had declared it as a dependency. This would satisfy the component import. (In practice the way I imagine we'd work this is just to wac everything in the target env's polyfill section onto every component, quietly ignoring cases where the component didn't import the interface. But others probably have a better picture of this than I do.) Karthik describes a better way in his comment below.

The polyfill/stub would have to work within the Spin world. For example, it might print a message to the console, or return a dummy value; in that sense maybe I should call them stubs.

Considerations:

• What if someone tries to polyfill an interface that Spin already implements? That seems rude but I am not sure how easy it is to detect and disallow it. We can't rely on the namespace because there are wasi: specs we don't implement.
• What is the permissions model here? E.g. if the polyfill wants to make network requests or access files? Allowing Bob to declare permissions, or even inheritance, in the target env file breaks our usual "only permissions explicitly granted by the developer" model. But we don't want the manifest cluttered with stuff that only exists for the purposes of these laughable fakes.

[karthik2804]

@itowlson thanks for the write-up here. The approach I was experimenting with differs in that, I chose the route of having a new factor that would load components and add it to the linker instead of composing it and the host would mediate the calls instead of the components being composed together. The benefit I see with the factor approach is that the component can possibly be re-used to allow sharing state between different components of the app. The example for my approach was a caching interface. I did also briefly toy with the idea of letting the wasm-components that provide host capabilities to have file-write access. I do imagine the capabilities for these can be provided in the runtime-config or possible a new file which the runtime-config points to but I have not put too much thought into this yet.

The approach I took was to configure Spin to support arbitrary interfaces by adding configuration to the runtime configuration. I was also looking to keep the capability model of Spin by create a new field in spin.toml which was a table of tables keyed by the interface-name and left the parsing of the configuration to the stub component.

[itowlson]

Ooh that's a great point about shared state. I agree, doing it in the linker is a better approach. And that avoids the "permissions can only be inherited" problem - we still need a trust/permissions story but it decouples it from any parent component, and your proposal of runtime config or some other permissions file makes sense to me.

I'm fine with allowing users to manually map stubs in runtime config, but I do think leveraging environments would make for a more transparent developer experience.

[itowlson]

@karthik2804 Could you share a link to your experiment please? I looked at your fork but couldn't see a branch that seemed to be it, sorry. Thanks!

[karthik2804]

@itowlson I had not pushed it up earlier and here it is https://github.com/karthik2804/spin/tree/wasm_host_components. It still has some half baked ideas.

[lann]

Another way to think about this: polyfills are basically just dependencies (right?). Maybe this feature should be "suggested dependencies", and when we have a target environment validation failure we could suggest fixes from this set. A certain doctor could even perform the necessary surgery for the user...

[lann]

With the above you'd also (presumably) want a targets field on dependencies to allow applying the polyfill dependency to only certain targets.