Add support for ESM / node 14?

[tripodsan]

see adobe/helix-deploy#227

Some of the dependencies major versions are now modules and are no longer provided as CommonJS modules.
https://github.com/adobe/helix-shared/pull/487/files#diff-7b5c8955fc544a11b4b74eddb4115f9cc51c9cf162dbffa60d37eeed82a55a57

If we would want to use them, we would need to enable the ESM loader for building and testing. which would affect all our libraries and services. also see the note in https://nodejs.org/en/blog/release/v14.0.0/:

As of Node.js 14 there is no longer this warning when using ESM in Node.js. However, the ESM implementation in Node.js remains experimental. As per our stability index: “The feature is not subject to Semantic Versioning rules. Non-backward compatible changes or removal may occur in any future release.” Users should be cautious when using the feature in production environments.

i.e. I would only feel comfortable to enable ESM, if we also require node14.

As for deployment, we can either babel+webpack and create a CommonJS bundle, then we don't need use ESM on the runtimes,
or we can create ESM bundle.

Also note, that we need to rewrite ALL code of the services that use ESM dependencies and switch them to modules (because there is no way to require a module dependency.

action items:

• verify that all 3rd party environments support node 14 for production
• create POC using pure ESM and use a ESM-only dependency and deploy it to all runtimes
• create POC using babel+webpack and use a ESM-only dependency and deploy it to all runtimes

[tripodsan]

btw, I converted word2md to esm:
https://github.com/adobe/helix-word2md/pull/648

it's more or less straightforward. the biggest problem are some of the missing features like __dirname and module loading of a json. the 2nd one is rather a problem with the webpack interoperability. maybe using rollup instead would solve that problem. (see adobe/helix-deploy#228)

[stefan-guggisberg]

Some of the dependencies major versions are now modules and are no longer provided as CommonJS modules.
https://github.com/adobe/helix-shared/pull/487/files#diff-7b5c8955fc544a11b4b74eddb4115f9cc51c9cf162dbffa60d37eeed82a55a57

The list above seems rather manageable and the packages don't appear to be essential, i.e. I guess there are plenty of alternatives.

What exactly would be the benefit of rewriting our entire code base to support ESM, considering the risks (experimental node support) and effort involved?

[tripodsan]

Well, that doesn't really convince me ;-)

yeah..it doesn't come for free :-(

BTW: An IMO excellent article on the subject: https://redfin.engineering/node-modules-at-war-why-commonjs-and-es-modules-cant-get-along-9617135eeca1

very nice article. we could also look into creating hybrid libraries as highlighted in the article. don't know how feasible this is.

[trieloff]

We can create hybrid libraries as long as all our dependencies are hybrid or CJS. I don't think that's worth the effort for the most part. Instead I'd start at the leaf nodes of our dependency tree, convert to ESM and not turn back.

Btw, the biggest driver for me to adopt ESM would be that it is the more standard syntax, used by browsers and something that opens the door to runtimes like CF workers or deno.

[tripodsan]

Instead I'd start at the leaf nodes of our dependency tree, convert to ESM and not turn back.

unfortunately, the library with the most esm deps is helix-pipeline, which will force helix-cli and the pre.js functions to be modules. this will be fun!

[trieloff]

I don't think that ESM helix-pipeline will do that:

• helix-cli should use helix-deploy Remove own build process and use helix-deploy instead helix-cli#1744 and so we only need to convert the UniversalTemplate.js to ESM, the rest of the CLI can stay CJS.
• as the pre.js functions are imported by the Universal Template and ESM can import CJS, most can stay CJS, too – unless they have ESM dependencies themselves

[tripodsan]

• as the pre.js functions are imported by the Universal Template and ESM can import CJS, most can stay CJS, too – unless they have ESM dependencies themselves

there might be a problem when they re-use the steps from the pipeline.