td.replaceEsm() without namedExportStubs throws weird error under windows

[jishi]

Description

I was evaluating testdouble to do mocking with native ES modules, but I run into a weird error that seems to be a bug.

Issue

it throws

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at new NodeError (internal/errors.js:322:7)
    at defaultResolve (internal/modules/esm/resolve.js:814:11)
    at resolve (file:///C:/Users/jishi/Documents/source/lovsta/mynewsdesk/node_modules/quibble/lib/quibble.mjs:12:25)
    at resolve (file:///C:/Users/jishi/Documents/source/lovsta/mynewsdesk/node_modules/quibble/lib/quibble.mjs:28:12)
    at Loader.resolve (internal/modules/esm/loader.js:89:40)
    at Loader.getModuleJob (internal/modules/esm/loader.js:242:28)
    at Loader.import (internal/modules/esm/loader.js:177:28)
    at importModuleDynamically (internal/modules/cjs/loader.js:1028:27)
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:30:14)
    at Object.exports.importOriginalModule (node_modules\quibble\lib\esm-import-functions.js:22:74)
    at Function.esmImportWithPath (node_modules\quibble\lib\quibble.js:128:41)
    at Object.replaceEsModule (node_modules\testdouble\lib\replace\module\index.js:39:50)
    at Module.replaceEsm (node_modules\testdouble\lib\replace\index.js:23:21)
    at Context.<anonymous> (file:///C:/Users/jishi/Documents/source/lovsta/mynewsdesk/test/acceptance/sync.mjs:6:14)
    at processImmediate (internal/timers.js:464:21)

When doing the following:

import * as td from 'testdouble';

describe('test/acceptance/sync.mjs', function () {

  beforeEach(async function () {
    await td.replaceEsm('../../src/mynewsdesk.mjs');
  });

  afterEach(function () {
    td.reset();
  });

  it('should fetch and upsert entries', async function () {

  });

});

If I add a replacement object as second argument, it doesn't fail:

await td.replaceEsm('../../src/mynewsdesk.mjs', {});

Looking at the source, if I have 2 or more arguments, it just transparently forwards down to quibble.

Environment

• node -v output: v14.18.1
• npm -v (or yarn --version) output: 6.14.15
• npm ls testdouble (or yarn list testdouble) version: [email protected]

Windows 10.

The same code, doesn't throw error if running in Linux, via WSL. Even weirder though is that the code just works, without specifying a loader, however maybe the loader only makes the stubbing works so without an assertion I won't notice it.

[searls]

Interesting! Other than tagging @giltayar for help, I wonder if this issue also exists under Windows under the latest Node 16.13.0?

[giltayar]

Will look into it.

[giltayar]

Took me a while, but I looked. Unfortunately, the code works under Mac, and I have no Windows machine.

I did research it a bit, and when you pass an undefined named exports, it will transform the mocked file into an empty file with no named and no default export. Just zero bytes. Maybe this is what trips the loader mechanism in Windows?

Not sure what I can do with this. Looks like a bug in the Windows version of Node.js. Maybe open a bug for Node?

[joselcvarela]

What solved for me, was to do the replaceEsm at the end of beforeEach, like so

test.beforeEach(async () => {
  //service = await td.replaceEsm('../../');  // moved from here
  ...
  ...
  service = await td.replaceEsm('../../'); // to here
})

I believe the issue is on starting things.
If we have an  `await` before calling `replaceEsm` I think it solves the problem.

[RexMorgan]

I've been investigating this issue and I believe the root is that in quibble.js, hackErrorStackToGetCallerFile returns something like this: /C:/git/test/fixtures.js with the forward slash at the beginning. Then, in esmImportWithPath, it calls path.resolve(path.dirname(callerFile), importPath), and ends up trying to import C:\C:\git\logger.js.

I hacked in a quick fix in hackErrorStackToGetCallerFile to do a .replace(/^\/(\w:\//g, '$1') (to strip out the forward slash if it's followed by (LETTER):/) and then prepend file:// to the import and I'm starting to get errors w/ mocha, so it looks like I broke it in some other way and it may not even be getting to my original failing code anymore... 🤦‍♂️

    return _.flow([
      _.invokeMap('getFileName'),
      _.compact,
      _.map(convertUrlToPath),
      _.reject(function (f) {
        return includeGlobalIgnores && _.includes(f, ignoredCallerFiles)
      }),
      _.filter(path.isAbsolute),
      _.find(function (f) {
        return f !== currentFile
      })
    ])(e.stack).replace(/^\/(\w:\/)/g, '$1');
...
  const fullImportPath = importPathIsBareSpecifier
    ? importPath
    : `file://${modulePath}`;

I've spun up a linux vm, and I'm working from there for now to get past this, but I may end up converting my code back to CommonJS. I wasn't aware ESM imports were so finicky when I decided to use them.

[searls]

@RexMorgan thanks for looking into this. I think that a PR that used a more durable pattern (or, if Windows always behaves this way, even for network shares, a branching condition on os.platform()) would resolve this issue. Would you be interested in helping us with that?

[RexMorgan]

@searls Sure, I can take a deep dive into this when I get some down time. I ended up switching back to the commonjs require calls, but I'd love to be able to use ESM imports :)

Also, I think this will be a fix for quibble, not testdouble.js

[cwienands1]

Any news on this issue? I have a new ES6 hobby project for which I would like to use testdouble, and I ran into exactly the same issue. My development machine is Windows. I am attempting something as simple as loginWithAwsModule = await td.replaceEsm('../src/login-with-aws.js'); but recceive the same message as the OP:

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'

I played around with my own code as well as the the quibble code. Passing in an absolute path in the form of c:\home\... did not work. Breaking in quibble to remove the duplicate 'c:\c:\...' didn't work either. Once I got the code to run through the entire replaceEsm() method (with several breakpoints and variable value changes) but the module didn't seem to get successfully mocked.

This is really the only post that I can find related to this issue. I am really astounded that there are not more. I mean, are developers with Windows machines using ES6 and modules, and requiring testdouble, really that rare?

[searls]

@cwienands1 it's been a few years since I've heard of anyone using Node.js outside Windows Subystem for Linux, for what it's worth, which resulted in me hearing a lot less about Windows-specific issues. Assuming this fails only under the Windows command line?

I would like to fix this, I just don't have a development environment set up with Windows.

[RexMorgan]

I'm no longer developing on Windows and am on macos, now. So, I'm unable to look at this anymore.

[giltayar]

I now have a Windows environment, so if you send me a sample repo reproducing this error, I'll gladly look into this.