Question about cross compile Node-API addon to wasm with emnapi and emscripten on Windows/macOS

[toyobayashi]

Hi! I'm the creator of emnapi which is a Node-API implementation for emscripten/wasi-sdk, it can be used for porting existing Node-API native addon to WebAssembly. I'm wondering if node-gyp can be used for building wasm with emscripten, then I have a test repo https://github.com/toyobayashi/emnapi-node-gyp-test here, seems that node-gyp cross compiling with emscripten only works on Linux, both Windows and macOS are failed.

• Node.js version: 18
• npm version: 8

Steps to reproduce:

git clone https://github.com/toyobayashi/emnapi-node-gyp-test
cd emnapi-node-gyp-test
npm install --ignore-scripts
node-gyp clean
node-gyp configure --nodedir=./wasm -- -f make
emmake make -C build

Logs and reason:

Windows 11 22H2 22621.1413

Failed due to the generated Makefile still use unix tools, even if execute make under WSL 2 then failed due to \\ path seperator.

> npm install --ignore-scripts

add 3 packages, and audited 4 packages in 8s

found 0 vulnerabilities

> node-gyp clean
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.12.1 | win32 | x64
gyp info ok

> node-gyp configure --nodedir=./wasm -- -f make
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.12.1 | win32 | x64
gyp info find Python using Python version 3.9.7 found at "C:\Users\toyobayashi\AppData\Local\Programs\Python\Python39\python.exe"
gyp info find VS using VS2022 (17.2.32516.85) found at:
gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
gyp info find VS run with --verbose for detailed information
gyp WARN read config.gypi ENOENT: no such file or directory, open 'C:\Users\toyobayashi\Projects\emnapi-node-gyp-test\wasm\include\node\config.gypi'
gyp info spawn C:\Users\toyobayashi\AppData\Local\Programs\Python\Python39\python.exe
gyp info spawn args [
gyp info spawn args   'C:\\Users\\toyobayashi\\app\\nvm\\v18.12.1\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\toyobayashi\\Projects\\emnapi-node-gyp-test\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\toyobayashi\\app\\nvm\\v18.12.1\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\toyobayashi\\Projects\\emnapi-node-gyp-test\\wasm\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=./wasm',
gyp info spawn args   '-Dnode_gyp_dir=C:\\Users\\toyobayashi\\app\\nvm\\v18.12.1\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=wasm\\\\$(Configuration)\\\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=C:\\Users\\toyobayashi\\Projects\\emnapi-node-gyp-test',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'C:\\Users\\toyobayashi\\Projects\\emnapi-node-gyp-test\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info ok

> emmake make -C build
make: C:\Users\toyobayashi\app\make\mingw32-make.exe -C build
mingw32-make: Entering directory 'C:/Users/toyobayashi/Projects/emnapi-node-gyp-test/build'
process_begin: CreateProcess(NULL, printf %s\n "  CC(target) Release/obj.target/binding/node_modules\emnapi\src\js_native_api.o", ...) failed.
make (e=2): 系统找不到指定的文件。
mingw32-make: *** [binding.target.mk:110: Release/obj.target/binding/node_modules\emnapi\src\js_native_api.o] Error 2
mingw32-make: Leaving directory 'C:/Users/toyobayashi/Projects/emnapi-node-gyp-test/build'
emmake: error: 'C:\Users\toyobayashi\app\make\mingw32-make.exe -C build' failed (returned 2)

> wsl
$ emmake make -C build
make: make -C build
make: Entering directory '/mnt/c/Users/toyobayashi/Projects/emnapi-node-gyp-test/build'
make: *** No rule to make target '../wasm\common.gypi', needed by 'Makefile'.  Stop.
make: Leaving directory '/mnt/c/Users/toyobayashi/Projects/emnapi-node-gyp-test/build'
emmake: error: 'make -C build' failed (returned 2)

WSL 2 Ubuntu 20.04 (Success)

$ npm install --ignore-scripts

added 3 packages in 418ms

$ node-gyp clean
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.14.0 | linux | x64
gyp info ok

$ node-gyp configure --nodedir=./wasm -- -f make
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.14.0 | linux | x64
gyp info find Python using Python version 3.8.10 found at "/usr/bin/python3"
gyp WARN read config.gypi ENOENT: no such file or directory, open '/home/toyobayashi/Projects/emnapi-node-gyp-test/wasm/include/node/config.gypi'
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/home/toyobayashi/.nvm/versions/node/v18.14.0/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/toyobayashi/Projects/emnapi-node-gyp-test/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/toyobayashi/.nvm/versions/node/v18.14.0/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/toyobayashi/Projects/emnapi-node-gyp-test/wasm/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=./wasm',
gyp info spawn args   '-Dnode_gyp_dir=/home/toyobayashi/.nvm/versions/node/v18.14.0/lib/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=wasm/$(Configuration)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/toyobayashi/Projects/emnapi-node-gyp-test',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info ok

$ emmake make -C build
make: make -C build
make: Entering directory '/home/toyobayashi/Projects/emnapi-node-gyp-test/build'
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/js_native_api.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/node_api.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/emnapi.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/async_cleanup_hook.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/async_context.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/async_work.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/threadsafe_function.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/uv-common.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/threadpool.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/unix/loop.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/unix/thread.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/unix/async.o
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/uv/unix/core.o
  CC(target) Release/obj.target/binding/src/binding.o
  LINK(target) Release/binding.js
em++: warning: USE_PTHREADS + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]
cache:INFO: generating system asset: symbol_lists/7f16a0cf42f34d504060010d577a8705337deb0d.txt... (this will be cached in "/home/toyobayashi/Projects/emsdk/upstream/emscripten/cache/symbol_lists/7f16a0cf42f34d504060010d577a8705337deb0d.txt" for subsequent builds)
cache:INFO:  - ok
make: Leaving directory '/home/toyobayashi/Projects/emnapi-node-gyp-test/build'

macOS 13.2.1 (M2 Pro)

Failed due to emcc doesn't support -arch arm64 compile flag.

$ npm install --ignore-scripts

added 3 packages in 593ms

$ node-gyp clean
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.12.1 | darwin | arm64
gyp info ok

$ node-gyp configure --nodedir=./wasm -- -f make
gyp info it worked if it ends with ok
gyp info using node-gyp@9.3.1
gyp info using node@18.12.1 | darwin | arm64
gyp info find Python using Python version 3.9.6 found at "/Library/Developer/CommandLineTools/usr/bin/python3"
gyp WARN read config.gypi ENOENT: no such file or directory, open '/Users/toyobayashi/code/github/emnapi-node-gyp-test/wasm/include/node/config.gypi'
gyp info spawn /Library/Developer/CommandLineTools/usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/Users/toyobayashi/.nvm/versions/node/v18.12.1/lib/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/toyobayashi/code/github/emnapi-node-gyp-test/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/toyobayashi/.nvm/versions/node/v18.12.1/lib/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/toyobayashi/code/github/emnapi-node-gyp-test/wasm/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=./wasm',
gyp info spawn args   '-Dnode_gyp_dir=/Users/toyobayashi/.nvm/versions/node/v18.12.1/lib/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=wasm/$(Configuration)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/toyobayashi/code/github/emnapi-node-gyp-test',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info ok

$ emmake make -C build
make: make -C build
  CC(target) Release/obj.target/binding/node_modules/emnapi/src/js_native_api.o
emcc: error: arm64: No such file or directory ("arm64" was expected to be an input file, based on the commandline arguments provided)
make: *** [Release/obj.target/binding/node_modules/emnapi/src/js_native_api.o] Error 1
emmake: error: 'make -C build' failed (returned 2)

Is there any way to workaround such gyp default behavior on different OS? Does it require upstream gyp code change to support? Or is there any way to solve these problem via configuration or some command line flags?

[toyobayashi]

anyone could help?

[mhdawson]

@toyobayashi just noticed this issue. May not be the right group to discuss the issues around node-gyp but if you come to the Node-api team meeting on Friday at 11 ET (https://zoom.us/j/363665824
https://hangouts.google.com/hangouts/_/event/c0eevtrlajniu7h8cjrdk0f56c8

I think we'd be intersted in better understanding what you are trying to do and the current issues. Might be one way to find people who might be interested in helping.

[toyobayashi]

@mhdawson Thanks for your invitation! But I'm sorry I think my English listening and speaking skills are not enough for me to attend the meeting.

I just want to find a way that make node-gyp can work fine on win/linux/mac for emscripten target, currently it only works on linux via --nodedir and -- -f make.

[mhdawson]

I'm not sure if its a reasonable suggestion but I think @legendecas is in the same region as you and maybe you two could talk more easily? Just a suggestions as I do know there are still different languages in the region.

One thing I wondered was how hard the dependency on node-gyp was versus using something like cmake which I know is also a possiblity for node-addon-api modules.

[toyobayashi]

@mhdawson CMake is easy to integrate cause emscripten has already provided the toolchain file, while node-gyp no. emnapi support user to use CMake build Node-API wasm module. On the other hand, not only does node-gyp not have a toolchain config file for emscripten, the generator will also generate platform-specific compilation options based on the operating system, for example on Windows it will generate VS solution by default, even if I tell it generate makefile, the path separator is incorrect, and on macOS it generates -arch arm64 which doesn't work in emcc. I don't know how to control these behavior.

Although CMake can indeed easily build Node-API wasm modules, most native modules in Node.js community are built using node-gyp. For example, sharp is using emnapi to build wasm that can run on StackBlitz, they still prefer using node-gyp (https://github.com/lovell/sharp/blob/main/src/emscripten/common.gypi) instead of maintaining another CMake configuration for emnapi. So I think it is so nice if node-gyp can work on all platform cause this can reduce the cost for native module maintainers to compile Node-API into wasm modules.

@legendecas Are you interested in taking a look?

[legendecas]

I am not familiar with windows builds but I would try this out on mac. Also, please feel free to message me on OpenJS Slack.

[toyobayashi]

@legendecas The link seems to be broken.

Just now I found out how to make it work on mac, pushed emnapi-node-gyp-test. I added these environment variables and place link flags in xcode_settings.OTHER_LDFLAGS then it work.

GYP_CROSSCOMPILE=1
AR_host=ar
CC_host=clang
CXX_host=clang++
AR_target=emar
CC_target=emcc
CXX_target=em++

emmake node-gyp rebuild --verbose --nodedir=./wasm -- -f make 

[legendecas]

Does the same trick address the problem on Windows as well?

[legendecas]

For what it's worth, I can build with https://github.com/toyobayashi/emnapi-node-gyp-test/blob/main/build.sh with following patch:

export GYP_CROSSCOMPILE=1
export AR_host=ar
export CC_host=clang
export CXX_host=clang++
export AR_target=emar
export CC_target=emcc
export CXX_target=em++

emmake node-gyp rebuild --verbose --nodedir=./wasm -- -f make 

[toyobayashi]

Does the same trick address the problem on Windows as well?

No, makefile generator generates linux commands which is not supported on windows. I tried to use Cygwin but the path separator "" generated in makefile doesn't work.

[toyobayashi]

After some hard trying, finally build successfully on Windows with some tricks. Pushed emnapi-node-gyp-test.

1. Install make and cygwin

2. npm install -D node-gyp@10.0.1

3. Modify node_modules/node-gyp/gyp/pylib/gyp/generator/make.py, line 862

   - self.output = self.output_binary = self.ComputeOutput(spec)
   + self.output = self.output_binary = self.ComputeOutput(spec).replace('\\', '/')

4. build.bat

   @echo off
   
   set GYP_CROSSCOMPILE=1
   set AR_target=emar.bat
   set CC_target=emcc.bat
   set CXX_target=em++.bat
   
   call npx.cmd node-gyp clean
   call npx.cmd node-gyp configure --nodedir=./wasm -- -f make
   
   node ./scripts/replace-sep.js
   
   call emmake.bat make -C build V=1

5. scripts/replace-sep.js

   const path = require('path')
   const fs = require('fs')
   
   const buildDir = path.join(__dirname, '../build')
   
   fs.readdirSync(buildDir)
     .filter(p => p.endsWith('.target.mk'))
     .map((p) => path.join(buildDir, p))
     .map((p) => {
       const content = fs.readFileSync(p, 'utf8').replace(/\\|\\\\/g, '/').replace(/\/(\r?\n)/g, '\\$1')
       fs.writeFileSync(p, content, 'utf8')
     })

6. Run build.bat in Cygwin terminal.

This is just for such a simple example, if for a more complex project, I think there will be more gyp source code that needs to be modified.

Can gyp consider make generator/make.py generate correct path on windows? Or add nmake generator?