src: add config file support

[marco-ippolito]

Refs: #53787

Introducing node.json

node --experimental-config-file=noderc.json file.js

Why a config file?

With the introduction of test runner, sea, and other feature that require a lot of flags, a json config flag would improve by a lot the developer experience and increase adoption.
Example:

node --experimental-config-file=noderc.json --test --experimental-test-coverage

{
 "test-coverage-lines": 80,
 "test-coverage-branches": 60
}

Current solutions

1. CLI Flags
2. NODE_OPTIONS
3. --env-file with NODE_OPTIONS

All of these solutions dont have a good developer experience.

Why JSON?

Simply because we already support json parsing (simdjson) and its kinda the default in the js ecosystem.

Do we have to support all the configurations possible?

NO

Not all flag need to be supported by the config file, and not all flags CAN be supported.
The config file will only support flags that can be passed through NODE_OPTIONS:
https://nodejs.org/api/cli.html#node_optionsoptions

How does it work?

This is an early development, it work similarly to the --env-file flag. It will parse the provided file with --experimental-config-file and add to NODE_OPTIONS the recognized flags. Unrecognized flags will be ignored.

Versioning

json_schema!
When we release a new version, we also release a json schema file.

{
  "$schema": "https://nodejs.org/dist/v20.18.3/docs/node_config_json_schema.json",
}

The users can has autocompletion and validation of their config file specific to the version they are using.
Since the config file mirrors the features that the current version has, it would be impossible to provide semver major backwards compatibility (immagine a feature gets removed we also have to remove it from the config file).

Examples in the ecosystem

• deno.json
• bun toml

Security

Just like .env files, Node.js entrusts the user to provide a secure and valid configuration.

@nodejs/tsc for visibility since its a big feature

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/startup

[anonrig]

I took a somewhat similar approach last year. Happy to talk in private if you need any help. Ref: anonrig#4

[AugustinMauroy]

Should this configuration file include experimental options?
I don't think so, it should just allow config to be used from feature mark to stable.

[AugustinMauroy]

And if this configuration file could support the version of node. It could issue a warning if the wrong version is used for the project.
But that would require node to be able to resolve semver ranges.
And I think that would really add a degree of complexity to the file.

[marco-ippolito]

I don't see the downside of supporting experimental features. Users can validate with the json schema we provide for the version they are using (you can read in the docs how)

[AugustinMauroy]

I think that enabling feature experimental support will increase their adoption, and so in the event of a breaking change, it will impact more users.

[codecov]

Codecov Report

Attention: Patch coverage is 72.48062% with 71 lines in your changes missing coverage. Please review.

Project coverage is 90.35%. Comparing base (3b0fce1) to head (a7ee706).
Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
src/node_config_file.cc	63.02%	33 Missing and 11 partials ⚠️
src/node_options.cc	62.50%	14 Missing and 10 partials ⚠️
src/node.cc	83.33%	2 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #57016      +/-   ##
==========================================
- Coverage   90.38%   90.35%   -0.03%     
==========================================
  Files         628      629       +1     
  Lines      184007   184265     +258     
  Branches    35952    36008      +56     
==========================================
+ Hits       166314   166499     +185     
- Misses      10869    10907      +38     
- Partials     6824     6859      +35     

Files with missing lines	Coverage Δ
lib/internal/options.js	100.00% <100.00%> (ø)
lib/internal/process/pre_execution.js	93.31% <100.00%> (+1.65%)	⬆️
src/node_errors.h	85.00% <ø> (ø)
src/node_options.h	98.32% <ø> (ø)
src/node.cc	73.96% <83.33%> (+0.38%)	⬆️
src/node_options.cc	85.32% <62.50%> (-2.05%)	⬇️
src/node_config_file.cc	63.02% <63.02%> (ø)

... and 21 files with indirect coverage changes

[jasnell]

Historically I've been fairly anti-config file but I actually think I've come around to the idea. I do think, however, that we need to be careful about making things too confusing for users with some options being passed as command line flags, others being passed as environment variables, and now others possibly being passed by config file. Ideally these mechanisms would integrate with each other such that we can add a configuration option once and it just works (much like we can do today with adding a command line flag and indicate whether it can be used in NODE_OPTIONS or not). It would even be ideal for us to either be able to add an option to the JSON schema and automatically have the code generated in the runtime to support the option or add the option programmatically and provide a way for node.js to generate the json schema dynamically from that. Either way, I've come around to supporting moving in this direction (even if while kicking and screaming).

[jasnell]

@lemire ... I'm curious, does simdjson support jsonc? (JSON with comments). Would it be difficult to add support for jsonc if not? If we're going to use json as a configuration format, then supporting comments is ideal. (note that both deno and bun support comments in their respective formats)

[marco-ippolito]

Maybe some builds (without intl or crypto) don't expose some flags 🤔 thats why comparing the json schema with the available flags fails

[richardlau]

Maybe some builds (without intl or crypto) don't expose some flags 🤔 thats why comparing the json schema with the available flags fails

Yes, for example some options depend on OpenSSL being present:

node/src/node_options.cc

Lines 1105 to 1111 in f6ce486

	#if HAVE_OPENSSL
	AddOption("--openssl-config",
	"load OpenSSL configuration from the specified file "
	"(overrides OPENSSL_CONF)",
	&PerProcessOptions::openssl_config,
	kAllowedInEnvvar);
	AddOption("--tls-cipher-list",

[ShogunPanda]

LGTM!

[marco-ippolito]

I had to rebase after #57142 😭

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/65344/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/65347/

[nodejs-github-bot]

Landed in 5ab7c4c...772c609

[bengl]

Would it be possible in a future commit to allow for (but maybe not require) options to be namespaced in a "node_options" property?

e.g.

{
  "node_options": {
    "test-coverage-lines": 80,
    "test-coverage-branches": 60
  }
}

This would allow for re-using existing JSON files currently used for other kinds of config.

[marco-ippolito]

Unless we decide to support other use cases in addition to flags, imho it does not make sense to namespace it.