Semantics of `Process.exec` on Windows

[HertzDevil]

The current documentation for Process.exec is:

Replaces the current process with a new one. This function never returns.

Raises IO::Error if executing the command fails (for example if the executable doesn't exist).

But on Windows it doesn't preserve the process ID like on other systems, so calling it "replaces" is probably not accurate. The Windows documentation for the _exec family doesn't contain any references about "replacing the current process" either. This is a fundamental limitation of the Win32 API, and it is unlikely we could match the POSIX semantics here. For reference, Wine simply calls CreateProcessW followed by _exit(0).

So I think we should change the documentation to make it more platform-agnostic, as well as elaborate on whether opened files (Win32 handles? C file descriptors?) and the environment are inherited.

[straight-shoota]

Besides not actually replacing the parent process, Process.exec on Windows has another significant limitation:
It does not propagate the exit code of the child process. This seems even more critical in practice.
The reason for this seems to be that _wexecvp exits directly after spawning the child process. So it cannot know its exit code.

$ cat exec.cr
Process.exec(ARGV[0], ARGV[1...])
$ crystal build exec.cr
$ ./exec powershell -Command exit 123
$ echo $LASTEXITCODE
0
$ powershell -Command exit 123
$ echo $LASTEXITCODE
123

We can see the effect in #15209 and crystal-lang/shards#686 specs that test the exit code of an exec process fail.

Stdlib process_spec.cr is seriously undertesting Process.exec and thus this issue hasn't surfaced there yet.

[straight-shoota]

I believe the most common use case for Process.exec is to switch control to a different process, which then determines the exit status.
Returning early and swallowing the exit status should rarely be an accepted behaviour.

So I suppose the behaviour of Process.exec(...) should be more similar to exit Process.run(...).exit_code.

But in order to work as expected, we must also make sure to shut down everything in the parent process, i.e. all threads must stop. No other fiber can execute after spawning the sub process and while waiting for its completion. Just as it would be if the parent process would exit directly.
And we should try to release as many resources (memory, handles, threads) as possible to lower the resource usage of the parent process while it's just waiting for the child to exit.

A related topic is cleaning up resources nicely before exec. This affects both Unix and Windows systems, though.

[straight-shoota]

It might actually be best to drop Process.exec from stdlib. It's a POSIX-specific concept. There is no equivalent in Windows because process semantics are very different. All we can do is a completely inadequate emulation of some of its aspects.

Instead we could consider introducing portable APIs dedicated to some of the use cases for exec, but which work cross-platform. exec might serve as implementation on Unix systems, but that would be an implementation detail. We've done similar things before, for example signal handlers (#13983) and exit status (#12284).

The main use case in the issues I mentioned above was the goal of running a different program and returning its exit status after completion.
This currently doesn't work correctly when the child process exits with a signal (crystal-lang/shards#685).

If we had a convenient solution for this problem, many use cases for exec could probably just use Process.run + exit.

This alone is maybe not as efficient as exec because the current process keeps hogging some resources. But in many cases that should not be much of a problem.
And we can certainly improve on top of that, for example by shutting down most of the runtime functionality in the parent process, as mentioned in the previous comment.

[straight-shoota]

A potential implementation for a portable exit based on a Process::Status

if exit_code = status.exit_code?
  exit exit_code
elsif signal = status.exit_signal
  Process.signal(signal, Process.pid)

  # The same signal that killed the child process might not kill the parent.
  # We have to exit explicitly. The exist status adding 128 is a convention.
  exit 128 + signal.value
else
  # Abnormal exit without a signal (can happen on Windows)
  exit status.system_exit_status
end

UPDATE: Incorporated feedback from crystal-lang/shards#686 (comment), that Windows has abnormal error statuses which are neither signal nor code.

[ysbaddaden]

We could just leave Process.exec unimplemented on Windows.

[straight-shoota]

I think it should be treated similarly to fork (see #6421).

Such platform-specific features could be provided by a platform-specific shard, not as part of stdlib.
For comparison, few programming languages provide an exec feature in their standard library (even though there are other, more general process management APIs): Rust, Go, Java are all designed with cross-platform intentions and thus, there is no exec at all.

The reason Process.exec exists in Crystal is probably due to the Ruby heritage. But Ruby's system API is very Unix-oriented and known for its bad Windows support. It's not a good role model.

We can provide some portable abstractions to help implement some common use cases for Process.exec in a way that works on all platforms.