dist: make installation asynchronous

[djc]

cc @FranciscoTGouveia

async-install.mov

[rami3l]

These changes look pretty reasonable and definitely an improvement over #4567’s attempt of spilling Arc everywhere.

I imagine that if correctly implemented, it would be equivalent to #4471 in terms of performance because as long as we are spawning the installation task, the extra overhead introduced by #4567 should be minimal.

It’s worth another bench nonetheless though.

PS: Would you mind investigating the CI failures? This implementation doesn’t seem to have covered all the edge cases.

[djc]

Running time rustup-init toolchain install beta (compiled with --release) on #4471:

4.78s user 14.11s system 101% cpu 18.604 total

On this PR:

5.12s user 14.54s system 138% cpu 14.214 total

So it looks like this PR is a bit faster, by making better use of parallelism.

On main:

5.21s user 14.69s system 103% cpu 19.154 total

[djc]

PS: Would you mind investigating the CI failures? This implementation doesn’t seem to have covered all the edge cases.

Oops, solved those.

[rami3l]

Thanks! I really want to test this with my own machine but I don't have my fiber installed until tomorrow... I hope you won't mind my delaying merging this for a bit just to be sure 🙏

[djc]

No problem! Hope the fiber install will work out better for you tomorrow!