External runners for Windows

[jaimergp]

Comes from Quansight/open-gpu-server#31

---

@wolfv and @baszalmstra asked in today's core call about the possibility of using the recently added Cirun + self-hosted GHA runner infra to add beefy Windows runners for the most demanding feedstocks. I think the idea is to sponsor this for PyTorch packages, in principle.

From the top of my head, this is what needs to happen, roughly:

1. Choose a Cirun-compatible cloud provider for the Windows runners.
2. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.
3. Configure the cloud provider accordingly, and register the credentials in the Cirun app for conda-forge.
4. Add support for registration and access control in the admin-requests + .cirun infra.
5. Optional. Figure out the legal bits for liabilities and service abuse (e.g. restrict to approved users or maintainers).

[carterbox]

2. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.

I worked on creating Docker containers a few years back (2022). AFAIR, the biggest two obstacles were that (1) Windows containers were large: around 11GB after you got all the stuff in there and (2) the "nano" Windows base image was 64-bit only and conda still required some 32-bit dependencies (that's probably changed?).

Maybe the Docker part isn't relevant to this issue, but there's some good links in that PR about how to install the MSVC tool chain without a GUI.

[aktech]

Thanks for creating the issue @jaimergp. I have just received an Azure account from @wolfv (thanks for that 🎉 ). I'll soon start the work and update here.

[aktech]

I have created a PR for the same in the access control repo here: conda-forge/.cirun#14

[aktech]

I believe this can be closed as well.

[jaimergp]

I think we should document the main outcomes before we close, namely:

• How external runners have been implemented so far and what the contribution process is
• Examples of this process being used in real world cases

[aktech]

That makes sense, is there a page in docs where this would be suitable? https://conda-forge.org/docs/maintainer/infrastructure/ ?

Or did you mean to just document here in the issue?

[jaimergp]

That makes sense, is there a page in docs where this would be suitable? conda-forge.org/docs/maintainer/infrastructure ?

Yep, under https://conda-forge.org/docs/maintainer/infrastructure/#ci-build-services would be perfect.

[aktech]

Thanks, I'll create a PR.