[WinRT] Start exposing Windows Runtime headers to Swift

[egorzhdan]

This adds an initial Clang modulemap for the WinRT module. The modulemap is injected into the winrt/ include search path using VFS.

For now, the modulemap only exposes a small number of utility headers from WinRT. It will be extended in the future to cover more of the WinRT API surface.

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please smoke test

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please build toolchain Windows

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please smoke test

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please build toolchain Windows

[compnerd]

Please do a smoke test on the windows toolchain before merging

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please build toolchain Windows ARM64

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please smoke test Linux

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please smoke test macOS

[egorzhdan]

swiftlang/swift-installer-scripts#465

@swift-ci please build toolchain Windows ARM64

[compnerd]

swiftlang/swift-installer-scripts#465

@swift-ci please build toolchain Windows

[stevenbrix]

It will be extended in the future to cover more of the WinRT API surface.

how far does this plan on going? will it eventually include all Windows.* headers in the WinRT folder?

[stevenbrix]

winsdk_winrt seems like a more consistent name? it would be ideal to import WinSDK.WinRT as opposed to import WinRT. the latter can cause lots of collisions with other WINMD based projection tools which will generate code in the WinRT module

[compnerd]

Is WinRT considered part of the Win32 SDK? I thought it was a separate parallel piece. The submodule support isn't sufficient to support that in Swift I think? So this will need to be a top-level module.

[stevenbrix]

there is just the "Windows SDK" . the distinction between Win32 and WinRT are the APIs used. but since the module is called WinSDK it seems appropriate to keep them all together? if WinSDK was named Win32 i would feel differently

[stevenbrix]

there are other submodules in WinSDK, so that should be fine?

[egorzhdan]

WinRT can't be a submodule of WinSDK mostly for build performance reasons. WinRT headers are huge, for instance for WinUI headers alone Clang produces a .pcm that is 300MB+ and takes over 10 minutes to precompile on my machine (with weak specs). Clang builds .pcms per top-level module, so all submodules are effectively merged into one. Similarly, Swift merges all submodules into a single module when importing them. That means that for a project that only uses Win32, clang would spend significant time prebuilding WinRT headers, and then Swift would spend time importing WinRT decls into Swift.

For this reason, I'm going to propose splitting WinRT headers into many top-level modules, e.g. having a top-level module WindowsXyz per windows.xyz.*.h. But that is out of scope of this PR ;)

the latter can cause lots of collisions with other WINMD based projection tools which will generate code in the WinRT module

Is WinRT a conventional name for some of the modules that are generated by other tools?

[stevenbrix]

For this reason, I'm going to propose splitting WinRT headers into many top-level modules, e.g. having a top-level module WindowsXyz per windows.xyz.*.h. But that is out of scope of this PR ;)

Does the fact that namespaces in WinRT can have cyclical dependencies affect this at all?

[egorzhdan]

Are you asking about namespaces that are redeclared across several different modules? Those aren't a problem, we run into the same situation with module std / module std_code / module std_config all redeclaring namespace std in libc++.

[stevenbrix]

No, i mean module Windows.xyz has a dependency on Windows.abc, but Windows.abc also has a dependency on Windows.xyz

namespace Windows.xyz {

   class XyzExample {
     void TakeAbc(Windows.abc abcClass);
   }
}

namespace Windows.abc {

  class AbcExample: XyzExample {
 
  }
}

[egorzhdan]

Oh, do you mean that they could have circular #includes? Then we'd have to make such headers a single module, yeah. Anyway, that would be an interesting question for future changes to the modulemap.

[egorzhdan]

how far does this plan on going? will it eventually include all Windows.* headers in the WinRT folder?

I think that would be the ideal outcome, yes! However I don't yet have a concrete plan to list every single WinRT header in the modulemap, and I'm not sure if Swift will actually be able to correctly digest all of them. For now I'm mostly interested in WinUI and its dependencies.

[stevenbrix]

For now I'm mostly interested in WinUI and its dependencies.

The most interesting WinRT development is done in WindowsAppSDK (with UWP you're limited to the sandbox, which may be ok, but for most it isn't) and those don't ship in the Windows SDK.

[stevenbrix]

I'm curious, why does swift want to go down this route of including windows.* headers from the SDK, as opposed to using a winmd based code generator tool? those headers in the windows SDK aren't really meant for consumption.

unless you have plans for neat tricks, writing Swift from those headers would feel very much like writing WRL in C++

[egorzhdan]

I'm curious, why does swift want to go down this route of including windows.* headers from the SDK, as opposed to using a winmd based code generator tool?

One of the main goals that we have is incremental adoption. We don't expect large C/C++ codebases that are starting to adopt Swift to rewrite all of their code in Swift overnight. To achieve that, we try to make Swift interoperate better with existing C/C++ code. Users will need to pass WinRT objects between C/C++ and Swift, so reusing the same type definitions that are already used in C/C++ seems like a natural solution. Swift is already really good at interoperating with C-family languages, so we might as well build upon that.

those headers in the windows SDK aren't really meant for consumption.

The winrt/ directory is one of the default include search paths, meaning that a sample C++ project will have those headers available for inclusion. That means that existing C/C++ projects that #include <windows.xyz.h> can already expose those types to Swift, they just wouldn't always be exposed correctly because of incorrect (missing) modularization. So we're not using any hidden or private API here.

unless you have plans for neat tricks

We're thinking about importing COM types into Swift as Swift classes. There are multiple different ways to go about this. We could use the existing foreign reference type support for this, or we could come up with something bespoke for COM. But this would be a large feature that isn't going to be part of this PR.

writing Swift from those headers would feel very much like writing WRL in C++

Can you give an example of what would be an issue?

[stevenbrix]

I'm gonna ramble here a bit, so I apologize for the lengthy post. I have lots of thoughts/ideas/opinions and experience in this space :)

One of the main goals that we have is incremental adoption. We don't expect large C/C++ codebases that are starting to adopt Swift to rewrite all of their code in Swift overnight. To achieve that, we try to make Swift interoperate better with existing C/C++ code.

That makes a lot of sense! I should've also clarified I don't think (or want) Swift to be using a WinMD based generator tool, rather that it should lay the groundwork for one to exist on top of whatever Swift has support for.

I know most of Apple's Windows apps are using C++/WinRT, but I'm sure there is a healthy mix of non C++/WinRT as well. They are all using WindowsAppSDK, so I'm not sure the types in Windows.UI.Xaml.* namespace are that interesting. Using that as the prime example feels strange to me as that is more or less a dead platform (there are tons of other Windows.* APIs which are very relevant though).

FWIW we have a large C++ code base which we interop with Swift through WinRT, much in the same way we use ObjCpp for Mac. We've wanted to remove these middle layers and rely on C++ interop for this, but it hasn't yet met our goals. I'd love for it to get there though!

Users will need to pass WinRT objects between C/C++ and Swift, so reusing the same type definitions that are already used in C/C++ seems like a natural solution.

What does this look like in practice? It would be awesome to see some examples.

We're thinking about importing COM types into Swift as Swift classes. There are multiple different ways to go about this. We could use the existing foreign reference type support for this, or we could come up with something bespoke for COM. But this would be a large feature that isn't going to be part of this PR.

If they are imported as Swift classes, how would you derive from multiple COM interfaces? Or is this something where you think a WinMD based tool would provide a better experience on top of?

writing Swift from those headers would feel very much like writing WRL in C++

Can you give an example of what would be an issue?

The headers don't define the class structures that WinMD does, so if you're going based off headers, you're writing code like this (assuming all of the nice conversions of out params):

let button = try! RoActivateInstance(RuntimeClass_Windows_UI_XAML_Controls_Button)
let buttonAsFE = button as IFrameworkElement
try! buttonAsFE.put_Width(10)

as opposed to:

let button = Button()
button.width = 10

But just to be clear, I don't think the Swift toolchain should be using a winmd based generator tool, but that should be the desired outcome (at least for consuming WinRT from Swift). I just want to make sure that the work done here is something we can build winmd generator tools off of

Final thoughts

If we take the above examples, let's say there is some C++ code base which takes a Button. In the first example, you could simply do this:

cppTakeButton(button)

which is cool, but I think the verbosity of writing Swift code like that doesn't outweigh that maybe the way to pass the native Button to C++ would be something like this;

cppTakeButton(button.get())

Or even better, if Swift defined some protocol for bridging the WinMD based type to the WinRT type, then you could just do cppTakeButton(button), and the projection could do the right thing

[al45tair]

If they are imported as Swift classes, how would you derive from multiple COM interfaces?

IMO they should be imported as protocols, not classes, for that (and other) reasons. I appreciate that that may not be what the C++ importer does now.

The goal for Swift should be to provide a natural way to use COM objects from Swift, without any weird rough edges.

[stevenbrix]

The goal for Swift should be to provide a natural way to use COM objects from Swift, without any weird rough edges.

WinRT expands on COM, so is the basic support limited to COM or you also hoping for natural WinRT?

[egorzhdan]

@stevenbrix

I should've also clarified I don't think (or want) Swift to be using a WinMD based generator tool, rather that it should lay the groundwork for one to exist on top of whatever Swift has support for.

Ah I see, That sounds entirely reasonable to me!

What does this look like in practice? It would be awesome to see some examples.

Here's one rough example of using WinUI from Swift with C++ interop, with a number of compiler changes. Some of those we might not actually want in production, also a lot of the code there would belong in the overlays or should be compiler-generated, such as the IID fields:
https://github.com/egorzhdan/swift/blob/egorzhdan/frt-experiments/test/Interop/Cxx/foreign-reference/com/main.swift

This example also shows how we could solve many of the ergonomics issues you've mentioned by using Swift protocols with some automated conformances.

If they are imported as Swift classes, how would you derive from multiple COM interfaces? Or is this something where you think a WinMD based tool would provide a better experience on top of?

These are great questions, I don't have a concrete answer here yet!

[egorzhdan]

@al45tair

IMO they should be imported as protocols, not classes, for that (and other) reasons

Yeap, that's also a very reasonable approach, I should've mentioned it explicitly in my comment above. Either way, we're going to need ClangImporter to see the type definitions to bridge them from C/C++ correctly, so I think this change will be useful.

[stevenbrix]

Agreed, protocols makes the most sense!

This example also shows how we could solve many of the ergonomics issues you've mentioned by using Swift protocols with some automated conformances.

I'm not sure i can really see how swift can solve all the ergonomic issues i laid out. How would the following work?

1. Classes
2. Custom constructors
3. Properties
4. Events
5. Async methods
6. Static methods
7. Delegates

in theory, 3-5 could be done based on naming conventions:

1. put_Xyz/get_Xyz for properties
2. add_Xyz / remove_Xyz for events
3. IAsyncAction, IAsyncOperation<Sender, Args>, IAsyncOperationWithProgress for async methods

But I struggle to see how you could do the others without parsing the WinMD. These relationships aren't defined in the header files.

[egorzhdan]

I think those are very good questions, although mostly outside of scope of this PR.

1. Classes
2. Custom constructors

Could you elaborate on what you mean here?

3. Properties

C++ interop can already transform a pair of getter/setter methods into a Swift property via the SWIFT_COMPUTED_PROPERTY attribute. We might need to tweak the logic a bit to support the specific conventions of WinRT, but that would be a small change to the compiler.

6. Static methods

These pretty much work in my rough example, although they aren't bridged to Swift as static methods, they become instance methods on a distinct XyzStatics class instead. This can definitely be improved.

I think a lot of these topics would warrant a more visible discussion on the Swift Forums – there might be great ideas coming from the community on this, and I don't think many people would see the discussion on this PR.

To keep this thread focused, let's consider this question: do we agree that whatever solution we come up with here, we would need ClangImporter to see the definitions of the WinRT types? I think the answer is yes, and if that is the case, we should merge this PR in some form, but I'm happy to hear other thoughts!

[stevenbrix]

Classes
Custom constructors

Could you elaborate on what you mean here?

There's no concept of a class in WinRT headers (aside from the class name). You can't tell what interfaces the class implements to know which APIs to provide. You can't even tell which interfaces are related (i.e. IVector derives from IIterable). However, WinRT is designed to model modern programming languages where you interact with classes which have properties, methods, events, custom constructors (i.e. constructor with parameters), and static methods on those classes.

I think a lot of these topics would warrant a more visible discussion on the Swift Forums – there might be great ideas coming from the community on this, and I don't think many people would see the discussion on this PR.

Yes, i definitely think there needs to be a formal pitch/design for WinRT support!

To keep this thread focused, let's consider this question: do we agree that whatever solution we come up with here, we would need ClangImporter to see the definitions of the WinRT types? I think the answer is yes, and if that is the case, we should merge this PR in some form, but I'm happy to hear other thoughts!

I don't personally agree that this is needed for WinRT support. You're welcome to check this in as-is, but I think we're putting the cart before the horse w/o having a formal pitch/design in mind of how this would work end-to-end. I can definitely see a world where there is no need for the swift toolchain to have special knowledge about anything WinRT related.

I'm not trying to say I have all the answers, and you guys ultimately get to make the call. I have lots of experience working on the Windows developer platform, where i worked very closely with the C++/WinRT and C#/WinRT teams and being the main contributor to https://github.com/thebrowsercompany/swift-winrt.

My only goal is for Swift to be the best place to do Windows development