Fix data loader build on Windows, re-enable pybind job #13564
Conversation
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13564
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
GregoryComer
changed the title
GregoryComer
requested review from
larryliu0820 and
kirklandsign
as code owners
|
Note that CI failures are pre-existing. |
tools/cmake/preset/pybind.cmake
Outdated
| ) | ||
| endif() | ||
|
|
||
| # These XNNPACK options don't currently build on Windows with Clang. |
There was a problem hiding this comment.
Where do these magic incantations come from
There was a problem hiding this comment.
These are 4 optional XNNPACK features (exposed by XNNPACK's CMakeLists) that clang-cl has issues with due to the /vlen compiler argument.
These disable a small subset of the available kernels, though they are relatively niche and should have minimal perf impact. It should be fixed or gated upstream in XNNPACK itself. I intend to file an issue there and maybe just fix it myself. They have some gating logic already (https://github.com/google/XNNPACK/blob/923253023555f75c38d96511aa7fa59b9ef9c25c/CMakeLists.txt#L176)
I could also move these into the XNNPACK backend CMakeLists, rather than the presets, as they are functionally an internal detail, and it would also ensure that they are always disabled on Windows.
There was a problem hiding this comment.
Thinking through it more, I think I will move these into the XNNPACK backend CMakeLists, rather than the preset.
extension/data_loader/CMakeLists.txt
Outdated
| @@ -24,6 +24,11 @@ if(NOT ET_HAVE_SYS_MMAN_H AND NOT WIN32) | |||
| "extension/data_loader/mmap_data_loader.cpp" | |||
| ) | |||
| endif() | |||
| if(WIN32) | |||
There was a problem hiding this comment.
Why is this necessary? I thought scott already selectively included this source file in build whatever.bzl
There was a problem hiding this comment.
Apparently not. I might be thinking of selectively removing the header from the public include if not on windows or something
There was a problem hiding this comment.
Yeah, I chatted with Scott briefly on this. His recommendation was to manage it locally. The main build variables list has to be mechanically parsable from both CMake and Buck, so it can't include more complex conditional logic.
|
I'm not sure, why ghstack merged this when re-ordering PRs. I'll re-open as a new PR. |
…t build CI job (#13702) **Note: This is a cherry-pick of #13564 and #13259. I inadvertently broke the ghstack when attempting to re-order the stack, leading to the old PR being merged by ghstack into the ghstack-created branch (not main).** Add mman_windows.cpp to the CMake sources. Fix a few build-blocking issues with XNNPACK on Windows. Enable XNNPACK on the Windows preset. This also gives CI coverage. I've disabled 4 microkernel families that don't build currently. These likely need to be gated or fixed upstream, but I'm disabling them locally for now to unblock usage. They are relatively niche and should have little to no perf impact on the average user. With this change, the pybind preset works and we can re-enable it in CI.
Add mman_windows.cpp to the CMake sources. With this change, the pybind preset works and we can re-enable it in CI.