Building IMAS Core UDA backend fails on Windows

[Louwrensth]

The ITER community would very much like to build a Python wheel of IMAS-Core with the UDA backend enabled. This currently works on Linux, but fails on Windows.

To reproduce

A) I've been building a github actions workflow that uses cibuildwheel (some good ideas borrowed from UDA's GHA workflow). Here is a link to a recent job: https://github.com/Louwrensth/IMAS-Core/actions/runs/17311074087/job/49145201122
But this is a bit of pain to read, here is a link to the logs.

B) You can also run it on a Windows machine yourself too:

# (make sure to install cibuildwheel package in your python env)
$ python -m cibuildwheel --output-dir wheelhouse --config-file pyproject.toml 2>&1 |tee cibw-log.log

I attach the log of this command:
cibw-log.log

e.g.

$ tail cibw-log.log 
uda_serialisation.lib(capnp_serialisation.obj) : error LNK2019: unresolved external symbol "void __cdecl capnp::writePackedMessage(class kj::BufferedOutputStream &,class kj::ArrayPtr<class kj::ArrayPtr<class capnp::word const > const >)" (?writePackedMessage@capnp@@YAXAEAVBufferedOutputStream@kj@@V?$ArrayPtr@$$CBV?$ArrayPtr@$$CBVword@capnp@@@kj@@@3@@Z) referenced in function uda_capnp_serialise [C:\Users\User\AppData\Local\Temp\tmpihsx7noe\build\al.vcxproj]
C:\Users\User\AppData\Local\Temp\tmpihsx7noe\build\Release\al.dll : fatal error LNK1120: 31 unresolved externals [C:\Users\User\AppData\Local\Temp\tmpihsx7noe\build\al.vcxproj]

*** CMake build failed

ERROR Backend subprocess exited when trying to invoke build_wheel

                                                                     ✕ 106.96s
cibuildwheel: error: Command ['C:\\Users\\User\\AppData\\Local\\Temp\\cibw-run-sza432az\\cp311-win_amd64\\build\\venv\\Scripts\\python.EXE', '-m', 'build', 'C:\\Users\\User\\GIT\\ITER\\iterorg\\imas-core', '--wheel', '--outdir=C:\\Users\\User\\AppData\\Local\\Temp\\cibw-run-sza432az\\cp311-win_amd64\\built_wheel'] failed with code 1.

I suspect something is off in the way I am setting it all up with vcpkg and cibuildwheels so that the linking goes awry after compiling succeeded? or is it the UDA backend code?

Please let me know what you need to look in to this.

[stephen-dixon]

I don't have access to either the iter IMAS-core repo or your fork to look closer or reproduce. Can you do the basic buld step on windows (with vcpkg and MSVC) outside of cibuildwheel? Something like the choice of vcpkg triplet or the version it uses and compatibility of that with your project/build options could maybe cause the linker issues but I'm afraid I'm not an expert.

I'm not super familiar with the uda backend to imas anymore but Jonathan will be back next week and may be able to say more about whether there could be any issues coming from there.

[jholloc]

The error output suggests it is missing an object from capnp. I don't know if the IMAS build uses the UDA pkg-config file, or if we are missing capnp as dependency in that file.

You could try adding -lcapnp when you link against the UDA library?

[Louwrensth]

Thanks for your replies.
I've added @stephen-dixon and @jholloc to access my fork.

Indeed some things are wrong with the Windows/vcpkg build of UDA and/or the UDA Backend in IMAS-Core, something isn't linking right.
I want to highlight this patch to the uda-cpp.pc file I made to make it compile on windows (MSVC compiler),

    find ./ -name "uda-cpp.pc" | xargs sed -i -e "s| capnp||g;s| -std=c++17| /std:c++20|"

I don't remember why I removed Requires: capnp from the .pc file before (I think because capnp.pc it isn't installed via the vcpkg installation of capnp), so I'm testing again for confirmation...

[Louwrensth]

I don't remember why I removed Requires: capnp from the .pc file before (I think because capnp.pc it isn't installed via the vcpkg installation of capnp), so I'm testing again for confirmation...

Indeed:

  -- Checking for module 'uda-cpp'
  --   Package 'capnp', required by 'uda-cpp', not found
  CMake Error at C:/Program Files/CMake/share/cmake-3.31/Modules/FindPkgConfig.cmake:645 (message):
    The following required packages were not found:
  -- Configuring incomplete, errors occurred!
  
     - uda-cpp

[stephen-dixon]

    find ./ -name "uda-cpp.pc" | xargs sed -i -e "s| capnp||g;s| -std=c++17| /std:c++20|"

Apologies for the delayed response. Can I just clarify the current status and the individual issues?

• Your PR in imas-core has now been merged with the patch above to the uda package-config file and your MSVC uda/imas build now works on windows for creating python wheels

The issues are:

• uda package config files are not currently portable to windows (particularly hardcoded -std=c++17)
• Requires: capnp in pkgconfig breaks things if you have a capnp install that doesn't provide its own pkgconfig files
• built uda client libraries not available for linux (gcc?) or windows (msvc?)

Questions:

• how have you got past the capnp linking requirement? is it just added manually in cmake or through another library linking in cmake?
• would an uda cmake module exported from the uda build be better to work with than pkgconfig files for you, if available?
• would binary distributions be better still, or would you still prefer to have control over platorm/compiler versions and use pkgconfig/cmake to compile uda yourself? do you know which platforms would need to be supported?

Exporting cmake targets is planned for v3.0 (a little way off) and distributing packages (rpm/deb) on release is planned within the next few months. So possibly just looking at windows-compatible pkg-config files will mean there are some shorter and longer term options for supporting this use case better?