Platform specific constants accessed in enum_string_N.go causes compilation errors

[johanhenriksson]

Hey!

First of all, exciting to see a new VK wrapper for Go! Im interested in evaluating it for my https://github.com/johanhenriksson/goworld project, and willing to contribute if it works out well :) I'd like to depend on a library that I could keep up to date myself if required.

I noticed there's been little testing on OSX, and indeed it does not seem to work at the moment. Tried to run one of the samples quickly, and im running in to these compilation errrors (excluded a bunch more similar ones):

# github.com/bbredesen/go-vk
../../../go/pkg/mod/github.com/bbredesen/go-vk@v0.0.0-20230304041510-33ed978b8bfd/enum_win32_string_0.go:21:9: undefined: FullScreenExclusiveEXT
../../../go/pkg/mod/github.com/bbredesen/go-vk@v0.0.0-20230304041510-33ed978b8bfd/enum_string_5.go:496:8: undefined: ERROR_FULL_SCREEN_EXCLUSIVE_MODE_LOST_EXT
../../../go/pkg/mod/github.com/bbredesen/go-vk@v0.0.0-20230304041510-33ed978b8bfd/enum_string_6.go:975:8: undefined: STRUCTURE_TYPE_EXPORT_FENCE_WIN32_HANDLE_INFO_KHR
../../../go/pkg/mod/github.com/bbredesen/go-vk@v0.0.0-20230304041510-33ed978b8bfd/enum_string_6.go:976:8: undefined: STRUCTURE_TYPE_EXPORT_SEMAPHORE_WIN32_HANDLE_INFO_KHR

Most of them have _WIN32 or _NV in their names, so I assume they are just platform specific constants not available on my platform. Perhaps they need to be moved into conditionally compiled files?

I haven't had a chance to look at the code generator yet, is there any support for keeping certain defines/code in platform specific go files?

[bbredesen]

Thanks for testing and reporting this.

There is platform-specific handling in the generator, and it is worth noting that this repository currently only has the win32 platform-specific extensions built. You could run vk-gen against darwin, but it wouldn't fix the compile errors you're reporting above.

There are a couple of related issues here, caused by how stringer works:

1. enum_win32_string_0.go should be platform guarded, just like enum_win32.go.

That file is referencing constants that the compiler is ignoring on your platform. I don't think there is a way to have stringer recognize the go:build annotation in the source, or to force it to add an annotation to its output.

One fix is to have vk-gen add a go:build line after the stringer file is generated. An alternative is to just move "_win32" to the end of the generated filename, which the compiler will also recognize and ignore on non-windows builds. Fix decision TBD on this.

2. The error code and structure types on the next three lines are win32 specific consts, and guarded by go:build as above, but stringer is picking them up along with the cross platform values of Result and StructureType. (i.e. stringer saw the values but your compiler doesn't.) I don't think it would hurt anything to define those constants on all platforms, they would just never get used. I'm not sure if that's the right approach though...I need to think a little more about this.

I have no idea if stringer recognizes go:build and would ignore those values if you ran it on your platform...I'd be surprised if that was the case, but it's worth trying. Install stringer if you don't already have it (go install golang.org/x/tools/cmd/stringer), delete all of the _string files in your local copy of the library, and then "go generate"...I'm planning to start testing against mac next week, and I'll take a look at this too.

In the short term, assuming that doesn't work, I think you will be able to compile against a local copy of the library if you do the following:

• delete all of the string.go files
• comment out the function definition for Result.Error() in static_common.go (lines 38-40). That is the only place that String() is used in the library, and that Error function is not called anywhere else (it is an artifact left over from early development, playing with Result as an error).
• rename enum_win32.go to something that doesn't end in "win32" and remove the go:build annotation in that file

[bbredesen]

Existing feature request that would partially address this in Go x/tools: golang/go#47179

[johanhenriksson]

I did manage compile by generating new code on darwin and simply not running the stringer thingy :) Had to fix a couple of errors but most of them had issues already. Nice work! Got the minimal sample to run!

I might try to convert a sample to GLFW or SDL tomorrow, so that we can test cross platform

[hherman1]

@johanhenriksson would love to reuse your work getting it compiling on darwin if you're keen to share

[johanhenriksson]

@hherman1 Sure, I published a draft PR here bbredesen/go-vk#8

[johanhenriksson]

To get the sample to compile I simply replaced go-vk with my own version in the sample project's go.mod

[bbredesen]

@hherman1 @johanhenriksson I'm planning to dive in on the code generation for Mac next week. I suspect the short-term fix will be to have all constants values unguarded, and only protect the platform-specific commands with go:build tags.

In the meantime, you may want to re-fetch vk-gen, I've made several bugfixes over the last few days where go-vk had compile errors out of the box.

*Edit - Fixes addressed general compile errors out of the box, have not looked at anything platform-specific yet. I did seevk-gen/#20 that @johanhenriksson submitted.

[johanhenriksson]

@bbredesen Awesome, looking forward to any improvements. I'd like to help but it feels daunting to dig in to the actual code generation without guidance.

Was going to give sample 01 a shot, but I noticed that my regenerated bindings seem to miss a bunch of swapchain functions and constants, presumably another macos issue. Would you like me to keep creating issues as I encounter them? Or wait until you had a chance to try it on mac?

[bbredesen]

Yes, please create issues as you find them, probably over on vk-gen if something is missing from the binding. Issues created here will eventually get moved over once I start working on them (and assuming they are related to code gen).

The generator feels complex because it covers a lot of edge cases from vk.xml and just grew organically as I ran into those during development. I would still encourage you to pull down a copy and trace generation of e.g. a single struct type with a debugger to see what is happening.

If you want to dig into the generator, there are four broad phases that it goes through:

1. Reading the xml and exceptions files and collecting the "raw" information on each type and value. These all get inserted into global maps (TypeRegistry and ValueRegistry are the map type definitions).
2. Reading the exceptions and platform definitions, essentially determining what types, commands and values are required for a given extension or Vulkan version.
3. "Resolving" everything by walking down the tree of required types for a platform. When resolving a struct, for example, the tool checks for the types of each member, recursively resolves those types, and inserts them into the set of features.
4. Printing everything out. The tool separates the feature set by type category (handles, structs, commands, etc.) and platform, then asks each Type/ValueDefiner to print its Go type definition, type conversion functions where needed (Vulkanize/Goify), and constant values.

[johanhenriksson]

Thanks for the pointers. I found the design notes in the vk-gen repo, which also look very helpful. I might try to write some tests or something to get into it. Im thinking it would be nice to have some simple tests that verify a certain piece of XML generates the expected code.

[bbredesen]

The plan on the base issue here is is to remove go:build tags from the enum type files, but to leave them in place around commands, structs, extensions, etc.

The enum files will still have their Vulkan platform names, and the platform-specific types will be generated to their own platform_string.go files, just without the build guards.

I'll also modify the default flags to generate against win32, macos and metal.