Upgrade from messagepack 3.2.0 to 7.0.0

[pranavshridhar]

Hey, y'all!

Finally upgraded MessagePack to the latest dependency of 7.0.0. This should solve #317, but it's not quite good to merge yet. I am sure that there are better ways to do this.

Problem: Data/DataConstRef receives the wrong type

The error faced by the user stated in #317 is a runtime type error, and after a lot of digging I know why: MessagePack 4.0.0, for the sake of minimizing message sizes, packs floating point types with no fractional component as the smallest integer type it can (e.g. 2.0 becomes 2, while 2.3 stays 2.3). I have tried like hell to debug MUSCLE3 to stop this (there is a way, but it involves more time than it should take), but without success.

Solution: Upgrade to the most recent version

At least this way, we're not locking ourselves in time with a version that doesn't work, but there are changes to how the build system deals with MessagePack.

MessagePack C/C++ has split

One of the breaking changes in 4.0.0 is the split between the C and the C++ libraries, which are msgpack-c and msgpack-cxx respectively. Thus, the GitHub URL had to be changed to match this.

msgpack-cxx does not use pkgconf

There is no such thing as a .pc file or a .pc.in within the msgpack-cxx library, so there is no way to use pkg-config to get any information about it. Only the C library has it. I was a bit surprised that any compilation was successful under 4.0.0. However, libmsgpack-dev lists a libmsgpack-cxx-dev as a dependency. I have tried to install this to see if that would be usable with pkg-config, but I was unsuccessful, which tells us that the C++ library cannot be used with it at all. The C++ library is also header-only, which might explain the lack of use of the pkgconf template files.

Future detection and installation:

There is a way that is described in the msgpack-cxx README which is described as the canonical cmake way, but I'm guessing that's not something we want to do.
To that end, the changes in the build system locally download and extract msgpack regardless if the current system has it or not. 4.0.0 has that problematic type compression behavior mentioned earlier, and it's the same version that's the latest available on apt, so this is probably for the better.

---

There are a lot of lines here that I commented out rather than deleted out of anticipation that we might need to keep it on hand. However, after all issues have been ironed out in this PR, I want to get rid of these lines before merging.

What needs to be done now:

• The minimum required version for MessagePack HAS to be 7.0.0 in the build system. Anything else will invite the same behavior again.
• Every instance of the string msgack in the build system HAS to be changed to msgpack-cxx, if it isn't already.
• An MPI issue in the test suite similar to the one solved in Improved MPI Compilation behavior on MacOS environments #328 is showing up again. Not to be solved here, but it needs its own fix.
• Suggested by @LourensVeen: We are going to install msgpack, so might as well do away with detection and skip to installing.
• Suggested by @LourensVeen: The documentation should reflect this change as well.
• Suggested by @LourensVeen: (see his comment below) Remove msgpack as a dependency listed in libmuscle.pc and libmuscle_mpi.pc to fix make test_examples.

[LourensVeen]

The number of lines changed in this PR belies the amount of work that went into it. Very nice, and thank you!

[LourensVeen]

Is the CMAKE_POLICY_VERSION_MINIMUM still needed when building 7.0.0.?

[pranavshridhar]

CMake does NOT like when I do that, so my recommendation is to keep it for now.

[LourensVeen]

I've seen this issue of CMake-built packages insisting that everyone else use CMake too if they want to use the library. It's worrying. On the other hand, if this is just about standardising on CMAKE_PREFIX_PATH then it's easy enough to add support to other build systems I suppose. Well, maybe not relevant here if we'll just always build locally.

I've found two more issues while testing this locally:

• The documentation still lists MsgPack 3.2.0 or later as a dependency, that should either have its version updated, or perhaps we should update it to point out that it will always be downloaded automatically?
• make test_examples fails, because MUSCLE3's own pkg-config files still list msgpack as a dependency. See libmuscle/cpp/build/libmuscle/libmuscle.pc and libmuscle_mpi.pc. I think that can be removed if it's header-only.

[pranavshridhar]

I am genuinely surprised that this is passing as many test cases as it is now. I am facing issues with Ubuntu regarding libmuscle's pc files, but the good news is the Mac build works!