failing installation of Rust 1.52.1 with GCCcore/10.3.0 on top of EESSI 2023.04 compat layer (due to OpenSSL 3.0.x)

[boegel]

Before we were filtering out OpenSSL/1.1 (cfr. #256), the installation of Rust-1.52.1-GCCcore-10.3.0.eb (a build dependency for Python-3.9.5-GCCcore-10.3.0.eb) was failing with errors like:

/tmp/easybuild/build/Rust/1.52.1/GCCcore-10.3.0/rustc-1.52.1-src/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/liblibssh2_sys-7661e85ab2ef197a.rlib(openssl.o):openssl.c:function _libssh2_pub_priv_keyfilememory: error: undefined reference to 'EVP_PKEY_id'
collect2: error: ld returned 1 exit status

error: aborting due to previous error
error: could not compile `cargo`

It seems like that's caused by some mismatch between the OpenSSL/1.1 dependency, and the OpenSSL 3.0.x in the 2023.04 compat layer - perhaps the headers of OpenSSL 1.1 are picked up, but the actual linking is done against OpenSSL 3.0.x.

When using OpenSSL in filter-deps in the EasyBuild configuration, the error changes to:

The following warnings were emitted during compilation:

warning: build/expando.c:4:24: error: pasting "RUST_VERSION_OPENSSL_" and "(" does not give a valid preprocessing token
warning:     4 | #define VERSION2(n, v) RUST_VERSION_##n##_##v
warning:       |                        ^~~~~~~~~~~~~
warning: build/expando.c:5:23: note: in expansion of macro VERSION2
warning:     5 | #define VERSION(n, v) VERSION2(n, v)
warning:       |                       ^~~~~~~~
warning: build/expando.c:10:1: note: in expansion of macro VERSION
warning:    10 | VERSION(OPENSSL, OPENSSL_VERSION_NUMBER)
warning:       | ^~~~~~~

error: failed to run custom build command for `openssl-sys v0.9.58`

...

  Failed to find OpenSSL development headers.

  You can try fixing this setting the `OPENSSL_DIR` environment variable
  pointing to your OpenSSL installation or installing OpenSSL headers package
  specific to your distribution:

...

  See rust-openssl README for more information:

      https://github.com/sfackler/rust-openssl#linux

This is a good indication that the openssl-sys crate being installed is not compatible yet with OpenSSL 3.0.x.

This is confirmed by https://github.com/sfackler/rust-openssl/blob/master/openssl-sys/CHANGELOG.md#v0964---2021-06-18, which shows that support for OpenSSL 3.0.x was only added in openssl-sys v0.9.64.

[boegel]

Two possible ways out here are:

1. Trying to patch Rust 1.52.1 to fix the build on top of OpenSSL 3.0.x . This may be as simple as bumping the version of the openssl-crate to a more recent version in the Cargo.lock in the Rust 1.52.1 sources, but it's probably way more painful than that. At the very least, it seems that openssl-src will need to be bumped too, and perhaps also openssl-probe.
2. Do an in-place update of Rust to a newer version, as was done by NESSI using the already available Rust-1.60.0-GCCcore-10.3.0.eb, cfr. Nessi.no 2023.04 replace Rust/1.52.1 by Rust/1.60.0 NorESSI/software-layer#106

The latter is OK in this case, since Rust is typically only a build dependency, and significantly less effort than 1 (and NESSI has demonstrated it works).

[boegel]

The same problem will arise in the 2021b generation of easyconfigs, where Rust/1.54.0-GCCcore-11.2.0 is used as a build dependency for Python/3.9.6-GCCcore-11.2.0.

https://github.com/rust-lang/rust/blob/1.54.0/Cargo.lock#L2439-L2440 shows that the version of the openssl-sys crate used there (v0.9.61) is not recent enough to be compatible with OpenSSL 3.0.x.

The existing Rust-1.56.0-GCCcore-11.2.0.eb easyconfig could also be an escape hatch here, since that does include a sufficiently recent version of openssl-sys, see https://github.com/rust-lang/rust/blob/1.56.0/Cargo.lock#L2383-L2384 .

From the 2022a generation onwards, we should be fine, since Rust 1.60.0 or newer is used there, and Rust-1.60.0-GCCcore-10.3.0.eb shows that's sufficiently recent.

For older generations (2020b, 2020a) this problem won't present itself, since Rust is not a build dependency yet for Python there because the cryptography extension didn't require Rust yet (cfr. pyca/cryptography#5771 🍿 ).

[boegel]

Hooks implemented in boegel@6d35be1 fixes this by bumping Rust version, PR coming soon (in company of an easystack file for 2023.04).

[boegel]

Problem fixed in new EESSI pilot version 2023.06 by sticking to OpenSSL 1.1.1 for now in compat layer, cfr. EESSI/compatibility-layer#188