PDL::GIS::Proj can't load, Makefile has strange LD_RUN_PATH #391
Comments
|
As a workaround, would it be workable to try instead with the Homebrew PROJ? |
|
Also this might be related: Perl-Toolchain-Gang/ExtUtils-MakeMaker#402 |
|
LD_RUN_PATH should be a colon separated list of paths, should it not? If so then that line is correct and something else is treating it as a single string. Is it being quoted somewhere along the line? |
|
That was my understanding too. @d-lamb How sure are you that the |
|
I don't currently have the tuits to look into this, but I mentioned in IRC that I have some notes on linking/rpath here and @mohawk2 suggested I post them: https://orbital-transfer.github.io/doc/development/os/macos/. |
This part of the error looks like it is using it as a single path. (Broken into several lines for clarity) However, there are two lines for the first so maybe it is just not breaking up the array when it gives feedback. If so then it could be a more generic issue with linking, which comes back to shawnlaffan/perl-alien-proj#7 . @d-lamb - What do you see when you run otool on the file? otool -L /Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib |
That was my thinking, too. I'm not sure that colon-separated LD_RUN_PATH is wrong (it's just different than anything else I see in PDL--but so too is the proj installation!). But looking for a file with a colon in the name seemed wrong, and LD_RUN_PATH is to where I (accurately or not??) traced it back.
I see this: $ otool -L /Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib
/Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib:
@rpath/libproj.25.dylib (compatibility version 25.0.0, current version 25.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
/usr/lib/libsqlite3.dylib (compatibility version 9.0.0, current version 331.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0) |
|
That all looks OK to my untrained eye. I suspect @mohawk2 or @zmughal can shed more light. The other thing to check is which version of Alien::proj you are using. The build system for shared builds was changed to CMake in 1.17 and this might have knock-on effects. There have also been changes in proj related to rpaths in version 9.0. What do you get for |
|
Also, could we see output of |
|
Also, could we see |
|
I've added to latest git |
|
OK, here's all the output everybody asked for, so far! Both Alien::proj and Proj are the latest version, since I didn't have either installed before this week: perl -MAlien::proj -E 'say Alien::proj->VERSION();'
1.20I don't see otool -l <snip>/auto/share/dist/Alien-proj/lib/libproj.25.dylib |grep -i 'cmd lc'
cmd LC_SEGMENT_64
cmd LC_SEGMENT_64
cmd LC_SEGMENT_64
cmd LC_SEGMENT_64
cmd LC_ID_DYLIB
cmd LC_DYLD_CHAINED_FIXUPS
cmd LC_DYLD_EXPORTS_TRIE
cmd LC_SYMTAB
cmd LC_DYSYMTAB
cmd LC_UUID
cmd LC_BUILD_VERSION
cmd LC_SOURCE_VERSION
cmd LC_LOAD_DYLIB
cmd LC_LOAD_DYLIB
cmd LC_LOAD_DYLIB
cmd LC_FUNCTION_STARTS
cmd LC_DATA_IN_CODEThe only mention of rpath in that output is perl -V
Summary of my perl5 (revision 5 version 28 subversion 3) configuration:
Platform:
osname=darwin
osvers=21.3.0
archname=darwin-thread-multi-2level
uname='darwin simurgh 21.3.0 darwin kernel version 21.3.0: wed jan 5 21:37:58 pst 2022; root:xnu-8019.80.24~20release_x86_64 x86_64 '
config_args='-des -Dprefix=/opt/local -Dscriptdir=/opt/local/bin -Dvendorprefix=/opt/local -Dusemultiplicity=y -Dusethreads -Duseshrplib -Dcc=/usr/bin/cc -Dman1ext=1pm -Dman3ext=3pm -Dinstallstyle=lib/perl5 -Dman1dir=/opt/local/share/man/man1p -Dman3dir=/opt/local/share/man/man3p -Dsitebin=/opt/local/libexec/perl5.28/sitebin -Dsiteman1dir=/opt/local/share/perl5.28/siteman/man1 -Dsiteman3dir=/opt/local/share/perl5.28/siteman/man3 -Dvendorbin=/opt/local/libexec/perl5.28 -Dvendorman1dir=/opt/local/share/perl5.28/man/man1 -Dvendorman3dir=/opt/local/share/perl5.28/man/man3 -Dpager=/usr/bin/less -sR -Dperlpath=/opt/local/bin/perl5.28 -Dstartperl=#!/opt/local/bin/perl5.28 -Acppflags=-I/opt/local/include -Accflags=-pipe -Os -Alddlflags=-L/opt/local/lib -Wl,-headerpad_max_install_names -Aldflags=-L/opt/local/lib -Wl,-headerpad_max_install_names '
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='/usr/bin/cc'
ccflags ='-fno-common -DPERL_DARWIN -pipe -Os -fno-strict-aliasing -fstack-protector-strong -I/opt/local/include -DPERL_USE_SAFE_PUTENV'
optimize='-O3'
cppflags='-I/opt/local/include -fno-common -DPERL_DARWIN -pipe -Os -fno-strict-aliasing -fstack-protector-strong -I/opt/local/include'
ccversion=''
gccversion='Apple LLVM 13.0.0 (clang-1300.0.29.30)'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='/usr/bin/cc'
ldflags =' -L/opt/local/lib -Wl,-headerpad_max_install_names -fstack-protector-strong'
libpth=/opt/local/lib /Library/Developer/CommandLineTools/usr/lib/clang/13.0.0/lib /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/lib /Library/Developer/CommandLineTools/usr/lib /usr/lib
libs=-lgdbm
perllibs=
libc=
so=dylib
useshrplib=true
libperl=libperl.dylib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=bundle
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags=' -bundle -undefined dynamic_lookup -L/opt/local/lib -Wl,-headerpad_max_install_names -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_SAFE_PUTENV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under darwin
Compiled at Mar 7 2022 13:19:10
%ENV:
PERL5LIB="/Users/derek/pdl/SWAMIS-EF:/Users/derek/local/lib/perl5"
PERLBREW_HOME="/Users/derek/.perlbrew"
PERLBREW_ROOT="/Users/derek/perl5/perlbrew"
PERLBREW_SHELLRC_VERSION="0.84"
@INC:
/Users/derek/pdl/SWAMIS-EF
/Users/derek/local/lib/perl5/darwin-thread-multi-2level
/Users/derek/local/lib/perl5
/opt/local/lib/perl5/site_perl/5.28/darwin-thread-multi-2level
/opt/local/lib/perl5/site_perl/5.28
/opt/local/lib/perl5/vendor_perl/5.28/darwin-thread-multi-2level
/opt/local/lib/perl5/vendor_perl/5.28
/opt/local/lib/perl5/5.28/darwin-thread-multi-2level
/opt/local/lib/perl5/5.28
derek@simurgh:PDL$ perl -V|grep path
config_args='-des -Dprefix=/opt/local -Dscriptdir=/opt/local/bin -Dvendorprefix=/opt/local -Dusemultiplicity=y -Dusethreads -Duseshrplib -Dcc=/usr/bin/cc -Dman1ext=1pm -Dman3ext=3pm -Dinstallstyle=lib/perl5 -Dman1dir=/opt/local/share/man/man1p -Dman3dir=/opt/local/share/man/man3p -Dsitebin=/opt/local/libexec/perl5.28/sitebin -Dsiteman1dir=/opt/local/share/perl5.28/siteman/man1 -Dsiteman3dir=/opt/local/share/perl5.28/siteman/man3 -Dvendorbin=/opt/local/libexec/perl5.28 -Dvendorman1dir=/opt/local/share/perl5.28/man/man1 -Dvendorman3dir=/opt/local/share/perl5.28/man/man3 -Dpager=/usr/bin/less -sR -Dperlpath=/opt/local/bin/perl5.28 -Dstartperl=#!/opt/local/bin/perl5.28 -Acppflags=-I/opt/local/include -Accflags=-pipe -Os -Alddlflags=-L/opt/local/lib -Wl,-headerpad_max_install_names -Aldflags=-L/opt/local/lib -Wl,-headerpad_max_install_names 'derek@simurgh:PDL$ perl -V:path_sep
path_sep=':';$ head -n 207 /opt/local/lib/perl5/5.28/darwin-thread-multi-2level/DynaLoader.pm | tail -n 20
# Many dynamic extension loading problems will appear to come from
# this section of code: XYZ failed at line 123 of DynaLoader.pm.
# Often these errors are actually occurring in the initialisation
# C code of the extension XS file. Perl reports the error as being
# in this perl code simply because this was the last perl code
# it executed.
my $flags = $module->dl_load_flags;
###LINE 197 is next
my $libref = dl_load_file($file, $flags) or
croak("Can't load '$file' for module $module: ".dl_error());
push(@dl_librefs,$libref); # record loaded object
$boot_symbol_ref = dl_find_symbol($libref, $bootname) or
croak("Can't find '$bootname' symbol in $file\n");
push(@dl_modules, $module); # record loaded module
boot:Compiling and testing latest git master, with the env var output, gives this: t/gis_proj.t .. # ENV LD_LIBRARY_PATH = undef
# ENV DYLD_LIBRARY_PATH = undef
# ENV LDFLAGS = undef
# ENV CFLAGS = undef
# ENV CXXFLAGS = undef
# ENV LD_RUN_PATH = undef
# ENV LC_RUN_PATH = undef
Can't load '/Users/derek/Build/PDL/Libtmp/GIS/Proj/../../../blib/arch/auto/PDL/GIS/Proj/Proj.bundle' for module PDL::GIS::Proj: dlopen(/Users/derek/Build/PDL/Libtmp/GIS/Proj/../../../blib/arch/auto/PDL/GIS/Proj/Proj.bundle, 0x0002): Library not loaded: @rpath/libproj.25.dylib
Referenced from: /Users/derek/Build/PDL/blib/arch/auto/PDL/GIS/Proj/Proj.bundle
Reason: tried: '/opt/local/lib:/Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib' (no such file), '/opt/local/lib:/Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib' (no such file), '/usr/local/lib/libproj.25.dylib' (no such file), '/usr/lib/libproj.25.dylib' (no such file) at /opt/local/lib/perl5/5.28/darwin-thread-multi-2level/DynaLoader.pm line 197.
at t/gis_proj.t line 18. |
|
From what I see the These should be separate entries, achievable using Resulting in From here |
|
Re-reading this, I think Alien::proj reveals a flaw in the general Perl/EUMM on MacOS handling of multiple dynamic runtime dep-locations, to which Roy has found a solution. Here, Alien::proj loads both sqlite from A workaround for here might be to install MacPorts PROJ (which does exist, not sure if it's up to date: https://ports.macports.org/port/proj4/) which would then probably live in the same location as sqlite, be found as a "system install" by Alien::proj, and dodge this. |
|
Alien::proj does not have its own sqlite. It gets it from Alien::sqlite, which in this case appears to be a system install as there are no paths containing Alien-sqlite in the various logging data. This can be checked using The Proj4 macport will not work as it uses proj version 4, which PDL has migrated away from. The proj port covers version 5. However, Alien::proj is set to use proj version 6 as a minimum because this is what GDAL 3 requires (used for Geo::GDAL::FFI). An alternative is to use homebrew, which is what I am using in CI for the GDAL stack if an example is needed. |
|
It seems (as Roy found) there are later, version-pinned MacPorts, including https://ports.macports.org/port/proj8/ and https://ports.macports.org/port/proj9/. @d-lamb I'd imagine they would provide a workaround that Alien::proj would find. |
|
And FWIW, proj should be stable at version 9 for the next few years so there won't need to be changes to the PDL interface for a while. https://lists.osgeo.org/pipermail/proj/2022-March/010571.html |
|
A further thought: if we could static-link the Alien::proj build, then it wouldn't need the runtime path. @kiwiroy Would you be able to try modifying your GIS::Proj so it uses https://metacpan.org/pod/Alien::Build#libs_static to static-link PROJ? |
|
@d-lamb Shawn reports on shawnlaffan/perl-alien-proj#7 an update - could you try that version and see if you still see the problem? |
|
Assuming this is sorted. Please reopen if not! |
Now resolved Perl-Toolchain-Gang/ExtUtils-MakeMaker#448 |
I after installing Alien::proj from CPAN, PDL tries to build PDL::GIS::Proj, but fails
make testand even a simple module load:It is trying to find libproj.25.dylib in a directory with a colon in it:
/opt/local/lib:/Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib.The correct location is:
/Users/derek/local/lib/perl5/darwin-thread-multi-2level/auto/share/dist/Alien-proj/lib/libproj.25.dylib.blib/arch/auto/PDL/GIS/Proj/Proj.bundlehas that same string in it.Libtmp/GIS/Proj/Makefilehas the lineLooking at the Makefile.PL, I don't see anything suspicious. The two values pulled from Alien::proj are correct:
All the other Makefiles in the PDL tree have either empty or sensible LD_RUN_PATH values, such as
/usr/local/libor/opt/local/lib.I'm using Perl 5.28.3, EUMM 7.64.
The text was updated successfully, but these errors were encountered: