Issues with finding 'cygmagic-1'

[jakubczaplicki]

Environment: Windows 7, Python 2.7, Most recent Cygwin x86 (DLL 2.2.1-1)

OK. So I got python-magic running only after moving the Cygwin PATH to the beginning of the %PATH% env.var. :

1. Install the cygwin-libmagic in Cygwin 
2. Install https://github.com/ahupp/python-magic (python setup.py install)
3. Place c:\cygwin\bin\   right at the beginning of PATH env.var. 
4. Open python, import magic

Without the step 3 I got:

   >>> import magic
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "C:\Python27\lib\site-packages\magic.py", line 184, in <module>
       raise ImportError('failed to find libmagic.  Check your installation')
   ImportError: failed to find libmagic.  Check your installation

and:

    >>> import ctypes
    >>> import ctypes.util
    >>> print ctypes.util.find_library('cygmagic-1')
    None
    >>> print ctypes.util.find_library('c')
    msvcr90.dll

It looks like the find_library() method can’t be bothered to look for the libs in other places from the %PATH%. The dll it finds is in a Webex folder...

Once I've put c:\cygwin\bin at the beginning of %PATH%, all my life problems were immediately solved :

   >>> import ctypes.util
   >>> ctypes.util.find_library('cygmagic-1')
   'c:\\cygwin\\bin\\cygmagic-1.dll'
   >>> import os
   >>> os.getenv("PATH")
   'c:\\cygwin\\bin\\;C:\\Python27\\;C:\\Python27\\Scripts; <snip>

So I wanted to check if it really was causing the problem, moved the path to Cygwin back to the end of %PATH%, rerun cmd and opened python console, and guess what ? It still works ! Why ? :

   >>> import ctypes.util
   >>> ctypes.util.find_library('cygmagic-1')
   'c:\\cygwin\\bin\\cygmagic-1.dll'
   >>> import os
   >>> os.getenv("PATH")
   <snap>;C:\\Program Files (x86)\\gnuplot\\bin;c:\\cygwin\\bin'

[ahupp]

Hah, I have no idea. The dll load path stuff is mysterious (i don't actually own a windows machine). If you find a good solution please submit it and I'll add to the docs.

[ahupp]

Or rather, find an answer. It's hard to imagine that the position in the path really effects this.

[ghost]

Apparently find_library may not be the appropriate lookup tool here as, according to Martin Panter,
find_library() is documented as emulating the build-time linker. (and not the run-time loader)

Please see the discussion http://bugs.python.org/issue9998.

That is, originally to get over similar issues on SunOS, the following patch to python seemed okay http://bugs.python.org/issue19317, but this was refused upstream.
Hence we, too, are back to square 1 with py-magic.

Updating the logic to take into consideration that the program is installed into a different $PREFIX than the system default (like '/usr') and use something equivalent to a build time option '--path-to-libmagic=<>' would be greatly appreciated.

[ahupp]

Does it find the lib if you put that path in /etc/ld.so.conf and re-run ldconfig? I don't understand your proposal. Libraries should not be handling command line args.

[ghost]

My point was that consideration could be made at build (or config) time, that is prior to installing, not run-time. This is how most libraries that have dependencies do it, for run-time needs.

I don't believe cluttering up ld.config (or equivalent) is an endearing approach.

If find_library() is used, since it emulates a build-time linker, it would need to be able
take into account any necessary library paths and usually the associated runpath specifications
for dependencies...

If py-magic were a linked binary, one could simply specify explicitly one or more runpaths at link-time and things like CDLL should then simply work without resorting to find_library() at all.

The problem stated in this issue is common to probably all packaging systems which
targetting where there isn't a native libmagic library (or equivalent)...

... and unfortunately it works improperly if the target system indeed has a native version that shouldn't be used by policy.

[ahupp]

What is "link time" in this context though? In python we don't really have that distinction. I'm tempted to just define an env variable that people can defined to override the location then I can get rid of all the janky platform-specific paths. Would be a breaking change though.

[ghost]

Mercurial employs a simple approach that perhaps is useful here.
Take a look at how setup.py deals with replacement of the @libdir@ token in hg itself

[ahupp]

Merging into #293