Develop Reference

MacOS, c++ python extension cannot find a dynamic library

I'm building a c++ python extension. So far, I created versions for Linux and Windows. Currently, I'm struggling with the MacOS version. CMake produces 2 libraries:
43898860 Aug 29 13:40 libslide_io.dylib
214876 Aug 29 13:40 slideio.cpython-35m-darwin.so
I pack them to a whl file. After installation with pip, when I try to import the package, I'm getting the following error:
(sld-35)dist % python -c "import slideio"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: dlopen(/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so, 2): Library not loaded: libslide_io.dylib
Referenced from: /Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/slideio.cpython-35m-darwin.so
Reason: image not found
Both libraries are correctly placed to the python environment directory:
/Users/.../opt/anaconda3/envs/sld-35/lib/python3.5/site-packages/
Moreover, if I unzip the whl file and import the package from the current directory, it works.
I suspect that it is something with rpath settings in my cmake files, but cannot find the correct solution. All my attempts are failed.
I would appreciate any help.

If somebody is interested. I solved the problem by adding a post-build with the execution of install_name utility with the #loader_path parameter. The command instructs the system looking for the library at the loader (in my case - the python package) folder.
if (APPLE)
add_custom_command(TARGET ${BINDLIB_NAME}
POST_BUILD COMMAND
${CMAKE_INSTALL_NAME_TOOL} -change libslide_io.dylib #loader_path/libslide_io.dylib
$<TARGET_FILE:${BINDLIB_NAME}>)
endif()

Kivy/Buildozer Import Error - pymssql.so is 64-bit instead of 32-bit

I'm using the 64-bit Kivy/Buildozer VM to build and deploy a Kivy app to Android Lollipop (armeabi-v7a 32-bit arch).
The Buildozer build is successful but on execution of the app and import of the pymssql package (used to connect to a remote MS SQL server) I get the following error:
I/python ( 5335): File "/build/phytogp/android/app/main.py", line 4, in <module>
I/python ( 5335): File "/build/phytogp/android/app/views/apps/phyto_app.py", line 45, in <module>
I/python ( 5335): File "/build/phytogp/android/app/services/twilio_service.py", line 2, in <module>
I/python ( 5335): ImportError: dlopen failed: "/data/data/com.phyto.phytogp/files/app/lib/python2.7/site-packages/pymssql.so" is 64-bit instead of 32-bit
As I understand it, the pre-built pymssql-2.1.3-cp27-cp27mu-manylinux1_x86_64.whl wheel for pymssql should support both 64 and 32-bit arch.
How can I get a pymssql.so that supports 32-bit?

There is 2 things to checks:
ensure you have the 32 bits version packaged. You said the wheel should support 64 and 32 bits, but you referenced the x86_64 wheel version. On their pypi page, they have a i686 version, that's your 32 bits version that you want.
even if it's 32 or 64, it must be ARM >= v7a compiled. Unless your wheels have been compiled specifically for android, there is a great chance it just doesn't work, because the libc is not the same on Linux and Android. So you'll hit binary incompatibility.
That's why for that kind of libraries, you need to write a recipe in python-for-android to compile the library for Android.

MySQLdb module works when running Python script from the command line, but not as a cron job

I am trying to create a cron job to run a python script I wrote, but for some reason the script only runs when I'm running it from the command line. It worked before, but I accidentally deleted my crontab. When the crontab I just wrote tries to run it, I get an error about the MySQLdb module being the wrong architecture:
Traceback (most recent call last):
File "/Users/myuser/Documents/EE_SFDC Integration/csv_reader/Weekly_Updates/Weekly_Confirmation/weekly_confirm.py", line 7, in <module>
import MySQLdb
File "/Library/Python/2.7/site-packages/MySQLdb/__init__.py", line 19, in <module>
import _mysql
ImportError: dlopen(/Library/Python/2.7/site-packages/_mysql.so, 2): no suitable image found. Did find:
/Library/Python/2.7/site-packages/_mysql.so: mach-o, but wrong architecture
In my crontab, I've tried using the full path to several python versions (2.7 and 3.4), but I still get the issue. When I use 3.4, I don't get an error about MySQLdb, but I do get an error on a print .forma() statement being the wrong syntax.
But again, that doesn't happen when I run the script from the command line, so I expect this is not the correct version either.
The cron job looks like this:
14 12 * * * today=`date +"\%Y-\%m-\%d"` && /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2 /Users/myuser/Documents/EE_SFDC\ Integration/csv_reader/Weekly_Updates/Weekly_Confirmation/weekly_confirm.py > /Users/myuser/Documents/EE/run_logs/weekly_sync_runlog_$today.txt 2>&1
Any help is much appreciated!

The error message typically means that you have a mismatch between the MySQLdb (or a dependency) and Python architecture.
That is, one of them are most likely 64-bit, while the other is 32-bit.
What happens if you try the following in a terminal?
$ /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2
(Take note of the exact version)
>>> import platform
>>> print platform.architecture()
>>> import MySQLdb
Do it again, but this time with python from your path and see if they differ
$ python
>>> import platform
>>> print platform.architecture()
>>> import MySQLdb
Edit:
You might also check with file, what architecture the module is built for.
$ file /Library/Python/2.7/site-packages/_mysql.so
/Library/Python/2.7/site-packages/_mysql.so: Mach-O universal binary with 2 architectures
/Library/Python/2.7/site-packages/_mysql.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/_mysql.so (for architecture i386): Mach-O bundle i386

Mac OS X 10.6 Python 2.7 pytidylib utidylib could not find libtidy

I am answering my own question, but because I stayed up all night figuring it out, I will hopefully save others some pain. If you are getting either of the following messages after properly installing pytidylib or utidylib, this answer may help.
Learning Python on Snow Leopard, I had installed the 32-bit version of Python 2.7 so that I could use the IDLE interpreter/editor. Stackoverflow has a great explanation why I had to do this.
When I installed utidylib, I got the following error from 'import tidy':
'OSError: Couldn't find libtidy, please make sure it is installed.'
With pytidylib, I got this error when I tried 'from tidylib import tidy_document':
'OSError: Could not load libtidy using any of these names: libtidy,libtidy.so,libtidy-0.99.so.0,cygtidy-0-99-0,tidylib,libtidy.dylib,tidy.'
If you are getting these errors, please read this answer. I hope it helps you.

I had a similar issue (see stacktrace below) on Linux.
With pytidylib==0.2.1 installed with pip in a virtualenv on both Ubuntu 11.10 (Python 2.7) and Gentoo (Python 2.6).
Installing apt-get install libtidy-dev on Ubuntu and emerge app-text/htmltidy on Gentoo, solved it.
----------------------------------------------------------------------
File "/home/andre/Progz/some_site/djcode/apps/default/tests.py", line ?, in default.tests.__test__.doctest
Failed example:
from tidylib import tidy_document
Exception raised:
Traceback (most recent call last):
File "/home/andre/Progz/some_site/lib/python2.6/site-packages/django/test/_doctest.py", line 1267, in __run
compileflags, 1) in test.globs
File "", line 1, in
from tidylib import tidy_document
File "/home/andre/Progz/some_site/lib/python2.6/site-packages/tidylib/__init__.py", line 71, in
raise OSError("Could not load libtidy using any of these names: %s" % (",".join(LIB_NAMES)))
OSError: Could not load libtidy using any of these names: libtidy,libtidy.so,libtidy-0.99.so.0,cygtidy-0-99-0,tidylib,libtidy.dylib,tidy
----------------------------------------------------------------------

Although it looked like the Python packages could not find the files, what was really happening was that I compiled libtidy for the wrong architecture: x86_64 instead of i386. Using the 32-bit Python installer made them incompatible.
For neophytes like me, lipo will tell you the architecture of your files:
$ lipo -info /Library/Frameworks/Python.framework/Versions/2.7/bin/python
Architectures in the fat file: /Library/Frameworks/Python.framework/Versions/2.7/bin/python are: ppc i386
$ lipo -info /usr/lib/libtidy.dylib
Non-fat file: /usr/lib/libtidy.dylib is architecture: x86_64
You may have to locate your libtidy.dylib and python files to compare. To fix the problem, you have to re-compile libtidy for i386 (or link to an i386-compiled libtidy already on your system). To re-compile for i386, use these instructions with the following modification:
cvs -z3 -d:pserver:anonymous#tidy.cvs.sourceforge.net:/cvsroot/tidy co -P tidy
cd tidy
sh build/gnuauto/setup.sh
./configure --prefix=/usr CFLAGS="-arch i386"
make && sudo make install
This worked for me. I hope it also works for you.

I had a same issue.I installed tidylib,but still come out error as followed.
raise OSError("Could not load libtidy using any of these names: %s" % (",".join(LIB_NAMES)))
OSError: Could not load libtidy using any of these names: libtidy,libtidy.so,libtidy-0.99.so.0,cygtidy-0-99-0,tidylib,libtidy.dylib,tidy
then I copy /usr/bin/tidy to my project folder ,and it got solved.

Mac 10.6 Universal Binary scipy: cephes/specfun "_aswfa_" symbol not found

I can't get scipy to function in 32 bit mode when compiled as a i386/x86_64 universal binary, and executed on my 64 bit 10.6.2 MacPro1,1.
My python setup
With the help of this answer, I built a 32/64 bit intel universal binary of python 2.6.4 with the intention of using the arch command to select between the architectures. (I managed to make some universal binaries of a few libraries I wanted using lipo.) That all works. I then installed scipy according to the instructions on hyperjeff's article, only with more up-to-date numpy (1.4.0) and skipping the bit about moving numpy aside briefly during the installation of scipy.
Now, everything except scipy seems to be working as far as I can tell, and I can indeed select between 32 and 64 bit mode using arch -i386 python and arch -x86_64 python.
The error
Scipy complains in 32 bit mode:
$ arch -x86_64 python -c "import scipy.interpolate; print 'success'"
success
$ arch -i386 python -c "import scipy.interpolate; print 'success'"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/interpolate/__init__.py", line 7, in <module>
from interpolate import *
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/interpolate/interpolate.py", line 13, in <module>
import scipy.special as spec
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/__init__.py", line 8, in <module>
from basic import *
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/basic.py", line 8, in <module>
from _cephes import *
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so, 2): Symbol not found: _aswfa_
Referenced from: /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so
Attempt at tracking down the problem
It looks like scipy.interpolate imports something called _cephes, which looks for a symbol called _aswfa_ but can't find it in 32 bit mode. Browsing through scipy's source, I find an ASWFA subroutine in specfun.f. The only scipy product file with a similar name is specfun.so, but both that and _cephes.so appear to be universal binaries:
$ cd /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/
$ file _cephes.so specfun.so
_cephes.so: Mach-O universal binary with 2 architectures
_cephes.so (for architecture i386): Mach-O bundle i386
_cephes.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
specfun.so: Mach-O universal binary with 2 architectures
specfun.so (for architecture i386): Mach-O bundle i386
specfun.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
Ho hum. I'm stuck. Things I may try but haven't figured out how yet include compiling specfun.so myself manually, somehow.
I would imagine that scipy isn't broken for all 32 bit machines, so I guess something is wrong with the way I've installed it, but I can't figure out what.
I don't really expect a full answer given my fairly unique (?) setup, but if anyone has any clues that might point me in the right direction, they'd be greatly appreciated.
(edit) More details to address questions:
I'm using gfortran (GNU Fortran from GCC 4.2.1 Apple Inc. build 5646).
Python 2.6.4 was installed more-or-less like so:
cd /tmp
curl -O http://www.python.org/ftp/python/2.6.4/Python-2.6.4.tar.bz2
tar xf Python-2.6.4.tar.bz2
cd Python-2.6.4
# Now replace buggy pythonw.c file with one that supports the "arch" command:
curl http://bugs.python.org/file14949/pythonw.c | sed s/2.7/2.6/ > Mac/Tools/pythonw.c
./configure --enable-framework=/Library/Frameworks --enable-universalsdk=/ --with-universal-archs=intel
make -j4
sudo make frameworkinstall
Scipy 0.7.1 was installed pretty much as described as here, but it boils down to a simple sudo python setup.py install.
It would indeed appear that the symbol is undefined in the i386 architecture if you look at the _cephes library with nm, as suggested by David Cournapeau:
$ nm -arch x86_64 /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so | grep _aswfa_
00000000000d4950 T _aswfa_
000000000011e4b0 d _oblate_aswfa_data
000000000011e510 d _oblate_aswfa_nocv_data
(snip)
$ nm -arch i386 /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/special/_cephes.so | grep _aswfa_
U _aswfa_
0002e96c d _oblate_aswfa_data
0002e99c d _oblate_aswfa_nocv_data
(snip)
however, I can't yet explain its absence.

Have you tried using scipy compiled using macports?
sudo port install scipy +universal
(of course you must have the rest of the chain, python, py26-numpycompiled with the same option)
I get:
$ arch -x86_64 /opt/local/bin/python -c "import scipy.interpolate; print 'success'"
success
$ arch -i386 /opt/local/bin/python -c "import scipy.interpolate; print 'success'"
success
you may then use the setting and knowledge that the macports maintainers used to make your own compilation.

How did you install scipy, for which python version, and with which fortran compiler ?
You may also want to check that the missing symbol is indeed in both archs (I don't remember off-hand where the function is, but you should be able to find ti by yourself pretty easily using a combination of nm/otool).