I'm trying to install PyCurl in my local environment which has python 2.7 and gcc-4.2 on OS X 10.7 Lion. I've tried doing this based on this answer Error installing PyCurl:
sudo env ARCHFLAGS="-arch x86_64" pip install pycurl
Which fails because I have gcc-4.2 installed via Xcode:
error: command 'gcc-4.0' failed with exit status 1
I've also tried downloading the source and building a setup.py (I modified this based on Problem trying to install PyCurl on Mac Snow Leopard):
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -pipe -DHAVE_CURL_SSL=1 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/pycurl.c
This results in the same error as above. I have verified that I do indeed have gcc-4.2 and that it is linked to my /usr/bin.
I'm thinking that it will work if I compile it correctly so that it knows to use gcc-4.2 when installing instead of gcc-4.0. However, I don't know how to do this and have not found something to explain passing an argument to use a particular gcc. I want to avoid overriding system defaults if possible.
Chances are you have a 32-bit-only Python 2.7 installed on your system (possibly downloaded from python.org) which was built with gcc-4.0 and includes a PPC universal variant. Building C extension modules with these Pythons is very problematic with Xcode 4 installed (the default for 10.7 and optional for 10.6) because gcc-4.0 and PPC support have both been removed. The easiest and best long-term solution is to install a 64-bit/32-bit Python build (see the python.org download page for current releases) or simply use the Apple-supplied Python 2.7.1 (/usr/bin/python2.7) in 10.7.
Related
Hit problem when convert python code to shared object by Cython.
setup file here:
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize("hello.py")
)
So everything works fine on my Ubuntu desktop util transferred to CentOS.
Got error:
undefined symbol: PyUnicodeUCS4_DecodeUTF8
I googled and find there are many questions on this, but, almost all of them say the root cause is python with UCS2 or UCS4, and I understand this, didn't find one show the way to solve this.
IMO, ways to solve:
rebuild python to get the right version by "--enable-unicode=ucs4/ucs2"
But I need to reinstall all packages
Compile the code from another desktop whose python with the right UCS
Now, I wanna if there is way to set Cython to compile with specified UCS mode.
Any suggestions is great appreciated.
Thanks.
First, to answer your actual question:
I wanna if there is way to set Cython to compile with specified UCS mode.
You can build a separate python installation from source and link Cython against its headers. To find the headers, you can use the python-config tool (or python3-config for Python 3). It is usually located in the bin directory where the python executable is:
$ # system python on my machine (macos):
$ which python-config
/usr/bin/python-config
$ # python 3 installation
$ which python3-config
/Library/Frameworks/Python.framework/Versions/3.6/bin/python3-config
$ python-config --cflags
-I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE
$ python-config --ldflags
-L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 -ldl -framework CoreFoundation
Copy the output to the setup.py:
from setuptools import setup
from setuptools.extension import Extension
from Cython.Build import cythonize
cflags_ucs4 = [
'-I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m',
'-I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m',
...
]
ldflags_ucs4 = [
'-L/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/config-3.6m-darwin',
'-lpython3.6m',
...
]
cflags_ucs4 = [...]
ldflags_ucs2 = [...]
should_build_ucs2 = False # i.e. could be passed via sys.argv
if should_build_ucs2:
cflags = cflags_ucs2
ldflags = ldflags_ucs2
else:
cflags = cflags_ucs4
ldflags = ldflags_ucs4
extensions = [
Extension('hello.py', extra_compile_args=cflags, extra_link_args=ldflags),
]
setup(
ext_modules = cythonize(extensions)
)
However, I do not recommend doing that as you won't win anything by doing that - you will still need to build and distribute two separate packages (one for UCS2, another for UCS4) which is messy to maintain.
Instead, if you are building a wheel that should be installable on a wide range of Linux distros (what is most probably your actual goal), I would suggest to make your build compliable with PEP 513 (manylinux1 packages).I suggest you to read it through as it was very helpful for me when I faced the problem of distributing Linux-compliant wheels.
Now, one way to get a manylinux1-compliant wheel is to build the wheel on your machine, then running auditwheel to check for platform-specific issues and trying to resolve them:
$ pip install auditwheel
$ python setup.py bdist_wheel
$ # there should be now a mypkg-myver-cp36-cp36m-linux_x86_64.whl file in your dist directory
$ auditwheel show dist/mypkg-myver-cp36-cp36m-linux_x86_64.whl
$ # check what warnings auditwheel produced
$ # if there are warnings, try to repair them:
$ auditwheel repair dist/mypkg-myver-cp36-cp36m-linux_x86_64.whl
This should generate a wheel file named mypkg-myver-cp36-cp36m-manylinux1_x86_64.whl in a wheelhouse directory. Check again that everything is fine now by running auditwheel show wheelhouse/mypkg-myver-cp36-cp36m-manylinux1_x86_64.whl. If the wheel is now consistent with manylinux1, you can distribute it and it should work on most Linux distros (at least those with glibc; distros with musl like Alpine won't work, you will need to build a separate wheel if you want to support it).
What should you do if auditwheel can't repair your wheel? The best way is to pull a special docker container provided by PyPA for building manylinux1-compliant wheels (this is what I'm using myself):
$ docker pull https://quay.io/repository/pypa/manylinux1_x86_64
A wheel built inside this container will work on most of the Linux distros (excluding some exotic ones like Alpine).
I am trying to install PyCrypto on OS X 10.11.3 (El Capitan). I am using Python 3.5.1. I downloaded the gzip file from https://pypi.python.org/pypi/pycrypto and decompressed it. Then I ran python setup.py build like the instructions said and it appeared to do something, then it produced this output:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/MD2.c -o build/temp.macosx-10.6-intel-2.7/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I tried python3 setup.py build and got some very similar output:
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -fwrapv -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c src/MD2.c -o build/temp.macosx-10.6-intel-3.5/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I tried Googling to figure out what to do, but I couldn't find anything useful. How can I install PyCrypto?
EDIT: I also tried several other things like pip install pycrypto and sudo pip3 install pycrypto and they didn't work. #l'L'l helped me get it to work by doing several strange, complex things that I never would have though have myself. They are summarized in the answer below.
Overview:
The manual build you're trying looks like it might be failing because it's referencing the OS X 10.6 SDK, which you likely don't have, and is outdated for the most part. Also, SDKs are now stored in a completely different location than when the 10.6 SDK was in it's prime.
New SDKs location:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/
Old SDKs location:
/Developer/SDKs/
Non-existent / outdated SDK:
Since it appears that when trying to build PyCrypto it's referencing the MacOSX10.6.sdk there are several things to consider:
Why does it reference an outdated SDK
Where is the SDK it's referencing set
What should be done to correct the issue
Unless we audit the source code carefully we might not know exactly where the incorrect flags are set, but we can do our best to work with the information we have. From the error we can see that there are several instances where the 10.6 SDK's name pops up:
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -fwrapv -Wall -Wstrict-prototypes -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library
Building from source:
/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/MD2.c -o build/temp.macosx-10.6-intel-2.7/src/MD2.o
src/MD2.c:30:10: fatal error: 'string.h' file not found
#include <string.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
Analyzing this we can see that the PyCrypto's MD2.c file is trying to be built using the flag -isysroot /Developer/SDKs/MacOSX10.6.sdk. It might be worth trying pip instead:
Installing with pip:
...
fatal error: 'string.h' file not found #include <string.h>
...
Same error; we should probably find out if the <string.h> header even exists on the system — Let's make a quick test C application to find out:
Testing the C headers:
$ echo "#include <string.h>
#include <stdio.h>
int main() { printf(\"TEST\n\"); return 0; }" > t.c
$ clang t.c -o t
$ ./t
TEST
It's apparent the header does exist because the test worked fine. This tells us that the problem is more likely related directly to the 10.6 SDK (which doesn't seem to exist on the system).
Symlinking (non-existing) 10.6 SDK to 10.11 SDK:
Since we haven't determined where the SDK is actually getting set we'll go ahead and try to create symlinks so that any reference of the old 10.6 SDK links to the latest SDK (10.11 at this time):
$ cd /Developer/SDKs
$ sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk MacOSX10.6.sdk
We can verify the symlink by issuing the following command:
$ ls -lat
total 8
drwxr-xr-x 3 root wheel 102 Feb 21 15:54 .
lrwxr-xr-x 1 root wheel 99 Feb 21 15:54 MacOSX10.6.sdk -> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk
drwxr-xr-x 3 root wheel 102 Feb 21 15:52 ..
Now that we've successfully created the symlink let's try installing PyCrypto with pip once more:
$ sudo pip install pycrypto
Collecting pycrypto
Downloading pycrypto-2.6.1.tar.gz (446kB)
100% |████████████████████████████████| 446kB 1.2GB/s
Installing collected packages: pycrypto
Running setup.py install for pycrypto ... done
Successfully installed pycrypto-2.6.1
No errors! It looks like our problem is solved! Well, almost...
We still need to figure out what is responsible for setting the wrong (10.6) SDK during builds. Let's use the xcrun tool to see what the defaults are set at:
$ xcrun --show-sdk-version
10.11
The system default SDK is set to 10.11, so it must be get set incorrectly to 10.6 by Python, PyCrypto, or some other anomaly we might not have considered.
UPDATE:
After doing some recon it was discovered that Python 3 appears to be built with the OS X 10.6 SDK. In addition it's also setting the SDK to 10.6 and setting the (outdated) path in numerous places throughout the Python_Framework. There are so many references I won't bother listing them all, although here's an example:
Python_Framework Folder/Versions/3.5/lib/python3.5/config-3.5m/Makefile:79:CONFIGURE_CFLAGS= -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk
I can only assume the developers were trying to be as backward compatible as possible, however, it's unfortunately breaking forward compatibility in the process.
Notes:
Installing Python packages with pip can make life much easier in a lot of ways (package management, updating, uninstalling, etc.). For example installing PyCrypto should just be a matter of issuing the command:
$ sudo pip install pycrypto
If you have multiple Python's you can use the version number to install for that Python accordingly:
$ sudo pip3.5 install pycrypto
↳ https://pip.pypa.io/en/stable/installing/
I had what might be a related issue on OSX El Capitan when I would run pip install pycrypto. I was seeing RuntimeError: autoconf error. All I had to do was run sudo xcodebuild -license and type agree after reviewing the licence agreement. Afterward I was able to use pip to install pycrypto.
I have installed python-2.7-macosx10.5.dmg from python.org on Mac os x 10.5.8.
I'm issuing: python setup.py build for pyPortMidi-0.0.3
And getting:
Found darwin (OS X) platform
running build
running build_ext
pyrexc pypm.pyx --> pypm.c
/Users/baz/Downloads/pyPortMidi-0.0.3/pypm.pyx:357:21: Type 'PmError' not acceptable as a boolean
building 'pypm' extension
creating build/temp.macosx-10.5-intel-2.7
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -O3 -arch i386 -arch x86_64 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pypm.c -o build/temp.macosx-10.5-intel-2.7/pypm.o
pypm.c:1:2: error: #error Do not use this file, it is the result of a failed Pyrex compilation.
pypm.c:1:2: error: #error Do not use this file, it is the result of a failed Pyrex compilation.
lipo: can't figure out the architecture type of: /var/folders/oO/oO1flrWgHAC8u6KdoO0Wq++++TI/-Tmp-//ccTcgy0s.out
error: command 'gcc-4.0' failed with exit status 1
Can anyone help me to resolve this?
I found the easiest way was to build the version of pyPortMidi included in pygame, which has some fixes applied.
You can use the following pattern to import it at the top of your file, preferring the standard version, but falling back to the pygame bundled version.
try:
import pypm
except ImportError:
from pygame import pypm
Using MacPorts, it was easy to install using:
port install py27-game +portmidi
I don't know if you need something special to include portmidi in the build if you build by other methods.
Looking at the link below it seems that stuff is broken and missing. Don't know if they are going to fix it anytime soon... Been bothering me too for some time...
https://groups.google.com/forum/#!topic/pygame-mirror-on-google-groups/sf3I8Q-wYQA
I want to run python's ndimage to do some image analysis. I have a 64-bit Mac running OSX Lion and Python 2.7. When I tried to run commands from ndimage commands I found out that I need to install PIL.
I downloaded that and unzipped it (for now into the downloads folder, is there a better place to do it?). I run setup.py and get these messages:
running install
running build
running build_py
running build_ext
--- using frameworks at /System/Library/Frameworks
building '_imaging' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/Library/Frameworks/Python.framework/Versions/2.7/include -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imaging.c -o build/temp.macosx-10.6-intel-2.7/_imaging.o
unable to execute gcc-4.2: No such file or directory
error: command 'gcc-4.2' failed with exit status 1
I read Failed to build PIL on Mac OS X 10.7 Lion and made sure that I have the 32/64 bit version of python 2.7 installed (Mac OS X 64-bit/32-bit x86-64/i386 Installer (2.7.2) for Mac OS X 10.6 and 10.7 ). I also have Xcode installed. Am I doing something stupid here?
Edit 1:
looking further, I have found this gcc-4.2 failed with exit status 1. I've tried entering this when I get the error:
llvm-gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/Library/Frameworks/Python.framework/Versions/2.7/include -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imaging.c -o build/temp.macosx-10.6-intel-2.7/_imaging.o
and I get these messages:
_imaging.c:3017: warning: initialization from incompatible pointer type
_imaging.c:3077: warning: initialization from incompatible pointer type
_imaging.c:3017: warning: initialization from incompatible pointer type
_imaging.c:3077: warning: initialization from incompatible pointer type
Edit 2:
I'm not sure if this is the right way to go, but I found a post on installing pil on OSX Leopard and, following its advice, got rid of the -arch i386 part of the command and entered:
llvm-gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch x86_64 -g -O2 -DNDEBUG -g -O3 -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -IlibImaging -I/Library/Frameworks/Python.framework/Versions/2.7/include -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imaging.c -o build/temp.macosx-10.6-intel-2.7/_imaging.o
Now I only get two errors:
_imaging.c:3017: warning: initialization from incompatible pointer type
_imaging.c:3077: warning: initialization from incompatible pointer type
From what I can tell at this point the program poops out and doesn't finish compiling. Can anyone help me take it from here?
https://github.com/kennethreitz/osx-gcc-installer/downloads
Download GCC for lion. It solved all the error: command 'gcc-4.2' failed with exit status 1
problems.
I have had very good success with the MacPorts Python Imaging Library (PIL), Pandas, Numpy and other numerical analysis packages on both Lion and Mountain Lion.
Recently there were some significant upgrades with gcc for integration with the latest numerical Python modules on MacPorts. Looked like a very significant effort. I recommend MacPorts unless you are determined to hash through a native install of PIL on Lion.
I had compiling problems with PIL with Mountain Lion and python 2.7. I used Pillow instead, it's a friendly PIL fork with wider platform support.
regardins your setup questions:
I use macports for python 2.7 and other open source stuff on my mac (libpng, libjpeg, etc for PILLOW)
virtualenv for creating a virtual python environment (venv will be built-in into python 3.3)
I put source code under ~/src (instead Downloads folder)
I'm an extremely amateur programmer; I've done some recreational algorithmics programming, but I honestly have no idea how libraries and programming languages really fit together. I'm supposed to work on a project that requires some image processing, so I've been trying to install PIL for a while, but I haven't been able to.
I went to http://www.pythonware.com/products/pil/ and downloaded "Python Imaging Library 1.1.6 Source Kit (all platforms) (440k TAR GZ) (December 3, 2006)". Then I opened the folder in my command prompt and ran
$ python setup.py build_ext -i .
This was the output I got:
running build_ext
--- using frameworks at /System/Library/Frameworks
building '_imaging' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -Os -Wall -Wstrict-prototypes -DMACOSX -I/usr/include/ffi -DENABLE_DTRACE -arch i386 -arch ppc -pipe -DHAVE_LIBZ -IlibImaging -I/opt/local/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c _imaging.c -o build/temp.macosx-10.5-i386-2.5/_imaging.o
unable to execute gcc: No such file or directory
error: command 'gcc' failed with exit status 1
"import Image" produced an error when I tried it.
Do you guys have any idea what's going on? I'm using a MacBook Pro with a Core 2 Duo.
And I'm honestly sorry if this is ridiculously stupid.
Actually, assuming you're still using the default 2.5.x Python that comes with OS X (at least as of 10.5.6), there's a pre-built installer package for it (download the dmg for PIL).
Otherwise, you'll need to either build it from source -- which does require the mac dev tools -- or install it with MacPorts or fink
edit: mono makes a good point, you'll still need the dev tools unless you use the pre-built installer.
You need to install the developer tools that come on your Mac OS X install DVD.
GCC is the GNU compiler. It's a very useful thing to have. You just need to install it in whatever mac-friendly way exists.
http://www.tech-recipes.com/rx/726/mac-os-x-install-gcc-compiler/
So this is from awhile ago, but I just ran into the problem.
The issues lies with ->
/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/sysconfig.py
or wherever your python install is.
there is a line that sets the compile flags:
archflags = '-arch i386 -arch ppc -arch x86_64'
I just removed it from that line and went on my merry way. Now there is obviously a way to configure this from the line above:
os.environ['ARCHFLAGS']
but I don't know about that, and didn't want to mess with it.