I'm trying to compile SciPy for both 32-bit and 64-bit architectures, since some of the applications that use the library are only one arch or the other, following advice from a few SO questions. This command has worked for me in a similar build about two months ago, but now it seems that some of the compiled shared libraries are only targeting x86_64:
sudo ARCHFLAGS="-arch i386 -arch x86_64" pip install scipy
I've tried several versions of this command, including setting the environment variables before sudo, including -m32 -m64 in FFlags, using the OSX Command Line Tools, a full Xcode install, using clang instead of the LLVM that comes with OSX following the official SciPy install instructions, easy_install instead of pip and even ./configure'ing separately, etc., but I can't seem to get it to work. Oddly, some of the .so's are building with both architectures, but some are not:
dhcp-10-249-71-202:~ kastman$ file /Library/Python/2.7/site-packages/scipy/optimize/*.so
/Library/Python/2.7/site-packages/scipy/optimize/_cobyla.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/_lbfgsb.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/_minpack.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/_nnls.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/_slsqp.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/_zeros.so: Mach-O universal binary with 2 architectures
/Library/Python/2.7/site-packages/scipy/optimize/_zeros.so (for architecture i386): Mach-O bundle i386
/Library/Python/2.7/site-packages/scipy/optimize/_zeros.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/minpack2.so: Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/scipy/optimize/moduleTNC.so: Mach-O universal binary with 2 architectures
/Library/Python/2.7/site-packages/scipy/optimize/moduleTNC.so (for architecture i386): Mach-O bundle i386
/Library/Python/2.7/site-packages/scipy/optimize/moduleTNC.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
Looking at the make log, it looks like the args are being passed successfully for both the libraries that seem to work and those that don't:
# Minpack doesn't build fat binaries
/usr/local/bin/gfortran -Wall -Wall -undefined dynamic_lookup -bundle build/temp.macosx-10.7-intel-2.7/build/src.macosx-10.7-intel-2.7/scipy/optimize/minpack2/minpack2module.o build/temp.macosx-10.7-intel-2.7/build/src.macosx-10.7-intel-2.7/fortranobject.o build/temp.macosx-10.7-intel-2.7/scipy/optimize/minpack2/dcsrch.o build/temp.macosx-10.7-intel-2.7/scipy/optimize/minpack2/dcstep.o -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin11/4.6.2 -Lbuild/temp.macosx-10.7-intel-2.7 -lgfortran -o build/lib.macosx-10.7-intel-2.7/scipy/optimize/minpack2.so
building 'scipy.optimize._slsqp' extension
compiling C sources
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe
#... but moduleTNC does.
llvm-gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -Wl,-F. -arch i386 -arch x86_64 build/temp.macosx-10.7-intel-2.7/scipy/optimize/tnc/moduleTNC.o build/temp.macosx-10.7-intel-2.7/scipy/optimize/tnc/tnc.o -Lbuild/temp.macosx-10.7-intel-2.7 -o build/lib.macosx-10.7-intel-2.7/scipy/optimize/moduleTNC.so
building 'scipy.optimize._cobyla' extension
compiling C sources
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe
Is it possible that the difference is the gfortran compiler I'm using? I used brew install gfortran which is supposed to be good according to the SciPy documentation. I would think the important line would be the C compiler: clang line, which appears to be nearly identical, including both -arch's.
The last most detailed build I've tried was:
sudo env ARCHFLAGS="-arch i386 -arch x86_64" ARCH="i386 x86_64" CC="clang" CXX="clang" FFLAGS="-ff2c -m32 -m64" pip install scipy
Does anyone have suggestions about how to further diagnose this?
(OSX 10.7, recent MacBookPro, compiling with gcc from the CLI tools and from Xcode)
Pulling from source and building appears to correctly create fat binaries. I'm not sure if this is a bug that's already been fixed (I didn't see anything on the scipy dev list) or if it has to do with pip/easy_install, but here's what I did to get it to work:
git clone https://github.com/scipy/scipy.git; cd scipy
export ARCHFLAGS='-arch i386 -arch x86_64'
python setup.py config
python setup.py build
sudo python setup.py install
No special CC environment variables, gfortran was from brew install gfortran, and surprisingly it looks like gcc was llvm-gcc.
Related
Here is the error log:
building 'psycopg2._psycopg' extension
creating build/temp.macosx-10.9-intel-2.7
creating build/temp.macosx-10.9-intel-2.7/psycopg
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.5 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090301 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I. -I/Library/PostgreSQL/9.3/include -I/Library/PostgreSQL/9.3/include/postgresql/server -c psycopg/psycopgmodule.c -o build/temp.macosx-10.9-intel-2.7/psycopg/psycopgmodule.o
clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
error: command 'cc' failed with exit status 1
Any suggestions? any workaround? I am entirely blocked by this for about a day.
This one works for me:
export CFLAGS=-Qunused-arguments
export CPPFLAGS=-Qunused-arguments
clang error: unknown argument: '-mno-fused-madd' (python package installation failure)
Try:
sudo ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install psycopg2
Im pretty sure pyscopg2 doesnt support 9.3 yet. Here's the documentation http://initd.org/psycopg/docs/news.html
I see references for 9.1 and 9.2 but nothing for 9.3 especially since it was just released a few months ago. 9.3 incorporates it's own Foreign Writable wrapper which other modules like Multicorn now utilize.
http://www.postgresql.org/about/news/1481/
I could be wrong, just a possibility. Try using 9.2 and see if it works.
I've been spending a day in order to install simplejson package. Always I'm without success!
When I type :
pip install simplejson
It raised me error :
Installing collected packages: simplejson
Running setup.py install for simplejson
building 'simplejson._speedups' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o
clang: warning: argument unused during compilation: '-mno-fused-madd'
Has some one ever met error like this, or can give me advice what i can do in order to install it?
So we managed to fix the problem on the chat. It was problem related to upgrading to OS X mavericks and this article was really helpful http://attentionshard.wordpress.com/2013/10/25/os-x-mavericks-fixing-broken-python-development-environments/.
I am trying to install PIL using PIP installer, and I'm getting this:
llvm-gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -IlibImaging -I/System/Library/Frameworks/Python.framework/Versions/2.7/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imaging.c -o build/temp.macosx-10.7-intel-2.7/_imaging.o
unable to execute llvm-gcc-4.2: No such file or directory
error: command 'llvm-gcc-4.2' failed with exit status 1
Any clues what the problem might be ?
I have Mac OS X Lion and LLVM is installed in the XCODE Developer tools... What can be wrong ?
i got the same error.
but additionally need to install the "commandline tools" under settings
edit:
just a few keywords for google, if someone is looking for this!
easy_install pip gcc not found mac osx xcode
may be a moderator add this keywords to the question
Are you sure you have actually run the Xcode installer (usually found in /Applications)? Downloading it from the Mac App Store doesn't automatically run the installer.
Check that you have installed XCode. Try installing a PIL from homebrew package manager
Xcode4 dropped PPC support, so when I try building PIL, it throws hate:
Bens-MacBook-Air:Imaging-1.1.7 bkeating$ python setup.py build
running buildrunning build_pyrunning build_ext
--- using frameworks at /System/Library/Frameworks
building '_imaging' extension
/usr/bin/gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch ppc -arch x86_64 -pipe -DHAVE_LIBJPEG -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/usr/local/include/freetype2 -IlibImaging -I/System/Library/Frameworks/Python.framework/Versions/2.6/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c decode.c -o build/temp.macosx-10.6-universal-2.6/decode.o
/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
Installed assemblers are:
/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
decode.c:688: fatal error: error writing to -: Broken pipe
compilation terminated.
lipo: can't open input file: /var/folders/jW/jW0VkC8uEb4lAjcJou+OaU+++TI/-Tmp-//ccEmCpUp.out (No such file or directory)
error: command '/usr/bin/gcc-4.0' failed with exit status 1
I ran that first line, manually, and removed the -arch ppc option and it didn't report back with any errors, but rerunning the build process ignores this manual attempt. `gcc-4.0 also does not exist, I symlinked it to gcc-4.2.
I could go ahead and install Xcode3 and then 4, but I want to muck up my fresh install.
Is there anyway to tell PIL to not compile for PPC? I did a grep across the distribution but i did not find a single mention of PPC. Any ideas?
The solution has nothing to do with PIL but rather setting gcc's ARCHFLAGS:
ARCHFLAGS="-arch i386 -arch x86_64" sudo pip install PIL
ARCHFLAGS doesn't seem to get passed into sudo. I had to do
sudo -s
then ARCHFLAGS="-arch i386 -arch x86_64" pip install PIL
to make it work.
A better way to solve this issue, in my opinion, would be to edit your ~/.profile or /etc/bashrc and add the line:
export ARCHFLAGS="-arch i386 -arch x86_64"
Will save messing around with any future installations (I've just had to do this for installing lots of Perl modules in CPAN)!
Just to help others...
I had to use ARCHFLAGS="-arch x86_64" before building imaging with my Snow Leopard/XCode 4 system i.e. having "-arch i386" in their stopped it working for me.
I'm trying to install PIL on an Intel Mac OS X Leopard machine. Unfortunately, "setup.py build" thinks it should be compiling for ppc.
gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3 -DHAVE_LIBJPEG -DHAVE_LIBZ -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/opt/local/include/freetype2 -IlibImaging -I/sw/include -I/opt/local/include -I/Users/adam/Development/pinax-env/include -I/usr/local/include -I/usr/include -I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c libImaging/GifEncode.c -o build/temp.macosx-10.3-i386-2.5/libImaging/GifEncode.o
This fails because I don't have the ppc arch files available on my machine (nor do I want to install them). How can I tell setup.py to only do i386?
I've looked in /Library/Frameworks/Python.framework for a config file to no avail.
The solution that worked for me was:
ARCHFLAGS="-arch i386 -arch x86_64" python setup.py build
Just pass the values for the flag right in the command line.
The easiest solution (for a single or a few C files) is to copy the compiler line, edit it, and invoke it manually, then run setup.py again - it should notice that this step was already done.
To make setup.py not use these options anymore, you need to change the Makefile in Python's config directory, and remove the options.