I'm trying to convert a Python 3 script into C and then compile that C file into an executable.
I have this simple python script:
def greet(name = ""):
print("Hello {0}".format(name if len(name) > 0 else "World"))
greet("Mango")
I've converted this script into C using:
cython greet.py -o greet.c
Then I've compiled the C file using:
cc greet.c -o greet
After I entered the last command I got the error:
fatal error: Python.h: No such file or directory compilation terminated.
After I got the error I went back and realised that I was using Python3 and that I had forgot the "3" after "cython".
So re-compiled the python script using:
cython3 greet.py -o greet.c
Then attempted to re-compile the C file using:
cc greet.c -o greet
Again this failed and threw the same error so I went searching on SO and Google and found these questions:
fatal error: Python.h: No such file or directory
I have Python on my Ubuntu system, but gcc can't find Python.h
https://askubuntu.com/questions/526708/fatal-error-python-h-no-file-or-directory
None of these answers in these questions work.
I've made sure that I have installed cython all of the correct dependencies using apt-get install and pip install sadly thought it still does not seem to work.
Check the documentation. It's not enough to do gcc x.c -o x.
This page explains compilation: http://docs.cython.org/src/reference/compilation.html
There's a lot more to it, but a direct answer is:
Compiling your .c files will vary depending on your operating system. Python documentation for writing extension modules should have some details for your system. Here we give an example on a Linux system:
$ gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing
-I/usr/include/python2.7 -o yourmod.so yourmod.c
Of course in your situation it's going to be something closer to -I/usr/include/python3.4, or even $(pkg-config --libs --cflags python3). And you're not building with -shared, because you want an executable.
Shortest "this has to work" set of commands is:
cython3 --embed greet.py -o greet.c
gcc $(pkg-config --libs --cflags python3) greet.c -o greet
You need to install pkg-config if it's missing.
As #viraptor's answer shows you and as per my comment, your main problem is that you need to tell your C compiler (e.g. gcc) where to find the python headers required (pyconfig.h and Python.h). To do this, you need to pass a -I option to gcc.
The other answer suggests using pkg-config to add this to your command line. However, like you, with Ubuntu 14.04, cython3 and python3-dev installs, using this method leads the compiled program to exit with a segmentation fault for me.
So, I suggest you go back to basics. After
cython greet.py -o greet.c
Run the following command. It assumes that Python.h and friends are in the standard place (i.e. you've done a standard install of python3-dev)
gcc -I/usr/include/python3.4m -o greet greet.c -lpython3.4m
If that doesn't work - use find / -iname Python.h to find the location of the necessary files and alter the -I path accordingly.
In time, when you want to use cython on more complex programs, such as those that link to other C libraries, you'll need to learn about the other options you need to pass to gcc to get it to compile and link correctly. To get you going, though, the above should work (tested on Ubuntu 14.04 as per your spec)
P.S. I'm not sure why the pkg-config suggestion doesn't work - but for me it seems to add in an extra path to -I which breaks things.
Python 2:
python -m pip install --upgrade cython
Python 3:
python3 -m pip install --upgrade cython
Related
I am trying to create Python-3.2.0 virtual environment for upgrading my tool.
The tool was earlier built with 2.7.15 version. Now I want to upgrade it to Python-3.2.0
These are the steps which I followed on Cent-OS:
(a) Download the Python source code from the official repository
cd /tmp
wget https://www.python.org/ftp/python/3.2/Python-3.2.tgz
tar -xvf Python-3.2
cd Python-3.2
(b) Compile Python with the required flags
./configure --enable-optimizations --enable-shared --
prefix=/opt/python LDFLAGS=-Wl,-rpath=/opt/python/lib
sudo make install
I got the following errors:
/usr/bin/install -c python-config /opt/python/bin/python3.2m-config
rm python-config
LD_LIBRARY_PATH=/tmp/Python-3.2: ./python -E ./setup.py install \
--prefix=/opt/python \
--install-scripts=/opt/python/bin \
--install-platlib=/opt/python/lib/python3.2/lib-dynload \
--root=/
running install
running build
running build_ext
building dbm using ndbm
INFO: Can't locate Tcl/Tk libs and/or headers
building '_dbm' extension
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DHAVE_NDBM_H -IInclude -I/opt/python/include -I. -I./Include -I/usr/local/include -I/tmp/Python-3.2 -c /tmp/Python-3.2/Modules/_dbmmodule.c -o build/temp.linux-x86_64-3.2/tmp/Python-3.2/Modules/_dbmmodule.o
I n file included from Include/Python.h:111:0,
from /tmp/Python-3.2/Modules/_dbmmodule.c:6:
Include/modsupport.h:27:1: warning: ‘_PyArg_ParseTuple_SizeT’ is an unrecognized format function type [-Wformat=]
PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...) Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
^
gcc -pthread -shared -Wl,-rpath=/opt/python/lib build/temp.linux-x86_64-3.2/tmp/Python-3.2/Modules/_dbmmodule.o -L. -L/opt/python/lib -L/usr/local/lib -lpython3.2m -o build/lib.linux-x86_64-3.2/_dbm.cpython-32m.so
*** WARNING: renaming "_dbm" since importing it failed: build/lib.linux-x86_64-3.2/_dbm.cpython-32m.so: undefined symbol: dbm_nextkey
Python build finished, but the necessary bits to build these modules were not found:
_tkinter bz2 ossaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Failed to build these modules:
_dbm
running build_scripts
copying and adjusting /tmp/Python-3.2/Tools/scripts/pydoc3 -> build/scripts-3.2
copying and adjusting /tmp/Python-3.2/Tools/scripts/idle3 -> build/scripts-3.2
copying and adjusting /tmp/Python-3.2/Tools/scripts/2to3 -> build/scripts-3.2
changing mode of build/scripts-3.2/pydoc3 from 644 to 755
I want to start by saying that what you're trying to do, is an exercise of futility. Check:
[Python]: PEP 373 -- Python 2.7 Release Schedule
[Python]: PEP 392 -- Python 3.2 Release Schedule
So:
You're trying to "upgrade" from a version that's going out of support at the end of the year (your particular flavor (v2.7.15) released last year) to a version that has been dead for several years
More: you're attempting v3.2.0 which is the very 1st one of that series
Quickly searching for your error revealed:
[Python.Bugs]: _dbm not building on Fedora 17
[Python.Bugs]: Failure to build _dbm with ndbm on Arch Linux
Now, this may or may not be the cause in your case. If it is, there is a fix, but you won't benefit from it because of #2..
A couple of ideas:
Generally, a software's 1st version of a series is likely to have more bugs, because it hasn't been tested much "in the real world" (as it hasn't been released yet). The chances of something going wrong increase if there is other software that is built on top of it (in this case, other Python 3rd-party modules). As an example (although not related to the current scenario), you could check [SO]: PyWin32 and Python 3.8.0
You should use a (Python) version that's supported and maintained (e.g. v3.8, or v3.7), so that you could have a real chance of getting help if running into problems
If for some reasons (that I fail to find logical) you need to stick to the v3.2, try at least using the latest one ([Python]: Python-3.2.6.tgz)
I seem to be one of the many people struggling to install gensim on windows. I have trawled through countless forums but the errors poster there never appear to match my errors. So hopefully someone can point me in the right direction!
I am running Windows Server 2012 R2 Standard 64-bit. I have installed MinGW & Anaconda 2.2.0 (64-bit), which comes with Python 2.7.9.
I have added a file distutils.cfg into C:\Users\Sam\Anaconda\Lib\distutils with the contents:
[build]
compiler=mingw32
I have added C:\MinGW\bin to my Environment variables.
If I install gensim using pip I do not get any errors, until I try to run Word2Vec when I get the error:
C:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\gensim\models\word2vec.py:459: UserWarning: C extension com
pilation failed, training will be slow. Install a C compiler and reinstall gensim for fast training.
So I have uninstalled gensim and tried to re-install using the mingw32 compiler, but this gives me this error:
python setup.py build --compiler=mingw32
c:\users\sam.passmore\appdata\local\continuum\anaconda\lib\site-packages\setuptools-14.3-py2.7.egg\setuptools\dist.py:282: UserWarni
ng: Normalizing '0.11.1-1' to '0.11.1.post1'
running build
running build_ext
building 'gensim.models.word2vec_inner' extension
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -Igensim\models -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\include -IC:
\Users\sam.passmore\AppData\Local\Continuum\Anaconda\PC -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\n
umpy\core\include -c ./gensim/models/word2vec_inner.c -o build\temp.win-amd64-2.7\Release\.\gensim\models\word2vec_inner.o
gcc: error: ./gensim/models/word2vec_inner.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
setup.py:82: UserWarning:
********************************************************************
WARNING: %s could not
be compiled. No C extensions are essential for gensim to run,
although they do result in significant speed improvements for some modules.
%s
Here are some hints for popular operating systems:
If you are seeing this message on Linux you probably need to
install GCC and/or the Python development package for your
version of Python.
Debian and Ubuntu users should issue the following command:
$ sudo apt-get install build-essential python-dev
RedHat, CentOS, and Fedora users should issue the following command:
$ sudo yum install gcc python-devel
If you are seeing this message on OSX please read the documentation
here:
http://api.mongodb.org/python/current/installation.html#osx
********************************************************************
The gensim.models.word2vec_inner extension moduleThe output above this warning shows how the compilation failed.
"The output above this warning shows how the compilation failed.")
building 'gensim.models.doc2vec_inner' extension
C:\MinGW\bin\gcc.exe -DMS_WIN64 -mdll -O -Wall -Igensim\models -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\include -IC:
\Users\sam.passmore\AppData\Local\Continuum\Anaconda\PC -IC:\Users\sam.passmore\AppData\Local\Continuum\Anaconda\lib\site-packages\n
umpy\core\include -c ./gensim/models/doc2vec_inner.c -o build\temp.win-amd64-2.7\Release\.\gensim\models\doc2vec_inner.o
gcc: error: ./gensim/models/doc2vec_inner.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
setup.py:82: UserWarning:
********************************************************************
WARNING: %s could not
be compiled. No C extensions are essential for gensim to run,
although they do result in significant speed improvements for some modules.
%s
Here are some hints for popular operating systems:
If you are seeing this message on Linux you probably need to
install GCC and/or the Python development package for your
version of Python.
Debian and Ubuntu users should issue the following command:
$ sudo apt-get install build-essential python-dev
RedHat, CentOS, and Fedora users should issue the following command:
$ sudo yum install gcc python-devel
If you are seeing this message on OSX please read the documentation
here:
http://api.mongodb.org/python/current/installation.html#osx
********************************************************************
The gensim.models.doc2vec_inner extension moduleThe output above this warning shows how the compilation failed.
"The output above this warning shows how the compilation failed."
I have exhausted all options I can think of or find, so if anyone could give some advice it would be much appreciated.
I managed to solve this after using conda install for gensim, rather than pip.
conda install gensim
I am not sure what other steps I have included above have contributed to the answer, but this was the last thing I did before I no longer was getting the 'Install a C compiler and reinstall gensim for fast training.' message.
During my research in trying to solve this problem I saw that the most common methods were adding the lines
[build]
compiler=mingw32
to the distutils.cfg file as well as ensuring MinGW is in your path. Also ensuring that the MinGW bit version is the same as your python version.
I'd like to start by saying that asking a question here isn't something I do lightly. I've now been attempting to install Pygame 1.9.1 from source for four hours, and I've run into several problems but was able to overcome each one. This is the first one I'm completely stumped on.
After I unpack pygame-1.9.1release.tar.gz and cd to the folder, I run 'python3 setup.py install'
/Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_syswm.h:58:10: fatal error: 'X11/Xlib.h' file not found
#include <X11/Xlib.h>
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
After some online research, I read that X11 is no longer included with OSX 10.9, so I installed XQuartz in the hope that it could serve as a replacement.
Xquartz does include 'Xlib.h', but it's filepath is /opt/X11/include/X11/Xlib.h, and the setup script doesn't find it here. I've tried several ways to fix this.
The error process specifically comes up in the build part of the install. I've tried using
python3 setup.py config --include-dirs /opt/X11/include
as well as
python3 setup.py config --include-dirs /opt/X11/include/X11
but get the same error.
I read online that installing the Xcode command line tools could fix this problem. I'm ~95% sure I had the command line tools before, but I tried to get them again anyway with 'xcode-select --install' only to be told that 'Can't install the software because it is currently unavailable from the Software Update server'. "No problem," I tell myself and just download the package from the Mac developers site. It doesn't help anything.
I've also tried symlinking with the command
sudo ln -s /usr/include/X11 /opt/X11/include/X11
Of everything I've tried, I understand this the least, do I could very well be doing the symlink command wrong.
Two other commands I tried are
export C_INCLUDE_PATH=/opt/X11/include
export CPLUS_INCLUDE_PATH=/opt/X11/include
Just looking at the documentation, I came across a debug option for the startup script.
The output of 'python3 setup.py build -g' is
running build
running build_py
running build_ext
building 'pygame.display' extension
/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -g -Ddarwin -I/Library/Frameworks/SDL.framework/Versions/Current/Headers -I/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -c src/display.c -o build/temp.macosx-10.6-intel-3.3/src/display.o
In file included from src/display.c:30:
/Library/Frameworks/SDL.framework/Versions/Current/Headers/SDL_syswm.h:58:10: fatal error: 'X11/Xlib.h' file not found
#include <X11/Xlib.h>
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
I'm not sure what else I can do, but I'd really appreciate some help with this. Thanks in advance!
setup.py build|install commands accepts CFLAGS and LDFLAGS compiler options, as far as I know.
Thus here, you can do
CFLAGS=-I/opt/X11/include python3 setup.py install
You probably need the LDFLAGS as well; I'm not sure about the exact location, but something like
CFLAGS=-I/opt/X11/include LDFLAGS=-L/opt/X11/lib python3 setup.py install
seems logical.
After that, you may have to set your DYLD_LIBRARY_PATH to point to your X11 libraries. In case your installation proceeds but your PyGame script won't run and complains about not finding X11 libraries. Thus
export DYLD_LIBRARY_PATH=/opt/X11/lib
As for the symbolic link you tried to create: it's the wrong way 'round: the two paths should have been switched, so I guess you got an error because /opt/X11/include/X11 already exists. But with the above, no symlink is necessary.
I ran into a similar error when trying to install pygame into my PyCharm project's virtualenv. I solved it by cding to the SDL Framework directory where the error originated and copying the X11 header files into it:
$ cd /Library/Frameworks/SDL.framework/Versions/Current/Headers
$ cp -R /opt/X11/include/X11 ./
then $ pip install pygame ran without errors.
I can't seem to be able to get the python ldap module installed on my OS X Mavericks 10.9.1 machine.
Kernel details:
uname -a
Darwin 13.0.0 Darwin Kernel Version 13.0.0: Thu Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64 x86_64
I tried what was suggested here:
http://projects.skurfer.com/posts/2011/python_ldap_lion/
But when I try to use pip I get a different error
Modules/LDAPObject.c:18:10: fatal error: 'sasl.h' file not found
*#include sasl.h
I also tried what was suggested here:
python-ldap OS X 10.6 and Python 2.6
But with the same error.
I am hoping someone could help me out here.
using pieces from both #hharnisc and #mick-t answers.
pip install python-ldap \
--global-option=build_ext \
--global-option="-I$(xcrun --show-sdk-path)/usr/include/sasl"
A workaround
/usr/include appears to have moved
$ xcrun --show-sdk-path
$ sudo ln -s <the_path_from_above_command>/usr/include /usr/include
Now run pip install!
In my particular case, I couldn't simply use the pip arguments noted in other answers because I'm using it with tox to install dependencies from a requirements.txt file, and I need my tox.ini to remain compatible with non-Mac environments.
I was able to resolve this in much simpler fashion: exporting CFLAGS such that it adds an include path to the sasl headers already installed by Xcode:
$ pip install python-ldap
...
building '_ldap' extension
creating build/temp.macosx-10.10-x86_64-2.7
creating build/temp.macosx-10.10-x86_64-2.7/Modules
clang -fno-strict-aliasing -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DHAVE_SASL -DHAVE_TLS -DHAVE_LIBLDAP_R -DHAVE_LIBLDAP_R -DLDAPMODULE_VERSION=2.4.19 -IModules -I/opt/openldap-RE24/include -I/usr/include/sasl -I/usr/include -I/Users/bc/.pyenv/versions/2.7.10/include/python2.7 -c Modules/LDAPObject.c -o build/temp.macosx-10.10-x86_64-2.7/Modules/LDAPObject.o
Modules/LDAPObject.c:18:10: fatal error: 'sasl.h' file not found
#include <sasl.h>
^
1 error generated.
error: command 'clang' failed with exit status 1
$ export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include/sasl"
$ pip install python-ldap
...
Successfully installed python-ldap-2.4.19
Depending on whether or not you use any userspace-friendly Python tools (I use pyenv), you may have to prefix your pip commands with sudo.
I had the same problem. I'm using Macports on my Mac and I have cyrus-sasl2 installed which provides sasl.h in /opt/local/include/sasl/. You can pass options to build_ext using pip's global-option argument. To pass the include PATH to /opt/local/include/sasl/sasl.h run pip like this:
pip install python-ldap --global-option=build_ext --global-option="-I/opt/local/include/sasl"
Alternatively you could point it to whatever the output from xcrun --show-sdk-path provides. On my box that's:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
Then you need to determine the PATH to the sasl header files. For me that's:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/sasl/
Let me know if that helps or you need a hand.
I used a combination of posts I found about this problem (including this one) to eventually come up with this (copied from a larger script):
export XC_SDK=$(xcrun --show-sdk-path)
export USR_INC=$XC_SDK/usr/include
export PATH=$USR_INC:$PATH
echo "installing python-ldap"
ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install python-ldap
You can test it with python -c "import ldap"
The main reason I didn't follow the advice of #hharnisc was that on my local machine /usr/local had not moved, so I just temporarily put $XC_SDK before it on the path, and that seems to work.
some sources:
how to install PIL on Macosx 10.9?
I got this error when running buildout.
I fixed it, first finding the sasl.h file:
mdfind -name sasl.h
then defining the corresponding CFLAGS environment variable:
export CFLAGS="-I/opt/local/include/sasl"
and finally running buildout again.
I am trying to install the python library pyinterval. It requires the crlibm C headers which I have installed without error, but seem to be at the root of the problem.
When I run:
$ sudo easy_install pyinterval
I get the following:
Searching for pyinterval
Reading http://pypi.python.org/simple/pyinterval/
Reading http://pyinterval.googlecode.com/
Best match: pyinterval 1.0b21
Downloading http://pypi.python.org/packages/source/p/pyinterval/pyinterval-1.0b21.tar.gz#md5=a65fe9855d3b6b0a9ddcc5b2f1e1e421
Processing pyinterval-1.0b21.tar.gz
Running pyinterval-1.0b21/setup.py -q bdist_egg --dist-dir /tmp/easy_install-K58WK9/pyinterval-1.0b21/egg-dist-tmp-Tp03Mb
ext/crlibmmodule.c: In function ‘crlibm_cospi_rn’:
ext/crlibmmodule.c:45:1: warning: implicit declaration of function ‘cospi_rn’
ext/crlibmmodule.c: In function ‘crlibm_cospi_ru’:
ext/crlibmmodule.c:45:1: warning: implicit declaration of function ‘cospi_ru’
...
ext/crlibmmodule.c: In function ‘crlibm_log1p_rz’:
ext/crlibmmodule.c:59:1: warning: implicit declaration of function ‘log1p_rz’
/usr/bin/ld: /usr/local/lib/libcrlibm.a(addition_scs.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/libcrlibm.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
error: Setup script exited with error: command 'gcc' failed with exit status 1
I have called:
$ sudo apt-get install python-all-dev
and that did not fix the problem.
In case anyone still has problems with this, you need to set "CPPFLAGS = -fPIC" in "scs_lib/Makefile", not "./Makefile".
I had been having the same problem, and I found the issue was with crlibm (the flags -fPIC are mentioned in the error).
I ended up installing crlibm from source, and was able to get the the installation working. Once I'd run ./configure for crlibm, I manually edited the Makefile by changing the line "CPPFLAGS = " to "CPPFLAGS = -fPIC". From here I ran "make", "make install", and then "sudo easy_install pyinterval".
I'll add that I'm not 100% confident in this solution, and its not very elegant. I'm not sure about the technical details of the -fPIC flag, and what effect it really has.
But it does work.
It seems to work ok for me:
wget http://lipforge.ens-lyon.fr/frs/download.php/152/crlibm-1.0beta3.tar.gz
tar vfxz crlibm-1.0beta3.tar.gz
cd crlibm-1.0beta3
export CPPFLAGS=-fPIC
./configure
make
sudo make install
sudo pip install pyinterval
python
>>> from interval import *
Thanks for the answer.
I just went through installing pyinterval on ubuntu 12.10 using the above suggestions for crlibm.
I tried adding -fPIC to CPPFLAGS in the scs_lib Makefile but it didn't work. I think later versions of crlibm (I am using 1.0beta-4) require that crlibm_private.o is also compiled with -fPIC so the flag needs to be added to CPPFLAGS in the Makefile of the base directory