I would like to compare Python with Cython in term of time execution, so I wrote two files:
fac.py
def factorial(n):
if n >= 1:
return n*factorial(n - 1)
return 1
fastfac.pyx
cpdef long fastfactorial(long n):
if n>= 1:
return n*fastfactorial(n - 1)
return 1
Then I wrote a setup file:
setup.py
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules = cythonize('fastfac.pyx'))
From the Powershell I executed the two commands:
pip install Cython
python setup.py build_ext --inplace
From the second command I get the following message:
Compiling fastfac.pyx because it changed.
[1/1] Cythonizing fastfac.pyx
C:\Users\.....\venv\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\.....\fastfac.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'fastfac' extension
error: Unable to find vcvarsall.bat
However I tried to make a comparison, so I wrote the file:
comparison.py
from fastfac import fastfactorial
from fac import factorial
from timeit import timeit
print(timeit('fastfactorial(20)', globals = globals(), number = 10000))
print(timeit('factorial(20)', globals = globals(), number = 10000))
When I run it, I get this error message:
Traceback (most recent call last):
File "C:/Users/...../comparison.py", line 1, in <module>
from fastfac import fastfactorial
ModuleNotFoundError: No module named 'fastfac'
It seems that in the file python.pyx the definition cpdef long fastfactorial(long n) is not recognized as a regular function definition but as a syntax error; in fact, if I try to run that file I get the error message:
File "C:/Users/...../fastfac.pyx", line 1
cpdef long fastfactorial(long n):
^
SyntaxError: invalid syntax
How can I solve? How can I correctly define a cpdef inside a .pyx file?
what am I missing?
The problem is not your definition of fastfactorial, it is the fact that your setup.py exited with an error and, presumably, without compiling fastfac into a c library. In general, you should always fix such errors.
Your error appears to be happening because you don't have a Microsoft Visual C++ compiler installed. You can follow the instructions in this answer to choose a version of Visual C++ to install.
You also have a warning about the language_level not being set. You shouldn't ignore warnings either so it is worth explicitly stating the level in your setup.py.
setup(ext_modules=cythonize('fastfac.pyx'), language_level=3)
Related
I'm having some trouble with cython. I recently found out about it and decided to use it in a project, but for some reason I'm unable to import the module/function that I've just compiled.
Here is what my sum.pyx file looks like:
def add(x, y):
return x+y
And this is my setup.py file:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules = cythonize('image_converter.pyx'))
When I run python setup.py build_ext --inplace, the output shows this error:
Compiling sum.pyx because it changed.
[1/1] Cythonizing sum.pyx
C:\Users\User-name\AppData\Local\Programs\Python\Python39\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\User-name\Documents\Cython\sum.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
running build_ext
building 'sum' extension
error: Unable to find vcvarsall.bat
Only one file was created called sum.c, but I'm aware that there should be some .so file created in a new directory but I couldn't find any, so I just continued and created the test.py file to see if the cython file worked:
import sum
print(sum.add(5,6))
But if I run this I get the error:
ModuleNotFoundError: No module named 'sum'
I am using Python 3.9.7 and Visual Studio Code, and I'm using the g++ compiler (MSYS2).
Thanks in advance.
I'm trying to convert a simple, and stupid, script from Python to C:
#script.pyx
import os
import sys
import numpy as np
from datetime import datetime
x = 5
So, following the tutorial I created setup.py file:
from distutils.core import setup
from Cython.Build import cythonize
setup(ext_modules = cythonize('script.pyx'))
When I try to run:
python setup.py build_ext --inplace
I have a correct c file like output but I have also this error:
tree = Parsing.p_module(s, pxd, full_module_name)
error: [WinError 2] Cannot find the specified file
From this result I got a doubt: with cython can I convert any Python script? Even the most complex? What are the limits? If I have a file that imports library and / or other classes?
Thank you
Sometimes we need to fully specify where the script is, instead of just typing the name of the file, you must enter the full path, or you gotta cd to the right directory,
"/home/userrr/Desktop/folder45/script.pyx"
Instead of "script.pyx"
Of it still does not work, uninstall python, re install python and the reinstall Cython, another issue could be the pyx source code, pyx source code should be different from Normal python language, check the cdef function, cdef Cythonize Language also
I haven't been able to find much about this, but when attempting to compile one of my python scripts with cython, it gives me this error:
Error compiling Cython file:
------------------------------------------------------------
...
import traceback #line:24
import bcrypt #line:25
Y =str (os .path .dirname (os .path .abspath (__file__ )))#line:28
IH =open #line:29
IA =str #line:30
IK =print #line:31
^
------------------------------------------------------------
headlessobfu.pyx:29:4: Expected an identifier or literal
Traceback (most recent call last):
File "setup.py", line 5, in <module>
ext_modules = cythonize("headlessobfu.pyx")
File "C:\Users\justi\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Cython\Build\Dependencies.py", line 1026, in cythonize
cythonize_one(*args)
File "C:\Users\justi\AppData\Local\Programs\Python\Python36-32\lib\site-packages\Cython\Build\Dependencies.py", line 1146, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: headlessobfu.pyx
I haven't been able to find the exact cause of this error. I was able to get simple scripts to compile just fine. The python runs just fine on it's own. Is it a problem with my python formatting?
Here is the command line argument I am running:
py setup.py build_ext --inplace
If anyone has a solution please let me know. Thanks.
By default Cython assumes Python 2 syntax, even when you're using Python 3 (edit 2021: Cython 3 will change this and largely assume Python 3 syntax by default). Here's the minimum, complete example you should have created:
cy_print_test.pyx
x = print
If I compile it with Cython
cython cy_print_test.pyx
I get the error message that you do. print is not an identifier or literal because under Python 2 syntax it is a special statement.
However, if I compile it with Cython set to use Python 3 syntax:
cython -3 cy_print_test.pyx
it works fine - under Python 3 syntax print is a function and so this makes perfect sense.
Alternative ways of getting Python 3 syntax would be to add the following line to the start of your .pyx file
#cython: language_level=3
or to specify it as a compiler directive in setup.py:
from distutils.core import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize('cy_print_test.pyx', compiler_directives={'language_level': 3}),
)
(note that your setup.py should also form part of the minimum, complete example in your question)
Ok so this has to do with cython and python formatting. The code has to be totally obfuscated and with no errors or the compiler won't work. So work through the compiler and fix each error as it arises. Hope this helps.
we need to install the required dependencies, and I guess you are missing out the python3-dev.
Try these commands on terminal (cmd)
$ pip3 install cython
$ sudo apt install update
$ sudo apt install build-essential
$ sudo apt install python3-dev
then run the $ python3 setup.py build_ext --inplace again. it should work.
Ultimately I am trying to install CVXPY for Python 2.7. CVXPY has a few requirements including CVXOPT. I have tried two approaches, installing from Gohlke's website, where he hosts Windows Binaries for Python Extensions: http://www.lfd.uci.edu/~gohlke/pythonlibs/ and building from source myself.
When using the binaries provided from Gohlke, I get the following error when trying to import in python:
import cvxpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
...
File "C:\Python27\Lib\site-packages\cvxpy\interface\cvxopt_interface\dense_matrix_interface.py", line 26, in <module>
import cvxopt
File "C:\Python27\Lib\site-packages\cvxopt\__init__.py", line 32, in <module>
import cvxopt.base
ImportError: DLL load failed: The specified module could not be found.
I have checked that the base.pyd file is present in cvxopt/
When building from source I followed the instructions from the site and when eventually building CVXOPT with python setup.py build --compiler=mingw32 I got the following error:
C:\MinGW\bin\gcc.exe -mdll -O -Wall -Isrc/C/SuiteSparse/AMD/Include -Isrc/C/SuiteSparse/SuiteSparse_config -IC:\Python27\include -IC:\Python27\PC -c src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c -o build\temp.win32-2.7\Release\src\c\suitesparse\suitesparse_config\suitesparse_config.o
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c: In function 'SuiteSparse_tic':
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c:358:21: error: storage size of 't' isn't known
struct timespec t ;
^
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c:359:5: warning: implicit declaration of function 'clock_gettime' [-Wimplicit-function-declaration]
clock_gettime (CLOCK_MONOTONIC, &t) ;
^
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c:359:20: error: 'CLOCK_MONOTONIC' undeclared (first use in this function)
clock_gettime (CLOCK_MONOTONIC, &t) ;
^
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c:359:20: note: each undeclared identifier is reported only once for each function it appears in
src/C/SuiteSparse/SuiteSparse_config/SuiteSparse_config.c:358:21: warning: unused variable 't' [-Wunused-variable]
struct timespec t ;
^
error: command 'C:\\MinGW\\bin\\gcc.exe' failed with exit status 1
I did some research into this and error: storage size of 't' isn't known but i can't determine if it's CVXOPT's fault or Windows?
I'm using Python2.7 on Windows 10. Any suggestions or clarity on some of these issues would be great. Like I mentioned, just trying to get CVXPY working, but seem to be hung up on CVXOPT. Thanks!
UPDATE
I was able to get a working installation of CVXOPT by uninstalling the current numpy version I had and downloading / installing the numpy-mkl whl from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy. I then installed CVXOPT and CVXPY from the site as well. Finally was able to get a successful import cvxpy without the cvxopt.base DLL error.
Still would be curious about the compilation error I was having.
thank you for this. My 2 cents on this:
With Anaconda on Windows 7 I just used the omnia repo conda install -c omnia cvxopt. It looks like it's working. Will update here if it breaks.
I have downloaded Pandas source and now trying to debug it.
I modified Makefile:
sparse: pandas/src/sparse.pyx
python-dbg setup.py build_ext --inplace --pyrex-gdb
build: clean_pyc
python-dbg setup.py build_ext --inplace --pyrex-gdb
develop: build
-python-dbg setup.py develop --pyrex-gdb
Also I have a very simple script:
from numpy import asarray
from pandas import algos
v = [4171.0, 0.0]
expAverage = algos.ewma(asarray(v), 50, 1)
print expAverage
When I try to run it with python-dbg test1.py, this is what I get:
/tmp/1/pandas/pandas/hashtable.so: undefined symbol: Py_InitModule4_64
Traceback (most recent call last):
File "test1.py", line 2, in <module>
from pandas import algos
File "/tmp/1/pandas/pandas/__init__.py", line 6, in <module>
from . import hashtable, tslib, lib
ImportError: /tmp/1/pandas/pandas/hashtable.so: undefined symbol: Py_InitModule4_64
[94423 refs]
What is wrong?
Obviously, at least one of your (C) extensions that is being loaded has not been compiled with debug info in a way that python-dbg can use.
This explanation has the details:
http://hustoknow.blogspot.co.uk/2013/06/why-your-python-program-cant-start-when.html
To me it seems like --with-pydebug flag is not equivalent / triggers the same actions as --pyrex-gdb. BTW, it seems that --pyrex-gdb has been renamed to --cython-gdb.
Can you use cygdb or cython --gdb instead? It seems like the flag that you're using has been reported not to work: https://groups.google.com/forum/#!topic/cython-users/K6sjhzUX5JA