I'm trying to run Hadoopy, which has a file _main.pyx, and import _main is failing with module not found in __init__.py.
I'm trying to run this on OS X w/ standard python 2.7.
Add this code before you try to import _main:
import pyximport
pyximport.install()
Note that pyximport is part of Cython, so you'll have to install that if it isn't already.
You need to make sure you have followed all steps:
Install the Cython package using pip
pip install Cython
Create a Cython file bbox.pyx
cimport cython
import numpy as np
cimport numpy as np
DTYPE = np.float32
ctypedef np.float32_t DTYPE_t
#cython.boundscheck(False)
def compare_bboxes(
np.ndarray[DTYPE_t, ndim=2] boxes1,
np.ndarray[DTYPE_t, ndim=2] boxes2):
...
Create setup.py in the same directory
from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy
package = Extension('bbox', ['bbox.pyx'], include_dirs=[numpy.get_include()])
setup(ext_modules=cythonize([package]))
Build the Cython
python3 setup.py build_ext --inplace
Create your main python script run.py in the same directory
import pyximport
pyximport.install(setup_args={"script_args" : ["--verbose"]})
from bbox import compare_bboxes
def main(args):
boxes1 = args.boxes1
boxes2 = args.boxes2
result = compare_bboxes(boxes1, boxes2)
Run your main script in the same directory
python run.py
Related
I apply ray with Cython. It shows an error:
TypeError: The #ray.remote decorator must be applied to either a function or to a class.
How to solve this problem? Thanks.
The setup.py:
from distutils.core import setup, Extension
from Cython.Build import cythonize
import numpy
setup(
ext_modules=cythonize("ray_cython.pyx"),
include_dirs=[numpy.get_include()]
)
setup(
ext_modules=[
Extension("ray_cython", ["ray_cython.c"],
include_dirs=[numpy.get_include()]),
],
)
# compile by python setup.py build_ext --inplace
# Then run compare.py
The ray_cython.pyx file:
from numpy import linalg as LA
import time
#Import package
import ray
cimport numpy as np
np.import_array()
DTYPE = np.int
ctypedef np.int_t DTYPE_t
#ray.remote
def myfunction():
...
https://docs.ray.io/en/latest/ray-core/advanced.html#cython-code-in-ray
You must include the following two lines at the top of any *.pyx file:
#!python
# cython: embedsignature=True, binding=True
You cannot decorate Cython functions within a *.pyx file (there are ways around this, but creates a leaky abstraction between Cython and Python that would be very challenging to support generally). Instead, prefer the following in your Python code:
some_cython_func = ray.remote(some_cython_module.some_cython_func)
I'm working on a python code. This code uses a library that is written in cython.
when I run, I face this error:
I use python3.7. I've searched a lot but I couldn't find a useful way to solve it. I think that visual cpp has not been matched with python. First I had visual cpp 2017 but didn't work and I uninstalled it. After that I installed only visualcppbuildtools but the result didn't change. Can anyone help me?
(Also I added the below part to my code but the result didn't change.)
This is setup.py :
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
ext_modules=[
Extension("lib2",
["lib2.pyx"],
language="c++",
libraries=["m"],
extra_compile_args = ["-std=c++11", "-O3", "-ffast-math", "-
march=native", "-fopenmp" ],
extra_link_args=['-fopenmp'],
include_dirs=[numpy.get_include()]
)
]
setup(
name = "lib2",
cmdclass = {"build_ext": build_ext},
ext_modules = ext_modules,
include_dirs=[numpy.get_include()]
)
this the main.py:
import sys
sys.path.append('E:\EDU\Hipp\septo-hippocampal-model-master\septo-
hippocampal-model-master\cython_code')
import pyximport
pyximport.install()
import lib2 as lib
and the last one is lib2.pyx:
from libc.math cimport exp, cos
from libcpp.map cimport map
from libcpp.pair cimport pair
from libcpp.string cimport string
from libcpp.vector cimport vector
from libcpp cimport bool
from cython.operator cimport dereference, preincrement
import numpy as np
cimport numpy as np
from libcpp.queue cimport queue
from cython.parallel cimport parallel, prange
cimport cython
By those simple codes I still receive the above error.
Here's my question in learning sklearn.
There are some cython code in sklearn , for example:
sklearn/tree/
_utils.pyd
_tree.pyd
_test.pyx
the _test.pyx includes code:
from ._utils cimport log
Now I want to complie _test.pyx to _test.pyd by using Cython.
But it doesn't work, the error is:
Error compiling Cython file:
------------------------------------------------------------
...
import numpy as np
cimport numpy as np
np.import_array()
from ._utils cimport log
^
------------------------------------------------------------
sklearn\tree\_test.pyx:30:0: 'sklearn\tree\_utils.pxd' not found
Here is my compilation code in cythonbuild.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
import numpy
extensions = [
Extension("tree", ["_criterion.pyx"],
include_dirs = [numpy.get_include()],
)
]
setup(
name = "test",
ext_modules = cythonize(extensions),
)
Then I ran the command in cmd:
python cythonbuild build_ext --inplace
How can I fix this problem?
I ran into the problem described here (What is this import_umath function?) and wanted to know if there is a fix for it? I have the exact same case where compiling Cython code that uses numpy with the following code:
import numpy as np
cimport numpy as np
np.import_array()
generates many warnings about _import_umath not being used:
/usr/local/lib/python2.7/dist-packages/numpy-1.6.2-py2.7-linux-x86_64.egg/numpy/core/include/numpy/__ufunc_api.h:226:1: warning: ‘_import_umath’ defined but not used [-Wunused-function]
removing np.import_array() does not change the result. Like one of the posters suggested in the above thread, I tried adding this in my .pxd/.pyx file:
cdef extern from *:
import_umath()
this also made no difference. How can this warnings be eliminated?
You can pass arguments to the C compiler using the keyword extra_compile_args in your setup.py. For example, this will not generate warnings:
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
import numpy
extensions=[
Extension("abc",
["abc.pyx"],
include_dirs=[numpy.get_include()],
extra_compile_args=["-w"]
)
]
setup(
ext_modules=cythonize(extensions),
)
In Cython Tricks and Tips they explain that you need:
cdef extern from *:
pass
when you import extern packages. I already needed this trick to write a wrapper, maybe it will work for you too...
I am trying to access the new parallel features of Cython 0.15 (using
Cython 0.15.1). However, if I try this minimal example (testp.py), taken from http://docs.cython.org/src/userguide/parallelism.html:
from cython.parallel import prange, parallel, threadid
cdef int i
cdef int sum = 0
for i in prange(n, nogil=True):
sum += i
print sum
with this setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
ext = Extension("testp", ["testp.pyx"], include_dirs=[numpy.get_include()],
extra_compile_args=['-fopenmp'], extra_link_args ['-fopenmp'])
setup(ext_modules=[ext], cmdclass={'build_ext': build_ext})
when I import testp, Python tells me: ImportError: No module named
parallel. And in fact, if I browse the Cython package in the
site-packages, I cannot find any file or directory that is called
parallel. But I thought it should be included somewhere in the
release? Could someone please clarify for a confused user?
I'm using Cython 0.15+
cython.parallel exists in Shadow.py:
import sys
sys.modules['cython.parallel'] = CythonDotParallel()
And the Shadow.py can be located in your Python's dist-packages directory like /usr/local/lib/python2.6/dist-packages/ in Linux
You can check all of your python modules in python command-line using:
>>> help('modules')
And then try to install/reinstall cython using easy_install or pip.