I'm trying to teach myself Cython but having problems accessing numpy. The problems seem to be when I use 'cimport'.
For example when importing the following function:
cimport numpy
def cy_sum(x):
cdef numpy.ndarray[int, ndim=1] arr = x
cdef int i, s = 0
for i in range(arr.shape[0]):
s += arr[i]
return s
I get error:
/Users/Daniel/.pyxbld/temp.macosx-10.6-x86_64-2.7/pyrex/test.c:314:10: fatal error: 'numpy/arrayobject.h' file not found
include "numpy/arrayobject.h"
1 error generated.
Any simple instructions on how to resolve this would be much appreciated!
Ok, as has been pointed out above, similar questions have been asked before.
Eg: Make distutils look for numpy header files in the correct place.
I got my code to work by using a setup file with the line
include_dirs = [np.get_include()],
As in:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy as np
ext = [Extension( "cy_test", sources=["cy_test.pyx"] )]
setup(
name = "testing",
cmdclass={'build_ext' : build_ext},
include_dirs = [np.get_include()],
ext_modules=ext
)
I haven't yet tried using pyximport so I'm not sure whether another fix is required for that.
Related
I know this might seem like a duplicated question, but I really could not find what I'm doing wrong... I wrote a .pyx file in order to compile it into a .pyd with cython. Long story short, it compiles my file just fine and creates a .pyd file. However, when I try to import that .pyd file I get an error saying No module named: "name_of_module". Note that this is my first time trying cython...
I am using venv with python3.9 on windows 10. I have cython installed along with minGW. To compile it into the .pyd file, I imply type in the command prompt in the same directory as the .pyx file:
python setup.py build_ext --inplace
Here is my setup.py file to cythonize my .pyx file:
from setuptools import setup, Extension
from Cython.Build import cythonize
extensions = [Extension('negamax_cy', ['negamax_cy.pyx'])]
setup(ext_modules=cythonize(extensions, language_level=3))
This is my .pyx file:
from connect4.constants import COLS, ROWS
from .position import Position
cdef int COLUMN_ORDER[7]
cdef int x
for x in range(COLS):
COLUMN_ORDER.append(COLS // 2 + (1 - 2 * (x % 2)) * (1 + x) // 2)
cpdef int negamax(pos, int depth, int alpha, int beta):
if pos.can_win(): # Check if current player can win this move
return 1000 - pos.moves*2
cdef long next_move = pos.possible_non_loosing_moves()
if next_move == 0: # Check for moves which are not losing moves
return -1000 + pos.moves # If we have 2 or more forcing moves we lose
if depth == 0: # Check if we have reached max depth
return 0
if pos.moves == ROWS * COLS: # Check for a draw game
return 0
cdef int col = 0
for col in COLUMN_ORDER[col]:
if next_move & Position.column_mask(col):
pos_cp = Position(position=pos)
pos_cp.play_col(col)
score = -negamax(pos_cp, depth - 1, -beta, -alpha)
if score >= beta:
return score
alpha = max(alpha, score)
return alpha
My project structure is as follow (I am trying to do a connect4 game with an A.I. in pygame):
connect4
/venv
/ai
__init__.py
setup.py
file_where_pyd_is_imported.py
negamax_cy.pyx
negamax_cy.pyd
negamax_cy.c
/connect4
__init__.py
other_files.py
__init__.py
main.py
Note that main.py imports file_where_pyd_is_imported.py
When I import I simply type:
import negamax_cy
this is the error I get:
Traceback (most recent call last):
File "D:\Users\dalla\Documents\Coding Projects\python\games\connect4\main.py", line 5, in <module>
from ai.negamax import mp_best_move, best_move, print_avg
File "D:\Users\dalla\Documents\Coding Projects\python\games\connect4\ai\negamax.py", line 7, in <module>
import negamax_cy
ModuleNotFoundError: No module named 'negamax_cy'
As I said I don't know what is wrong. Maybe it has to do with my project structure or something, or my setup.py file, but I am unsure... If anyone has an idea let me know.
Well turns out I'm really stupid and in python3 I have to upload like this:
from . import negamax_cy
Sorry for wasting anyone's time...
Below is my setup file to setup a python wrapper. The issue I am having is that in my c code I am writing is making calls to clock_gettime for profiling reasons. The thing is when I try to import the module I get the following: error undefined symbol: clock_gettime. I understand that I need to compile with -lrt, but obviously my compiler is not getting that flag. What am I doing wrong?
from distutils.core import setup, Extension
import os
module1 = Extension('relaymod',
extra_compile_args = ["-lrt"], #flag so compiler links to realtime lib
sources=['relaymodule.c']
)
setup (name = 'relaymod',
version = '1.0',
description = "CTec Relay Board controller",
author='Richard Kelly',
url='site',
ext_modules=[module1])
EDIT:
looking at the distutils.core documentation I believe I need to set extra_link_args Below is my new change, but I am now getting this error: NameError: name 'extra_link_args' is not defined
EDIT2: ok the code below is now working. Had a few things going on. after I removed the build folder and rebuilt this worked.
from distutils.core import setup, Extension
import os
module1 = Extension('relaymod',
extra_link_args=["-lrt"],
sources=['relaymodule.c']
)
setup (name = 'relaymod',
version = '1.0',
description = "CTec Relay Board controller",
author='Richard Kelly',
url='site',
ext_modules=[module1])
you are missing the equal (=) you need to say extra_link_args=[your list of link args]
Updated per the comments:
delete the build folder before retrying
I have a working cpp project which builds fine with cmake. Now I need to create a Python wrapper for the same. So I chose cython to bridge the gap between C++ and Python.
Instead of writing the CMakeLists.txt logic in pyx file, I tried compiling the library with cmake itself and later wrap the Cython wrapper around it. Following is the setup.py file:
import os
import sys
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
try:
build_dir = os.path.join(os.getcwd(), "build")
print("building bobcore.a")
if not os.path.exists(build_dir):
os.makedirs(build_dir)
if not os.path.exists(build_dir):
raise Exception("Not able to create `build` directory")
os.chdir(build_dir)
assert os.system("cmake ..") == 0
assert os.system("make -j4") == 0
os.chdir(os.path.abspath(os.path.join(os.getcwd(), os.pardir)))
except:
if not os.path.exists("build/bobcli"):
print("Error building external library, please create libsample.a manually.")
sys.exit(1)
_ext_modules = cythonize([
Extension("call_core",
sources=["call_core.pyx"],
language="c++",
extra_compile_args=["-std=c++11", "-lpthread", "-lz", "-ldl"],
extra_link_args=["-std=c++11"],
include_dirs=[os.path.join(os.getcwd(), "src")], # path to .h file(s)
library_dirs=[os.path.join(os.getcwd(), "build")], # path to .a or .so file(s)
libraries=["bobcli"] # name of pre-built .so file.
)
])
setup(
name='Demos',
ext_modules=_ext_modules,
)
Here is the call_core.pyx file:
cdef extern from "src/api/Api.h" namespace "cli":
cdef cppclass Api:
int getVal()
cdef Api *api = new Api()
def getHelp():
return api.getVal()
del api
Now this file works fine if I implement the getVal() method in the header file itself. But as soon as I move the implementation to .cpp file, Cython compiler shows the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: /usr/local/lib/python2.7/dist-packages/call_core.so: undefined symbol: _ZN3cli9Api6getValEv
NOTE: The above snippets work perfectly for the functions with their implementation in the .h files.
I try run v8 (google javascript engine) in python. Try do this like this example https://developers.google.com/v8/get_started:
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext
import os
from distutils.sysconfig import get_config_vars
(opt,) = get_config_vars('OPT')
os.environ['OPT'] = " ".join(
flag for flag in opt.split() if flag != '-Wstrict-prototypes'
)
setup(
name = 'helloworld',
ext_modules = cythonize(
Extension(
"helloworld",
["helloworld.pyx"],
language = "c++",
libraries = ["v8"],
library_dirs = ["v8"],
include_dirs = ["v8/include"],
extra_compile_args = ["-std=c++11"],
extra_link_args = ["-std=c++11"],
cmdclass = {'build_ext': build_ext}
)
)
)
helloworld.pyx
from libcpp cimport bool
cdef extern from "v8.h":
cdef bool c_initx "v8::V8::InitializeICU"()
cdef class Script:
def test(self):
c_initx()
return "wow"
Compilation process is OK but i can't import helloworld:
Traceback (most recent call last):
File "test.py", line 1, in <module>
import helloworld
ImportError: /opt/cython/helloworld.so: undefined symbol: _ZN2v82V813InitializeICUEPKc
I use latest v8 version from github https://github.com/v8/v8
I'd like to build a static Cython library using distutils. I don't care about having it be a true Python extension module that can be import'ed. I just want to compile the code and put the objects in a static library. The code to create a dynamic library is very simple,
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
cmdclass = {'build_ext':build_ext},
ext_modules = [Extension("test",["test.pyx"])]
)
Is there a simple way to make it static instead?
Distutils is very limited and not set up for static builds. I would advise you to use something else to compile the static library part of your project.
If your use case is to call into Cython code from other C code, then you want to use the public or api declarations along with your cdef declared functions and variables in your Cython code. Cython will allow the so-declared objects to be called from external C code, and it will generate a .h file alongside the .c file for you.
Fyi, this works using numpy distutils, but obviously is nowhere near the simplicity or probably portability of the original code for a shared library,
from Cython.Compiler.Main import compile
from numpy.distutils.misc_util import Configuration
compile('test.pyx')
config = Configuration(...)
config.add_installed_library('test',
['test.c'],
'test',
{'include_dirs':[get_python_inc()]})
Assuming that you have a sources, include_dirs and build_dir in your setup.py, this is how you can build a static library
from distutils.ccompiler import new_compiler
from sysconfig import get_paths
import os
project_name = "slimey_project"
source = ['source1.c']
include_dirs = ['include']
build_dir = os.path.join(os.path.dirname(__file__), 'build')
class StaticLib(Command):
description = 'build static lib'
user_options = [] # do not remove, needs to be stubbed out!
python_info = get_paths()
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
# Create compiler with default options
c = new_compiler()
# Optionally add include directories etc.
for d in include_dirs:
c.add_include_dir(d)
c.add_include_dir(self.python_info['include'])
# Compile into .o files
objects = c.compile(sources)
# Create static or shared library
c.create_static_lib(objects, project_name, output_dir=build_dir)
Source: https://gist.github.com/udnaan/d549950a33fd82d13f9e6ba4aae82964