I have a C++ function which returns a pointer double** - a high dimensional matrix in particular - and I'd like to wrap it to some python code using Cython. How should I act?
Here an example with a function with a double* pointer for sake of simplicity.
My C++ fib.cpp code:
double add(double a, double b)
{
return a+b;
}
double p[]= {1,2,3,4};
double* mult(double a)
{
p[0]=p[0]*a;
p[1]=p[1]*a;
return p;
}
Then there is the fib.hpp file:
double add(double a,double b);
double* mult(double a);
Then the pxd file fib.pxd:
cdef extern from "fib.hpp":
double add(double a,double b);
double* mult(double a);
In the end the pyx file
# distutils: language = c++
# distutils: sources = fib.cpp
cimport fib
def add(a,b):
return fib.add(a,b)
def mult(a): # dropping these lines
return fib.mult(a) # the code works without the double* function
Everything is compiled with the rather standard setup.py :
from distutils.core import setup, Extension
from Cython.Build import cythonize
ext = Extension("fib2cpp",
sources=["fib.pyx", "fib.cpp"],
language="c++")
setup(name="fib",
ext_modules=cythonize(ext))
When I compile the code:
setup.py build_ext -if
Cannot convert 'double *' to python object.
When I try with a double** function I get the same error.
What should I do?
I have found a solution for the case double *. First of all, the file fib.pxd is useless.
Then we need a new fib.pyx file:
# distutils: language = c++
# distutils: sources = fib.cpp
import numpy as np
cimport numpy as cnp
cdef extern from "fib.hpp":
double fib(int n)
double add (double a, double b)
double* mult(double a)
def make_mult(double a):
cdef double[:] mv = <double[:4]> mult(a) # the 4 stands for
return np.asarray(mv) # the dimension of the array defined in fib.cpp
In the case of a function which returns a nrows*ncols matrix; such as double* make_mat(int nrows, int ncols) , the second line of the function make_mult has to be rewritten as:
cdef double[:,:] mv=<double[:nrows,:ncols]> make_mat(nrows, ncols)
Regrettably, If I have a function double** make_mat(int nrows, int ncols) which always returns a matrix, then the previous code rises the error :
Pointer base type does not match cython.array base type
Related
I am writing a Cython wrapper for the NAG C library.
In one of the header files from the NAG C library is the macros:
#define NAG_FREE(x) x04bdc((Pointer *)&(x))
Pointer is void*
x04bdc is:
extern void NAG_CALL x04bdc(Pointer *ptr);
NAG_FREE is the NAG library equivalent of free(), to free up memory.
Here is the extract from my lib_nag_integrate.pxd file:
cdef extern from "<nagx04.h>":
void x04bdc(Pointer *ptr)
x04bdc is a "fancy" free (malloc) routine. I cant access this code.
I then create a cdef function in my .pyx file:
cdef void NAG_FREE(double *x):
x04bdc(<Pointer *>&x)
Here i have type casted x to a double pointer, as that is what I am trying to free from memory, however the NAG library examples seem to use it for any type of pointer.
When running the python script which calls a cpdef function which eventually uses NAG_FREE, I get the following error:
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
If i comment out the NAG_FREE calls then it works fine, however NAG say it is necessary to use NAG_FREE.
The cdef function using NAG_FREE is:
cdef (double,double,Integer,Integer) dim1_fin_gen(lib_nag_integrate.NAG_D01SJC_FUN objfun,double a, double b,double epsabs, double epsrel,
Integer max_num_subint,Nag_User *comm,integration_out *output):
"""
:param f: user function
:type f: function
:param a: lower limit of integration
:type a: real float
:param b: upper limit of integration
:type b: real float
:param epsabs: user requested absolute error
:type epsabs: integer
:param epsrel: user requested relative error
:type epsrel: integer
:param max_num_subint: maximum number of subintervals
:type max_num_subint: real integer
:return: integration value of user function f
:rtype: real float
"""
cdef lib_nag_integrate.Nag_QuadProgress _qp
cdef lib_nag_integrate.NagError _fail
cdef double result
cdef double abserr
_fail.print = True
_fail.code = 0
_fail.errnum = 0
_fail.handler = NULL
lib_nag_integrate.d01sjc(objfun, a, b, epsabs, epsrel,
max_num_subint, &result, &abserr,
&_qp, comm, &_fail)
if _fail.code > 0 :
errorMessage = _fail.message
raise NagException(_fail.code,errorMessage)
print(_fail.message)
else:
output[0].result = result
output[0].abserr = abserr
output[0].fun_count = _qp.fun_count
output[0].num_subint = _qp.num_subint
NAG_FREE(_qp.sub_int_beg_pts)
NAG_FREE(_qp.sub_int_end_pts)
NAG_FREE(_qp.sub_int_result)
NAG_FREE(_qp.sub_int_error)
My libnag_integrate.pxd header file imports the following from the c library:
cdef extern from "<nag_types.h>":
ctypedef bint Nag_Boolean
ctypedef long Integer
ctypedef void* Pointer
ctypedef struct NagError:
int code
bint print "print"
char message[512]
Integer errnum
void (*handler)(char*,int,char*)
ctypedef struct Nag_User:
Pointer p
ctypedef struct Nag_QuadProgress:
Integer num_subint
Integer fun_count
double *sub_int_beg_pts
double *sub_int_end_pts
double *sub_int_result
double *sub_int_error
cdef extern from "<nagx04.h>":
(void*) NAG_ALLOC "x04bjc" (size_t size)
void x04bdc(Pointer *ptr)
cdef extern from "<nagd01.h>":
void d01sjc(NAG_D01SJC_FUN f, double a, double b,
double epsabs, double epsrel, Integer max_num_subint, double *result,
double *abserr, Nag_QuadProgress *qp, Nag_User *comm,
NagError *fail)
d01sjc is an integration routine which I cannot access. It allocates the memory of qp.sub_int_beg_pts etc inside.
I think I have a corrupt pointer causing the error. If so, where is it and how to I fix it?
many thanks
Upon further inspection of the structure '_qp'. The same error occurs when dereferencing e.g:
x = _qp.sub_int_end_pts[0]
so its the dereferencing of _qp which is causing the error.
The struct type Nag_QuadProgress is imported from its NAG header file into my .pxd as follows:
cdef extern from "<nag_types.h>":
ctypedef struct Nag_QuadProgress:
Integer num_subint
Integer fun_count
double *sub_int_beg_pts
double *sub_int_end_pts
double *sub_int_result
double *sub_int_error
Any ideas why dereferencing the pointers in this structure causes the error?
From Cython's point of view you're using NAG_FREE as a function, so that's what you should declare it as. It doesn't really matter that it's a really macro, and it certainly doesn't help to attempt to reimplement it.
cdef extern from "whatever_the_nag_header_is":
void NAG_FREE(Pointer x)
# or
void NAG_FREE(void *x)
# or
void NAG_FREE(...) # accepts anything - Cython doesn't attempt to match types
You may have to play around a bit with the type of the arguments to get it to work - I've suggested three options.
Really all you're aiming to do is to give Cython enough information that it can generate the right C code, and the right C code is NAG_FREE(your_variable), as if it's a function call.
With your code:
(<integration_out*>output)[0] suggests you're doing something very wrong. output is an integration_out pointer so why are you casting it? It either does nothing or introduces a potential error.
Despite claiming to return a C tuple type you actually don't bother to return anything.
I have a C function that takes a function as an input. I would like to to be able to call this from python. The final goal is to have the input function be defined in python but evaluated in C. At the moment I'm just trying to get a toy example working. So here's what I've done...
I have a header file that I call 'Test.h'. There is one function here that simply evaluates a function with two arguments.
#include <stdio.h>
double cEval( double ( *f )( double, double ), double n, double m ){
double k1 = f( n, m );
return k1;
};
I then wrote a cython file to wrap my function. This file is called 'Test.pyx'.
cdef extern from "Test.h":
double cEval( double ( *f )( double, double ), double n, double m )
def Eval( f, n, m ):
return cEval( f, n, m )
Finally, I wrote the setup file.
from distutils.core import setup, Extension
from Cython.Build import cythonize
setup(ext_modules=cythonize('Test.pyx'))
I then compile 'setup.py' using the command 'python setup.py build_ext --inplace', which returns the following error:
Compiling Test.pyx because it changed.
[1/1] Cythonizing Test.pyx
/usr/lib64/python2.7/site-packages/Cython/Compiler/Main.py:367: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /maybehome/jritchie/c_interaction_example/Fun_Input/Test.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "Test.h":
double cEval( double ( *f )( double, double ), double n, double m )
def Eval( f, n, m ):
return cEval( f, n, m ) ^
------------------------------------------------------------
Test.pyx:5:15: Cannot convert Python object to 'double (*)(double, double)'
Traceback (most recent call last):
File "setup.py", line 4, in <module>
setup(ext_modules=cythonize('Test.pyx'))
File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1086, in cythonize
cythonize_one(*args)
File "/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py", line 1209, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: Test.pyx
Does anyone know what I am doing wrong and how I can fix this?
I am fairly new to Cython, so this is probably fairly trivial, but I haven't been able to find the answer anywhere.
I've defined a struct type and I want to write a function that will initialize all the fields properly and return a pointer to the new struct.
from cpython.mem import PyMem_Malloc
ctypedef struct cell_t:
DTYPE_t[2] min_bounds
DTYPE_t[2] max_bounds
DTYPE_t size
bint is_leaf
cell_t * children[4]
DTYPE_t[2] center_of_mass
UINT32_t count
cdef cell_t * make_cell(DTYPE_t[2] min_bounds, DTYPE_t[2] max_bounds):
cdef cell_t * cell = <cell_t *>PyMem_Malloc(sizeof(cell_t)) # <- Fails here
if not cell:
MemoryError()
cell.min_bounds[:] = min_bounds
cell.max_bounds[:] = max_bounds
cell.size = min_bounds[0] - max_bounds[0]
cell.is_leaf = True
cell.center_of_mass[:] = [0, 0]
cell.count = 0
return cell
However, when I try to compile this, I get the following two errors during compilation:
cdef cell_t * make_cell(DTYPE_t[2] min_bounds, DTYPE_t[2] max_bounds):
cdef cell_t * cell = <cell_t *>PyMem_Malloc(sizeof(cell_t))
^
Casting temporary Python object to non-numeric non-Python type
------------------------------------------------------------
cdef cell_t * make_cell(DTYPE_t[2] min_bounds, DTYPE_t[2] max_bounds):
cdef cell_t * cell = <cell_t *>PyMem_Malloc(sizeof(cell_t))
^
Storing unsafe C derivative of temporary Python reference
------------------------------------------------------------
Now, I've looked all over, and from what I can gather, cell is actually stored in a temporary variable that gets deallocated at the end of the function.
Any help would be greatly appreciated.
cell.min_bounds = min_bounds
This doesn't do what you think it does (although I'm not 100% sure what it does do). You need to copy arrays element by element:
cell.min_bounds[0] = min_bounds[0]
cell.min_bounds[1] = min_bounds[1]
Same for max_bounds.
The line that I suspect is giving you that error message is:
cell.center_of_mass = [0, 0]
This is trying to assign a Python list to a C array (remembering that arrays and pointers are somewhat interchangeable in C), which doesn't make much sense. Again, you'd do
cell.center_of_mass[0] = 0
cell.center_of_mass[1] = 0
All this is largely consistent with the C behaviour that there aren't operators to copy whole arrays into each other, you need to copy element by element.
Edit:
However that's not your immediate problem. You haven't declared PyMem_Malloc so it's assumed to be a Python function. You should do
from cpython.mem cimport PyMem_Malloc
Make sure it's cimported, not imported
Edit2:
The following compiles fine for me:
from cpython.mem cimport PyMem_Malloc
ctypedef double DTYPE_t
ctypedef struct cell_t:
DTYPE_t[2] min_bounds
DTYPE_t[2] max_bounds
cdef cell_t * make_cell(DTYPE_t[2] min_bounds, DTYPE_t[2] max_bounds) except NULL:
cdef cell_t * cell = <cell_t *>PyMem_Malloc(sizeof(cell_t))
if not cell:
raise MemoryError()
return cell
I've cut down cell_t a bit (just to avoid having to make declarations of UINT32_t). I've also given the cdef function an except NULL to allow it to signal an error if needed and added a raise before MemoryError(). I don't think either of these changes are directly related to your error.
I am building a Python wrapper to a C code.
In the .c code I have the following defined function
double myfunction_at_k_and_z(struct background * pba,
struct spectra * psp,
struct perturbs *ppt,
double k,
double z,
double * pk_ic)
{ ...body of the function ...}
and in the .h file I have
struct background ba;
struct perturbs pt;
struct spectra sp;
Now I implement a function in my .pyx file, inside a class myclass
def get_myfunction(self, double k, double z):
cdef double dummy
result = myfunction_at_k_and_z(&self.ba, &self.sp, &self.pt, k, z, &dummy)
return result
and in the .pxd file I write:
cdef struct background: ....
cdef struct spectra: ...
cdef struct perturbs: ...
double myfunction_at_k_and_z (background * pba, spectra * psp, perturbs *ppt, double k, double z, double * pk_ic)
where the dots ... denote the components of the structures, which are not relevant here apart from one case: in struct spectra there are
int ln_k_size
double * ln_k
Now my problem is that despite everything compiling, when I run with Python my wrapper and use get_myfunction through e.g. myclass.get_myfunction(1., 1.), there seems to be a problem with the array ln_k, which apparently seems to have ln_k_size=0. This causes segmentation error.
Am I missing something in terms of allocation of memory?
Any way out?
Below is a simple C program square.c, which is a simplification of a complex one:
#include <stdio.h>
#include <stdlib.h>
struct Square {
float length;
float width;
};
typedef struct Square *sq;
float calc_area(sq a) {
float s;
s = a->length * a->width;
return s;
}
int main() {
sq a;
a = malloc(sizeof(struct Square));
a->length = 10.0;
a->width = 3.0;
printf("%f\n", calc_area(a));
free(a);
}
I want to wrap the calc_area() function and call it in Python. I am not quite familiar with C nor Cython, but it seems that the difficulty is that the argument of calc_area() is a pointer struct.
I tried to use Cython, and below is my effort:
First, I wrote a header file square.h:
#include <stdio.h>
#include <stdlib.h>
struct Square {
float length;
float width;
};
typedef struct Square *sq;
float calc_area(sq a);
Second, I wrote the square_wrapper.pyx as below
cdef extern from 'square.h':
struct Square:
float length
float width
ctypedef struct Square *sq
float calc_area(sq a)
def c_area(a):
return calc_area(sq a)
It seems that ctypedef struct Square *sq is not correct, but I have no idea how to modify it - this is the first time I try to wrap a C program and I cannot find similar examples to my case.
How could I fix this using Cython or any other tools? Thanks in advance!