Following these answers, I've currently defined a Rust 1.0 function as follows, in order to be callable from Python using ctypes:
use std::vec;
extern crate libc;
use libc::{c_int, c_float, size_t};
use std::slice;
#[no_mangle]
pub extern fn convert_vec(input_lon: *const c_float,
lon_size: size_t,
input_lat: *const c_float,
lat_size: size_t) -> Vec<(i32, i32)> {
let input_lon = unsafe {
slice::from_raw_parts(input_lon, lon_size as usize)
};
let input_lat = unsafe {
slice::from_raw_parts(input_lat, lat_size as usize)
};
let combined: Vec<(i32, i32)> = input_lon
.iter()
.zip(input_lat.iter())
.map(|each| convert(*each.0, *each.1))
.collect();
return combined
}
And I'm setting up the Python part like so:
from ctypes import *
class Int32_2(Structure):
_fields_ = [("array", c_int32 * 2)]
rust_bng_vec = lib.convert_vec_py
rust_bng_vec.argtypes = [POINTER(c_float), c_size_t,
POINTER(c_float), c_size_t]
rust_bng_vec.restype = POINTER(Int32_2)
This seems to be OK, but I'm:
Not sure how to transform combined (a Vec<(i32, i32)>) to a C-compatible structure, so it can be returned to my Python script.
Not sure whether I should be returning a reference (return &combined?) and how I would have to annotate the function with the appropriate lifetime specifier if I did
The most important thing to note is that there is no such thing as a tuple in C. C is the lingua franca of library interoperability, and you will be required to restrict yourself to abilities of this language. It doesn't matter if you are talking between Rust and another high-level language; you have to speak C.
There may not be tuples in C, but there are structs. A two-element tuple is just a struct with two members!
Let's start with the C code that we would write:
#include <stdio.h>
#include <stdint.h>
typedef struct {
uint32_t a;
uint32_t b;
} tuple_t;
typedef struct {
void *data;
size_t len;
} array_t;
extern array_t convert_vec(array_t lat, array_t lon);
int main() {
uint32_t lats[3] = {0, 1, 2};
uint32_t lons[3] = {9, 8, 7};
array_t lat = { .data = lats, .len = 3 };
array_t lon = { .data = lons, .len = 3 };
array_t fixed = convert_vec(lat, lon);
tuple_t *real = fixed.data;
for (int i = 0; i < fixed.len; i++) {
printf("%d, %d\n", real[i].a, real[i].b);
}
return 0;
}
We've defined two structs — one to represent our tuple, and another to represent an array, as we will be passing those back and forth a bit.
We will follow this up by defining the exact same structs in Rust and define them to have the exact same members (types, ordering, names). Importantly, we use #[repr(C)] to let the Rust compiler know to not do anything funky with reordering the data.
extern crate libc;
use std::slice;
use std::mem;
#[repr(C)]
pub struct Tuple {
a: libc::uint32_t,
b: libc::uint32_t,
}
#[repr(C)]
pub struct Array {
data: *const libc::c_void,
len: libc::size_t,
}
impl Array {
unsafe fn as_u32_slice(&self) -> &[u32] {
assert!(!self.data.is_null());
slice::from_raw_parts(self.data as *const u32, self.len as usize)
}
fn from_vec<T>(mut vec: Vec<T>) -> Array {
// Important to make length and capacity match
// A better solution is to track both length and capacity
vec.shrink_to_fit();
let array = Array { data: vec.as_ptr() as *const libc::c_void, len: vec.len() as libc::size_t };
// Whee! Leak the memory, and now the raw pointer (and
// eventually C) is the owner.
mem::forget(vec);
array
}
}
#[no_mangle]
pub extern fn convert_vec(lon: Array, lat: Array) -> Array {
let lon = unsafe { lon.as_u32_slice() };
let lat = unsafe { lat.as_u32_slice() };
let vec =
lat.iter().zip(lon.iter())
.map(|(&lat, &lon)| Tuple { a: lat, b: lon })
.collect();
Array::from_vec(vec)
}
We must never accept or return non-repr(C) types across the FFI boundary, so we pass across our Array. Note that there's a good amount of unsafe code, as we have to convert an unknown pointer to data (c_void) to a specific type. That's the price of being generic in C world.
Let's turn our eye to Python now. Basically, we just have to mimic what the C code did:
import ctypes
class FFITuple(ctypes.Structure):
_fields_ = [("a", ctypes.c_uint32),
("b", ctypes.c_uint32)]
class FFIArray(ctypes.Structure):
_fields_ = [("data", ctypes.c_void_p),
("len", ctypes.c_size_t)]
# Allow implicit conversions from a sequence of 32-bit unsigned
# integers.
#classmethod
def from_param(cls, seq):
return cls(seq)
# Wrap sequence of values. You can specify another type besides a
# 32-bit unsigned integer.
def __init__(self, seq, data_type = ctypes.c_uint32):
array_type = data_type * len(seq)
raw_seq = array_type(*seq)
self.data = ctypes.cast(raw_seq, ctypes.c_void_p)
self.len = len(seq)
# A conversion function that cleans up the result value to make it
# nicer to consume.
def void_array_to_tuple_list(array, _func, _args):
tuple_array = ctypes.cast(array.data, ctypes.POINTER(FFITuple))
return [tuple_array[i] for i in range(0, array.len)]
lib = ctypes.cdll.LoadLibrary("./target/debug/libtupleffi.dylib")
lib.convert_vec.argtypes = (FFIArray, FFIArray)
lib.convert_vec.restype = FFIArray
lib.convert_vec.errcheck = void_array_to_tuple_list
for tupl in lib.convert_vec([1,2,3], [9,8,7]):
print tupl.a, tupl.b
Forgive my rudimentary Python. I'm sure an experienced Pythonista could make this look a lot prettier! Thanks to #eryksun for some nice advice on how to make the consumer side of calling the method much nicer.
A word about ownership and memory leaks
In this example code, we've leaked the memory allocated by the Vec. Theoretically, the FFI code now owns the memory, but realistically, it can't do anything useful with it. To have a fully correct example, you'd need to add another method that would accept the pointer back from the callee, transform it back into a Vec, then allow Rust to drop the value. This is the only safe way, as Rust is almost guaranteed to use a different memory allocator than the one your FFI language is using.
Not sure whether I should be returning a reference and how I would have to annotate the function with the appropriate lifetime specifier if I did
No, you don't want to (read: can't) return a reference. If you could, then the ownership of the item would end with the function call, and the reference would point to nothing. This is why we need to do the two-step dance with mem::forget and returning a raw pointer.
Related
I have a C++ class with a member function that takes an unsigned char* buffer and an unsigned int length as arguments and operates on them. I've wrapped this class with Boost::Python and would like to pass a pre-populated buffer to the class from a Python script. The Python-side buffer is created with struct.pack. I can't figure out how to make the argument type match and keep getting Boost.Python.ArgumentError.
include/Example.h
#ifndef EXAMPLECLASS_H_
#define EXAMPLECLASS_H_
#include <cstdio>
class ExampleClass
{
public:
ExampleClass() {}
virtual ~ExampleClass() {}
void printBuffer(unsigned char* buffer, unsigned int length)
{
for (unsigned int i = 0; i < length; ++i)
{
printf("%c", buffer[i]);
}
printf("\n");
}
};
#endif
src/example.cpp
#include "Example.h"
int main(int argc, char** argv)
{
unsigned char buf[4];
buf[0] = 0x41;
buf[1] = 0x42;
buf[2] = 0x43;
buf[3] = 0x44;
ExampleClass e;
e.printBuffer(buf, 4);
return 0;
}
src/Example_py.cpp
#include <boost/python.hpp>
#include "Example.h"
using namespace boost::python;
BOOST_PYTHON_MODULE(example_py)
{
class_<ExampleClass>("ExampleClass")
.def("printBuffer", &ExampleClass::printBuffer)
;
}
scripts/example.py
#!/usr/bin/env python
import example_py
import struct
import ctypes
buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
print 'python:'
print buf
e = example_py.ExampleClass()
print 'c++:'
print e.printBuffer(ctypes.cast(ctypes.c_char_p(buf), ctypes.POINTER(ctypes.c_ubyte)), len(buf))
CMakeLists.txt (incomplete)
include_directories(
include
${Boost_INCLUDE_DIRS}
${PYTHON_INCLUDE_DIRS}
)
add_library(example_py
src/Example_py.cpp
)
target_link_libraries(example_py ${Boost_LIBRARIES} ${PYTHON_LIBRARIES})
set_target_properties(example_py PROPERTIES PREFIX "")
add_executable(example src/example.cpp)
target_link_libraries(example example_py)
Output
$ ./example
ABCD
$ ./scripts/example.py
python: ABCD
c++:
Traceback (most recent call last):
File "/home/dustingooding/example/scripts/example.py", line 13, in <module>
print 'c++:', e.printBuffer(ctypes.cast(ctypes.c_char_p(buf), ctypes.POINTER(ctypes.c_ubyte)), len(buf))
Boost.Python.ArgumentError: Python argument types in
ExampleClass.printBuffer(ExampleClass, LP_c_ubyte, int)
did not match C++ signature:
printBuffer(ExampleClass {lvalue}, unsigned char*, unsigned int)
I've tried a number of different approaches (passing 'buf' directly, passing 'buf' as a ctypes.c_char_p, creating a ctypes.ubyte array and populating it with the contents of 'buf' and passing it), but none seem to work.
I don't understand why 'LP_c_ubyte' and 'unsigned char*' don't match.
EDIT
Here's a Github project with a ready-to-go codebase. Feel free to use this. I've added #Tanner's fix. https://github.com/dustingooding/boost_python_ucharp_example
It may be worth considering exposing a Pythonic auxiliary function as the ExampleClass.printBuffer method to Python, that delegates to the c-ish ExampleClass::printBuffer member function. For instance, this would allow the Python users to invoke:
import example
import struct
buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
e.printBuffer(buf)
Rather than requiring the user to perform the correct ctypes cast and sizing.
The struct.pack() method returns a str object in Python2 and a bytes object in Python3, so the auxiliary C++ function would need to populate a continuous block of memory with the elements of from either str or bytes. The boost::python::stl_input_iterator can provide a convenient way to construct C++ containers, such as std::vector<char>, from a Python object, such as str or bytes. The only oddity is that stl_input_iterator expects the Python type to support the iterable protocol, which str does not do. However, the builtin iter() Python method can be used to create an iterable object.
/// #brief Auxiliary function used to allow a Python iterable object with char
/// elements to be passed to ExampleClass.printBuffer().
void example_class_print_buffer_wrap(
ExampleClass& self,
boost::python::object py_buffer)
{
namespace python = boost::python;
// `str` objects do not implement the iterator protcol (__iter__),
// but do implement the sequence protocol (__getitem__). Use the
// `iter()` builtin to create an iterator for the buffer.
// >>> __builtins__.iter(py_buffer)
python::object locals(python::borrowed(PyEval_GetLocals()));
python::object py_iter = locals["__builtins__"].attr("iter");
python::stl_input_iterator<char> begin(
py_iter(py_buffer)), end;
// Copy the py_buffer into a local buffer with known continguous memory.
std::vector<char> buffer(begin, end);
// Cast and delegate to the printBuffer member function.
self.printBuffer(
reinterpret_cast<unsigned char*>(&buffer[0]),
buffer.size());
}
With the auxiliary function created, one just needs to expose it as the ExampleClass.printBuffer method:
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<ExampleClass>("ExampleClass")
.def("printBuffer", &example_class_print_buffer_wrap)
;
}
Here is a complete example demonstrating this approach:
#include <cstdio>
#include <vector>
#include <boost/python.hpp>
#include <boost/python/stl_iterator.hpp>
// Mocks...
/// #brief Legacy class that cannot be changed.
class ExampleClass
{
public:
void printBuffer(unsigned char* buffer, unsigned int length)
{
for (unsigned int i = 0; i < length; ++i)
{
printf("%c", buffer[i]);
}
printf("\n");
}
};
/// #brief Auxiliary function used to allow a Python iterable object with char
/// elements to be passed to ExampleClass.printBuffer().
void example_class_print_buffer_wrap(
ExampleClass& self,
boost::python::object py_buffer)
{
namespace python = boost::python;
// `str` objects do not implement the iterator protcol (__iter__),
// but do implement the sequence protocol (__getitem__). Use the
// `iter()` builtin to create an iterator for the buffer.
// >>> __builtins__.iter(py_buffer)
python::object locals(python::borrowed(PyEval_GetLocals()));
python::object py_iter = locals["__builtins__"].attr("iter");
python::stl_input_iterator<char> begin(
py_iter(py_buffer)), end;
// Copy the py_buffer into a local buffer with known continguous memory.
std::vector<char> buffer(begin, end);
// Cast and delegate to the printBuffer member function.
self.printBuffer(
reinterpret_cast<unsigned char*>(&buffer[0]),
buffer.size());
}
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<ExampleClass>("ExampleClass")
.def("printBuffer", &example_class_print_buffer_wrap)
;
}
Interactive usage:
>>> import example
>>> import struct
>>> buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
>>> print 'python:', buf
python: ABCD
>>> e = example.ExampleClass()
>>> e.printBuffer(buf)
ABCD
The python documentation lists the following in the chapter Fundamental Data Types:
class ctypes.c_char_p
Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer
that may also point to binary data, POINTER(c_char) must be used. The
constructor accepts an integer address, or a string.
shows that you should probably use a c_char_p type. If you use the POINTER() functions this will be a LP_c_char_p.
The type
LP_c_ubyte /* corresponds to */ unsigned char;
you should probably use
LP_c_char_p /* which corresponds to */ char *;
Update:
I've corrected the types above. Also: I'm not a python expert, so I might have it wrong. There is also this answer.
Im calling a cpp function from dll with ctypes
the function definition is
int foo(strc *mystrc, int *varsize);
And the structure:
typedef struct
{
int type;
int count;
void *value;
} strc;
So what I tried in python was to define:
class strc(ctypes.Structure):
_fields_ = [('type', ctypes.c_int),
('count', ctypes.c_int),
('value', ctypes.c_void_p)]
And to call the function as
varsize = ctypes.c_int()
mystrc = strc()
foo(ctypes.byref(mystrc), ctypes.byref(varsize))
I can perfectly call the function and retrieve all values except for the "value". It should be an array of variables indicated by the "type", have size "varsize" and be an array of "count" variables.
How can I retrieve what is indicated by the void pointer?
template<class T> void iterate_strc_value(const void* value, int size)
{
const T* element = reinterpret_cast<const T*>(value);
for(int offset = 0; offset != size; ++offset)
*(element + offset) // at this point you have the element at the offset+1th position
}
switch(strc_var.type)
{
case 0: iterate_strc_value<char>(strc_var.value, element_count); break;
case 1: iterate_strc_value<int>(strc_var.value, element_count); break;
case 2: iterate_strc_value<std::string>(strc_var.value, element_count); break;
default: // invalid type -> maybe throw exception within python?!
}
The value pointer is the pointer that you also named value, while size specifies the amount of elements within the array. The size of the type is not needed, as the size of your types is known at compile time.
Basically the function just converts the void* to a pointer of your desired type and then uses pointer arithmetic to iterate over the array. This approach is general enough to just iterate over any void* which points to an array as long as you know it's size.
You could also add a third argument as a callback which performs the given action on each element to keep specialized code outside of the template function.
Consider this dummy Cython code:
#!python
#cython: boundscheck=False
#cython: wraparound=False
#cython: initializedcheck=False
#cython: cdivision=True
#cython: nonecheck=False
import numpy as np
# iterator function
cdef double[:] f(double[:] data):
data[0] *= 1.01
data[1] *= 1.02
return data
# looping function
cdef double[:] _call_me(int bignumber, double[:] data):
cdef int ii
for ii in range(bignumber):
data = f(data)
return data
# helper function to allow calls from Python
def call_me(bignumber):
cdef double[:] data = np.ones(2)
return _call_me(bignumber, data)
Now, if I do a cython -a on this, it shows the return statements in yellow. I'm doing something similar in a very performance-critical program, and according to profiling this is really slowing my code down. So, why does cython need python for these return statements? The annotated file gives a hint:
PyErr_SetString(PyExc_TypeError,"Memoryview return value is not initialized");
Amazingly, a google search for cython "Memoryview return value is not initialized" gives zero results.
The slow part isn't what you think it is. The slow part is (well... primarily)
data = f(data)
Not the f(data). The data =.
This assigns a struct, which is defined as so
typedef struct {
struct __pyx_memoryview_obj *memview;
char *data;
Py_ssize_t shape[8];
Py_ssize_t strides[8];
Py_ssize_t suboffsets[8];
} __Pyx_memviewslice;
and the assignment mentioned does
__pyx_t_3 = __pyx_f_3cyt_f(__pyx_v_data);
where __pyx_t_3 is of that type. If this is done heavily in a loop as it is, it takes far longer to copy the structs than to do the trivial body of the function. I've done a timing in pure C and it gives similar numbers.
(Edit note: The assigning is actually primarily a problem because it also causes generation of structs and other copies to not be optimised out.)
However, the whole thing seems silly. The only reason to copy the struct is for if something has changed, but nothing has. The memory points at the same place, the data points at the same place and the shape, strides and offsets are the same.
The only way I see to avoid the struct copy is to not change any of what it references (aka. always return the memoryview given in). That's only possible in circumstances where returning is pointless anyway, like here. Or you can hack at the C, I guess, like I was. Just don't cry if you break something.
Also note that you can make your function nogil, so it can't have anything to do with harking back to Python.
EDIT
C's optimising compiler was throwing me slightly off. Basically, I removed some assigning and it removed loads of other things. Basically the slow path is this:
#include<stdio.h>
struct __pyx_memoryview_obj;
typedef struct {
struct __pyx_memoryview_obj *memview;
char *data;
ssize_t shape[8];
ssize_t strides[8];
ssize_t suboffsets[8];
} __Pyx_memviewslice;
static __Pyx_memviewslice __pyx_f_3cyt_f(__Pyx_memviewslice __pyx_v_data) {
__Pyx_memviewslice __pyx_r = { 0, 0, { 0 }, { 0 }, { 0 } };
__pyx_r = __pyx_v_data;
return __pyx_r;
}
main() {
int i;
__Pyx_memviewslice __pyx_v_data = {0, 0, { 0 }, { 0 }, { 0 }};
for (i=0; i<10000000; i++) {
__pyx_v_data = __pyx_f_3cyt_f(__pyx_v_data);
}
}
(compile with no optimisations). I'm no C programmer, so apologies if what I've done sucks in some way not directly linked to the fact I've copied computer-generated code.
I know this doesn't help, but I did my best, OK?
Refering to http://mail.python.org/pipermail/python-dev/2009-June/090210.html
AND http://dan.iel.fm/posts/python-c-extensions/
and here is other places i searched regarding my question:
http://article.gmane.org/gmane.comp.python.general/424736
http://joyrex.spc.uchicago.edu/bookshelves/python/cookbook/pythoncook-CHP-16-SECT-3.html
http://docs.python.org/2/c-api/sequence.html#PySequence_Check
Python extension module with variable number of arguments
I am inexperienced in Python/C API.
I have the following code:
sm_int_list = (1,20,3)
c_int_array = (ctypes.c_int * len(sm_int_list))(*sm_int_list)
sm_str_tuple = ('some','text', 'here')
On the C extension side, i have done something like this:
static PyObject* stuff_here(PyObject *self, PyObject *args)
{
char* input;
int *i1, *i2;
char *s1, *s2;
// args = (('some','text', 'here'), [1,20,3], ('some','text', 'here'), [1,20,3])
**PyArg_ParseTuple(args, "(s#:):#(i:)#(s#:):#(i:)#", &s1, &i1, &s2, &i2)**;
/*stuff*/
}
such that:
stuff.here(('some','text', 'here'), [1,20,3], ('some','text', 'here'), [1,20,3])
returns data in the same form as args after some computation.
I would like to know the PyArg_ParseTuple expression, is it the proper way to parse
an array of varying string
an array of integers
UPDATE NEW
Is this the correct way?:
static PyObject* stuff_here(PyObject *self, PyObject *args)
unsigned int tint[], cint[];
ttotal=0, ctotal=0;
char *tstr, *cstr;
int *t_counts, *c_counts;
Py_ssize_t size;
PyObject *t_str1, *t_int1, *c_str2, *c_int2; //the C var that takes in the py variable value
PyObject *tseq, cseq;
int t_seqlen=0, c_seqlen=0;
if (!PyArg_ParseTuple(args, "OOiOOi", &t_str1, &t_int1, &ttotal, &c_str2, &c_int2, &ctotal))
{
return NULL;
}
if (!PySequence_Check(tag_str1) && !PySequence_Check(cat_str2)) return NULL;
else:
{
//All things t
tseq = PySequence_Fast(t_str1, "iterable");
t_seqlen = PySequence_Fast_GET_SIZE(tseq);
t_counts = PySequence_Fast(t_int1);
//All things c
cseq = PySequence_Fast(c_str2);
c_seqlen = PySequence_Fast_GET_SIZE(cseq);
c_counts = PySequence_Fast(c_int2);
//Make c arrays of all things tag and cat
for (i=0; i<t_seqlen; i++)
{
tstr[i] = PySequence_Fast_GET_ITEM(tseq, i);
tcounts[i] = PySequence_Fast_GET_ITEM(t_counts, i);
}
for (i=0; i<c_seqlen; i++)
{
cstr[i] = PySequence_Fast_GET_ITEM(cseq, i);
ccounts[i] = PySequence_Fast_GET_ITEM(c_counts, i);
}
}
OR
PyArg_ParseTuple(args, "(s:)(i:)(s:)(i:)", &s1, &i1, &s2, &i2)
And then again while returning,
Py_BuildValue("sisi", arr_str1,arr_int1,arr_str2,arr_int2) ??
Infact if someone could in detail clarify the various PyArg_ParseTuple function that would be of great benefit. the Python C API, as i find it in the documentation, is not exactly a tutorial on things to do.
You can use PyArg_ParseTuple to parse a real tuple, that has a fixed structure. Especially the number of items in the subtuples cannot change.
As the 2.7.5 documentation says, your format "(s#:):#(i:)#(s#:):#(i:)#" is wrong since : cannot occur in nested parenthesis. The format "(sss)(iii)(sss)(iii)", along with total of 12 pointer arguments should match your arguments. Likewise for Py_BuildValue you can use the same format string (which creates 4 tuples within 1 tuple), or "(sss)[iii](sss)[iii]" if the type matters (this makes the integers to be in lists instead of tuples).
I have a self-made C library that I want to access using python. The problem is that the code consists essentially of two parts, an initialization to read in data from a number of files and a few calculations that need to be done only once. The other part is called in a loop and uses the data generated before repeatedly. To this function I want to pass parameters from python.
My idea was to write two C wrapper functions, "init" and "loop" - "init" reads the data and returns a void pointer to a structure that "loop" can use together with additional parameters that I can pass on from python. Something like
void *init() {
struct *mystruct ret = (mystruct *)malloc(sizeof(mystruct));
/* Fill ret with data */
return ret;
}
float loop(void *data, float par1, float par2) {
/* do stuff with data, par1, par2, return result */
}
I tried calling "init" from python as a c_void_p, but since "loop" changes some of the contents of "data" and ctypes' void pointers are immutable, this did not work.
Other solutions to similar problems I saw seem to require knowledge of how much memory "init" would use, and I do not know that.
Is there a way to pass data from one C function to another through python without telling python exactly what or how much it is? Or is there another way to solve my problem?
I tried (and failed) to write a minimum crashing example, and after some debugging it turned out there was a bug in my C code. Thanks to everyone who replied!
Hoping that this might help other people, here is a sort-of-minimal working version (still without separate 'free' - sorry):
pybug.c:
#include <stdio.h>
#include <stdlib.h>
typedef struct inner_struct_s {
int length;
float *array;
} inner_struct_t;
typedef struct mystruct_S {
int id;
float start;
float end;
inner_struct_t *inner;
} mystruct_t;
void init(void **data) {
int i;
mystruct_t *mystruct = (mystruct_t *)malloc(sizeof(mystruct_t));
inner_struct_t *inner = (inner_struct_t *)malloc(sizeof(inner_struct_t));
inner->length = 10;
inner->array = calloc(inner->length, sizeof(float));
for (i=0; i<inner->length; i++)
inner->array[i] = 2*i;
mystruct->id = 0;
mystruct->start = 0;
mystruct->end = inner->length;
mystruct->inner = inner;
*data = mystruct;
}
float loop(void *data, float par1, float par2, int newsize) {
mystruct_t *str = data;
inner_struct_t *inner = str->inner;
int i;
inner->length = newsize;
inner->array = realloc(inner->array, newsize * sizeof(float));
for (i=0; i<inner->length; i++)
inner->array[i] = par1 + i * par2;
return inner->array[inner->length-1];
}
compile as
cc -c -fPIC pybug.c
cc -shared -o libbug.so pybug.o
Run in python:
from ctypes import *
sl = CDLL('libbug.so')
# What arguments do functions take / return?
sl.init.argtype = c_void_p
sl.loop.restype = c_float
sl.loop.argtypes = [c_void_p, c_float, c_float, c_int]
# Init takes a pointer to a pointer
px = c_void_p()
sl.init(byref(px))
# Call the loop a couple of times
for i in range(10):
print sl.loop(px, i, 5, 10*i+5)
You should have a corresponding function to free the data buffer when the caller is done. Otherwise I don't see the issue. Just pass the pointer to loop that you get from init.
init.restype = c_void_p
loop.argtypes = [c_void_p, c_float, c_float]
loop.restype = c_float
I'm not sure what you mean by "ctypes' void pointers are immutable", unless you're talking about c_char_p and c_wchar_p. The issue there is if you pass a Python string as an argument it uses Python's private pointer to the string buffer. If a function can change the string, you should first copy it to a c_char or c_wchar array.
Here's a simple example showing the problem of passing a Python string (2.x byte string) as an argument to a function that modifies it. In this case it changes index 0 to '\x00':
>>> import os
>>> from ctypes import *
>>> open('tmp.c', 'w').write("void f(char *s) {s[0] = 0;}")
>>> os.system('gcc -shared -fPIC -o tmp.so tmp.c')
0
>>> tmp = CDLL('./tmp.so')
>>> tmp.f.argtypes = [c_void_p]
>>> tmp.f.restype = None
>>> tmp.f('a')
>>> 'a'
'\x00'
>>> s = 'abc'
>>> tmp.f(s)
>>> s
'\x00bc'
This is specific to passing Python strings as arguments. It isn't a problem to pass pointers to data structures that are intended to be mutable, either ctypes data objects such as a Structure, or pointers returned by libraries.
Is your C code in a DLL? If so can might consider creating a global pointer in there. init() will do any initialization required and set the pointer equal to newly allocated memory and loop() will operate on that memory. Also don't forget to free it up with a close() function