I have a c++ class written and I am using SWIG to make a Python version of my class. I would like to overload the constructor so that it can take in Python lists. For example:
>>> import example
>>> a = example.Array([1,2,3,4])
I was attempting to use the typemap feature in swig, but the scope of typemap does not include code in extend
Here is a similar example to what I have...
%typemap(in) double[]
{
if (!PyList_Check($input))
return NULL;
int size = PyList_Size($input);
int i = 0;
$1 = (double *) malloc((size+1)*sizeof(double));
for (i = 0; i < size; i++)
{
PyObject *o = PyList_GetItem($input,i);
if (PyNumber_Check(o))
$1[i] = PyFloat_AsDouble(o);
else
{
PyErr_SetString(PyExc_TypeError,"list must contain numbers");
free($1);
return NULL;
}
}
$1[i] = 0;
}
%include "Array.h"
%extend Array
{
Array(double lst[])
{
Array *a = new Array();
...
/* do stuff with lst[] */
...
return a;
}
}
I know the typemap is working correctly (I wrote a small test function that just prints out elements in the double[]).
I attempted putting the typemap inside the extend clause, but that did not solve the problem.
Maybe there is another way to use Python Lists inside of the extend, but I could not find any examples.
Thanks for the help in advance.
You're really close: instead of a double lst[], extend with std::list<double>:
%include "std_list.i" // or std_vector.i
%include "Array.h"
%extend Array
{
Array(const std::list<double>& numbers) {
Array* arr = new Array;
...put numbers list items in "arr", then
return a; // interpreter will take ownership
}
}
SWIG should automatically convert the Python list to the std::list.
Related
I have this class with an int and int[2] members, and I have a getMember accessor method that takes an index of a member and void* and fills the (pre-allocated) space after void* with the member:
foobar.h:
class Foobar {
public:
void getMember(int index, void* data) {
switch (index) {
case 0:
*(int *) data = member0;
break;
case 1:
*(int*) data = member1[0];
*((int*) data + 1) = member1[1];
break;
}
}
int member0;
int member1[2];
};
I can then write a SWIG interface to this:
%{
#include "foobar.h"
%}
%include "foobar.h"
Now, if I also add
%include <cpointer.i>
%pointer_functions(int, intp)
I can then do the following in Python:
>>>p = new_intp()
>>>f = Foobar()
>>>f.member0 = 2
>>>f.getMember(0, p)
>>>intp_value(p)
2
Question 1. I have a void* declared and I am passing intp and yet the whole thing works. Why??
Question 2. Assuming you explain to me how the above works, then how do I accomplish the same for member1 ?? That is, I added the pointer_functions code to make the above work (magically). Then what similar thing I need to add and what pointer p1 to pass so that
>>>f.getMember(1, p1)
works?
Well, I still can't answer Question 1, nevertheless I found the new "magic" way to answer Question 2:
%include <carrays.i>
%array_functions(int, inta);
Now Question 1 is unchanged, and Question 2 becomes, why does it work?
I'd have to check the code generated by SWIG, but my guess is that for void* function parameter declaration, SWIG can't do any type checking so any type given to the function will be accepted and passed on to the function. Then in the code, you cast the void* to an int* so as long as the type really is an int*, all is good. If you passed something not an int* then you would get undefined behavior such as crash since you would be overwriting part of an object or, worse, a short or such.
So this should tell you that what you are doing is rather dangerous. I don't see why you can declare your function to take an int* since the pointer can refer to one item or an array:
void getMember(int index, int* data) {
switch (index) {
case 0:
*data = member0;
break;
case 1:
*data = member1[0];
*(data + 1) = member1[1];
break;
}
}
Then SWIG will generate code that will check that the passed-in type is int* and will throw otherwise. I don't know if SWIG will know that your inta type is compatible with intp type. If not, you could %extend Foo with an adapter in your .i file:
%extend Foobar %{
void getMember(int index, int[2] data) {
getMember(index, data); // C++ knows this is ok
}
}
Refering to http://mail.python.org/pipermail/python-dev/2009-June/090210.html
AND http://dan.iel.fm/posts/python-c-extensions/
and here is other places i searched regarding my question:
http://article.gmane.org/gmane.comp.python.general/424736
http://joyrex.spc.uchicago.edu/bookshelves/python/cookbook/pythoncook-CHP-16-SECT-3.html
http://docs.python.org/2/c-api/sequence.html#PySequence_Check
Python extension module with variable number of arguments
I am inexperienced in Python/C API.
I have the following code:
sm_int_list = (1,20,3)
c_int_array = (ctypes.c_int * len(sm_int_list))(*sm_int_list)
sm_str_tuple = ('some','text', 'here')
On the C extension side, i have done something like this:
static PyObject* stuff_here(PyObject *self, PyObject *args)
{
char* input;
int *i1, *i2;
char *s1, *s2;
// args = (('some','text', 'here'), [1,20,3], ('some','text', 'here'), [1,20,3])
**PyArg_ParseTuple(args, "(s#:):#(i:)#(s#:):#(i:)#", &s1, &i1, &s2, &i2)**;
/*stuff*/
}
such that:
stuff.here(('some','text', 'here'), [1,20,3], ('some','text', 'here'), [1,20,3])
returns data in the same form as args after some computation.
I would like to know the PyArg_ParseTuple expression, is it the proper way to parse
an array of varying string
an array of integers
UPDATE NEW
Is this the correct way?:
static PyObject* stuff_here(PyObject *self, PyObject *args)
unsigned int tint[], cint[];
ttotal=0, ctotal=0;
char *tstr, *cstr;
int *t_counts, *c_counts;
Py_ssize_t size;
PyObject *t_str1, *t_int1, *c_str2, *c_int2; //the C var that takes in the py variable value
PyObject *tseq, cseq;
int t_seqlen=0, c_seqlen=0;
if (!PyArg_ParseTuple(args, "OOiOOi", &t_str1, &t_int1, &ttotal, &c_str2, &c_int2, &ctotal))
{
return NULL;
}
if (!PySequence_Check(tag_str1) && !PySequence_Check(cat_str2)) return NULL;
else:
{
//All things t
tseq = PySequence_Fast(t_str1, "iterable");
t_seqlen = PySequence_Fast_GET_SIZE(tseq);
t_counts = PySequence_Fast(t_int1);
//All things c
cseq = PySequence_Fast(c_str2);
c_seqlen = PySequence_Fast_GET_SIZE(cseq);
c_counts = PySequence_Fast(c_int2);
//Make c arrays of all things tag and cat
for (i=0; i<t_seqlen; i++)
{
tstr[i] = PySequence_Fast_GET_ITEM(tseq, i);
tcounts[i] = PySequence_Fast_GET_ITEM(t_counts, i);
}
for (i=0; i<c_seqlen; i++)
{
cstr[i] = PySequence_Fast_GET_ITEM(cseq, i);
ccounts[i] = PySequence_Fast_GET_ITEM(c_counts, i);
}
}
OR
PyArg_ParseTuple(args, "(s:)(i:)(s:)(i:)", &s1, &i1, &s2, &i2)
And then again while returning,
Py_BuildValue("sisi", arr_str1,arr_int1,arr_str2,arr_int2) ??
Infact if someone could in detail clarify the various PyArg_ParseTuple function that would be of great benefit. the Python C API, as i find it in the documentation, is not exactly a tutorial on things to do.
You can use PyArg_ParseTuple to parse a real tuple, that has a fixed structure. Especially the number of items in the subtuples cannot change.
As the 2.7.5 documentation says, your format "(s#:):#(i:)#(s#:):#(i:)#" is wrong since : cannot occur in nested parenthesis. The format "(sss)(iii)(sss)(iii)", along with total of 12 pointer arguments should match your arguments. Likewise for Py_BuildValue you can use the same format string (which creates 4 tuples within 1 tuple), or "(sss)[iii](sss)[iii]" if the type matters (this makes the integers to be in lists instead of tuples).
I am trying to output an array of values from a C function wrapped using SWIG for Python. The way I am trying to do is using the following typemap.
Pseudo code:
int oldmain() {
float *output = {0,1};
return output;
}
Typemap:
%typemap(out) float* {
int i;
$result = PyList_New($1_dim0);
for (i = 0; i < $1_dim0; i++) {
PyObject *o = PyFloat_FromDouble((double) $1[i]);
PyList_SetItem($result,i,o);
}
}
My code compiles well, but it hangs when I run access this function (with no more ways to debug it).
Any suggestions on where I am going wrong?
Thanks.
The easiest way to allow the length to vary is to add another output parameter that tells you the size of the array too:
%module test
%include <stdint.i>
%typemap(in,numinputs=0,noblock=1) size_t *len {
size_t templen;
$1 = &templen;
}
%typemap(out) float* oldmain {
int i;
$result = PyList_New(templen);
for (i = 0; i < templen; i++) {
PyObject *o = PyFloat_FromDouble((double)$1[i]);
PyList_SetItem($result,i,o);
}
}
%inline %{
float *oldmain(size_t *len) {
static float output[] = {0.f, 1.f, 2, 3, 4};
*len = sizeof output/sizeof *output;
return output;
}
%}
This is modified from this answer to add size_t *len which can be used to return the length of the array at run time. The typemap completely hides that output from the Python wrapper though and instead uses it in the %typemap(out) instead of a fixed size to control the length of the returned list.
This should get you going:
/* example.c */
float * oldmain() {
static float output[] = {0.,1.};
return output;
}
You are returning a pointer here, and swig has no idea about the size of it. Plain $1_dim0 would not work, so you would have to hard code or do some other magic. Something like this:
/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern float * oldmain();
%}
%typemap(out) float* oldmain {
int i;
//$1, $1_dim0, $1_dim1
$result = PyList_New(2);
for (i = 0; i < 2; i++) {
PyObject *o = PyFloat_FromDouble((double) $1[i]);
PyList_SetItem($result,i,o);
}
}
%include "example.c"
Then in python you should get:
>> import example
>> example.oldmain()
[0.0, 1.0]
When adding typemaps you may find -debug-tmsearch very handy, i.e.
swig -python -debug-tmsearch example.i
Should clearly indicate that your typemap is used when looking for a suitable 'out' typemap for float *oldmain. Also if you just like to access c global variable array you can do the same trick using typemap for varout instead of just out.
I have a little project that works beautifully with SWIG. In particular, some of my functions return std::vectors, which get translated to tuples in Python. Now, I do a lot of numerics, so I just have SWIG convert these to numpy arrays after they're returned from the c++ code. To do this, I use something like the following in SWIG.
%feature("pythonappend") My::Cool::Namespace::Data() const %{ if isinstance(val, tuple) : val = numpy.array(val) %}
(Actually, there are several functions named Data, some of which return floats, which is why I check that val is actually a tuple.) This works just beautifully.
But, I'd also like to use the -builtin flag that's now available. Calls to these Data functions are rare and mostly interactive, so their slowness is not a problem, but there are other slow loops that speed up significantly with the builtin option.
The problem is that when I use that flag, the pythonappend feature is silently ignored. Now, Data just returns a tuple again. Is there any way I could still return numpy arrays? I tried using typemaps, but it turned into a giant mess.
Edit:
Borealid has answered the question very nicely. Just for completeness, I include a couple related but subtly different typemaps that I need because I return by const reference and I use vectors of vectors (don't start!). These are different enough that I wouldn't want anyone else stumbling around trying to figure out the minor differences.
%typemap(out) std::vector<int>& {
npy_intp result_size = $1->size();
npy_intp dims[1] = { result_size };
PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
int* dat = (int*) PyArray_DATA(npy_arr);
for (size_t i = 0; i < result_size; ++i) { dat[i] = (*$1)[i]; }
$result = PyArray_Return(npy_arr);
}
%typemap(out) std::vector<std::vector<int> >& {
npy_intp result_size = $1->size();
npy_intp result_size2 = (result_size>0 ? (*$1)[0].size() : 0);
npy_intp dims[2] = { result_size, result_size2 };
PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_INT);
int* dat = (int*) PyArray_DATA(npy_arr);
for (size_t i = 0; i < result_size; ++i) { for (size_t j = 0; j < result_size2; ++j) { dat[i*result_size2+j] = (*$1)[i][j]; } }
$result = PyArray_Return(npy_arr);
}
Edit 2:
Though not quite what I was looking for, similar problems may also be solved using #MONK's approach (explained here).
I agree with you that using typemap gets a little messy, but it is the right way to accomplish this task. You are also right that the SWIG documentation does not directly say that %pythonappend is incompatible with -builtin, but it is strongly implied: %pythonappend adds to the Python proxy class, and the Python proxy class does not exist at all in conjunction with the -builtin flag.
Before, what you were doing was having SWIG convert the C++ std::vector objects into Python tuples, and then passing those tuples back down to numpy - where they were converted again.
What you really want to do is convert them once, at the C level.
Here's some code which will turn all std::vector<int> objects into NumPy integer arrays:
%{
#include "numpy/arrayobject.h"
%}
%init %{
import_array();
%}
%typemap(out) std::vector<int> {
npy_intp result_size = $1.size();
npy_intp dims[1] = { result_size };
PyArrayObject* npy_arr = (PyArrayObject*)PyArray_SimpleNew(1, dims, NPY_INT);
int* dat = (int*) PyArray_DATA(npy_arr);
for (size_t i = 0; i < result_size; ++i) {
dat[i] = $1[i];
}
$result = PyArray_Return(npy_arr);
}
This uses the C-level numpy functions to construct and return an array. In order, it:
Ensures NumPy's arrayobject.h file is included in the C++ output file
Causes import_array to be called when the Python module is loaded (otherwise, all NumPy methods will segfault)
Maps any returns of std::vector<int> into NumPy arrays with a typemap
This code should be placed before you %import the headers which contain the functions returning std::vector<int>. Other than that restriction, it's entirely self-contained, so it shouldn't add too much subjective "mess" to your codebase.
If you need other vector types, you can just change the NPY_INT and all the int* and int bits, otherwise duplicating the function above.
I'm writing a little Python extension in C/C++, and I've got a function like this:
void set_parameters(int first_param, std::list<double> param_list)
{
//do stuff
}
I'd like to be able to call it from Python like this:
set_parameters(f_param, [1.0, 0.5, 2.1])
Is there a reasonably easy way to make that conversion? Ideally, I'd like a way that doesn't need a whole lot of extra dependencies, but some things just aren't possible without extra stuff, so that's not as big a deal.
Take a look at Boost.Python. Question you've asked is covered in Iterators chapter of the tutorial
The point is, Boost.Python provides stl_input_iterator template that converts Python's iterable to stl's input_iterator, which can be used to fill your std::list.
It turned out to be less pain than I thought, once I found the docs that I probably should have read before I asked the question. I was able to get a PyList object in my wrapper function, then just iterate over it and push the values onto the vector I needed. The code looks like this:
static PyObject* py_set_perlin_parameters(PyObject* self, PyObject* args)
{
int octaves;
double persistence;
PyObject* zoom_list;
int zoom_count = 0;
std::vector<double> zoom_vector;
if(!PyArg_ParseTuple(args, "idO!:set_perlin_parameters", &octaves, &persistence, &PyList_Type, &zoom_list))
{
return NULL;
}
if(!PyList_Check(zoom_list))
{
PyErr_SetString(PyExc_TypeError, "set_perlin_parameters: third parameter must be a list");
return NULL;
}
zoom_count = PyList_Size(zoom_list);
for(int i = 0; i < zoom_count; i++)
{
PyObject* list_val;
double val;
list_val = PyList_GetItem(zoom_list, i);
if(list_val == NULL)
{
return NULL;
}
val = PyFloat_AsDouble(list_val);
zoom_vector.push_back(val);
}
set_perlin_parameters(octaves, persistence, zoom_vector);
return Py_None;
}