Develop Reference

SWIG TypeError: in method 'my_method', argument 1 of type 'double const []'

I am trying to wrap a c function into python using swig.
The function, defined in my_method.h and implemented in my_method.c, is:
double my_method(const double a[], const double b[], const long int len_a, const long int len_b){
...
}
my swig interface file (my_method.i) is:
/* file : my_method.i */
/* name of the python module*/
%module my_method
%{
#include "my_method.h"
%}
%include "my_method.h"
I use dist-utils to generate the wrapped function
#!/usr/bin/env python
"""
setup.py file for SWIG wrapping
"""
from distutils.core import setup, Extension
my_method_module = Extension('_my_method',
sources=['my_method.c', 'my_method_wrap.c'],
)
setup (name = 'my_method',
version = '0.1',
author = "author",
description = """my method""",
ext_modules = [my_method_module],
py_modules = ["my_method"],
)
and then
swig -python my_method.i
python setup.py build_ext --inplace
however when I run my main script in python:
a = np.ones(4)
b = np.ones(4)
res = my_method(a, b, len(a), len(b))
I get the following error:
return _my_method.my_method(a, b, len_a, len_b)
TypeError: in method 'my_method', argument 1 of type 'double const []'
any suggestion?

SWIG has no idea that double a[] represents one or many doubles or that len_a indicates the length. NumPy has a SWIG interface file that defines the typemaps needed to associate a pointer and size with a numpy array or Python list called numpy.i (docs) (download).
Here's a standalone example. You'll either need to reorder the parameters so pointer/size are together or add an adapter function. I've demonstrated the latter in case reordering is not possible:
%module test
%{
#define SWIG_FILE_WITH_INIT
// sample implementation of OP function.
// Just sum all the values in both arrays.
double my_method(const double a[], const double b[], const long int len_a, const long int len_b) {
double sum = 0.0;
for(long int aa = 0; aa < len_a; ++aa)
sum += a[aa];
for(long int bb = 0; bb < len_b; ++bb)
sum += b[bb];
return sum;
}
// adapter function in case re-ordering parameters is not possible
double my_method2(const double a[], int len_a, const double b[], int len_b) {
return my_method(a, b, len_a, len_b);
}
%}
%include "numpy.i"
%init %{
import_array();
%}
// apply the typemaps for numpy arrays to the two pairs of parameters representing the arrays
%apply (double* IN_ARRAY1, int DIM1) {(double* a, int len_a)};
%apply (double* IN_ARRAY1, int DIM1) {(double* b, int len_b)};
// process the adapter function and rename
%rename(my_method) my_method2;
double my_method2(double* a, int len_a, double* b, int len_b);
I built using MSVC on Windows. Note that include paths are needed to Python and its NumPy package headers:
swig -python test.i
cl /LD /W3 /Ic:\python310\include /Ic:\Python310\Lib\site-packages\numpy\core\include /Fe_test.pyd test_wrap.c -link /libpath:c:\python310\libs
Demo:
>>> import test
>>> test.my_method([1, 2, 3], [4, 5, 6])
21.0
>>> import numpy as np
>>> a = np.ones(4)
>>> b = np.ones(4)
>>> test.my_method(a, b)
8.0

Python dictionary to C++ map

I am very new to C++ and am trying to translate a dictionary into a C++ format. I can't quite seem to find the answer I am looking for from the previous questions submitted on here.
I have code as follows:
#include <iostream>
#include <map>
using namespace std;
typedef std::map<string, int> BasePairMap;
int main()
{
BasePairMap m;
m['power'] = 0;
m['select'] = 1;
m['backup'] = 2;
...
...
...
m['rewind'] = 71;
m['boxoffice'] = 240;
m['sky'] = 241;
return 0;
}
But I keep getting character overflow errors. How can I map string/int pairs together in C++?
Thanks

While many languages (such as Python) allow developers to use either single or double quotes for strings, in C++ you need to use double quotes (reference). Simple quotes are used for the char type which describes a single character (reference).
So your code should be:
#include <iostream>
#include <map>
using namespace std;
typedef std::map<string, int> BasePairMap;
int main()
{
BasePairMap m;
m["power"] = 0;
m["select"] = 1;
m["backup"] = 2;
// ...
m["rewind"] = 71;
m["boxoffice"] = 240;
m["sky"] = 241;
return 0;
}

Vector allocator argument error while using SWIG generated python library

I have used SWIG and generated a python library from C++ code and encountered the following error:
TypeError: in method 'new_SpikeGeneratorFromVector', argument 1 of type 'std::vector< int,std::allocator< int > >'
I have included the interface file std_vector.i and stl.i and a few more that seemed necessary. When I pass an integer list to the function, I get the above error.
Any help is appreciated.

It may help:
/* File : example.i */
%module example
%{
#include "example.h"
%}
%include "std_vector.i"
namespace std {
%template(IntVector) vector<int>;
}
%include "example.h"
/*example.h*/
void my_func(std::vector<int> v)
{
for (int i=0; i<v.size(; i++))
std::cout<<v[i]<<"\n";
}
/*in runme.py*/
import example
# call with a python list:
print example.my_func([1, 2, 3, 4])
#call with a python tuple:
print example.my_func((1, 2, 3, 4))
# ... or a wrapped std::vector<int>
v = example.IntVector(4)
for i in range(len(v)):
v[i] = i + 1
print example.my_func(v)

How do I pass a pre-populated "unsigned char*" buffer to a C++ method using boost.python?

I have a C++ class with a member function that takes an unsigned char* buffer and an unsigned int length as arguments and operates on them. I've wrapped this class with Boost::Python and would like to pass a pre-populated buffer to the class from a Python script. The Python-side buffer is created with struct.pack. I can't figure out how to make the argument type match and keep getting Boost.Python.ArgumentError.
include/Example.h
#ifndef EXAMPLECLASS_H_
#define EXAMPLECLASS_H_
#include <cstdio>
class ExampleClass
{
public:
ExampleClass() {}
virtual ~ExampleClass() {}
void printBuffer(unsigned char* buffer, unsigned int length)
{
for (unsigned int i = 0; i < length; ++i)
{
printf("%c", buffer[i]);
}
printf("\n");
}
};
#endif
src/example.cpp
#include "Example.h"
int main(int argc, char** argv)
{
unsigned char buf[4];
buf[0] = 0x41;
buf[1] = 0x42;
buf[2] = 0x43;
buf[3] = 0x44;
ExampleClass e;
e.printBuffer(buf, 4);
return 0;
}
src/Example_py.cpp
#include <boost/python.hpp>
#include "Example.h"
using namespace boost::python;
BOOST_PYTHON_MODULE(example_py)
{
class_<ExampleClass>("ExampleClass")
.def("printBuffer", &ExampleClass::printBuffer)
;
}
scripts/example.py
#!/usr/bin/env python
import example_py
import struct
import ctypes
buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
print 'python:'
print buf
e = example_py.ExampleClass()
print 'c++:'
print e.printBuffer(ctypes.cast(ctypes.c_char_p(buf), ctypes.POINTER(ctypes.c_ubyte)), len(buf))
CMakeLists.txt (incomplete)
include_directories(
include
${Boost_INCLUDE_DIRS}
${PYTHON_INCLUDE_DIRS}
)
add_library(example_py
src/Example_py.cpp
)
target_link_libraries(example_py ${Boost_LIBRARIES} ${PYTHON_LIBRARIES})
set_target_properties(example_py PROPERTIES PREFIX "")
add_executable(example src/example.cpp)
target_link_libraries(example example_py)
Output
$ ./example
ABCD
$ ./scripts/example.py
python: ABCD
c++:
Traceback (most recent call last):
File "/home/dustingooding/example/scripts/example.py", line 13, in <module>
print 'c++:', e.printBuffer(ctypes.cast(ctypes.c_char_p(buf), ctypes.POINTER(ctypes.c_ubyte)), len(buf))
Boost.Python.ArgumentError: Python argument types in
ExampleClass.printBuffer(ExampleClass, LP_c_ubyte, int)
did not match C++ signature:
printBuffer(ExampleClass {lvalue}, unsigned char*, unsigned int)
I've tried a number of different approaches (passing 'buf' directly, passing 'buf' as a ctypes.c_char_p, creating a ctypes.ubyte array and populating it with the contents of 'buf' and passing it), but none seem to work.
I don't understand why 'LP_c_ubyte' and 'unsigned char*' don't match.
EDIT
Here's a Github project with a ready-to-go codebase. Feel free to use this. I've added #Tanner's fix. https://github.com/dustingooding/boost_python_ucharp_example

It may be worth considering exposing a Pythonic auxiliary function as the ExampleClass.printBuffer method to Python, that delegates to the c-ish ExampleClass::printBuffer member function. For instance, this would allow the Python users to invoke:
import example
import struct
buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
e.printBuffer(buf)
Rather than requiring the user to perform the correct ctypes cast and sizing.
The struct.pack() method returns a str object in Python2 and a bytes object in Python3, so the auxiliary C++ function would need to populate a continuous block of memory with the elements of from either str or bytes. The boost::python::stl_input_iterator can provide a convenient way to construct C++ containers, such as std::vector<char>, from a Python object, such as str or bytes. The only oddity is that stl_input_iterator expects the Python type to support the iterable protocol, which str does not do. However, the builtin iter() Python method can be used to create an iterable object.
/// #brief Auxiliary function used to allow a Python iterable object with char
/// elements to be passed to ExampleClass.printBuffer().
void example_class_print_buffer_wrap(
ExampleClass& self,
boost::python::object py_buffer)
{
namespace python = boost::python;
// `str` objects do not implement the iterator protcol (__iter__),
// but do implement the sequence protocol (__getitem__). Use the
// `iter()` builtin to create an iterator for the buffer.
// >>> __builtins__.iter(py_buffer)
python::object locals(python::borrowed(PyEval_GetLocals()));
python::object py_iter = locals["__builtins__"].attr("iter");
python::stl_input_iterator<char> begin(
py_iter(py_buffer)), end;
// Copy the py_buffer into a local buffer with known continguous memory.
std::vector<char> buffer(begin, end);
// Cast and delegate to the printBuffer member function.
self.printBuffer(
reinterpret_cast<unsigned char*>(&buffer[0]),
buffer.size());
}
With the auxiliary function created, one just needs to expose it as the ExampleClass.printBuffer method:
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<ExampleClass>("ExampleClass")
.def("printBuffer", &example_class_print_buffer_wrap)
;
}
Here is a complete example demonstrating this approach:
#include <cstdio>
#include <vector>
#include <boost/python.hpp>
#include <boost/python/stl_iterator.hpp>
// Mocks...
/// #brief Legacy class that cannot be changed.
class ExampleClass
{
public:
void printBuffer(unsigned char* buffer, unsigned int length)
{
for (unsigned int i = 0; i < length; ++i)
{
printf("%c", buffer[i]);
}
printf("\n");
}
};
/// #brief Auxiliary function used to allow a Python iterable object with char
/// elements to be passed to ExampleClass.printBuffer().
void example_class_print_buffer_wrap(
ExampleClass& self,
boost::python::object py_buffer)
{
namespace python = boost::python;
// `str` objects do not implement the iterator protcol (__iter__),
// but do implement the sequence protocol (__getitem__). Use the
// `iter()` builtin to create an iterator for the buffer.
// >>> __builtins__.iter(py_buffer)
python::object locals(python::borrowed(PyEval_GetLocals()));
python::object py_iter = locals["__builtins__"].attr("iter");
python::stl_input_iterator<char> begin(
py_iter(py_buffer)), end;
// Copy the py_buffer into a local buffer with known continguous memory.
std::vector<char> buffer(begin, end);
// Cast and delegate to the printBuffer member function.
self.printBuffer(
reinterpret_cast<unsigned char*>(&buffer[0]),
buffer.size());
}
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<ExampleClass>("ExampleClass")
.def("printBuffer", &example_class_print_buffer_wrap)
;
}
Interactive usage:
>>> import example
>>> import struct
>>> buf = struct.pack('BBBB', 0x41, 0x42, 0x43, 0x44)
>>> print 'python:', buf
python: ABCD
>>> e = example.ExampleClass()
>>> e.printBuffer(buf)
ABCD

The python documentation lists the following in the chapter Fundamental Data Types:
class ctypes.c_char_p
Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer
that may also point to binary data, POINTER(c_char) must be used. The
constructor accepts an integer address, or a string.
shows that you should probably use a c_char_p type. If you use the POINTER() functions this will be a LP_c_char_p.
The type
LP_c_ubyte /* corresponds to */ unsigned char;
you should probably use
LP_c_char_p /* which corresponds to */ char *;
Update:
I've corrected the types above. Also: I'm not a python expert, so I might have it wrong. There is also this answer.

Python/SWIG: Output an array

I am trying to output an array of values from a C function wrapped using SWIG for Python. The way I am trying to do is using the following typemap.
Pseudo code:
int oldmain() {
float *output = {0,1};
return output;
}
Typemap:
%typemap(out) float* {
int i;
$result = PyList_New($1_dim0);
for (i = 0; i < $1_dim0; i++) {
PyObject *o = PyFloat_FromDouble((double) $1[i]);
PyList_SetItem($result,i,o);
}
}
My code compiles well, but it hangs when I run access this function (with no more ways to debug it).
Any suggestions on where I am going wrong?
Thanks.

The easiest way to allow the length to vary is to add another output parameter that tells you the size of the array too:
%module test
%include <stdint.i>
%typemap(in,numinputs=0,noblock=1) size_t *len {
size_t templen;
$1 = &templen;
}
%typemap(out) float* oldmain {
int i;
$result = PyList_New(templen);
for (i = 0; i < templen; i++) {
PyObject *o = PyFloat_FromDouble((double)$1[i]);
PyList_SetItem($result,i,o);
}
}
%inline %{
float *oldmain(size_t *len) {
static float output[] = {0.f, 1.f, 2, 3, 4};
*len = sizeof output/sizeof *output;
return output;
}
%}
This is modified from this answer to add size_t *len which can be used to return the length of the array at run time. The typemap completely hides that output from the Python wrapper though and instead uses it in the %typemap(out) instead of a fixed size to control the length of the returned list.

This should get you going:
/* example.c */
float * oldmain() {
static float output[] = {0.,1.};
return output;
}
You are returning a pointer here, and swig has no idea about the size of it. Plain $1_dim0 would not work, so you would have to hard code or do some other magic. Something like this:
/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern float * oldmain();
%}
%typemap(out) float* oldmain {
int i;
//$1, $1_dim0, $1_dim1
$result = PyList_New(2);
for (i = 0; i < 2; i++) {
PyObject *o = PyFloat_FromDouble((double) $1[i]);
PyList_SetItem($result,i,o);
}
}
%include "example.c"
Then in python you should get:
>> import example
>> example.oldmain()
[0.0, 1.0]
When adding typemaps you may find -debug-tmsearch very handy, i.e.
swig -python -debug-tmsearch example.i
Should clearly indicate that your typemap is used when looking for a suitable 'out' typemap for float *oldmain. Also if you just like to access c global variable array you can do the same trick using typemap for varout instead of just out.