Pass array as parameter from python to C function - python

the main idea of the question is How to pass array from python to the C function, get result (array to), and print this.
I created C function with algorithm for calculation convolution. (It doesn't not work correctly, but it works, and it's my first program on C, so it's not pretty good, don't judge harshly):
#include <stdio.h>
int * convolution(int a[], int b[], int inputArraySize)
{
int resArrayLength = inputArraySize*2-1;
int resArr[resArrayLength];
for (int i = 0; i < inputArraySize; i++)
{
int aPointer = 0;
int bPointer = i;
int res = 0;
for (int j = 0; j <= i; j++){
bPointer = i - j;
res = res + (b[bPointer] * a[aPointer]);
aPointer = aPointer + 1;
}
resArr[i] = res;
}
for(int i = 0; i<inputArraySize/2; i++){
int temp = a[i];
a[i] = a[inputArraySize-i-1];
a[inputArraySize-i-1] = temp;
}
for(int i = 0; i<inputArraySize/2; i++){
int temp = b[i];
b[i] = b[inputArraySize-i-1];
b[inputArraySize-i-1] = temp;
}
for (int i = 0; i < inputArraySize-1; i++)
{
int aPointer = 0;
int bPointer = i;
int res = 0;
for (int j = 0; j <= i; j++){
bPointer = i - j;
res = res + (b[bPointer] * a[aPointer]);
aPointer = aPointer + 1;
}
resArr[resArrayLength-1-i] = res;
}
return resArr;
}
I created .so file using this command cc -fPIC -shared -o convolution.so convolution.c.
Also I created python script (yeh, it's my first script on python) according to the guide from this article.
from ctypes import *
so_file = "./convolution.so"
convolution = CDLL(so_file)
print(type(convolution))
a = [1, 3, 5, 6]
b = [3, 5, 6, 7]
print(convolution.convolution(a, b, 4))
Here I try to create 2 arrays and pass to the C function.
But it doesn't work as I expect.
Output of the program:
<class 'ctypes.CDLL'>
Traceback (most recent call last):
File "t.py", line 10, in <module>
print(convolution.convolution(a, b, 4))
ctypes.ArgumentError: argument 1: <class 'TypeError'>: Don't know how to convert parameter 1
How can I pass this two arrays to the C function?
Also C function returns pointer to the first element of the result array to the Python script.
How to print all array elements using returned pointer from c function in the python?


Those are Python array, you have to convert them to C arrays before using them as parameters for a C function:
a = [1, 3, 5, 6]
b = [3, 5, 6, 7]
a_arr = (ctypes.c_int * len(a))(*a)
b_arr = (ctypes.c_int * len(b))(*b)
To get and print the resulting array, you can set the returning ctype of the function using .restype, call it, and then get a slice from the resulting array, to set the dimension ([0:4]):
convolution.convolution.restype = ctypes.POINTER(ctypes.c_int)
tmp = convolution.convolution(a, b, 4)
res = tmp[0:4]
print(res)

Related

My 2D array does not work properly in Ctypes

main.c:
#include <stdlib.h>
int **function() {
int **information = malloc(5 * sizeof(int));
for (int k = 0; k < 5; k++) {
information[k] = malloc(5 * sizeof(int));
for (int j = 0; j < 5; j++) {
information[k][j] = j;
}
}
return information;
}
main.py
import ctypes
from numpy.ctypeslib import ndpointer
lib = ctypes.CDLL("C:\\Users\\.....\\Desktop\\test9\\a.dll")
lib.function.restype = ndpointer(dtype=ctypes.c_int, shape=(5,5),flags='C')
res = lib.function()
print(res)
Result:
[[222866112 368 222866144 368 222866176]
[ 368 222866208 368 222866672 368]
[ 2 3 4 0 389116888]
[201333630 0 1 2 3]
[ 4 0 389116888 201333630 0]]
I do not know why it gives such an output and I have no problem with the same method in one-dimensional mode
How can I solve this problem?
Edited:
Both methods can be useful

Your code allocates an array of 5 pointers to arrays of 5 int. The allocation for the pointer array is incorrect: instead of int **information = malloc(5 * sizeof(int)); you should write: int **information = malloc(5 * sizeof(*information));
Yet for your purpose, the type int ** is incorrect. You should instead allocate an actual 2D array int array[5][5].
The syntax for a pointer to a dynamically allocated 2D array is cumbersome and the syntax for a function that returns such a pointer is a real challenge:
#include <stdlib.h>
int (*function(void))[5] {
// allocate an array of 5 arrays of 5 int
int (*information)[5] = malloc(5 * sizeof(*information));
// initialize all 5 rows to identical vectors { 0, 1, 2, 3, 4 }
for (int k = 0; k < 5; k++) {
for (int j = 0; j < 5; j++) {
information[k][j] = j;
}
}
return information;
}
The function prototype can be read using the spiral rule: start from the name, read the postfix operators then the prefix operators, switching direction at the parentheses:
function is a function of no arguments returning a pointer to arrays of 5 int.

To use ctypes with int** you can use POINTER(POINTER(c_int)) as the type and use slicing to access the elements of the array. To use ctypes with a C-contiguous 5x5 array using int* then numpy can be used efficiently.
test.c
#include <stdlib.h>
#ifdef _WIN32
# define API __declspec(dllexport)
#else
# define API
#endif
API int **function() {
int **information = malloc(5 * sizeof(int*)); // Note row element is int*
for (int k = 0; k < 5; k++) {
information[k] = malloc(5 * sizeof(int)); // Note col element is int
for (int j = 0; j < 5; j++) {
information[k][j] = k * 5 + j;
}
}
return information;
}
API int *function2() {
int *information = malloc(5 * 5 * sizeof(int)); // 5 x 5 C contiguous shape
for (int k = 0; k < 5; k++) {
for (int j = 0; j < 5; j++) {
information[k * 5 + j] = k * 5 + j;
}
}
return information;
}
test.py
import ctypes as ct
import numpy as np
from pprint import pprint
lib = ct.CDLL('./test')
lib.function.argtypes = ()
lib.function.restype = ct.POINTER(ct.POINTER(ct.c_int))
lib.function2.argtypes = ()
lib.function2.restype = np.ctypeslib.ndpointer(dtype=ct.c_int, shape=(5, 5), flags='C')
def function():
res = lib.function()
# slicing used to extract the correct number of items
return [row[:5] for row in res[:5]]
pprint(function())
pprint(lib.function2())
Output:
[[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]]
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]])
In both cases the memory is leaked for the mallocs. Ideally, let Python manage the memory and let C know the shape. Using np.empty below allocates memory without initialization then C initializes it. This is more flexible and can be used with different-sized arrays:
test.c
#include <stdlib.h>
#ifdef _WIN32
# define API __declspec(dllexport)
#else
# define API
#endif
API void function3(int* information, int row, int col) {
for (int k = 0; k < row; k++) {
for (int j = 0; j < col; j++) {
information[k * row + j] = k * 5 + j;
}
}
}
test.py
import ctypes as ct
import numpy as np
lib = ct.CDLL('./test')
lib.function3.argtypes = np.ctypeslib.ndpointer(dtype=ct.c_int),
lib.function3.restype = None
info = np.empty(dtype=ct.c_int,shape=(5,5))
lib.function3(info,5,5)
print(info)
Output:
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]

Pass a numpy array from Python to C

I'm having successfully embedded a Python script into a C module. The Python script produces a multi-dimensional Numpy array. Whereas the entire calculation in python takes 9 ms, the final tolist() conversion in order to return it to C takes 4 ms alone. I would like to change that by passing the Numpy array as reference and do the iterations in C again. But I can't currently figure out, how this can be done.
There are a lot of samples around, which use the other way around: Passing a Numpy array to a C function which is called from Python, but this is not my use case.
Any pointer welcome.

Ok, it's a while ago but I solved it like so:
My python process delivers an array, containing one array, containing one array, containing N arrays of M floats each. The input is a JPEG image.
Unwrapping it like so:
int predict(PyObject *pyFunction, unsigned char *image_pointer, unsigned long image_len) {
int result = -1;
PyObject *pImage = NULL;
PyObject *pList = NULL;
pImage = PyBytes_FromStringAndSize((const char *)image_pointer, image_len);
if (!pImage) {
fprintf(stderr, "Cannot provide image to python 'predict'\n");
return result;
}
pList = PyObject_CallFunctionObjArgs(pyFunction, pImage, NULL);
Py_DECREF(pImage);
PyArrayObject *pPrediction = reinterpret_cast<PyArrayObject *>(pList);
if (!pPrediction) {
fprintf(stderr, "Cannot predict, for whatever reason\n");
return result;
}
if (PyArray_NDIM(pPrediction) != 4) {
fprintf(stderr, "Prediction failed, returned array with wrong dimensions\n");
} else {
RESULTPTR pResult = reinterpret_cast<RESULTPTR>(PyArray_DATA(pPrediction));
int len0 = PyArray_SHAPE(pPrediction)[0];
int len1 = PyArray_SHAPE(pPrediction)[1];
int len2 = PyArray_SHAPE(pPrediction)[2];
int len3 = PyArray_SHAPE(pPrediction)[3];
for (int i = 0; i < len0; i++) {
int offs1 = i * len1;
for (int j = 0; j < len1; j++) {
int offs2 = j * len2;
for (int k = 0; k < len2; k++) {
int offs3 = k * len3;
for (int l = 0; l < len3; l++) {
float f = (*pResult)[offs1 + offs2 + offs3 + l];
//printf("data: %.8f\n", f);
}
}
}
}
result = 0;
}
Py_XDECREF(pList);
return result;
}
HTH

how to pass array from python to c using numpy

Let's say I have the following function written in C:
/* C-Code for calculating the average for a given dataset */
#define NOE 16
int calcAvg(float *data, float *avg)
{
float sum; sum = 0;
int i;
for (i = 0; i < NOE; i++)
{
data[i] = i;
sum += data[i];
}
avg = sum/n;
return 0;
}
Now I want to pass the data from an np.array to that C function "calAvg". Also, I want the result to be stored in "result" which I defined in python.
# Python Code
result = float(0)
a = np.array([1, 2, 3, 4])
myCfuncion.calc(a, result)
I have already created a C module and imported it into python. The problem I have is that I do not know how to pass pointers in such a way I showed.
Anybody does have an idea?

How to pass two dimensional array from python to function written in c++ using SWIG? [duplicate]

I have written a good bit of code in python and it works great. But now I'm scaling up the size of the problems that I'm analyzing and python is dreadfully slow. The slow part of the python code is
for i in range(0,H,1):
x1 = i - length
x2 = i + length
for j in range(0,W,1):
#print i, ',', j # check the limits
y1 = j - length
y2 = j + length
IntRed[i,j] = np.mean(RawRed[x1:x2,y1:y2])
With H and W equal to 1024 the function takes around 5 minutes to excute. I've written a simple c++ program/function that performs the same computation and it excutes in less than a second with the same data size.
double summ = 0;
double total_num = 0;
double tmp_num = 0 ;
int avesize = 2;
for( i = 0+avesize; i <X-avesize ;i++)
for(j = 0+avesize;j<Y-avesize;j++)
{
// loop through sub region of the matrix
// if the value is not zero add it to the sum
// and increment the counter.
for( int ii = -2; ii < 2; ii ++)
{
int iii = i + ii;
for( int jj = -2; jj < 2 ; jj ++ )
{
int jjj = j + jj;
tmp_num = gsl_matrix_get(m,iii,jjj);
if(tmp_num != 0 )
{
summ = summ + tmp_num;
total_num++;
}
}
}
gsl_matrix_set(Matrix_mean,i,j,summ/total_num);
summ = 0;
total_num = 0;
}
I have some other methods to perform on the 2D array. The one listed is a simple examples.
What I want to do is pass a python 2D array to my c++ function and return a 2D array back to python.
I've read a bit about swig, and have sereached pervious questions, and it seems like it's a possible solution. But I can't seem to figure out what I actually need to do.
Can I get any help? Thanks

You can use arrays as it is described here: Doc - 5.4.5 Arrays, the carray.i or std_vector.i from the SWIG library.
I find it easier to work with std::vector from the SWIG library std_vector.i to send a python list to a C++ SWIG extension. Though in your case where optimization matters, it may not be the optimal.
In your case you can define:
test.i
%module test
%{
#include "test.h"
%}
%include "std_vector.i"
namespace std {
%template(Line) vector < int >;
%template(Array) vector < vector < int> >;
}
void print_array(std::vector< std::vector < int > > myarray);
test.h
#ifndef TEST_H__
#define TEST_H__
#include <stdio.h>
#include <vector>
void print_array(std::vector< std::vector < int > > myarray);
#endif /* TEST_H__ */
test.cpp
#include "test.h"
void print_array(std::vector< std::vector < int > > myarray)
{
for (int i=0; i<2; i++)
for (int j=0; j<2; j++)
printf("[%d][%d] = [%d]\n", i, j, myarray[i][j]);
}
If you run the following python code (I used python 2.6.5), you can see that the C++ function can access the python list:
>>> import test
>>> a = test.Array()
>>> a = [[0, 1], [2, 3]]
>>> test.print_array(a)
[0][0] = [0]
[0][1] = [1]
[1][0] = [2]
[1][1] = [3]

zero terminated dynamic array as return value in Python

From python I am trying to use c function that returns dynamically allocated zero terminated array of integers:
int* my_func(void)
{
int i;
int *ret = (int*)malloc((LEN + 1) * sizeof(int));
for (i = 0; i < LEN; i++) ret[i] = 1;
ret[LEN] = 0
return ret;
}
I need something like
from ctypes import *
l = cdll.LoadLibrary("lib.so")
my_func = l.my_func
my_func.restype = c_int * LEN
for x in my_func(): print x
The problem is that LEN is unknown in python code, I need to read the array till first zero element.

Haven't really worked with ctypes yet, but what about:
from ctypes import *
l = cdll.LoadLibrary("lib.so")
my_func = l.my_func
my_func.restype = POINTER(c_int)
i = 0;
rv = my_func()
while rv[i]:
print rv[i]
i += 1

Categories

Resources