I have the following python code:
r = range(1,10)
r_squared = []
for item in r:
print item
r_squared.append(item*item)
How would I convert this code to C? Is there something like a mutable array in C or how would I do the equivalent of the python append?
simple array in c.Arrays in the C are Homogenous
int arr[10];
int i = 0;
for(i=0;i<sizeof(arr);i++)
{
arr[i] = i; // Initializing each element seperately
}
Try using vectors in C go through this link
/ vector-usage.c
#include <stdio.h>
#include "vector.h"
int main() {
// declare and initialize a new vector
Vector vector;
vector_init(&vector);
// fill it up with 150 arbitrary values
// this should expand capacity up to 200
int i;
for (i = 200; i > -50; i--) {
vector_append(&vector, i);
}
// set a value at an arbitrary index
// this will expand and zero-fill the vector to fit
vector_set(&vector, 4452, 21312984);
// print out an arbitrary value in the vector
printf("Heres the value at 27: %d\n", vector_get(&vector, 27));
// we're all done playing with our vector,
// so free its underlying data array
vector_free(&vector);
}
Arrays in C are mutable by default, in that you can write a[i] = 3, just like Python lists.
However, they're fixed-length, unlike Python lists.
For your problem, that should actually be fine. You know the final size you want; just create an array of that size, and assign to the members.
But of course there are problems for which you do need append.
Writing a simple library for appendable arrays (just like Python lists) is a pretty good learning project for C. You can also find plenty of ready-made implementations if that's what you want, but not in the standard library.
The key is to not use a stack array, but rather memory allocated on the heap with malloc. Keep track of the pointer to that memory, the capacity, and the used size. When the used size reaches the capacity, multiply it by some number (play with different numbers to get an idea of how they affect performance), then realloc. That's just about all there is to it. (And if you look at the CPython source for the list type, that's basically the same thing it's doing.)
Here's an example. You'll want to add some error handling (malloc and realloc can return NULL) and of course the rest of the API beyond append (especially a delete function, which will call free on the allocated memory), but this should be enough to show you the idea:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct {
int *i;
size_t len;
size_t capacity;
} IntArray;
IntArray int_array_make() {
IntArray a = {
.i = malloc(10 * sizeof(int)),
.len = 0,
.capacity = 10
};
return a;
}
void int_array_append(IntArray *a, int value) {
if (a->len+1 == a->capacity) {
size_t new_capacity = (int)(a->capacity * 1.6);
a->i = realloc(a->i, new_capacity * sizeof(int));
a->capacity = new_capacity;
}
a->i[a->len++] = value;
}
int main(int argc, char *argv[]) {
IntArray a = int_array_make();
for (int i = 0; i != 50; i++)
int_array_append(&a, i);
for (int i = 0; i != a.len; ++i)
printf("%d ", a.i[i]);
printf("\n");
}
c doesnt have any way of dynamically increasing the size of the array like in python. arrays here are of fixed length
if you know the size of the array that you will be using, u can use this kind of declaration, like this
int arr[10];
or if you would want to add memery on the fly (in runtime), use malloc call along with structure (linked lists)
Related
I am trying to instantiate an array in GPU memory using CuPy and then pass the pointer to this array to C++ using pybind11.
A minimal example of the problem I am running into is shown below.
Python
import demolib #compiled pybind11 library
import cupy as cp
x = cp.ones(100000)
y = cp.ones(100000)
demolib.pyadd(len(x),x.data.ptr,y.data.ptr)
C++/CUDA
#include <iostream>
#include <math.h>
#include <cuda_runtime.h>
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
namespace py = pybind11;
// Error Checking Function
#define gpuErrchk(ans) { gpuAssert((ans), __FILE__, __LINE__); }
inline void gpuAssert(cudaError_t code, const char *file, int line, bool abort=true)
{
if (code != cudaSuccess)
{
fprintf(stderr,"GPUassert: %s %s %d\n", cudaGetErrorString(code), file, line);
if (abort) exit(code);
}
}
// Simple CUDA kernel
__global__
void cuadd(int n, float *x, float *y)
{
int index = blockIdx.x * blockDim.x + threadIdx.x;
int stride = blockDim.x * gridDim.x;
for (int i = index; i < n; i += stride)
y[i] = x[i] + y[i];
}
// Simple wrapper function to be exposed to Python
int pyadd(int N, float *x, float *y)
{
// Run kernel on 1M elements on the GPU
int blockSize = 256;
int numBlocks = (N + blockSize - 1) / blockSize;
cuadd<<<numBlocks, blockSize>>>(N,x,y);
// Wait for GPU to finish before accessing on host
gpuErrchk( cudaPeekAtLastError() );
gpuErrchk( cudaDeviceSynchronize() );
return 0;
}
PYBIND11_MODULE(demolib, m) {
m.doc() = "pybind11 example plugin"; // optional module docstring
m.def("pyadd", &pyadd, "A function which adds two numbers");
}
The code throws the following error:
GPUassert: an illegal memory access was encountered /home/tbm/cuda/add_pybind.cu 47
I realize that this specific example could be implemented using a cupy user defined kernel, but the end goal is to be able to do zero-copy passes of cupy arrays into a larger codebase which would be prohibitive to rewrite in this paradigm.
I have also located this GitHub Issue, which is the the reverse of what I'm trying to do.
The fix was to change the argument types of pyadd to int and cast the int to float pointers as shown below. As pointed out in the comments, this was figured out by referencing another question.(unanswered at the time of posting)
int pyadd(int N, long px, long py)
{
float *x = reinterpret_cast<float*> (px);
float *y = reinterpret_cast<float*> (py);
.
.
.
I'm trying to figure out, how I could achieve this:
I'm having a Python script, which in the end produces a Numpy array, an array of arrays of floats, to be more specific. I have all properly set: I can pass parameters from C to Python, launch Py functions from C, and process returned scalar values in C.
What I'm currently not being able to do, is to return such a Numpy array as a result of a Py function to C.
Could somebody probably provide me a pointer, how to achieve this?
TIA
What you need to look at is Inter Process communication (IPC). There are several ways to perform it.
You can use one of:
Files (Easy to use)
Shared memory (really fast)
Named pipes
Sockets (slow)
See Wikipedia's IPC page and find the best approach for your needs.
Here's a small working demo example (1D, not 2D array! it's not perfect, adjust to your needs).
# file ./pyscript.py
import numpy as np
# print inline
np.set_printoptions(linewidth=np.inf)
x = np.random.random(10)
print(x)
# [0.52523722 0.29053534 0.95293405 0.7966214 0.77120688 0.22154705 0.29398872 0.47186567 0.3364234 0.38107864]
~ demo.c
// file ./demo.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
int main()
{
int fd[2];
pipe(fd); // create pipes
char buf[4096];
pid_t pid=fork();
if (pid==0) { // child process
dup2(fd[1],1);
close(fd[0]);
close(fd[1]);
char *pyscript = "./pyscript.py";
char *args[] = {"python3", pyscript, (char*)NULL};
execv("/usr/bin/python3",args);
}
else {
int status;
close(fd[1]);
int bytes = read(fd[0], buf, sizeof(buf));
printf("Python script output: %.*s\n", bytes, buf);
char* values[10];
int count = 0;
values[count++] = &buf[1]; // ignore the '[' coming from numpy array output
char* p = buf;
while (*p) {
if (*p == ' ') {
*p = 0;
values[count++] = p + 1;
}
p++;
}
float a[10];
float f;
for (int i = 0; i < 10; i++) {
printf("%f\n", f = atof(values[i]) ); // float values
a[i] = f;
}
waitpid(pid, &status, 0);
}
return 0;
}
Sample output
# cc demo.c
# ./a.out
Python script output: [0.23286839 0.54437959 0.37798547 0.17190732 0.49473837 0.48112695 0.93113395 0.20877592 0.96032973 0.30025713]
0.23286839
0.54437959
0.232868
0.544380
0.377985
0.171907
0.494738
0.481127
0.931134
0.208776
0.960330
0.300257
a will be your desired result, an array of float.
One has to use the PyList API for decoding list objects from Python to C
https://docs.python.org/3.3/c-api/list.html?highlight=m
Solved.
MNIST is the hello world of machine learning and I've practiced it with TensorFlow and with pure python and numpy.
For more practice I am trying to write it in C on my own with only the standard library because I am relatively new to C and it's a great way to learn.
It's taken three weeks, and a lot of SEGFAULTS but I get 81% accuracy. Not very good but it's for learning.
The most troubling stuff was of course malloc/free for the data in the matrix struct as below:
typedef struct matrix{
int rows, cols;
float *data;
} matrix;
The forward and backward passes have things like:
1) matrix dot product
2) matrix add
3) matrix subtract
4) activation function (sigmoid in this case)
To avoid memory leaks I pass in three structs like so:
void matrix_add(matrix *a, matrix *b, matrix *res);
If res requires a dimensions change from a previous layer, then I free it and do a new malloc like so:
void zero_out_data(matrix *res, int rows, int cols)
{
if (res->rows != rows || res->cols != cols)
{
if ((res->rows*res->cols) != (rows*cols))
{
free(res->data);
res->data = NULL;
free(res);
res = NULL;
res = malloc(sizeof(matrix));
// make_matrix will calloc the data based on rows*cols
// any other init stuff that could be needed
make_matrix(res, rows, cols);
}
res->rows = rows;
res->cols = cols;
}
else {
res->rows = rows;
res->cols = cols;
for (int i =0; i < (rows*cols); i++)
{
res->data[i] = 0.0;
}
}
}
Then I can use that like so:
void sigmoid(matrix *z, matrix *res)
{
zero_out_data(res, z->rows, z->cols);
for (int i = 0; i < (z->rows*z->cols); i++)
{
res->data[i] = 1.0/(1.0+exp(-z->data[i]));
}
}
This gets very messy because a single forward pass has the following:
/* forward pass */
for (int k=0; k < (network->num_layers-1); k++)
{
matrix_dot(network->weights[k], activation, dot);
matrix_add(dot, network->biases[k], zs[k]);
sigmoid(zs[k], activation);
sigmoid(zs[k], activations[k+1]);
}
/* end forward pass */
As you can imagine the backprop gets alot messier. I have to pre-create 8 different matrices, plus many more of those pointers to pointers of matrices like the activations and zs above, for the gradient descent.
What I would like to be able to do is return a matrix from a function like matrix_dot so that I can do:
sigmoid(matrix_add(matrix_dot(network->weights[k], activation), network->biases[k]));
That's kind of in the style of python/numpy.
Of course I can't return a local variable from a function because it's taken off the stack once the function returns.
If I return a pointer, then the above style will cause sever memory leaks.
Please note: I am not trying to write my own library/framework. I am simply trying to learn neural networks and coding in C. I have been a python developer for 7 years or so, and my C skills need improvement.
Memory leak in void zero_out_data(matrix *res, int rows, int cols)
matrix *res malloc out of the function and pass to zero_out_data. In zero_out_data, res is free and malloc again. If you want to change pointer res's value, then you need parameter like matrix **res.
If you want zero out data, no need malloc new matrix, just malloc the data part. I think your make_matrix function can malloc memory for data.
void zero_out_data(matrix *res, int rows, int col) {
if (res->data == NULL) {
make_matrix(res, rows, cols);
} else if (res->rows != rows || res->cols != cols) {
if ((res->rows*res->cols) != (rows*cols))
{
free(res->data);
res->data = NULL;
make_matrix(res, rows, cols);
}
}
res->rows = rows;
res->cols = cols;
for (int i =0; i < (rows*cols); i++)
{
res->data[i] = 0.0;
}
}
How to implement this: sigmoid(matrix_add(matrix_dot(network->weights[k], activation), network->biases[k])); ?
You can use static or global variables to implement what you want. This will not be thread safe and reentrant. Examples in below:
matrix *matrix_dot(matrix *in_a, matrix *in_b)
{
static matrix res = {0, 0, NULL}; // static variable
// calculate the res's cols and rows number
zero_out_data(&res, res_cols, res_rows); // malloc new data
// do some math.
return &res;
}
// matrix_add will be just like matrix_dot
// I was wrong about sigmod no need new matrix. sigmod can also do it like matrix_dot.
You can use global variable replace static variable.
If you want thread-safe or reentrant, then just use local variable, then you can do it like this.
matrix *matrix_dot(matrix *in_a, matrix *in_b, matrix *res)
{
zero_out_data(res, xxx, xxx);
// do some math
return res;
}
// matrix_add will be the same.
// define local variables.
matrix add_res, dot_res, sig_res;
add_res->data = NULL;
dot_res->data = NULL;
sig_res->data = NULL;
sigmod(matrix_add(matrix_dot(network->weights[k], activation, &dot_res), network->biases[k], &add_res), &sig_res)
// Now remember to free data in matrix
free(add_res->data);
free(dot_res->data);
free(sig_res->data);
I have a C++ function computing a large tensor which I would like to return to Python as a NumPy array via pybind11.
From the documentation of pybind11, it seems like using STL unique_ptr is desirable.
In the following example, the commented out version works, whereas the given one compiles but fails at runtime ("Unable to convert function return value to a Python type!").
Why is the smartpointer version failing? What is the canonical way to create and return a NumPy array?
PS: Due to program structure and size of the array, it is desirable to not copy memory but create the array from a given pointer. Memory ownership should be taken by Python.
typedef typename py::array_t<double, py::array::c_style | py::array::forcecast> py_cdarray_t;
// py_cd_array_t _test()
std::unique_ptr<py_cdarray_t> _test()
{
double * memory = new double[3]; memory[0] = 11; memory[1] = 12; memory[2] = 13;
py::buffer_info bufinfo (
memory, // pointer to memory buffer
sizeof(double), // size of underlying scalar type
py::format_descriptor<double>::format(), // python struct-style format descriptor
1, // number of dimensions
{ 3 }, // buffer dimensions
{ sizeof(double) } // strides (in bytes) for each index
);
//return py_cdarray_t(bufinfo);
return std::unique_ptr<py_cdarray_t>( new py_cdarray_t(bufinfo) );
}
A few comments (then a working implementation).
pybind11's C++ object wrappers around Python types (like pybind11::object, pybind11::list, and, in this case, pybind11::array_t<T>) are really just wrappers around an underlying Python object pointer. In this respect there are already taking on the role of a shared pointer wrapper, and so there's no point in wrapping that in a unique_ptr: returning the py::array_t<T> object directly is already essentially just returning a glorified pointer.
pybind11::array_t can be constructed directly from a data pointer, so you can skip the py::buffer_info intermediate step and just give the shape and strides directly to the pybind11::array_t constructor. A numpy array constructed this way won't own its own data, it'll just reference it (that is, the numpy owndata flag will be set to false).
Memory ownership can be tied to the life of a Python object, but you're still on the hook for doing the deallocation properly. Pybind11 provides a py::capsule class to help you do exactly this. What you want to do is make the numpy array depend on this capsule as its parent class by specifying it as the base argument to array_t. That will make the numpy array reference it, keeping it alive as long as the array itself is alive, and invoke the cleanup function when it is no longer referenced.
The c_style flag in the older (pre-2.2) releases only had an effect on new arrays, i.e. when not passing a value pointer. That was fixed in the 2.2 release to also affect the automatic strides if you specify only shapes but not strides. It has no effect at all if you specify the strides directly yourself (as I do in the example below).
So, putting the pieces together, this code is a complete pybind11 module that demonstrates how you can accomplish what you're looking for (and includes some C++ output to demonstrate that is indeed working correctly):
#include <iostream>
#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
namespace py = pybind11;
PYBIND11_PLUGIN(numpywrap) {
py::module m("numpywrap");
m.def("f", []() {
// Allocate and initialize some data; make this big so
// we can see the impact on the process memory use:
constexpr size_t size = 100*1000*1000;
double *foo = new double[size];
for (size_t i = 0; i < size; i++) {
foo[i] = (double) i;
}
// Create a Python object that will free the allocated
// memory when destroyed:
py::capsule free_when_done(foo, [](void *f) {
double *foo = reinterpret_cast<double *>(f);
std::cerr << "Element [0] = " << foo[0] << "\n";
std::cerr << "freeing memory # " << f << "\n";
delete[] foo;
});
return py::array_t<double>(
{100, 1000, 1000}, // shape
{1000*1000*8, 1000*8, 8}, // C-style contiguous strides for double
foo, // the data pointer
free_when_done); // numpy array references this parent
});
return m.ptr();
}
Compiling that and invoking it from Python shows it working:
>>> import numpywrap
>>> z = numpywrap.f()
>>> # the python process is now taking up a bit more than 800MB memory
>>> z[1,1,1]
1001001.0
>>> z[0,0,100]
100.0
>>> z[99,999,999]
99999999.0
>>> z[0,0,0] = 3.141592
>>> del z
Element [0] = 3.14159
freeing memory # 0x7fd769f12010
>>> # python process memory size has dropped back down
I recommend using ndarray. A foundational principle is that the underlying data is never copied unless explicitly requested (or you quickly end up with huge inefficiencies). Below is an example of it in use, but there are other features I haven't shown, including conversion to Eigen arrays (ndarray::asEigen(array)), which makes it pretty powerful.
Header:
#ifndef MYTENSORCODE_H
#define MYTENSORCODE_H
#include "ndarray_fwd.h"
namespace myTensorNamespace {
ndarray::Array<double, 2, 1> myTensorFunction(int param1, double param2);
} // namespace myTensorNamespace
#endif // include guard
Lib:
#include "ndarray.h"
#include "myTensorCode.h"
namespace myTensorNamespace {
ndarray::Array<double, 2, 1> myTensorFunction(int param1, double param2) {
std::size_t const size = calculateSize();
ndarray::Array<double, 2, 1> array = ndarray::allocate(size, size);
array.deep() = 0; // initialise
for (std::size_t ii = 0; ii < size; ++ii) {
array[ii][ndarray::view(ii, ii + 1)] = 1.0;
}
return array;
}
} // namespace myTensorNamespace
Wrapper:
#include "pybind11/pybind11.h"
#include "ndarray.h"
#include "ndarray/pybind11.h"
#include "myTensorCode.h"
namespace py = pybind11;
using namespace pybind11::literals;
namespace myTensorNamespace {
namespace {
PYBIND11_MODULE(myTensorModule, mod) {
mod.def("myTensorFunction", &myTensorFunction, "param1"_a, "param2"_a);
}
} // anonymous namespace
} // namespace myTensorNamespace
I have a C++ function returning a std::vector and I want to use it in python, so I'm using the C numpy api:
static PyObject *
py_integrate(PyObject *self, PyObject *args){
...
std::vector<double> integral;
cpp_function(integral); // This changes integral
npy_intp size = {integral.size()};
PyObject *out = PyArray_SimpleNewFromData(1, &size, NPY_DOUBLE, &(integral[0]));
return out;
}
Here's how I call it from python:
import matplotlib.pyplot as plt
a = py_integrate(parameters)
print a
fig = plt.figure()
ax = fig.add_subplot(111)
ax.plot(a)
print a
What happens is: The first print is ok, the values are correct. But when I plot a they are not; in the second print I see very strange values like 1E-308 1E-308 ... or 0 0 0 ... as an uninitialized memory. I don't understand why the first print is ok.
Partial solution (not working):
static void DeleteVector(void *ptr)
{
std::cout << "Delete" << std::endl;
vector * v = static_cast<std::vector<double> * >(ptr);
delete v;
return;
}
static PyObject *
cppfunction(PyObject *self, PyObject *args)
{
std::vector<double> *vector = new std::vector<double>();
vector->push_back(1.);
PyObject *py_integral = PyCObject_FromVoidPtr(vector, DeleteVector);
npy_intp size = {vector->size()};
PyArrayObject *out;
((PyArrayObject*) out)->base = py_integral;
return (PyObject*)(out);
}
Your std::vector object appears to be local to that function. PyArray_SimpleNewFromData does not make a copy of the data you pass it. It just keeps a pointer. So once your py_integrate function returns, the vector is deallocated. The print works the first time because nothing has written over the freed memory yet, but by the time you get to the next print, something else has used that memory, causing the values to be different.
You need to make a NumPy array that owns its own storage space and then copy the data into it.
Alternatively, allocate your vector on the heap. Then store a pointer to it in a CObject. Provide a destructor that deletes the vector. Then, take a look at the C-level PyArrayObject type. It has a PyObject * member called base. Store your CObject there. Then when the NumPy array is garbage collected, the reference count on this base object will be decremented, and assuming you haven't taken a copy of it elsewhere, your vector will be deleted thanks to the destructor you provided.
Fixer-upper
You forgot to actually create the PyArray. Try this:
(You didn't post DeleteVector, so I can only hope that it's right)
std::vector<double> *vector = new std::vector<double>();
vector->push_back(1.);
PyObject *py_integral = PyCObject_FromVoidPtr(vector, DeleteVector);
npy_intp size = {vector->size()};
PyObject *out = PyArray_SimpleNewFromData(1, &size, NPY_DOUBLE, &((*vector)[0]));
((PyArrayObject*) out)->base = py_integral;
return out;
Note: I'm not a C++ programmer, so I can only assume that &((*vector)[0]) works as intended with a pointer to a vector. I do know that the vector reallocate its storage area if you grow it, so don't increase its size after getting that pointer or it won't be valid anymore.
You will need to make a copy of the vector, since the vector will go out of scope and the memory will no longer be usable by the time you need it in Python (as stated by kwatford).
One way to make the Numpy array you need (by copying the data) is:
PyObject *out = nullptr;
std::vector<double> *vector = new std::vector<double>();
vector->push_back(1.);
npy_intp size = {vector.size()};
out = PyArray_SimpleNew(1, &size, NPY_DOUBLE);
memcpy(PyArray_DATA((PyArrayObject *) out), vector.data(), vector.size());