I am currently using Python to parse a C file using LibClang. I've encountered a problem while reading a C-array which size is defined by a define-directive-variable.
With node.get_children i can perfectly read the following array:
int myarray[20][30][10];
As soon as the array size is replaced with a variable, the array won't be read correctly. The following array code can't be read.
#define MAX 60;
int myarray[MAX][30][10];
Actually the parser stops at MAX and in the dump there is the error: invalid sloc.
How can I solve this?
Thanks
Run the code through a C preprocessor before trying to parse it. That will cause all preprocessor-symbols to be replaced by their values, i.e. your [MAX] will become [60].
Note that C code can also do this:
const int three[] = { 1, 2, 3 };
i.e. let the compiler deduce the length of the array from the number of initializer values given.
Or, from C99, even this:
const int hundred[] = { [99] = 4711 };
So a naive approach might still break, but I don't know anything about the capabilities of the parser you're using, of course.
Semicolon ; in the define directive way causing the error.
Related
I have a python script that I have to translate in c++, and 80 % of my python script is based on lists.
I have a file that I read, and put the data of that file in a list :
//Code to translate in c++
bloc = [line]
for b in range(11):
bloc.append(lines[i + 1])
i += 1
I make my stuff with that data and then, I do it again until I read the whole file.
And finally I want to be able to get data of this list doing something like :
#Python script
var = bloc[0, 1, 2, 3 ...]
I'll respond to any questions you need more infos
The C++ container closest to a python List is a std::vector. However contrary to python a std::vector contains only one type of element. You have to declare what the vector will hold.
In your case it would be std::string (reading from a file).
So:
std::vector<std::string> cpp_list; // container for lines (stored as string )from the file
is equivalent to python python_list = []
should get you started.
With a std::vector you do not strictly need to allocated storage upfront but for performance reason it is better to do is if you know the required size in advance.
if you use cpp_list.reserve(something) or do not do any memory allocation, you must push in the vector using cpp_list.push_back(...) which is similar to pyhton_list.append(...)
If you allocate memory upfront eg: std::vector<std::string> cpp_list(nb_lines)
You must use indexing as in python eg cpp_list[3] = something
I'm new to python and I'm trying to write some code using python wrappers for epanet which I believe are SWIG-wrapped. There is a method I need to use, as outlined below from the source. However I can't seem to get the method to accept any type of value for the 3rd argument "values:".. Anything I try I get the error: TypeError: in method 'ptrn_set', argument 3 of type 'double *'
How do I convert my value/values to 'double *' within Python? what is the solution?
Thankful for any help...
"""
ptrn_set(ph, index, values, len) -> int
Parameters
----------
ph: Handle
index: int
values: double *
len: int
"""
return _toolkit.ptrn_set(ph, index, values, len)
I found this here:
https://docs.python.org/2.4/lib/typesnumeric.html:
Floating point numbers are implemented using double in C
Did you try float?
Python itself does not support values of type double *. In C, that is a pointer-to-a-double-precision-floating-point-number. In Python, effectively every name is a pointer, but you can't have a (native) value that is a pointer.
From context, it looks like they actually want an array of doubles. They really ought to write that as double [], but double * is compatible with it. You could try
values = [1.0, 2.0]
_toolkit.ptrn_set(ph, index, values, len(values))
It's not very Pythonic to do that, but if it works, it works. If that doesn't work, see if your expanet python wrappers provide a way of constructing such values.
If your expanet Python wrappers don't provide a way to construct such values, you can try the cffi library, but it's quite tricky. I think x = ffi.new("double[10]") will give you an array of 10 doubles.
I know some time passed an you probably already solved it, but it might be useful for someone else.
you can use the following function to create an array that will be accepted by epanet:
def make_array(values):
dbl_arr = en.doubleArray(len(values))
for i in range(len(values)):
dbl_arr[i] = values[i]
return dbl_arr
the input is an array of float, like [1.2 , 0.8].
This assuming you imported the epanet python library as
from epanet import toolkit as en,
source: https://github.com/OpenWaterAnalytics/epanet-python/issues/57
I have a written a DLL in which I'm getting one of the paths:
//demo.h
__declspec(dllexport) void pathinfo(char * path);
Something is being done in the code to get this path.
And now, the python script that I have written to retrieve this path from the DLL is as shown:
//demo.py
import sys
import ctypes
from ctypes import *
class demo(object):
def __init__(self):
self.demoDLL=CDLL("demo.dll")
def pathinfo(self):
path=c_char()
self.demoDLL.pathinfo.argtypes(POINTER(c_char))
self.demoDLL.pathinfo.result=None
self.demoDLL.pathinfo(byref(path))
return path.value
if __name__=='__main__':
abc=demo()
path_info=abc.pathinfo()
print "information of the path:",path_info
But the value that I'm able to see is just the first character of the path instead of the whole string.
Can anybody help me with this problem?
The reason you see only the first character is that by calling c_char() you create a single char value that Python treats like a str (Python 2) object or bytes (Python 3) object of length 1. You are probably lucky that you do not get a segmentation fault. By writing more than 1 byte or a NULL-terminated string of length > 0 (e.g. with strcpy) in the C code, you actually produce an undetected buffer overflow. ctypes does not know how many bytes you have written at the pointer's memory location. path.value is still a str / bytes of length 1.
It would be better to change the C pathinfo function into someting like
size_t pathinfo(char* path, size_t bsize);
Use ctypes.create_string_buffer() to allocate memory in your Python code and let pathinfo return the length of the result. Of course you have to check, whether char* path is large enough using bsize in your C-Code.
The Python-code would look like this:
buf = ctypes.create_string_buffer(256)
result_len = pathinfo(buf, len(buf))
if result_len == len(buf):
# buffer may have been too short,
# try again with larger buffer
...
restlt_str = buf[0:result_len].decode('utf-8') # or another encoding
Also be aware of NULL-termination in the C domain, character encodings when converting python strings to char* and back, the changes regarding str / bytes in ctypes regarding Python 2 and Python 3.
Haven't done it, but have done the following:
To pass an array of ints, for example, use this:
Create a type of, say, 20 ints:
Ints20 = c_int * 20
Create an instance:
data = Ints20()
And then you can pass data.
Extract the numbers from data by using the list function:
values = list(data)
So may be you can do the same with chars:
Chars20 = c_char * 20
path = Chars20()
and then:
self.demoDLL.pathinfo(path)
In the case of c_char array there is no need to use the list function, but this works:
return path.value
I've been writing a Python extension that writes into a NumPy array from C. During testing, I noticed that certain very large arrays would generate a segfault when I tried to access some of their elements.
Specifically, the last line of the following code segment fails with a segfault:
// Size of buffer we will write to
npy_intp buffer_len_alt = BUFFER_LENGTH;
//
PyArray_Descr * dtype;
dtype = PyArray_DescrFromType(NPY_BYTE);
PyObject* column = PyArray_Zeros(1, &buffer_len_alt, dtype, 0);
//Check that array creation succeeds
if (column == NULL){
// This exit point is not reached, so it looks like everything is OK
return (PyObject *) NULL;
}
// Get the array's internal buffer so we can write to it
output_buffer = PyArray_BYTES((PyArrayObject *)column);
// Try writing to the buffer
output_buffer[0] = 'x'; //No segfault
output_buffer[((int) buffer_len_alt) - 1] = 'x'; // Segfault here
I checked and found that the error occurs only when I try to allocate an array of about 3GB (i.e. BUFFER_LENGTH is about 3*2^30). It's not surprising that an allocation of this size would fail, even if Python is using it's custom allocator. What really concerns me is that NumPy did not raise an error or otherwise indicate that the array creation did not go as planned.
I have already tried checking PyArray_ISCONTIGUOUS on the returned array, and using PyArray_GETCONTIGUOUS to ensure it is a single memory segment, but the segfault would still occur. NPY_ARRAY_DEFAULT creates contiguous arrays, so this shouldn't be necessary anyways.
Is there some error flag I should be checking? How can I detect/prevent this situation in the future? Setting BUFFER_LENGTH to a smaller value obviously works, but this value is determined at runtime and I would like to know the exact bounds.
EDIT:
As #DavidW pointed out, the error stems from casting buffer_len_alt to an int, since npy_intp can be a 64-bit number. Replacing the cast to int with a cast to 'unsigned long' fixes the problem for me.
The issue (diagnosed in the comments) was actually with the array lookup rather than the allocation of the array. Your code contained the line
output_buffer[((int) buffer_len_alt) - 1] = 'x'
When buffer_len_alt (approx value 3000000000) was cast to an (32 bit) int (maximum value 2147483647) you ended up with an invalid address, probably a large negative number.
The solution is just to use
output_buffer[buffer_len_alt - 1] = 'x'
(i.e. I don't see why you should need a cast at all).
Briefly:
Is there an efficient way to make a numpy array given a pointer in memory to the array, it's type, and the number of elements?
More detail:
I am working with a python framework which has an object.GetData() command that is supposed to return a pointer to the data (an array of 35,000 int8) of this object.
I'm supposed to be able efficiently load these integers to a numpy array through
arr = numpy.frombuffer(object.GetData(),count=35000,dtype="int8")
but this doesn't seem to work. I get an error message ValueError: buffer is smaller than requested size. Changing the length, I can get it to output an array, but typically less than 20 integers in length (usually 0 or 1 integers).
I believe I can access the pointer to the start of the array, in hex form, through
hex(id(object.GetData()))
which looks like it gives addresses (e.g. 0x10fd8c670) but I don't know if this is the actual address.
I'm more comfortable in python than c++, but there could be a bug in the c++ code. The c++ code for GetData is:
const _Tp* GetData() const
{
// Return a const pointer to the internal data
return (fData.size() > 0 ) ? &(fData)[0] : NULL;
}
where fdata is initialized as a VecType through:
VecType fData;
Right now I can access each element of the object's data through an object.At(i) command where i is the index of the data array of object, but it is very slow to load each element into a numpy array this way, and I'm dealing with a lot of data. For reference, the At command in the c++ code does this:
_Tp At(size_t i) const
{
return fData.at(i);
}
Any help would be appreciated. I don't have a ton of experience with pointers, and even less with pointers in python, but I would like to figure this out in python rather than re-write all my code in c++. Thanks!