I have a C function that takes a pointer (for output) to the following structure:
class MacFrame(ctypes.Structure):
_fields_ = [("buffer", ctypes.c_ubyte * 128),
("length", ctypes.c_ubyte),
("nodeid", ctypes.c_uint)]
Then, I need to make the following Python function that calls the C function converts the buffer and length field to a bytearray object to return it, but I am unclear on what the syntax should be:
def recv(self, bufsize):
frame = MacFrame()
self.Api.otListenerRead(ctypes.byref(frame))
return bytearray(frame.buffer, frame.length), frame.nodeid
I know the bytearray(frame.buffer, frame.length) is incorrect, but I don't know what the best way is.
Related
I have a structure defined using class type and I want to initialize test_arr like this:
from ctypes import *
class ST_DATA(Structure):
_fields_ = [("test1", c_int),
("test2", c_double),
("test_arr", c_double*2)]
stmyData = ST_DATA(1, 2, (3, 4))
print(stmyData.test1, stmyData.test2, stmyData.test_arr)
Result:
1 2.0 <__main__.c_double_Array_2 object at 0x11BB2AD8>
I don't understand this situation. How can I initialize this structure array in Python?
It is initialized, but the default print representation doesn't reflect the data...just the object's type and address.
It's a good habit to write a debug representation for classes so printing a class instance is more convenient. In this case, ctypes arrays can be converted to lists to see their content:
from ctypes import *
class ST_DATA(Structure):
_fields_ = [('test1', c_int),
('test2', c_double),
('test_arr', c_double * 2)]
# debug representation "magic" method
def __repr__(self):
return f'ST_DATA(test1={self.test1}, test2={self.test2}, test_arr={list(self.test_arr)})'
stmyData = ST_DATA(1, 2, (3, 4))
print(stmyData)
Output:
ST_DATA(test1=1, test2=2.0, test_arr=[3.0, 4.0])
See also:
object.__repr__(self)
Called by the repr() built-in function to compute the “official” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form <...some useful description...> should be returned. The return value must be a string object. If a class defines __repr__() but not __str__(), then __repr__() is also used when an “informal” string representation of instances of that class is required.
This is typically used for debugging, so it is important that the representation is information-rich and unambiguous.
object.__str__(self)
Called by str(object) and the built-in functions format() and print() to compute the “informal” or nicely printable string representation of an object. The return value must be a string object.
This method differs from object.__repr__() in that there is no expectation that __str__() return a valid Python expression: a more convenient or concise representation can be used.
The default implementation defined by the built-in type object calls object.__repr__().
This might be a silly question but I couldn't find a good answer in the docs or anywhere.
If I use struct to define a binary structure, the struct has 2 symmetrical methods for serialization and deserialization (pack and unpack) but it seems ctypes doesn't have a straightforward way to do this. Here's my solution, which feels wrong:
from ctypes import *
class Example(Structure):
_fields_ = [
("index", c_int),
("counter", c_int),
]
def Pack(ctype_instance):
buf = string_at(byref(ctype_instance), sizeof(ctype_instance))
return buf
def Unpack(ctype, buf):
cstring = create_string_buffer(buf)
ctype_instance = cast(pointer(cstring), POINTER(ctype)).contents
return ctype_instance
if __name__ == "__main__":
e = Example(12, 13)
buf = Pack(e)
e2 = Unpack(Example, buf)
assert(e.index == e2.index)
assert(e.counter == e2.counter)
# note: for some reason e == e2 is False...
The PythonInfo wiki has a solution for this.
FAQ: How do I copy bytes to Python from a ctypes.Structure?
def send(self):
return buffer(self)[:]
FAQ: How do I copy bytes to a ctypes.Structure from Python?
def receiveSome(self, bytes):
fit = min(len(bytes), ctypes.sizeof(self))
ctypes.memmove(ctypes.addressof(self), bytes, fit)
Their send is the (more-or-less) equivalent of pack, and receiveSome is sort of a pack_into. If you have a "safe" situation where you're unpacking into a struct of the same type as the original, you can one-line it like memmove(addressof(y), buffer(x)[:], sizeof(y)) to copy x into y. Of course, you'll probably have a variable as the second argument, rather than a literal packing of x.
Have a look at this link on binary i/o in python:
http://www.dabeaz.com/blog/2009/08/python-binary-io-handling.html
Based on this you can simply write the following to read from a buffer (not just files):
g = open("foo","rb")
q = Example()
g.readinto(q)
To write is simply:
g.write(q)
The same for using sockets:
s.send(q)
and
s.recv_into(q)
I did some testing with pack/unpack and ctypes and this approach is the fastest except for writing straight in C
Tested on Python3
e = Example(12, 13)
serialized = bytes(e)
deserialized = Example.from_buffer_copy(serialized)
I have function in C that reads byte by byte from a given buffer and returns the result of a mathematical formula.
I need to write the same function in Python
The buffer in C is struct and in python i used ctypes Structure class
my prototype in c is int calc_formula(char *buff,int len)
so calling the function in c is staright forward but how i define such function in Python?
I try to define the following and have some questions
def calc_formula(buff,len):
some code
In C I called the function with pointer to the strcut first char. How do I do it in Python? is buff passed as pointer? My buffer is very large and if it can't be done, I will use global variable (which is less preferred).
I need to read the buffer byte by byte, so in c I simply increment the buffer pointer. What's the way to do it in python? I read about ctypes union class that I can define over the Structure and go over it byte by byte. Do you have a better solution?
UPDATE
i tried bbrame solution :
def calc_formula(buff, len):
sum = 0
for curChar in buff:
numericByteValue = ord(curChar)
sum += numericByteValue
return sum
with When i try its code with calc_formula(input_buff,len) , i get the following:
"*error:TypeError: 't_input_buff' object is not iterable*" - input_buff is instance of t_input_buff that is Class(Structure) . what can be the problem?
(it give me the error when it try to do the for command)
In c, try using the type c_char_p rather than char* (see the ctypes documentation).
In python the parameter (buff) will be a python string. Loop through it as follows:
def calc_formula(buff, len):
sum = 0
for curChar in buff:
numericByteValue = ord(curChar)
sum += numericByteValue
return sum
UPDATE
i solve it with ctypes union class
for answer look in this question
I'm trying to convert a 16 byte blob of data returned by socket.inet_pton into a ctypes array of unsigned bytes. My data structure looks like this:
class in6_addr(ctypes.Structure):
_fields_ = (("Byte", ctypes.c_ubyte * 16),)
And the blob is just:
data = socket.inet_pton(socket.AF_INET6, "2001::3")
However, these attempts get errors:
sin6 = in6_addr()
# TypeError: expected c_ubyte_Array_16 instance, got str
sin6.Byte = data
# TypeError: cast() argument 2 must be a pointer type, not c_ubyte_Array_16
sin6.Byte = ctypes.cast(data, ctypes.c_ubyte * 16)
# TypeError: incompatible types, LP_c_ubyte instance instead of c_ubyte_Array_16 instance
sin6.Byte = ctypes.cast(data, ctypes.POINTER(ctypes.c_ubyte))
All of the code: http://codepad.org/2cjyVXBA
Any ideas what type I need to cast to?
I might be completely wrong here (and it does seem a bit complex) but this works for me:
sin6.Byte = (ctypes.c_ubyte*16)(*list(bytearray(data)))
I had to convert the data into a list of integers and unpack them for the constructor. There must be an easier way!
Arguably easier:
sin6.Byte = cast(data, ctypes.POINTER(ctypes.c_ubyte * 16)).contents
or:
sin6.Byte = (ctypes.c_ubyte * 16)(*[x for x in data])
Using bytes stream:
import io
io.BytesIO(data).readinto(sin6.Byte)
And since the considered structure contains the single field, the field name can be omited:
sin6 = cast(data, ctypes.POINTER(ctypes.c_ubyte * 16)).contents
sin6 = (ctypes.c_ubyte * 16)(*[x for x in data])
io.BytesIO(data).readinto(sin6)
I'm trying to figure out why this works after lots and lots of messing about with
obo.librar_version is a c function which requires char ** as the input and does a strcpy
to passed in char.
from ctypes import *
_OBO_C_DLL = 'obo.dll'
STRING = c_char_p
OBO_VERSION = _stdcall_libraries[_OBO_C_DLL].OBO_VERSION
OBO_VERSION.restype = c_int
OBO_VERSION.argtypes = [POINTER(STRING)]
def library_version():
s = create_string_buffer('\000' * 32)
t = cast(s, c_char_p)
res = obo.library_version(byref(t))
if res != 0:
raise Error("OBO error %r" % res)
return t.value, s.raw, s.value
library_version()
The above code returns
('OBO Version 1.0.1', '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', '')
What I don't understand is why 's' does not have any value? Anyone have any ideas? Thx
When you cast s to c_char_p you store a new object in t, not a reference. So when you pass t to your function by reference, s doesn't get updated.
UPDATE:
You are indeed correct:
cast takes two parameters, a ctypes
object that is or can be converted to
a pointer of some kind, and a ctypes
pointer type. It returns an instance
of the second argument, which
references the same memory block as
the first argument.
In order to get a reference to your string buffer, you need to use the following for your cast:
t = cast(s, POINTER(c_char*33))
I have no idea why c_char_p doesn't create a reference where this does, but there you go.
Because library_version requires a char**, they don't want you to allocate the characters (as you're doing with create_string_buffer. Instead, they just want you to pass in a reference to a pointer so they can return the address of where to find the version string.
So all you need to do is allocate the pointer, and then pass in a reference to that pointer.
The following code should work, although I don't have obo.dll (or know of a suitable replacement) to test it.
from ctypes import *
_OBO_C_DLL = 'obo.dll'
STRING = c_char_p
_stdcall_libraries = dict()
_stdcall_libraries[_OBO_C_DLL] = WinDLL(_OBO_C_DLL)
OBO_VERSION = _stdcall_libraries[_OBO_C_DLL].OBO_VERSION
OBO_VERSION.restype = c_int
OBO_VERSION.argtypes = [POINTER(STRING)]
def library_version():
s_res = c_char_p()
res = OBO_VERSION(byref(s_res))
if res != 0:
raise Error("OBO error %r" % res)
return s_res.value
library_version()
[Edit]
I've gone a step further and written my own DLL that implements a possible implementation of OBO_VERSION that does not require an allocated character buffer, and is not subject to any memory leaks.
int OBO_VERSION(char **pp_version)
{
static char result[] = "Version 2.0";
*pp_version = result;
return 0; // success
}
As you can see, OBO_VERSION simply sets the value of *pp_version to a pointer to a null-terminated character array. This is likely how the real OBO_VERSION works. I've tested this against my originally suggested technique above, and it works as prescribed.