Develop Reference

Trying to get a response from a dll function using ctypes python

I am trying to get a response from a dll library in python using the ctypes wrapper. The response of the function should return x and y coordinates as c_short. A relevant snippet of my code is as follows:
ethdll = ctypes.cdll.LoadLibrary('C:\\Users\\jgallacher\\Documents\\Software_Drivers\\RTC4eth V2 Software Release 2021-06-25\\DLL\\RTC4ethDLLx64.dll')
def get_xy_pos(ethdll):
x = ctypes.c_short()
y = ctypes.c_short()
res = ethdll.get_xy_pos(ctypes.byref(x), ctypes.byref(y))
print(res)
However, when I try this definition I get a Nonetype(0) as the return. Can anyone suggest what is wrong with my function call? I've attached the
I've attached a picture showing the response type to this question thread.
Thanks!
Jordan.

The function returns void, so capturing the return value does nothing. You have created x and y variables to hold the result and need to inspect them after calling the function. Here's a working example with a sample DLL function implementation:
test.c
__declspec(dllexport)
void get_xy_pos(short *xpos, short *ypos) {
*xpos = 5;
*ypos = 7;
}
test.py
import ctypes as ct
dll = ct.CDLL('./test')
# Good practice is to define .argtypes and .restype so ctypes can do type-checking
dll.get_xy_pos.argtypes = ct.POINTER(ct.c_short),ct.POINTER(ct.c_short)
dll.get_xy_pos.restype = None
def get_xy_pos():
x = ct.c_short() # storage for output parameters
y = ct.c_short()
dll.get_xy_pos(ct.byref(x), ct.byref(y)) # pass by reference
return x.value,y.value # inspect the return values
print(get_xy_pos())
Output:
(5, 7)

Error while calling C DLL function from Python

I'm trying to execute my Python Code that must call a C function, which execute some calculation and save the value in a pointer that must be accessible from Python code. I'd like to do that because i'm constructing a DLL and i want to validate the algebra inside the DLL function, therefore i'd like to use a python code to validate the DLL.
The Python Code
from ctypes import *
if __name__ == '__main__':
mydll = cdll.LoadLibrary("./dll_simples.dll")
funcao = mydll.simuser
funcao.argtypes = c_double,c_double,POINTER(c_double),POINTER(c_double)
a = 0
b = 0
input_1 = (c_double * 1)()
input_1[0] = 5
output_1 = (c_double * 1)()
funcao(a,b,input_1,output_1)
and my DLL
__declspec(dllexport) void simuser(double t, double delt, double* in, double* out)
{
out[0] = 2 * in[0];
}
after executing this code, i have the error
funcao(a,b,input_1,output_1)
OSError: exception: access violation reading 0x0000000000000018

Listing [Python 3.Docs]: ctypes - A foreign function library for Python.
So, you want to pass an array to a function that expects a pointer. For that case, ctypes.cast is required:
So, instead of:
funcao(a, b, input_1, output_1)
use:
funcao(a, b, cast(input_1, POINTER(c_double)), cast(output_1, POINTER(c_double)))
Looking at the existing C code, it only uses one value for the 2 pointers, case in which you won't be needing arrays at all (but I doubt that's the intent because then the input value shouldn't be a pointer):
# ...
input_1 = c_double(5)
output_1 = c_double(0)
funcao(a, b, byref(input_1), byref(output_1))
A working example: [SO]: Pointer from Python (ctypes) to C to save function output (#CristiFati's answer).

How to rebuild the libmem_crc32_direct CRC function in python?

I like to rebuild the libmem_crc32_direct function in python.
I used the crcmod python package before. So I like to setup the crc generator by using it.
the c-code looks like:
uint32_t crc_process_chunk(uint8_t* data, uint32_t len) {
return ~libmem_crc32_direct(data, len, 0xFFFFFFFF);
}
my python code looks so far:
def bit_not(n, numbits=8):
return (1 << numbits) - 1 - n
def getCRC(imageBA):
crcGen = crcmod.mkCrcFun(0x104C11DB7, initCrc=0xFFFFFFFF)
val = crcGen(imageBA)
val = bit_not(val, 32)
return val
The returned value of the python code is not equal of the one in c. So I guess I mad some error.
Any ideas?

Doesn't (1 << numbits) == 0? If this is two's complement math it should work as bit_not could be return 0-1-n. However, this isn't needed, since there is an optional xorOut parameter for crcmod. I'm thinking that since the optional rev parameter for reversed (reflected) input and output defaults to true, it needs to be set to false. I think the call to create the crc generator should be:
crcGen = crcmod.mkCrcFun(0x104C11DB7, initCrc=0xFFFFFFF, rev=False, xorOut=0xFFFFFFFF)

B bit tricky because 64Bit arithmetic on PC vs 32Bit arithmetic on ARM STM32F4, but finally this solution works:
def libmem_crc32_direct_with_xor(im, startAddr, l):
fw = im[startAddr:startAddr+l]
crcGen = crcmod.Crc(0x104C11DB7, initCrc=0xFFFFFFFF, rev = False)
crcGen.update(fw)
return (~crcGen.crcValue ) & 0xFFFFFFFF # 32bit xor

ctypes with callback: access violation on exit

I've asked this question before with no replies. I'm asking it again, much more simplified this time.
I have a dll called by Python ctypes, with a callback function. The callback works correctly all the way through (I can see it in operation if I step through the program in Visual Studio), but on exit Visual Studio throws an "access violation" exception. BUT if I remove the call to the callback from the dll, it exits normally without an access violation.
Is there something else I must do to exit from a dll with a callback? I have researched this for hours and I haven't found anything online that solves this.
Here's the ctypes code. I omitted the dll code to keep this short (it's written in NASM) but if it's needed I can post it, too.
def SimpleTestFunction_asm(X):
Input_Length_Array = []
Input_Length_Array.append(len(X)*8)
CA_X = (ctypes.c_double * len(X))(*X)
length_array_out = (ctypes.c_double * len(Input_Length_Array))(*Input_Length_Array)
hDLL = ctypes.WinDLL("C:/Test_Projects/SimpleTestFunction/SimpleTestFunction.dll")
CallName = hDLL.Main_Entry_fn
CallName.argtypes = [ctypes.POINTER(ctypes.c_double),ctypes.POINTER(ctypes.c_double),ctypes.POINTER(ctypes.c_longlong)]
CallName.restype = ctypes.POINTER(ctypes.c_int64)
#__________
#The callback function
LibraryCB = ctypes.WINFUNCTYPE(ctypes.c_double, ctypes.c_double)
def LibraryCall(ax):
bx = math.ceil(ax)
return (bx)
lib_call = LibraryCB(LibraryCall)
lib_call = ctypes.cast(lib_call,ctypes.POINTER(ctypes.c_longlong))
#__________
ret_ptr = CallName(CA_X,length_array_out,lib_call)
I would really REALLY appreciate any ideas on how to solve this. I hope this simplified post will help.
Thanks very much.

I made some minor changes to your code to make actually run (imports) and added a print to see the addresses of the objects passed and the return value, plus created an equivalent C DLL to ensure the pointers pass correctly and the callback works.
Python:
import ctypes
import math
def SimpleTestFunction_asm(X):
Input_Length_Array = []
Input_Length_Array.append(len(X)*8)
CA_X = (ctypes.c_double * len(X))(*X)
length_array_out = (ctypes.c_double * len(Input_Length_Array))(*Input_Length_Array)
hDLL = ctypes.WinDLL('test')
CallName = hDLL.Main_Entry_fn
CallName.argtypes = [ctypes.POINTER(ctypes.c_double),ctypes.POINTER(ctypes.c_double),ctypes.POINTER(ctypes.c_longlong)]
CallName.restype = ctypes.POINTER(ctypes.c_int64)
LibraryCB = ctypes.WINFUNCTYPE(ctypes.c_double, ctypes.c_double)
def LibraryCall(ax):
bx = math.ceil(ax)
return (bx)
lib_call = LibraryCB(LibraryCall)
lib_call = ctypes.cast(lib_call,ctypes.POINTER(ctypes.c_longlong))
ret_ptr = CallName(CA_X,length_array_out,lib_call)
print('{:016X} {:016X} {:016X} {}'.format(ctypes.addressof(CA_X),ctypes.addressof(length_array_out),ctypes.addressof(lib_call.contents),ret_ptr.contents))
SimpleTestFunction_asm([1.1,2.2,3.3])
Test.DLL source:
#include <inttypes.h>
#include <stdio.h>
typedef double (*CB)(double);
__declspec(dllexport) int64_t* __stdcall Main_Entry_fn(double* p1, double* p2, long long* p3)
{
static int64_t x = 123;
double out = ((CB)p3)(1.1);
printf("%p %p %p %lf\n",p1,p2,p3,out);
return &x;
}
Output:
0000021CC99B23A8 0000021CCBADAC10 0000021CCBC90FC0 2.000000
0000021CC99B23A8 0000021CCBADAC10 0000021CCBC90FC0 c_longlong(123)
You can see the pointers are the same and the callback return value and function return value are correct.
It is likely your NASM code isn't implementing the calling convention correctly or corrupting the stack accessing the arrays. I just did the minimum to make your Python code work. I did think it odd that length_array_out is always a length 1 double array with a value 8 times the length of the input array X. How does the NASM code know how long the arrays are?
You could be more type-correct and declare the following instead of casting the callback to a long long *:
CALLBACK = ctypes.WINFUNCTYPE(ctypes.c_double, ctypes.c_double)
CallName.argtypes = [ctypes.POINTER(ctypes.c_double),ctypes.POINTER(ctypes.c_double),CALLBACK]
CallName.restype = ctypes.POINTER(ctypes.c_int64)
#CALLBACK
def LibraryCall(ax):
bx = math.ceil(ax)
return (bx)
ret_ptr = CallName(CA_X,length_array_out,LibraryCall)

#Mark Tolonen, thank you very much for your detailed analysis. I'm posting this as an answer because the formatting of the code won't come out correctly in a comment -- but I chose your answer as the best answer.
I suspected that stack alignment may be the problem, and you eliminated ctypes as the source, so I focused on the stack. Here's what I did to make it work.
In the NASM code, I push rbp and rdi on entry, then restore them on exit. Here, before the call, I set the stack state back by popping rbp and rdi from the stack. Then I subtract 32 bytes (not 40) from rsp. When the call is finished, I restore the stack state:
pop rbp
pop rdi
sub rsp,32
call [CB_Pointer] ; The call to the callback function
add rsp,32
push rdi
push rbp
For an external function call (like to a C library function), I have to subtract 40 bytes, but for this callback I need only 32 bytes. Before your answer I had tried that with 40 bytes and it didn't work. I guess the reason is because it's not calling an external library, it's a callback to the ctypes code that's callling the dll in the first place.
One other thing. The call sends a floating-point value (xmm0) and returns an integer value, but the integer value is returned in the xmm0 register, not rax. Setting the prototype in ctypes to an integer return doesn't do it. It has to stay like this:
LibraryCB = ctypes.WINFUNCTYPE(ctypes.c_double, ctypes.c_double)
Thanks again for your reply. You showed me where to look.
P.S. length_array_out passes the length of the input array to NASM. If I pass more than one array, length_array_out will be longer with one qword for each length; currently I convert the qword to integer on entry.

Passing C++ double array to Python Results in a Crash

I'm running into an issue while trying to pass a double array from C++ to Python. I run a script to create a binary file with data, then read that data back into an array and am trying to pass the array to Python. I've followed advice here: how to return array from c function to python using ctypes among other pages I have found through google. I can write a generic example that works fine (like a similar array to the link above), but when I try to pass the array read from a binary file (code below), the program crashes with "Unhandled exception at ADDR (ucrtbase.dll) in python.exe: An invalid parameter was passed to a function that considers invalid parameters fatal." So, I'm wondering if anyone has any insight.
A word on methodology:
Right now, I'm just trying to learn - that's why I'm going through the convoluted process of saving to disk, loading, and passing to Python. Eventaully, I will use this in scientific simulations where the data read from disk needs to be generated by distributed computing/a super computer. I would like to use Python for its ease of plotting (matplotlib) and C++ for its speed (iterative calculations, etc).
So, on to my code. This generates the binary file:
for (int zzz = 0; zzz < arraysize; ++zzz)
{
for (int yyy = 0; yyy < arraysize; ++yyy)
{
for (int xxx = 0; xxx < arraysize; ++xxx)
{//totalBatP returns a 3 element std::vector<double> - dblArray3_t is basically that with a few overloaded operators (+,-,etc)
dblArray3_t BatP = B.totalBatP({ -5 + xxx * stepsize, -5 + yyy * stepsize, -5 + zzz * stepsize }, 37);
for (int bbb = 0; bbb < 3; ++bbb)
{
dataarray[loopind] = BatP[bbb];
++loopind;
...(end braces here)
FILE* binfile;
binfile = fopen("MBdata.bin", "wb");
fwrite(dataarray, 8, 3 * arraysize * arraysize * arraysize, binfile);
The code that reads the file:
DLLEXPORT double* readDblBin(const std::string filename, unsigned int numOfDblsToRead)
{
char* buffer = new char[numOfDblsToRead];
std::ifstream binFile;
binFile.open(filename, std::ios::in | std::ios::binary);
binFile.read(buffer, numOfDblsToRead);
double* dataArray = (double*)buffer;
binFile.close();
return dataArray;
}
And the Python Code that receives the array:
def readBDataWrapper(filename, numDblsToRead):
fileIO = ctypes.CDLL('./fileIO.dll')
fileIO.readDblBin.argtypes = (ctypes.c_char_p, ctypes.c_uint)
fileIO.readDblBin.restype = ctypes.POINTER(ctypes.c_double)
return fileIO.readDblBin(filename, numDblsToRead)

One possible problem is here
char* buffer = new char[numOfDblsToRead];
Here you allocate numOfDblsToRead bytes. You probably want numOfDblsToRead * sizeof(double).
Same with the reading from the file, you only read numOfDblsToRead bytes.

I figured it out - at least it appears to be working. The problem was with the binary files that were generated with the first code block. I swapped the c-style writing with ofstream. My assumption is perhaps I was using the code to write to disk wrong somehow. Anyway, it appears to work now.
Replaced:
FILE* binfile;
binfile = fopen("MBdata.bin", "wb");
fwrite(dataarray, 8, 3 * arraysize * arraysize * arraysize, binfile);
With:
std::ofstream binfile;
binfile.open("MBdata.bin", std::ios::binary | std::ios::out);
binfile.write(reinterpret_cast<const char*>(dataarray), std::streamsize(totaliter * sizeof(double)));
binfile.close();