Develop Reference

SecureZeroMemory in Python Ctypes

I'm trying to translate the following line of C code into ctypes. Here's a snippet from the C program I'm trying to translate:
pIfRow = (MIB_IF_ROW2 *) malloc(sizeof(MIB_IF_ROW2));
SecureZeroMemory((PVOID)pIfRow, sizeof(MIB_IF_ROW2));
(Note that MIB_IF_ROW2 is a struct, defined in Netioapi.h)
Anyway, I can translate the first line fine in ctypes, assuming MIB_IF_ROW2 has already been defined as a ctypes struct:
from ctypes import *
# Translate first line of C Code
buff = create_string_buffer(sizeof(MIB_IF_ROW2))
p_if_row = cast(buff, POINTER(MIB_IF_ROW2))
# Second Line... ?
But when I get to the second line, I get stuck. I can't find anything in the docs or online with a ctypes equivalent for the function. What is the best way to go about this?

SecureZeroMemory will just fill the memory you pass it with zeroes. You should get the exact same result with ZeroMemory/memset or a plain loop in python. The thing that makes it "secure" is that it is not supposed to be optimized away by the compiler (when programming at a lower level like C/C++).
Using it on memory you just malloc'ed is not its intended purpose (not harmful though), it is supposed to be used like this:
char password[100];
AskUserForPassword(password);
DoSomething(password);
SecureZeroMemory(password, sizeof(password)); // Make sure password is no longer visible in memory in case the application is paged out or creates a memory dump in a crash

How to Grab dll message-dispatching procedure and redirect to python stdout?

I'm trying to figure out how to proceed and if it is feasible or not in general.
I working with external DLL to control my mechanical delay line.
This API has it internal procedure for message output in separate window. I have a strong desire to catch this message flow and present in my python (PyQT5) written application.
In API description there is a function:
int LS_SetProcessMessagesProc(void *pProc);
Function returns either 0 or 1, if there is no error or it is present, respectively.
According to dll description
It enables the replacement of the internal message-dispatching procedure of the LStep API.
The LStep API processes during waiting for confirmation of the LStep in the main-thread messages. If you want to switch of the Message-Dispatching or replace with your onw Code, you can use SetProcessMessagesProc for using a callback-procedure.
pProc must be a pointer to a stdcall-procedure without a parameter:
void MyProcessMessages() {...}
Example:
LS.SetProcessMessagesProc(&MyProcessMessages);
As example if we take python stdout, how I can send the message to it?

I'm going to illustrate everything on the:
[MSDN]: EnumWindows function - which enumerates all windows on the screen, and for each of them calls a callback function - check next bullet
[MSDN]: EnumWindowsProc callback function - which is used to handle every enumerated window
which is (a slightly more complicated example of) what you need: a function defined in an external .dll which needs to call another custom function (written by you in Python), via [Python]: ctypes module (on Win).
The code:
import ctypes
from ctypes import wintypes
try:
from win32gui import GetWindowText
pywin32_present = True
except ImportError:
pywin32_present = False
def enum_windows_proc(hwnd, l_param):
print("HWND: {}\n".format(hwnd))
if pywin32_present:
txt = GetWindowText(hwnd)
if txt and "MSCTFIME UI" not in txt and "Default IME" not in txt:
print(" Window text: {}\n".format(txt))
return 1
def main():
user32_dll = ctypes.windll.LoadLibrary("user32.dll")
enum_windows = user32_dll.EnumWindows
WND_ENUM_PROC_TYPE = ctypes.WINFUNCTYPE(wintypes.BOOL, wintypes.HWND, wintypes.LPARAM)
enum_windows.argtypes = (WND_ENUM_PROC_TYPE, wintypes.LPARAM)
enum_windows.restype = wintypes.BOOL
enum_windows(WND_ENUM_PROC_TYPE(enum_windows_proc), wintypes.LPARAM(0))
if __name__ == "__main__":
main()
Notes:
imports:
wintypes is a ctypes sub-module that defines a bunch of Ms specific data (constants, structs, enums, ...)
[Python]: pywin32 is a Python wrapper over C Win functions, it's basically a more advanced (and pythonic) approach of what ctypes does. It doesn't come with Python by default, it must be manually installed; in our example, it's optional
def enum_windows_proc(hwnd, l_param)::
It's the Python form of BOOL CALLBACK EnumWindowsProc(_In_ HWND hwnd, _In_ LPARAM lParam);
Prints the hwnd (window handle) for each window (note that there will be lots of such windows, since most of them are "invisible" to the user)
If pywin32 module is installed, it will be used to extract each window's title(caption). Of course, that can also be done with ctypes but it's a little bit more complicated
The title filtering is to avoid printing useless text (for most of the windows). If you want more details, check: [SO]:
Get the title of a window of another program using the process name
The main function:
First, the .dll that contains the function (in our case user32.dll) needs to be loaded. This is done using [MSDN]: LoadLibrary function (Ux: [man]: dlopen). Also, the internal structure (needed for next line to look so simple) of the returned object (user32_dll) is initialized
The function (or better: a pointer to it) is being retrieved (enum_windows) using [MSDN]:
GetProcAddress function (Ux: [man]: dlsym)
The next 3 lines of code are used to let Python know the details about the loaded function pointer (return type and argument types). Note that (in some cases) there is a simpler way (codewise) to do all that, but for learning purposes, it's OK to go through the whole thing
Finally, call the external function, with our custom function as an argument
Since there will be lots of output (and will be mostly memory addresses), I won't paste it here.
Going to your problem, based on the function headers you pasted in the question, we can take the same approach (note that the code will not work copy/pasted OOTB):
import ctypes
from ctypes import wintypes
def my_process_messages():
# Your code here (delete the next (`pass`) line)
pass
dll_name = "your dll path (full or relative)"
dll_object = ctypes.windll.LoadLibrary(dll_name)
ls_set_process_messages_proc = dll_object.LS_SetProcessMessagesProc
PROCESS_MESSAGES_TYPE = ctypes.WINFUNCTYPE(None)
ls_set_process_messages_proc.argtypes = (PROCESS_MESSAGES_TYPE,)
ls_set_process_messages_proc.restype = ctypes.c_int
print("ls_set_process_messages_proc returned: {}\n".format(ls_set_process_messages_proc(PROCESS_MESSAGES_TYPE(my_process_messages))))
Note: The example is based on the fact that the external .dll:
Is Win style (uses stdcall calling convention). If that's not true (it uses cdecl), you need to change (for rigorousity's sake I'm going to say 2 things):
ctypes.windll to ctypes.cdll
ctypes.WINFUNCTYPE to ctypes.CFUNCTYPE
Exports C style functions (not C++ which mangles function names), which I'm almost 100% sure. But, if this is not the case then, sorry, nothing to do here. For more details on this topic, check: [SO]: Excel VBA, Can't Find DLL Entry Point from a DLL file.

Read about ctypes tutorial
Loading-dynamic-link-libraries
Accessing functions from loaded dlls
Calling functions
Linux example there, which uses the standard C library's qsort function:
Load the libc.so.6 dll.
from ctypes import *
libc = CDLL("libc.so.6")
Get a function pointer to qsort.
qsort = libc.qsort
qsort.restype = None
Create the type for the callback function
and implement the Python callback function.
CMPFUNC = CFUNCTYPE(c_int, POINTER(c_int), POINTER(c_int))
def py_cmp_func(a, b):
return a[0] - b[0]
cmp_func = CMPFUNC(py_cmp_func)
Define a C-Type Integer Array with values
and use qsort to sort the Array using cmp_func.
IntArray5 = c_int * 5
ia = IntArray5(5, 1, 7, 33, 99)
qsort(ia, len(ia), sizeof(c_int), cmp_func)
for i in ia:
print(i)
Output
1 5 7 33 99

AssertionError while generating python wrapper for dll file using comtypes.client.GetModule()

I'm trying to use "PortableDevice.PortableDevice" COM API for my python application. When I try to generate python wrapper as follow:
comtypes.client.GetModule("C:\\Windows\\system32\\PortableDeviceApi.dll")
I get following error message:
assert sizeof(__MIDL_IOleAutomationTypes_0004) == 16, sizeof(__MIDL_IOleAutomationTypes_0004)
AssertionError: 8
Can anyone please help me to troubleshoot this issue?

The main reason why this fails is due to a kludge in comtypes, where the DECIMAL type is not properly defined. As is, it needs 64 bits, or 8 bytes, for a double float, but it should really take 16 bytes, or 128 bits, for the actual struct.
For your current purpose, you can get along with any definition of DECIMAL that has the proper size, so here's one:
# comtypes/automation.py
class tagDEC(Structure):
_fields_ = [("wReserved", c_ushort),
("scale", c_ubyte),
("sign", c_ubyte),
("Hi32", c_ulong),
("Lo64", c_ulonglong)]
DECIMAL = tagDEC
# comtypes/tools/tlbparser.py
DECIMAL_type = typedesc.Structure("DECIMAL",
align=alignment(automation.DECIMAL)*8,
members=[], bases=[],
size=sizeof(automation.DECIMAL)*8)
However, you'll probably stump over the fact that some methods in the Portable Device API are not suitable for automation.
For instance, IPortableDeviceManager::GetDevices has the unique attribute (in the actual PortableDeviceApi.idl file, not in the documentation), which means you can actually pass NULL. However, type libraries don't capture this information.
That same argument can actually be an array which size is determined by the next argument. Again, type libraries don't support this, only single-object top-level pointers. Moreover, the actual IDL doesn't have a size_is attribute, which means either that the method call will not work across apartments or that this interface has a custom marshaler.
A quick look at the Portable Device API in general shows that this pattern is consistently applied in other methods that actually use arrays. It seems as if someone familiar with the Win32 API made these methods, because there are a bunch of Win32 functions that are overloaded to fetch the size of some array when that array argument is NULL. This is not the usual COM way at all, it would be better to have two methods (with the same race condition between knowing the number of elements, allocating enough memory and fetching them), a single method with only out arguments (no race condition, but no control over memory usage) or use enumerators (e.g. IEnumPortableDevice, harder, but much cleaner).
Anyway, you can take the code that comtypes.client.GetModule("…PortableDeviceApi.dll") generates as a first step. Then, follow these instructions to make the Python methods actually call the COM methods in the documented way. For instance, IPortableManager::GetDevices would become:
# comtypes/gen/_1F001332_1A57_4934_BE31_AFFC99F4EE0A_0_1_0.py
class IPortableDeviceManager(comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0.IUnknown):
# ...
def GetDevices(self):
cPnPDeviceIDs = c_ulong(0)
self.__com_GetDevices(None, byref(cPnPDeviceIDs))
PnPDeviceIDs = (WSTRING * cPnPDeviceIDs.value)()
self.__com_GetDevices(PnPDeviceIDs, byref(cPnPDeviceIDs))
deviceIDs = PnPDeviceIDs[:cPnPDeviceIDs.value]
for i in range(cPnPDeviceIDs.value):
windll.ole32.CoTaskMemFree(cast(PnPDeviceIDs, POINTER(c_void_p))[i])
return deviceIDs
# ...
IPortableDeviceManager._methods_ = [
COMMETHOD([], HRESULT, 'GetDevices',
( ['in'], POINTER(WSTRING), 'pPnPDeviceIDs' ),
( ['in'], POINTER(c_ulong), 'pcPnPDeviceIDs' )),
# ...
The following test runs successfully, although it returns an empty list in my case, since I don't have any connected device right now:
# First run
import os
from comtypes.client import GetModule
GetModule(os.getenv("WINDIR") + "\\system32\\PortableDeviceApi.dll")
# Quit python
# Edit comtypes/gen/_1F001332_1A57_4934_BE31_AFFC99F4EE0A_0_1_0.py
# Subsequent runs
from comtypes.client import CreateObject
from comtypes.gen.PortableDeviceApiLib import *
CreateObject(PortableDeviceManager).GetDevices()
I can't provide more help without scavenging further into comtypes. I suggest you contact its authors or maintainers.
EDIT: Meanwhile, I created a ticket at the SourceForge site. Since the project was transitioning out of SourceForge, it seemed to be forgotten, but it wasn't (here too).
Have you tried to import the module since then?

Differences in ctypes between Python 2 and 3

I have a working python 2.7 program that calls a DLL. I am trying to port the script to python 3.2. The DLL call seems to work (i.e. there is no error upon calling) but the returned data does not make sense.
Just in case it could be useful:
- The call takes three arguments: two int (input) and a pointer to a ushort array (output).
I have tried using both python and numpy arrays without success.
Can anyone enumerate the differences between Python 2.7 and 3.2 respecting ctypes?
Thanks in advance
EDIT
Here is some example code. The DLL is propietary so I do not have the code. But I do have the C header:
void example (int width, int height, unsigned short* pointer)
The python code is:
width, height = 40, 100
imagearray = np.zeros((width,height), dtype=np.dtype(np.ushort))
image = np.ascontiguousarray(imagearray)
ptrimage = image.ctypes.data_as(ct.POINTER(ct.c_ushort))
DLL.example(width, height, ptrimage)
This works in python 2.7 but not in 3.2.
EDIT 2
If the changes in ctypes are only those pointed out by Cedric, it does not make sense that python 3.2 will not work. So looking again at the code, I have found that there is a preparation function called before the function that I am mentioning. The signature is:
void prepare(char *table)
In python, I am calling by:
table = str(aNumber)
DLL.prepare(table)
Is it possible that the problem is due to the change in the Python string handling?

In Python 2.7, strings are byte-strings by default. In Python 3.x, they are unicode by default. Try explicitly making your string a byte string using .encode('ascii') before handing it to DLL.prepare.
Edit:
#another way of saying table=str(aNumber).encode('ascii')
table = bytes(str(aNumber), 'ascii')
DLL.prepare(table)

In our case, we had code looking like:
addr = clib.some_function()
data = some_struct.from_address(addr)
This worked in python 2 but not in 3. The reason turned out not to be any difference in ctypes, but rather a change in memory layout that unmasked a bug in the code above. In python 2, the address returned was always (by chance) small enough to fit inside a C int (32-bit), which is the default return type for all ctypes function calls. In python 3, the addresses were almost always too large, which caused the pointer address to become corrupted as it was coerced to int.
The solution is to set the function restype to a 64-bit integer type to ensure that it can accommodate the whole address, like:
clib.some_function.restype = c_longlong
or:
clib.some_function.restype = POINTER(some_struct)

According to the python documentation, the only changes between 2.7 and 3.2 is here
A new type, ctypes.c_ssize_t represents the C ssize_t datatype.
In 2.7, there was some other modifications introduced :
The ctypes module now always converts None to a C NULL pointer for
arguments declared as pointers. (Changed by Thomas Heller; issue
4606.) The underlying libffi library has been updated to version
3.0.9, containing various fixes for different platforms. (Updated by
Matthias Klose; issue 8142.)
I'm not sure it will explain the cause of your problem...

Python c_types .dll functions (pari library)

Alright, so a couple days ago I decided to try and write a primitive wrapper for the PARI library. Ever since then I've been playing with ctypes library in loading the dll and accessing the functions contained using code similar to the following:
from ctypes import *
libcyg=CDLL("<path/cygwin1.dll") #It needs cygwin to be loaded. Not sure why.
pari=CDLL("<path>/libpari-gmp-2.4.dll")
print pari.fibo #fibonacci function
#prints something like "<_FuncPtr object at 0x00BA5828>"
So the functions are there and they can potentially be accessed, but I always receive an access violation no matter what I try. For example:
pari.fibo(5) #access violation
pari.fibo(c_int(5)) #access violation
pari.fibo.argtypes = [c_long] #setting arguments manually
pari.fibo.restype = long #set the return type
pari.fibo(byref(c_int(5))) #access violation reading 0x04 consistently
and any variation on that, including setting argtypes to receive pointers.
The Pari .dll is written in C and the fibonacci function's syntax within the library is GEN fibo(long x).
Could it be the return type that's causing these errors, as it is not a standard int or long but a GEN type, which is unique to the PARI library? Any help would be appreciated. If anyone is able to successfully load the library and use ANY function from within python, please tell; I've been at this for hours now.
EDIT: Seems as though I was simply forgetting to initialize the library. After a quick pari.pari_init(4000000,500000) it stopped erroring. Now my problem lies in the in the fact that it returns a GEN object; which is fine, but whenever I try to reference the address to which it points, it's always 33554435, which I presume is still an address. I'm trying further commands and I'll update if I succeed in getting the correct value of something.

You have two problems here, one give fibo the correct return type and two convert the GEN return type to the value you are looking for.
Poking around the source code a bit, you'll find that GEN is defined as a pointer to a long. Also, at looks like the library provides some converting/printing GENs. I focused in on GENtostr since it would probably be safer for all the pari functions.
import cytpes
pari = ctypes.CDLL("./libpari.so.2.3.5") #I did this under linux
pari.fibo.restype = ctypes.POINTER(ctypes.c_long)
pari.GENtostr.restype = ctypes.POINTER(ctypes.c_char)
pari.pari_init(4000000,500000)
x = pari.fibo(100)
y = pari.GENtostr(x)
ctypes.string_at(y)
Results in:
'354224848179261915075'