Develop Reference

Python3 ctypes c_unit32 endianness

I'm trying to load PCI config space data into python for evaluation, but am running into an issue with using a c_unit32 for BAR0. I've managed to simplify the issue / script to show c_uint16 behaving as I expect, but the c_uint32 failing. Bar0 must be a c_unit32 as in the actual script, the Bar0 is decoded as a separate struct which uses bits 0-13 for various fields, then bits 14:31 for the lower BAR.
In the script, data is loaded from /sys/pci/bus/devices/x/config, so comes back as a string of hex characters, this is emulated in the example by loading a smaller string to represent only a few of the fields
Expected:
Input string: "86802010f6710000"
VendorID : 0x8086
DeviceID : 0x1020
Bar0 : 0x71f60000
Actual:
Input string: "86802010f6710000"
VendorID : 0x8086
DeviceID : 0x1020
Bar0 : 0x71f6
The VendorID and DeviceID fields are correctly swapped. I presume the "0000" is being byte swapped and being put infront of the f671, and the f671 is then byte swapped to be 71f6. The leading 00s are then eliminated creating "0x71f6". This is confirmed if I use "f6710100" which becomes 0x171f6.
How can I ensure when I use the c_uint32 that it is correctly interpreted as 0x71f60000?
Code:
from ctypes import *
class PCICfg(Structure):
_pack_ = 1
_fields_ = [
("VendorID", c_uint16),
("DeviceID", c_uint16),
("Bar0", c_uint32),
]
class PCIUnion(Union):
_pack_ = 1
_fields_ = [
("ConfigSpace", PCICfg),
("Bytes", c_ubyte * sizeof(PCICfg))
]
def from_str(self, input):
print("Input String: ", input)
hex_data = bytearray.fromhex(input)
for x in range(0, len(hex_data)):
self.Bytes[x] = hex_data[x]
if __name__ == '__main__':
cfg = PCIUnion()
cfg.from_str("86802010f6710100")
print("VID: %s" % hex(cfg.ConfigSpace.VendorID))
print("DID: %s" % hex(cfg.ConfigSpace.DeviceID))
print("BA0: %s" % hex(cfg.ConfigSpace.Bar0))

What you are seeing is the correct little-endian 32-bit result of your input, but if your input is really a hex representation of little-endian 16-bit words, then treat it that way:
from ctypes import *
from binascii import unhexlify
class PCICfg(Structure):
_pack_ = 1
_fields_ = [("VendorID", c_uint16),
("DeviceID", c_uint16),
("Bar0Hi", c_uint16),
("Bar0Lo", c_uint16)]
class PCIUnion(Union):
_pack_ = 1
_fields_ = [("ConfigSpace", PCICfg),
("Bytes", c_ubyte * sizeof(PCICfg))]
def from_str(self, inp):
print("Input String: ", inp)
self.Bytes[:] = unhexlify(inp)
if __name__ == '__main__':
cfg = PCIUnion()
cfg.from_str("86802010f6710100")
print("VID: {:#04x}".format(cfg.ConfigSpace.VendorID))
print("DID: {:#04x}".format(cfg.ConfigSpace.DeviceID))
print("BA0: {:#04x}{:04x}".format(cfg.ConfigSpace.Bar0Hi,cfg.ConfigSpace.Bar0Lo))
Input String: 86802010f6710100
VID: 0x8086
DID: 0x1020
BA0: 0x71f60001

how does toByte work while bit manipulation in python

The document for bit manipulation in python states
https://wiki.python.org/moin/BitManipulation
as example, the following code
import ctypes
c_uint8 = ctypes.c_uint8
class Flags_bits( ctypes.LittleEndianStructure ):
_fields_ = [
("logout", c_uint8, 1 ), # asByte & 1
("userswitch", c_uint8, 1 ), # asByte & 2
("suspend", c_uint8, 1 ), # asByte & 4
("idle", c_uint8, 1 ), # asByte & 8
]
class Flags( ctypes.Union ):
_anonymous_ = ("bit",)
_fields_ = [
("bit", Flags_bits ),
("asByte", c_uint8 )
]
flags = Flags()
flags.asByte = 0x2 # ->0010
print( "logout: %i" % flags.bit.logout )
# `bit` is defined as anonymous field, so its fields can also be accessed directly:
print( "logout: %i" % flags.logout )
print( "userswitch: %i" % flags.userswitch )
print( "suspend : %i" % flags.suspend )
print( "idle : %i" % flags.idle )
so i figured out that setting the value to asByte is leading to the value being parsed into the c_uint8, but i am unable to understand the logic behind this. When we set the asByte value, does python work back from the asByte value set to update the value for individual components that make the class. Also as a side note, i was not able to find documentation for asByte. If someone can point me to it, it'd be great.

Error using GetExtendedTcpTable in python

I've got some troubles with using "GetExtendedTcpTable". When I tried to run my script, i've got message like this:
AssertionError: [Error 0] The operation completed successfully
Rarely script working normally, I dont understand this message, Operation completed, what`s wrong?
This is code, i tried to execute:
from ctypes import *
from ctypes.wintypes import *
from socket import inet_aton, inet_ntoa, htons
AF_INET = 2
TCP_TABLE_BASIC_LISTENER = 0
TCP_TABLE_BASIC_CONNECTIONS = 1
TCP_TABLE_BASIC_ALL = 2
TCP_TABLE_OWNER_PID_LISTENER = 3
TCP_TABLE_OWNER_PID_CONNECTIONS = 4
TCP_TABLE_OWNER_PID_ALL = 5
TCP_TABLE_OWNER_MODULE_LISTENER = 6
TCP_TABLE_OWNER_MODULE_CONNECTIONS = 7
TCP_TABLE_OWNER_MODULE_ALL = 8
# for storing socket info python style.
class socket_info:
State = None
LocalAddr = None
LocalPort = None
RemoteAddr = None
RemotePort = None
def __init__ (self, **kwargs):
for key, word in kwargs.items():
setattr(self, key, word)
def formatip (ip):
ip = inet_aton (str(ip))
return inet_ntoa (ip[::-1])
states = {
1 : "TCP_STATE_CLOSED",
2 : "TCP_STATE_LISTEN",
3 : "TCP_STATE_SYN_SENT",
4 : "TCP_STATE_SYN_RCVD",
5 : "TCP_STATE_ESTAB",
6 : "TCP_STATE_FIN_WAIT",
7 : "TCP_STATE_FIN_WAIT2",
8 : "TCP_STATE_CLOSE_WAIT",
9 : "TCP_STATE_CLOSING",
10 : "TCP_STATE_LAST_ACK",
11 : "TCP_STATE_TIME_WAIT",
12 : "TCP_STATE_DELETE_TCB",
"TCP_STATE_CLOSED" : 1,
"TCP_STATE_LISTEN" : 2,
"TCP_STATE_SYN_SENT" : 3,
"TCP_STATE_SYN_RCVD" : 4,
"TCP_STATE_ESTAB" : 5,
"TCP_STATE_FIN_WAIT" : 6,
"TCP_STATE_FIN_WAIT2" : 7,
"TCP_STATE_CLOSE_WAIT" : 8,
"TCP_STATE_CLOSING" : 9,
"TCP_STATE_LAST_ACK" :10,
"TCP_STATE_TIME_WAIT" : 11,
"TCP_STATE_DELETE_TCB" : 12 }
class MIB_TCPROW_OWNER_PID(Structure):
_fields_ = [
("dwState", DWORD),
("dwLocalAddr", DWORD),
("dwLocalPort", DWORD),
("dwRemoteAddr", DWORD),
("dwRemotePort", DWORD),
("dwOwningPid", DWORD)
]
class MIB_TCPTABLE_OWNER_PID(Structure):
_fields_ = [
("dwNumEntries", DWORD),
("MIB_TCPROW_OWNER_PID", MIB_TCPROW_OWNER_PID * 100)
]
def GetExtendedTcpTable (vip=AF_INET):
table = MIB_TCPTABLE_OWNER_PID ()
so = sizeof (table)
size = DWORD (so)
order = c_int(1)
failure= windll.iphlpapi.GetExtendedTcpTable (
byref (table),
addressof (size),
order,
vip,
TCP_TABLE_OWNER_PID_ALL,
0 )
assert not failure, WinError (GetLastError ())
pytables = []
tables = table.MIB_TCPROW_OWNER_PID
for index in range(table.dwNumEntries):
table = tables [index]
pytables.append (
socket_info (
State=states.get (table.dwState, "UNKNOWN_STATE_%s" %(str(table.dwState))),
LocalAddr=formatip (table.dwLocalAddr),
LocalPort=htons(table.dwLocalPort),
RemoteAddr=formatip (table.dwRemoteAddr),
RemotePort=htons(table.dwRemotePort),
OwningPid = int (table.dwOwningPid)
)
)
return pytables
def GetTcpTableForPid (pid):
tables = GetExtendedTcpTable ()
for table in tables:
if table.OwningPid == pid: return table
raise "Cannot find tcp table for pid %s" %pid
dict_process = {}
pid_set =set()
pid_list = []
tcp_info_list = []
tcp_info = GetExtendedTcpTable()
for item in tcp_info:
LocalAddr = item.LocalAddr
LocalPort = item.LocalPort
RemoteAddr = item.RemoteAddr
RemotePort = item.RemotePort
OwningPid = item.OwningPid
print('local Addr: '+ LocalAddr,'local port: '+ str(LocalPort),'remote Addr: ' + RemoteAddr, 'Remote Port: ' + str(RemotePort), OwningPid)
The script is run from time to time. It can run for 5 minutes and then don't work about an hour with this stupid mistake. How to get around it?
I really dont know, what's with it. Please, help me, what i do wrong?
I use python 3.2 on Win7 SP1 x64
Thank you a lot!

You shouldn't use addressof(size). That returns a Python integer which will be cast as a 32-bit C int. Use byref(size) to create a pointer, which will be a 64-bit value if you're using 64-bit Python.
GetExtendedTcpTable doesn't call SetLastError. It returns a DWORD with one of the following codes:
NO_ERROR = 0
ERROR_INVALID_PARAMETER = 87
ERROR_INSUFFICIENT_BUFFER = 122
The pdwSize argument has the required size if the buffer was too small. One option here is to start with a length 0 array; then resize the struct; and finally cast the array to the correct size:
class MIB_TCPTABLE_OWNER_PID(Structure):
_fields_ = [
("dwNumEntries", DWORD),
("MIB_TCPROW_OWNER_PID", MIB_TCPROW_OWNER_PID * 0),
]
_GetExtendedTcpTable = windll.iphlpapi.GetExtendedTcpTable
def GetExtendedTcpTable(vip=AF_INET):
table = MIB_TCPTABLE_OWNER_PID()
size = DWORD()
order = 1
failure = _GetExtendedTcpTable(
byref(table),
byref(size),
order,
vip,
TCP_TABLE_OWNER_PID_ALL,
0)
if failure == ERROR_INSUFFICIENT_BUFFER:
resize(table, size.value)
memset(byref(table), 0, sizeof(table))
failure = _GetExtendedTcpTable(
byref(table),
byref(size),
order,
vip,
TCP_TABLE_OWNER_PID_ALL,
0)
if failure:
raise WinError(failure)
ptr_type = POINTER(MIB_TCPROW_OWNER_PID * table.dwNumEntries)
tables = cast(table.MIB_TCPROW_OWNER_PID, ptr_type)[0]
pytables = []
for table in tables:
# rest unchanged
Regarding the Win32 LastError value, in general you shouldn't rely on GetLastError in Python. You don't know if you're seeing an old error code from a previous call or if an intervening call modified the LastError value. If you're checking a single API call that uses LastError, then it should be OK to check GetLastError immediately afterward if the call failed. But more generally you may need to load the DLL with use_last_error=True:
iphlpapi = WinDLL('iphlpapi', use_last_error=True)
Function pointers created from this WinDLL instance will save LastError to thread local storage immediately after the call returns. Calling get_last_error returns the saved error code. Beforehand you can call set_last_error(0) to have 0 swapped in to LastError before the function is called.

Python ctypes access violation

I'm trying to access a dll from python, following this guide. The first function I try and call from the dll seems to work fine, so I must be most of the way there, but the second one gives me an access violation.
Python code:
#create open function
openProto = ctypes.WINFUNCTYPE( ctypes.c_int, ctypes.c_void_p )
openParams = ( 1, "handle", 0 ) ,
open = openProto ( ( "SensorOpen", pm ), openParams )
#create zero function
zeroProto = ctypes.WINFUNCTYPE( ctypes.c_int, ctypes.c_int )
zeroParams = ( 1, "handle", 0 ),
zero = zeroProto ( ( "SensorZero", pm ), zeroParams )
handle = ctypes.c_int ( 0 )
status = ctypes.c_int ( 0 )
open( ctypes.byref(handle) )
print handle
zero( handle )
The access violation occurs when I call the zero( handle ).
The C++ code that does this is here:
SENSOR_HANDLE seHandle; // Sensor handle
double power;
char buff[80];
// Get handle for sensor
SensorOpen (&seHandle);
printf ("Sensor connected\nZeroing, please wait...\n");
while (SensorZero (seHandle) == SENSOR_ZERO_FAILED)
{
printf ("Zero failed.\nCheck no power is being applied.\nPress return key to retry ");
gets (buff);
}
Where SENSOR_HANDLE is just a typedef'd int
Any ideas? I assume it's ok not to do anything with the returned value.

Problem solved.
The c++ code was sample code provided with some documentation. Needless to say the sample code and the documentation were incorrect (the zero function still wanted its argument as a pointer).

SendKeys for Python 3.1 on Windows

The latest Python Sendkeys module is for Python 2.6. I can't upgrade it myself as it needs the C module to be recompiled.
Does anyone know of a fairly easy alternative way to send keys to a window?
Using win32ui.FindWindow() I can find the right window, then make it active with PyCWnd.SetActiveWindow(), so all that's needed is a simple way to send keystrokes to the active window.
The purpose is to execute a menu item.
The app was written in Delphi, and doesn't have any inter-process interface that I know about.

Here is a working module that calls user32.SendInput().
Not perfect, but usable.
Edit:
Yesterday I did a version with a class, and am using it in a working tkinter app. Will put it here when i get time to clean it up.
Have added this in the doc string below:
[ It is OK if I work from a folder within my profile.
These problems happened when working on another partition.
File permissions were OK, so do not know what blocked SendInput. ]
SciTE still needs the full exact window title.
#!/usr/bin/python
# -*- coding: utf-8 -*-
''' send_input for python 3, from jh45dev#gmail.com
code from Daniel F is adapted here. The original is at:
http://mail.python.org/pipermail/python-win32/2005-April/003131.html
SendInput sends to the window that has the keyboard focus.
That window must not be minimized.
There seem to be some strange limitations with user32.SendInput()
Here is what happened in my testing (on Vista sp2).
[edit: It is OK if I work from a folder within my profile.
These problems happened when working on another partition.
File permissions were OK, so do not know what blocked SendInput.]
1
I opened Notepad from the Start menu,
then in Notepad opened test.txt,
and all worked fine.
2
I opened Notepad by opening test.txt in Explorer.
find_window() found Notepad, but user32.SendInput() had no effect.
If Notepad was minimized, it did not get restored or focussed.
The same happened with SciTE and Notepad2.
Another strangeness:
For SciTE I had to put in the whole window title, eg "test.txt - SciTE",
but for Notepad and Notepad2, only the app name, eg "Notepad".
'''
import ctypes as ct
from win32con import SW_MINIMIZE, SW_RESTORE
from win32ui import FindWindow, error as ui_err
from time import sleep
class cls_KeyBdInput(ct.Structure):
_fields_ = [
("wVk", ct.c_ushort),
("wScan", ct.c_ushort),
("dwFlags", ct.c_ulong),
("time", ct.c_ulong),
("dwExtraInfo", ct.POINTER(ct.c_ulong) )
]
class cls_HardwareInput(ct.Structure):
_fields_ = [
("uMsg", ct.c_ulong),
("wParamL", ct.c_short),
("wParamH", ct.c_ushort)
]
class cls_MouseInput(ct.Structure):
_fields_ = [
("dx", ct.c_long),
("dy", ct.c_long),
("mouseData", ct.c_ulong),
("dwFlags", ct.c_ulong),
("time", ct.c_ulong),
("dwExtraInfo", ct.POINTER(ct.c_ulong) )
]
class cls_Input_I(ct.Union):
_fields_ = [
("ki", cls_KeyBdInput),
("mi", cls_MouseInput),
("hi", cls_HardwareInput)
]
class cls_Input(ct.Structure):
_fields_ = [
("type", ct.c_ulong),
("ii", cls_Input_I)
]
def find_window( s_app_name ):
try:
window1 = FindWindow( None, s_app_name,)
return window1
except ui_err:
pass
except:
raise
try:
window1 = FindWindow( s_app_name, None, )
return window1
except ui_err:
return None
except:
raise
def make_input_objects( l_keys ):
p_ExtraInfo_0 = ct.pointer(ct.c_ulong(0))
l_inputs = [ ]
for n_key, n_updown in l_keys:
ki = cls_KeyBdInput( n_key, 0, n_updown, 0, p_ExtraInfo_0 )
ii = cls_Input_I()
ii.ki = ki
l_inputs.append( ii )
n_inputs = len(l_inputs)
l_inputs_2=[]
for ndx in range( 0, n_inputs ):
s2 = "(1, l_inputs[%s])" % ndx
l_inputs_2.append(s2)
s_inputs = ', '.join(l_inputs_2)
cls_input_array = cls_Input * n_inputs
o_input_array = eval( "cls_input_array( %s )" % s_inputs )
p_input_array = ct.pointer( o_input_array )
n_size_0 = ct.sizeof( o_input_array[0] )
# these are the args for user32.SendInput()
return ( n_inputs, p_input_array, n_size_0 )
'''It is interesting that o_input_array has gone out of scope
by the time p_input_array is used, but it works.'''
def send_input( window1, t_inputs, b_minimize=True ):
tpl1 = window1.GetWindowPlacement()
was_min = False
if tpl1[1] == 2:
was_min = True
window1.ShowWindow(SW_RESTORE)
sleep(0.2)
window1.SetForegroundWindow()
sleep(0.2)
window1.SetFocus()
sleep(0.2)
rv = ct.windll.user32.SendInput( *t_inputs )
if was_min and b_minimize:
sleep(0.3) # if the last input was Save, it may need time to take effect
window1.ShowWindow(SW_MINIMIZE)
return rv
# define some commonly-used key sequences
t_ctrl_s = ( # save in many apps
( 0x11, 0 ),
( 0x53, 0 ),
( 0x11, 2 ),
)
t_ctrl_r = ( # reload in some apps
( 0x11, 0 ),
( 0x52, 0 ),
( 0x11, 2 ),
)
def test():
# file > open; a non-invasive way to test
t_ctrl_o = ( ( 0x11, 0 ), ( 0x4F, 0 ), ( 0x11, 2 ), )
# writes "Hello\n"
# 0x10 is shift. note that to repeat a key, as with 4C here, you have to release it after the first press
t_hello = ( ( 0x10, 0 ), ( 0x48, 0 ), ( 0x10, 2 ), ( 0x45, 0 ), ( 0x4C, 0 ), ( 0x4C, 2 ), ( 0x4C, 0 ), ( 0x4F, 0 ), ( 0x0D, 0 ), )
l_keys = [ ]
## l_keys.extend( t_ctrl_o )
l_keys.extend( t_hello )
l_keys.extend( t_ctrl_s )
## s_app_name = "SciTE"
## s_app_name = "(Untitled) - SciTE"
s_app_name = "test.txt - SciTE"
## s_app_name = "Notepad2"
## s_app_name = "Notepad"
window1 = find_window( s_app_name )
if window1 == None:
print( "%r has no window." % s_app_name )
input( 'press enter to close' )
exit()
t_inputs = make_input_objects( l_keys )
n = send_input( window1, t_inputs )
## print( "SendInput returned: %r" % n )
## print( "GetLastError: %r" % ct.windll.kernel32.GetLastError() )
## input( 'press enter to close' )
if __name__ == '__main__':
test()

I rewrote the C bits of sendkeys in ctypes a while ago...
https://bitbucket.org/markm/sendkeysctypes
and I see someone else did it too:
http://code.google.com/p/sendkeys-ctypes/
If I remember it should be a drop in replacement for sendkeys (only the import line has to change)

Got something that works, using win32api.keybd_event.
Looks like SendInput would be safer, but pywin32 does not include it.
[ edit: See a SendInput version in the accepted answer below. I leave this message here in case someone prefers to use sendkeys. jh ]
This page helped: http://social.msdn.microsoft.com/Search/en-us/?Query=keybd_event
Code that works:
import win32api;
import win32ui
PyCWnd1 = win32ui.FindWindow( None, "an_app" )
PyCWnd1.SetForegroundWindow()
PyCWnd1.SetFocus()
win32api.keybd_event(0x12, 0, ) # Alt
win32api.keybd_event(0x12, 0, 2 ) # Alt release
win32api.keybd_event(0x46, 0, ) # F
win32api.keybd_event(0x52, 0, ) # R
This does File > Reload in the app. Does not work if the app is minimized
Without releasing Alt, after running this, the keyboard behaved like i was holding the Alt key!
Don't seem to need to release the other keys

You want the keybd_event API. My PushKeys program is syntax compatible with SendKeys and has a sleep function built in. Although it isn't in Python it should be translatable readily enough (its been translated into a lot of languages - at least it should show you the API to use). It is available here in a variety of languages.