I have a commercial package which has a COM interface. I am trying to control it via the COM interface from Python. Most things are working just fine with regular input parameters and outputs.
However one particular set of functions appear to take a pre-allocated data structure as input which they will then fill out with the results of the query. So: an out-parameter of type array in this instance.
Some helpful example VBA code which accompanies the product alongside an Excel spreadsheet seems to work just fine. It declares the input array as: Dim myArray(customsize) as Integer
This then gets passed directly into the call on the COM object: cominterface.GetContent(myArray)
However when I try something similar in Python, I either get errors or no results depending on how I try to pass the array in:
import comtypes
... code to generate bindings, create object, grab the interface ...
# create an array for storing the results
my_array_type = c_ulong * 1000
my_array_instance = my_array_type ()
# attempt to pass the array into the call on the COM interface
r = cominterface.GetContent(my_array_instance )
# expect to see id's 1,2,3,4
print(my_array_instance)
The above gives me error:
TypeError: Cannot put <__main__.c_ulong_Array_1000 object at 0x00000282514F5640> in VARIANT
So it would seem that the comtypes does not support ctype arrays for passing through as it tries to make it a VARIANT.
Thus a different attempt:
# create an array for storing the results
my_array_instance = [0] * 100
# attempt to pass the array into the call on the COM interface
r = cominterface.GetContent(my_array_instance )
# expect to see id's 1,2,3,4
print(my_array_instance)
The above call has a return code indicating success, but the array is unchanged and still contains the initial 0's it was preseaded with.
So I am assuming here that comtypes is somehow not transporting the written values back into the Python list. But thats a big assumption - I really don't know.
I have tried a number of things including using POINTER, byref() and various things. Almost everything results in some kind of error - either in the code doing the bindings or an error from the COM function I am calling to say the parameter does not meet its requirements.
If someone knows how I can pass in a pre-allocated array for this COM function to write to, I would be very much appreciative.
EDIT:
I rewrote the code in C# and it had the same problem, so I began to suspect the COM interface was not correct. By providing my own interface with modified function signatures (adding a 'ref' for the parameters), I was able to get the calls to work.
I suspect the tlb file was in error and happened to work with VBA, but I am unsure.
Related
I am using pywin32 to automate some tasks in software that has an Automation Server technology interface (formerly OLE Automation Server).
This software comes with a somewhat detailed manual with code examples in VBA, C++ or Matlab but no Python. I have built a Python library that can do most of the functionalities built into the software but there are some parts I cannot do in Python.
I cannot change the value of a property if this property is contained in a iterable COM object.
What I can do:
[Documentation for Visibility property]
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
app.Visibility = True
As an example, with this code, I can change the visibility parameter of the software: if it runs with or without GUI.
What I cannot do:
[Documentation for getting and setting current device]
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
app.CurrentDevice(0) = 'NAME OF DEVICE'
I then get the following error:
SyntaxError: cannot assign to function call here. Maybe you meant '==' instead of '='?
This error makes sense to me but I cannot find a way to set any of these software properties when they come in the form of an iterable object. As soon as I have to specify an index, I don't know how to set the value.
From what I understand, in C++ we are able to change the value because of pointers but how can we achieve the same thing in Python? Is it possible or do I have to use some C++ code in parallel to my Python to run my library? I don't know anything in C++ so if I could avoid doing that, it would be good.
What I have tried
Of course, the 1st thing I tried was to change () to [] or {} which logically didn't work.
Then I used the Evaluate function in PyCharms to see what was hiding behind my app.CurrentDevice. I was hoping to find sub-attributes that I could then set but I don't see anything inside the object:
[Result of Evaluate on the CurrentDevice object]
Finally, I have tried the following:
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
curr_device = app.CurrentDevice(0)
curr_device = 'NAME OF DEVICE'
I wanted to affect the object to a variable and then change the value but of course, this only rewrites the variable curr-device with 'NAME OF DEVICE' but loses any link to COM Object.
I feel like my questions are similar to the following unanswered question:
How can I set the value of an indexed property of a COM object in Python?
It looks as if win32com is struggling to set the property if there is an additional argument to the put function, which is a little surprising.
First thing to do is to use
app = win32com.client.gencache.EnsureDispatch('NAME_OF_APP')
This creates a Python wrapper for the COM object (rather than just firing function calls at the object and hoping). This may in itself clear up your issue.
If not, here is a quite ugly way of working around. As you have identified, the relevant part of the type library is:
[id(0x00000018),propput, helpstring("property CurrentDevice")]
HRESULT CurrentDevice([in] long lAcq, [in] VARIANT pVal);
And you can use this to set the property at a low level.
win32com dispatch objects are a wrapper for the PyIDispatch object. All dispatch objects support the Invoke method, and you can use this to call the function yourself. NB. Since I don't have access to your COM object, I can't test, so this answer may need some tweaking (!).
The PyIDispatch documentation
Try:
import win32com.client as wc
import pythoncom
app = wc.gencache.EnsureDispatch('NAME OF APP')
app.Visibility=TRUE
newVal = wc.VARIANT(pythoncom.VT_VARIANT,'NAME OF DEVICE')
app._oleobj_.Invoke(24,0,pythoncom.INVOKE_PROPERTYPUT,0,0,newVal)
There are a lot of 'magic' numbers here, but basically:
24 = 0x00000018 in decimal: this is the Id of the property
0 = the LCID, the Locale Id ... I always set it to 0
pythoncom.INVOKE_PROPERTYPUT = the type of call.
0 = whether you care about the return type (you probably don't = False)
0 = first parameter, lAcq, as in CurrentDevice(0)
newVal = second paramter,pVal, the new device name as a VARIANT
I haven't tried this, but pythoncom is pretty good about converting VARIANT types, so you might not need the VARIANT creation, and can just use NAME OF DEVICE directly as the parameter.
I am writing a program in Python that communicates with a spectrometer from Avantes. There are some proprietary dlls available whose code I don't access to, but they have some decent documentation. I am having some trouble to find a good way to store the data received via callbacks.
The proprietary shared library
Basically, the dll contains a function that I have to call to start measuring and that receives a callback function that will be called whenever the spectrometer has finished a measurement. The function is the following:
int AVS_MeasureCallback(AvsHandle a_hDevice,void (*__Done)(AvsHandle*, int*),short a_Nmsr)
The first argument is a handle object that identifies the spectrometer, the second is the actual callback function and the third is the amount of measurements to be made.
The callback function will receive then receive another type of handle identifying the spetrometer and information about the amount of data available after a measurement.
Python library
I am using a library that has Python wrappers for many equipments, including my spectrometer.
def measure_callback(self, num_measurements, callback=None):
self.sdk.AVS_MeasureCallback(self._handle, callback, num_measurements)
And they also have defined the following decorator:
MeasureCallback = FUNCTYPE(None, POINTER(c_int32), POINTER(c_int32))
The idea is that when the callback function is finally called, this will trigger the get_data() function that will retrieve data from the equipment.
The recommended example is
#MeasureCallback
def callback_fcn(handle, info):
print('The DLL handle is:', handle.contents.value)
if info.contents.value == 0: # equals 0 if everything is okay (see manual)
print(' callback data:', ava.get_data())
ava.measure_callback(-1, callback_fcn)
My problem
I have to store the received data in a 2D numpy array that I have created somewhere else in my main code, but I can't figure out what is the best way to update this array with the new data available inside the callback function.
I wondered if I could pass this numpy array as an argument for the callback function, but even in this case I cannot find a good way to do this since it is expected that the callback function will have only those 2 arguments.
Edit 1
I found a possible solution here but I am not sure it is the best way to do it. I'd rather not create a new class just to hold a single numpy array inside.
Edit 2
I actually changed my mind about my approach, because inside my callback I'd like to do many operations with the received data and save the results in many different variables. So, I went back to the class approach mentioned here, where I would basically have a class with all the variables that will somehow be used in the callback function and that would also inherit or have an object of the class ava.
However, as shown in this other question, the self parameter is a problem in this case.
If you don't want to create a new class, you can use a function closure:
# Initialize it however you want
numpy_array = ...
def callback_fcn(handle, info):
# Do what you want with the value of the variable
store_data(numpy_array, ...)
# After the callback is called, you can access the changes made to the object
print(get_data(numpy_array))
How this works is that when the callback_fcn is defined, it keeps a reference to the value of the variable numpy_array, so when it's called, it can manipulate it, as if it were passed as an argument to the function. So you get the effect of passing it in, without the callback caller having to worry about it.
I finally managed to solve my problem with a solution envolving a new class and also a closure function to deal with the self parameter that is described here. Besides that, another problem would appear by garbage collection of the new created method.
My final solution is:
class spectrometer():
def measurement_callback(self,handle,info):
if info.contents.value >= 0:
timestamp,spectrum = self.ava.get_data()
self.spectral_data[self.spectrum_index,:] = np.ctypeslib.as_array(spectrum[0:pixel_amount])
self.timestamps[self.spectrum_index] = timestamp
self.spectrum_index += 1
def __init__(self,ava):
self.ava = ava
self.measurement_callback = MeasureCallback(self.measurement_callback)
def register_callback(self,scans,pattern_amount,pixel_amount):
self.spectrum_index = 0
self.timestamps = np.empty((pattern_amount),dtype=np.uint32)
self.spectral_data = np.empty((pattern_amount,pixel_amount),dtype=np.float64)
self.ava.measure_callback(scans, self.measurement_callback)
I have a whole series of arrays with similar names mcmcdata.rho0, mcmcdata.rho1, ... and I want to be able to loop through them while updating their values. I can't figure out how this might be done or even what such a thing might be called.
I read my data in from file like this:
names1='l b rho0 rho1 rho2 rho3 rho4 rho5 rho6 rho7 rho8 rho9 rho10 rho11 rho12 rho13 rho14 rho15 rho16 rho17 rho18 rho19 rho20 rho21 rho22 rho23'.split()
mcmcdata=np.genfromtxt(filename,names=names1,dtype=None).view(np.recarray)
and I want to update the "rho" arrays later on after I do some calculations.
for jj in range(dbins):
mcmc_x, mcmc_y, mcmc_z = wf.lbd_to_xyz(mcmcdata.l,mcmcdata.b,d[jj],R_sun)
rho, thindisk, thickdisk, halo = wf.total_density_fithRthinhRthickhzthinhzthickhrfRiA( mcmc_x, mcmc_y, mcmc_z, R_sun,params)
eval("mcmcdata."+names1[2+jj]) = copy.deepcopy(rho)
eval("mcmcthin."+names1[2+jj]) = copy.deepcopy(thindisk)
eval("mcmcthick."+names1[2+jj]) = copy.deepcopy(thickdisk)
eval("mcmchalo."+names1[2+jj]) = copy.deepcopy(halo)
But the eval command is giving an error:
File "<ipython-input-133-30322c5e633d>", line 13
eval("mcmcdata."+names1[2+jj]) = copy.deepcopy(rho)
SyntaxError: can't assign to function call
How can I loop through my existing arrays and update their values?
or
How can identify the arrays by name so I can update them?
The eval command doesn't work the way you seem to think it does. You appear to be using it like a text-replacement macro, hoping that Python will read the given string and then pretend you wrote that text in the original source code. Instead, it receives a string, and then it executes that code. You're giving it an expression that refers to an attribute of an object, which is fine, but the result of evaluating that expression does not yield a thing you can assign to. It yields the value of that attribute.
Although Python provides eval, it also provides many other things that often obviate the need for eval. In the case of your code, Python provides setattr. You give it an object, the name of an attribute on that object, and a value, and it assigns that object's attribute to refer to the given value.
setattr(mcmcdata, names1[2+jj], copy.deepcopy(rho))
It might make the code more readable to get rid of the names1 portion, too. I might write the code like this:
setattr(mcmcdata, 'rho' + str(jj), copy.deepcopy(rho))
That way, it's clear that I'm assigning the rho-related attributes of the object without having to go look at what's held in the names1 list; the name names1 doesn't offer much information about what's in it.
I want to search surfs in all images in a given directory and save their keypoints and descriptors for future use. I decided to use pickle as shown below:
#!/usr/bin/env python
import os
import pickle
import cv2
class Frame:
def __init__(self, filename):
surf = cv2.SURF(500, 4, 2, True)
self.filename = filename
self.keypoints, self.descriptors = surf.detect(cv2.imread(filename, cv2.CV_LOAD_IMAGE_GRAYSCALE), None, False)
if __name__ == '__main__':
Fdb = open('db.dat', 'wb')
base_path = "img/"
frame_base = []
for filename in os.listdir(base_path):
frame_base.append(Frame(base_path+filename))
print filename
pickle.dump(frame_base,Fdb,-1)
Fdb.close()
When I try to execute, I get a following error:
File "src/pickle_test.py", line 23, in <module>
pickle.dump(frame_base,Fdb,-1)
...
pickle.PicklingError: Can't pickle <type 'cv2.KeyPoint'>: it's not the same object as cv2.KeyPoint
Does anybody know, what does it mean and how to fix it? I am using Python 2.6 and Opencv 2.3.1
Thank you a lot
The problem is that you cannot dump cv2.KeyPoint to a pickle file. I had the same issue, and managed to work around it by essentially serializing and deserializing the keypoints myself before dumping them with Pickle.
So represent every keypoint and its descriptor with a tuple:
temp = (point.pt, point.size, point.angle, point.response, point.octave,
point.class_id, desc)
Append all these points to some list that you then dump with Pickle.
Then when you want to retrieve the data again, load all the data with Pickle:
temp_feature = cv2.KeyPoint(x=point[0][0],y=point[0][1],_size=point[1], _angle=point[2],
_response=point[3], _octave=point[4], _class_id=point[5])
temp_descriptor = point[6]
Create a cv2.KeyPoint from this data using the above code, and you can then use these points to construct a list of features.
I suspect there is a neater way to do this, but the above works fine (and fast) for me. You might have to play around with your data format a bit, as my features are stored in format-specific lists. I tried to present the above using my idea at its generic base. I hope that this may help you.
Part of the issue is cv2.KeyPoint is a function in python that returns a cv2.KeyPoint object. Pickle is getting confused because, literally, "<type 'cv2.KeyPoint'> [is] not the same object as cv2.KeyPoint". That is, cv2.KeyPoint is a function object, while the type was cv2.KeyPoint. Why OpenCV is like that, I can only make guesses at unless I go digging. I have a feeling it has something to do with it being a wrapper around a C/C++ library.
Python does give you the ability to fix this yourself. I found the inspiration on this post about pickling methods of classes.
I actually use this clip of code, highly modified from the original in the post
import copyreg
import cv2
def _pickle_keypoints(point):
return cv2.KeyPoint, (*point.pt, point.size, point.angle,
point.response, point.octave, point.class_id)
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoints)
Key points of note:
In Python 2, you need to use copy_reg instead of copyreg and point.pt[0], point.pt[1] instead of *point.pt.
You can't directly access the cv2.KeyPoint class for some reason, so you make a temporary object and use that.
The copyreg patching will use the otherwise problematic cv2.KeyPoint function as I have specified in the output of _pickle_keypoints when unpickling, so we don't need to implement an unpickling routine.
And to be nauseatingly complete, cv2::KeyPoint::KeyPoint is an overloaded function in C++, but in Python, this isn't exactly a thing. Whereas in the C++, there's a function that takes the point for the first argument, in Python, it would try to interpret that as an int instead. The * unrolls the point into two arguments, x and y to match the only int argument constructor.
I had been using casper's excellent solution until I realized this was possible.
A similar solution to the one provided by Poik. Just call this once before pickling.
def patch_Keypoint_pickiling(self):
# Create the bundling between class and arguments to save for Keypoint class
# See : https://stackoverflow.com/questions/50337569/pickle-exception-for-cv2-boost-when-using-multiprocessing/50394788#50394788
def _pickle_keypoint(keypoint): # : cv2.KeyPoint
return cv2.KeyPoint, (
keypoint.pt[0],
keypoint.pt[1],
keypoint.size,
keypoint.angle,
keypoint.response,
keypoint.octave,
keypoint.class_id,
)
# C++ Constructor, notice order of arguments :
# KeyPoint (float x, float y, float _size, float _angle=-1, float _response=0, int _octave=0, int _class_id=-1)
# Apply the bundling to pickle
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoint)
More than for the code, this is for the incredibly clear explanation available there : https://stackoverflow.com/a/50394788/11094914
Please note that if you want to expand this idea to other "unpickable" class of openCV, you only need to build a similar function to "_pickle_keypoint". Be sure that you store attributes in the same order as the constructor. You can consider copying the C++ constructor, even in Python, as I did. Mostly C++ and Python constructors seems not to differ too much.
I has issue with the "pt" tuple. However, a C++ constructor exists for X and Y separated coordinates, and thus, allow this fix/workaround.
Hey! as you might have noticed I have an annoying issue with ctypes. I'm trying to communicate with an instrument and to do so I have to use ctypes to communicate with the DLL driver.
so far I've managed to export the DLL by doing this:
>>> from ctypes import *
>>>maury = WinDLL( 'MLibTuners')
>>> maury
(WinDLL 'MlibTuners', handle 10000000 at 9ef9d0)
>>> maury.get_tuner_driver_version()
(_FuncPtr object at 0x009F6738)
>>> version_string = create_string_buffer(80)
>>> maury.get_tuner_driver_version(version_string)
2258920
>>> print version_string.value
'Maury Microwave MT993V04 Tuner Driver DLL, Version 1.60.00, 07/25/2007'
And it works pretty well, according to the documentation it is supposed to save the Tuner Driver DLL in the 80 byte string given as a parameter.
However when I try to use the function called add_tuner it fails. This is what the documentation says:
short add_tuner(short tuner_number, char model[], short serial_number,
short ctlr_num, short ctlr_port, short *no_of_motors, long max_range[],
double *fmin, double *fmax, double *fcrossover, char error_string[])
this is how I tried to call the function above:
The parameters that are changed are all the pointers and max_range[], according to the manual the values below are correct too, i just don't know why I keep getting a windows access violation writing 0x00000000
no_motors = pointer(c_short())
f_min = pointer(c_double())
f_max = pointer(c_double())
f_crossover = pointer(c_double())
maury.add_tuner(c_short(0), c_char_p('MT982EU'), c_short(serial_number), c_short(0),
c_short(1),no_motors, c_long(),
f_min,f_max,f_crossover, create_string_buffer(80))
The serial number is given however censored by refering to a variable. Someone know what to do?, do you see any errors with my input?
Thanks
/Mazdak
I figure it's the value you pass at the long max_range[] argument. The function expects a pointer to a long integer there (it asks for an array of long integers), but you're passing a long value of zero (result of the c_long() call), which is implicitly cast to a null pointer. I suspect the function then tries to write to the address passed at max_range, ie. the null pointer, hence the access violation at address 0x00000000.
To create an array of longs to pass in max_range, you first create the array type by multiplying the array data type with the size of the array (somewhat verbose for clarity):
array_size = 3
ThreeLongsArrayType = c_long * array_size
You can then instantiate an array like you would with any other Python class:
array = ThreeLongsArrayType()