Develop Reference

Setting value of a iterable property of a Python COM object

I am using pywin32 to automate some tasks in software that has an Automation Server technology interface (formerly OLE Automation Server).
This software comes with a somewhat detailed manual with code examples in VBA, C++ or Matlab but no Python. I have built a Python library that can do most of the functionalities built into the software but there are some parts I cannot do in Python.
I cannot change the value of a property if this property is contained in a iterable COM object.
What I can do:
[Documentation for Visibility property]
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
app.Visibility = True
As an example, with this code, I can change the visibility parameter of the software: if it runs with or without GUI.
What I cannot do:
[Documentation for getting and setting current device]
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
app.CurrentDevice(0) = 'NAME OF DEVICE'
I then get the following error:
SyntaxError: cannot assign to function call here. Maybe you meant '==' instead of '='?
This error makes sense to me but I cannot find a way to set any of these software properties when they come in the form of an iterable object. As soon as I have to specify an index, I don't know how to set the value.
From what I understand, in C++ we are able to change the value because of pointers but how can we achieve the same thing in Python? Is it possible or do I have to use some C++ code in parallel to my Python to run my library? I don't know anything in C++ so if I could avoid doing that, it would be good.
What I have tried
Of course, the 1st thing I tried was to change () to [] or {} which logically didn't work.
Then I used the Evaluate function in PyCharms to see what was hiding behind my app.CurrentDevice. I was hoping to find sub-attributes that I could then set but I don't see anything inside the object:
[Result of Evaluate on the CurrentDevice object]
Finally, I have tried the following:
import win32com.client
app = win32com.client.Dispatch('NAME_OF_APP')
curr_device = app.CurrentDevice(0)
curr_device = 'NAME OF DEVICE'
I wanted to affect the object to a variable and then change the value but of course, this only rewrites the variable curr-device with 'NAME OF DEVICE' but loses any link to COM Object.
I feel like my questions are similar to the following unanswered question:
How can I set the value of an indexed property of a COM object in Python?

It looks as if win32com is struggling to set the property if there is an additional argument to the put function, which is a little surprising.
First thing to do is to use
app = win32com.client.gencache.EnsureDispatch('NAME_OF_APP')
This creates a Python wrapper for the COM object (rather than just firing function calls at the object and hoping). This may in itself clear up your issue.
If not, here is a quite ugly way of working around. As you have identified, the relevant part of the type library is:
[id(0x00000018),propput, helpstring("property CurrentDevice")]
HRESULT CurrentDevice([in] long lAcq, [in] VARIANT pVal);
And you can use this to set the property at a low level.
win32com dispatch objects are a wrapper for the PyIDispatch object. All dispatch objects support the Invoke method, and you can use this to call the function yourself. NB. Since I don't have access to your COM object, I can't test, so this answer may need some tweaking (!).
The PyIDispatch documentation
Try:
import win32com.client as wc
import pythoncom
app = wc.gencache.EnsureDispatch('NAME OF APP')
app.Visibility=TRUE
newVal = wc.VARIANT(pythoncom.VT_VARIANT,'NAME OF DEVICE')
app._oleobj_.Invoke(24,0,pythoncom.INVOKE_PROPERTYPUT,0,0,newVal)
There are a lot of 'magic' numbers here, but basically:
24 = 0x00000018 in decimal: this is the Id of the property
0 = the LCID, the Locale Id ... I always set it to 0
pythoncom.INVOKE_PROPERTYPUT = the type of call.
0 = whether you care about the return type (you probably don't = False)
0 = first parameter, lAcq, as in CurrentDevice(0)
newVal = second paramter,pVal, the new device name as a VARIANT
I haven't tried this, but pythoncom is pretty good about converting VARIANT types, so you might not need the VARIANT creation, and can just use NAME OF DEVICE directly as the parameter.

Unit testing an outgoing query message

I have adapted for Python an example taken from "The Magic Tricks of Testing" by Sandi Metz. Suppose I have this:
class Gear:
def __init__(self, wheel=Wheel()):
self.wheel = wheel
def get_gear_inches(self):
self._get_ratio() * self.wheel.get_diameter()
Now I want to write unit tests for the get_gear_inches() method.
In "The Magic Tricks of Testing", Sandi Metz suggests not to set expectations on the return value of wheel.get_diameter() (which is an outgoing query), because this would bind us to the implementation, while we only care about the returned results. So we are just supposed to test the returned value, like this:
def test_get_gear_inches():
gear = Gear()
assert gear.get_gear_inches() == 12 # For example
But I'm a bit confused.
Isn't this an integration test? It seems to me like we're testing the behaviour of two components (Gear and Wheel), not just one.
Also, what if this outgoing query (self.wheel.get_diameter()) is reaching the db and takes a long time?
What's the "correct" way to test this?
Note: my alternative would have been to set expectations and mock the behaviour of Wheel.get_diameter(), like this:
def test_get_gear_inches():
wheel_mock = Mock()
wheel_mock.get_diameter.return_value = 123 # We avoid calling the actual implementation
gear = Gear(wheel_mock)
assert Gear().get_gear_inches() == 12 # For example
But in the video this is discouraged as far as I understand.

She is saying that checking gear.wheel.diameter is antipattern because this way the test is going to be broken if the implementation of weel has changed. But it doesn't bother her that the initialization weel already makes the tests more fragile.
But the better solution would be to test gear together with wheel if this is possible. Or even to test them both together with something bigger.
But if these objects have complex behavior and you want to test them separately then the right solution is to mock wheel. Otherwise, you test wheel together with Gear when there is no such need because you test `wheel somewhere else.
All principles should be applied only in certain situations and shouldn't be applied when it is contrary to common sense. It is not always mentioned by those who teach.

Python `is_trait_type` doesn't work with `Date`

In the project I work for, we often need to convert text to the value of a trait. Generally, we use the is_trait_type method to do the appropriate conversion.
However, it doesn't work with Date traits. Here is a MWE:
from traits.has_traits import HasTraits
from traits.trait_types import Int, Date
class A(HasTraits):
a_date = Date
an_int = Int
a = A()
class_traits = a.class_traits()
print class_traits["an_int"].is_trait_type(Int)
print class_traits["a_date"].is_trait_type(Date)
The Int behave as expected but the Date fails with:
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types
We use Enthought traits module (version 4.1.0) under Ubuntu 14.04.

As mentioned in the comments, Date (and Time) trait types are not classes but instances. The is_trait_type(x) method checks whether self.trait_type is an instance of the provided class (i.e. the value of x), and hence it fails if x is not a class. In my opinion, this a bug in the API.
If you need a workaround, you can define a method like this:
def my_is_trait_type(trait, trait_type):
if isinstance(trait_type, BaseInstance):
return trait.is_trait_type(trait_type.__class__)
else:
return trait.is_trait_type(trait_type)
However, I would reconsider using is_trait_type() for the task of finding the appropriate conversion. Maybe a map would do, for instance.

Pickling cv2.KeyPoint causes PicklingError

I want to search surfs in all images in a given directory and save their keypoints and descriptors for future use. I decided to use pickle as shown below:
#!/usr/bin/env python
import os
import pickle
import cv2
class Frame:
def __init__(self, filename):
surf = cv2.SURF(500, 4, 2, True)
self.filename = filename
self.keypoints, self.descriptors = surf.detect(cv2.imread(filename, cv2.CV_LOAD_IMAGE_GRAYSCALE), None, False)
if __name__ == '__main__':
Fdb = open('db.dat', 'wb')
base_path = "img/"
frame_base = []
for filename in os.listdir(base_path):
frame_base.append(Frame(base_path+filename))
print filename
pickle.dump(frame_base,Fdb,-1)
Fdb.close()
When I try to execute, I get a following error:
File "src/pickle_test.py", line 23, in <module>
pickle.dump(frame_base,Fdb,-1)
...
pickle.PicklingError: Can't pickle <type 'cv2.KeyPoint'>: it's not the same object as cv2.KeyPoint
Does anybody know, what does it mean and how to fix it? I am using Python 2.6 and Opencv 2.3.1
Thank you a lot

The problem is that you cannot dump cv2.KeyPoint to a pickle file. I had the same issue, and managed to work around it by essentially serializing and deserializing the keypoints myself before dumping them with Pickle.
So represent every keypoint and its descriptor with a tuple:
temp = (point.pt, point.size, point.angle, point.response, point.octave,
point.class_id, desc)
Append all these points to some list that you then dump with Pickle.
Then when you want to retrieve the data again, load all the data with Pickle:
temp_feature = cv2.KeyPoint(x=point[0][0],y=point[0][1],_size=point[1], _angle=point[2],
_response=point[3], _octave=point[4], _class_id=point[5])
temp_descriptor = point[6]
Create a cv2.KeyPoint from this data using the above code, and you can then use these points to construct a list of features.
I suspect there is a neater way to do this, but the above works fine (and fast) for me. You might have to play around with your data format a bit, as my features are stored in format-specific lists. I tried to present the above using my idea at its generic base. I hope that this may help you.

Part of the issue is cv2.KeyPoint is a function in python that returns a cv2.KeyPoint object. Pickle is getting confused because, literally, "<type 'cv2.KeyPoint'> [is] not the same object as cv2.KeyPoint". That is, cv2.KeyPoint is a function object, while the type was cv2.KeyPoint. Why OpenCV is like that, I can only make guesses at unless I go digging. I have a feeling it has something to do with it being a wrapper around a C/C++ library.
Python does give you the ability to fix this yourself. I found the inspiration on this post about pickling methods of classes.
I actually use this clip of code, highly modified from the original in the post
import copyreg
import cv2
def _pickle_keypoints(point):
return cv2.KeyPoint, (*point.pt, point.size, point.angle,
point.response, point.octave, point.class_id)
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoints)
Key points of note:
In Python 2, you need to use copy_reg instead of copyreg and point.pt[0], point.pt[1] instead of *point.pt.
You can't directly access the cv2.KeyPoint class for some reason, so you make a temporary object and use that.
The copyreg patching will use the otherwise problematic cv2.KeyPoint function as I have specified in the output of _pickle_keypoints when unpickling, so we don't need to implement an unpickling routine.
And to be nauseatingly complete, cv2::KeyPoint::KeyPoint is an overloaded function in C++, but in Python, this isn't exactly a thing. Whereas in the C++, there's a function that takes the point for the first argument, in Python, it would try to interpret that as an int instead. The * unrolls the point into two arguments, x and y to match the only int argument constructor.
I had been using casper's excellent solution until I realized this was possible.

A similar solution to the one provided by Poik. Just call this once before pickling.
def patch_Keypoint_pickiling(self):
# Create the bundling between class and arguments to save for Keypoint class
# See : https://stackoverflow.com/questions/50337569/pickle-exception-for-cv2-boost-when-using-multiprocessing/50394788#50394788
def _pickle_keypoint(keypoint): # : cv2.KeyPoint
return cv2.KeyPoint, (
keypoint.pt[0],
keypoint.pt[1],
keypoint.size,
keypoint.angle,
keypoint.response,
keypoint.octave,
keypoint.class_id,
)
# C++ Constructor, notice order of arguments :
# KeyPoint (float x, float y, float _size, float _angle=-1, float _response=0, int _octave=0, int _class_id=-1)
# Apply the bundling to pickle
copyreg.pickle(cv2.KeyPoint().__class__, _pickle_keypoint)
More than for the code, this is for the incredibly clear explanation available there : https://stackoverflow.com/a/50394788/11094914
Please note that if you want to expand this idea to other "unpickable" class of openCV, you only need to build a similar function to "_pickle_keypoint". Be sure that you store attributes in the same order as the constructor. You can consider copying the C++ constructor, even in Python, as I did. Mostly C++ and Python constructors seems not to differ too much.
I has issue with the "pt" tuple. However, a C++ constructor exists for X and Y separated coordinates, and thus, allow this fix/workaround.

How could you swap out a particular database implementation in python?

If I have a seperate class for my db calls, and I create another implementation of the db layer but say with a different data store.
Is there a way for me to completly swap out the implementation without having to change allot of code?
i.e. I am starting a project, so I can design things properly to achieve this from the get-go.
Note: I will use this pattern for other parts of the site also, not just the db layer so its not really specific to db layer only.

As long as two modules implement exactly the same interface (classes with the same names, methods, and other attributes, functions with the same names and signatures, ...) you can pick one or the other at the time your application is starting up, for example on the basis of some configuration file, and import the chosen one under a fixed name. All the rest of your application can then use that fixed name and, net of the startup code, be blissfully unaware of any shenanigans that may have been done at the start.
For example, consider a simplified case:
# english.py
def greet(): return 'Hello!'
# italian.py
def greet(): return 'Ciao!'
# french.py
def greet(): return 'Salut!'
# config.py
langname = 'italian'
# startit.py
import config
import sys
lang = __import__(config.langname)
sys.modules['lang'] = lang
Now, all the rest of the application can just import lang, and it will be getting under that name the italian module, so, when calling lang.greet(), it will get the string 'Ciao!'.
Of course, in real life you'll have multiple modules, each with multiple functions, classes, and whatnot, but the general principles stay very similar. Just take special care about modules with qualified names (such as foo.bar), i.e., modules which must reside in a package (in this case, foo). For those, you can't just use __import__'s return value, but must use a slightly more roundabout approach, such as:
import sys
def importanyasname(actualname, fakename):
__import__(actualname)
sys.modules[fakename] = sys.modules[actualname]
that is, ignore __import__'s return value, and reach right for the value that's left (with the actual name as the key) in the sys.modules dictionary -- that is the module object you seek, and that you can set back into sys.modules with the "fake name" by which all the rest of the application will be able to blissfully import it any time.