I have classes that contain unpickleable attributes, e.g. COM interfaces. When I pickle the classes I can remove those attributes to make the whole instance of this class pickleable.
But when I unpickle the dumped data again, of course I want to re-establish those previously removed attributes. The easiest for me would be to provide the new values, e.g. the COM interfaces, as additional parameters to __setstate__() but of course this is not possible.
Is there another nice solution to this without setting all those attributes one by another from externally? Maybe something using __getnewargs__(), which I have not really understood yet?
Here is a minimal example:
import win32com.client as COM
import pickle
class Test:
def __init__(self, app):
self.app = app
def __getstate__(self):
attr = self.__dict__.copy()
# Remove the unpicklable entries.
attr['app'] = None
return attr
def __setstate__(self, attr):
self.__dict__.update(attr)
# FIXME: Re-establish COM interfaces here! Would be nice to provide
# app as parameter an do self.app = app here.
# This must be done outside the class since it is only that easy in this
# minimal example.
app = COM.Dispatch('designer.Application')
test = Test(app)
with open('Test.pickle', 'wb') as fid:
pickle.dump(test, fid)
with open('Test.pickle', 'rb') as fid:
test_new = pickle.load(fid)
# Of course now I could do
# test_new.app = app
# But this is cumbersome with a lot of such attributes to be set...
print(test.app, test_new.app)
Related
I use an external tool in my Python code. In order to initialize this tool, I have to create a couple of objects. The external tool in question provides two quite different APIs, and no one of these APIs is capable of creating all objects the tool needs. Let's say, the tool is trafic simulation tool, where car objects are created using API 1 and bikes are created using API 2.
I have played with inheritance, tried to pick an appropriate design pattern but all my solutions look ugly to me.
The most simple way to represent what I am trying to achieve is:
class ObjectHandler():
api_1_types = ('type1', 'foo')
api_2_types = ('type2', 'bar')
def __init__(self):
self.handler1 = ObjectHandler1()
self.handler2 = ObjectHandler2()
def create(self, obj_type):
if obj_type in self.api_1_types:
return self.handler1.create()
elif obj_type in self.api_2_types:
return self.handler2.create()
else:
raise NotImplementedError
class ObjectHandler1():
def __init__(self):
# load external module that defines API 1
def create(self):
# return an object created via API 1
class ObjectHandler2():
def __init__(self):
# load external module that defines API 2
def create(self):
# return an object created via API 2
if __name__ == '__main__':
handler = ObjectHandler()
object_1 = handler.create('type1') # must be created by ObjectHandler1
object_2 = handler.create('type2') # must be created by ObjectHandler2
I am now searching for a good OO and pythonic way to achieve this.
Your method looks ok. Should use sets for in tests but it doesn't really matter. An alternative could be the following but I don't know if it is better:
def __init__(self):
self.handlers = dict()
handler1 = ObjectHandler1()
for type in api_1_types:
# These won't be copied but simply be a reference to the object
self.handlers[type] = handler1
# Repeat for the other one
and
def create(self, obj_type):
try:
return self.handlers[obj_type].create()
except KeyError:
raise NotImplementedError
I want to make a singleton class that gathers data from a csv file, and in order to do that, it needs to have a data member of type DictReader; but I am not sure how I would initialize this member in the class definition, as it can only be initialized like so:
with open('sourceFile.csv') as source:
reader = csv.DictReader(source)
Because Python will not allow you to declare variables without initialization, I need to know how I can initialize the reader object in the class Singleton.
Are you looking for something like:
class MySingleton(object):
def __init__(self, source):
self.my_reader = DictReader(source)
if __name__ == '__main__':
singleton = MySingleton(sourcefile)
for row in singleton.my_reader:
# do stuff
traits_pickle_problem.py
from traits.api import HasTraits, List
import cPickle
class Client(HasTraits):
data = List
class Person(object):
def __init__(self):
self.client = Client()
# dynamic handler
self.client.on_trait_event(self.report,'data_items')
def report(self,obj,name,old,new):
print 'client added-- ' , new.added
if __name__ == '__main__':
p = Person()
p.client.data = [1,2,3]
p.client.data.append(10)
cPickle.dump(p,open('testTraits.pkl','wb'))
The above code reports a dynamic trait. Everything works as expected in this code. However, using a new python process and doing the following:
>>> from traits_pickle_problem import Person, Client
>>> p=cPickle.load(open('testTraits.pkl','rb'))
>>> p.client.data.append(1000)
causes no report of the list append. However, re-establishing the listener separately as follows:
>>> p.client.on_trait_event(p.report,'data_items')
>>> p.client.data.append(1000)
client added-- [1000]
makes it work again.
Am I missing something or does the handler need to be re-established in __setstate__ during the unpickling process.
Any help appreciated. This is for Python 2.7 (32-bit) on windows with traits version 4.30.
Running pickletools.dis(cPickle.dumps(p)), you can see the handler object being referenced:
...
213: c GLOBAL 'traits.trait_handlers TraitListObject'
...
But there's no further information on how it should be wired to the report method. So either the trait_handler doesn't pickle itself out properly, or it's an ephemeral thing like a file handle that can't be pickled in the first place.
In either case, your best option is to overload __setstate__ and re-wire the event handler when the object is re-created. It's not ideal, but at least everything is contained within the object.
class Person(object):
def __init__(self):
self.client = Client()
# dynamic handler
self.client.on_trait_event(self.report, 'data_items')
def __setstate__(self, d):
self.client = d['client']
self.client.on_trait_event(self.report, 'data_items')
def report(self, obj, name, old, new):
print 'client added-- ', new.added
Unpickling the file now correctly registers the event handler:
p=cPickle.load(open('testTraits.pkl','rb'))
p.client.data.append(1000)
>>> client added-- [1000]
You might find this talk Alex Gaynor did at PyCon interesting. It goes into the high points of how pickling work under the hood.
EDIT - initial response used on_trait_change - a typo that appears to work. Changed it back to on_trait_event for clarity.
I had the same problem but came around like this: Imaging I want to pickle only parts of a quiet big class and some of the objects has been set so transient=True so they're not pickled because there is nothing important to save, e.g.
class LineSpectrum(HasTraits):
andor_cam = Instance(ANDORiKonM, transient=True)
In difference to objects which should be saved, e.g.
spectrometer = Instance(SomeNiceSpectrometer)
In my LineSpectrum class, I have a
def __init__(self, f):
super(LineSpectrum, self).__init__()
self.load_spectrum(f)
def __setstate__(self, state): # WORKING!
print("LineSpectrum: __setstate__ with super(...) call")
self.__dict__.update(state)
super(LineSpectrum, self).__init__() # this has to be done, otherwise pickled sliders won't work, also first update __dict__!
self.from_pickle = True # is not needed by traits, need it for myself
self.andor_cam = ANDORiKonM(self.filename)
self.load_spectrum(self.filename)
In my case, this works perfectly - all sliders are working, all values set at the time the object has been pickled are set back.
Hope this works for you or anybody who's having the same problem. Got Anaconda Python 2.7.11, all packages updated.
PS: I know the thread is old, but didn't want to open a new one just for this.
I am updating some code from using libglade to GtkBuilder, which is supposed to be the way of the future.
With gtk.glade, you could call glade_xml.signal_autoconnect(...) repeatedly to connect signals onto objects of different classes corresponding to different windows in the program. However Builder.connect_signals seems to work only once, and (therefore) to give warnings about any handlers that aren't defined in the first class that's passed in.
I realize I can connect them manually but this seems a bit laborious. (Or for that matter I could use some getattr hackery to let it connect them through a proxy to all the objects...)
Is it a bug there's no function to hook up handlers across multiple objects? Or am I missing something?
Someone else has a similar problem http://www.gtkforums.com/about1514.html which I assume means this can't be done.
Here's what I currently have. Feel free to use it, or to suggest something better:
class HandlerFinder(object):
"""Searches for handler implementations across multiple objects.
"""
# See <http://stackoverflow.com/questions/4637792> for why this is
# necessary.
def __init__(self, backing_objects):
self.backing_objects = backing_objects
def __getattr__(self, name):
for o in self.backing_objects:
if hasattr(o, name):
return getattr(o, name)
else:
raise AttributeError("%r not found on any of %r"
% (name, self.backing_objects))
I have been looking for a solution to this for some time and found that it can be done by passing a dict of all the handlers to connect_signals.
The inspect module can extract methods using
inspect.getmembers(instance, predicate=inspect.ismethod
These can then be concatenated into a dictionary using d.update(d3), watching out for duplicate functions such as on_delete.
Example code:
import inspect
...
handlers = {}
for c in [win2, win3, win4, self]: # self is the main window
methods = inspect.getmembers(c, predicate=inspect.ismethod)
handlers.update(methods)
builder.connect_signals(handlers)
This will not pick up alias method names declared using #alias. For an example of how to do that, see the code for Builder.py, at def dict_from_callback_obj.
I'm only a novice but this is what I do, maybe it can inspire;-)
I instantiate the major components from a 'control' and pass the builder object so that the instantiated object can make use of any of the builder objects (mainwindow in example) or add to the builder (aboutDialog example). I also pass a dictionary (dic) where each component adds "signals" to it.
Then the 'connect_signals(dic)' is executed.
Of course I need to do some manual signal connecting when I need to pass user arguments to the callback method, but those are few.
#modules.control.py
class Control:
def __init__(self):
# Load the builder obj
guibuilder = gtk.Builder()
guibuilder.add_from_file("gui/mainwindow.ui")
# Create a dictionnary to store signal from loaded components
dic = {}
# Instanciate the components...
aboutdialog = modules.aboutdialog.AboutDialog(guibuilder, dic)
mainwin = modules.mainwindow.MainWindow(guibuilder, dic, self)
...
guibuilder.connect_signals(dic)
del dic
#modules/aboutdialog.py
class AboutDialog:
def __init__(self, builder, dic):
dic["on_OpenAboutWindow_activate"] = self.on_OpenAboutWindow_activate
self.builder = builder
def on_OpenAboutWindow_activate(self, menu_item):
self.builder.add_from_file("gui/aboutdialog.ui")
self.aboutdialog = self.builder.get_object("aboutdialog")
self.aboutdialog.run()
self.aboutdialog.destroy()
#modules/mainwindow.py
class MainWindow:
def __init__(self, builder, dic, controller):
self.control = controller
# get gui xml and/or signals
dic["on_file_new_activate"] = self.control.newFile
dic["on_file_open_activate"] = self.control.openFile
dic["on_file_save_activate"] = self.control.saveFile
dic["on_file_close_activate"] = self.control.closeFile
...
# get needed gui objects
self.mainWindow = builder.get_object("mainWindow")
...
Edit: alternative to auto attach signals to callbacks:
Untested code
def start_element(name, attrs):
if name == "signal":
if attrs["handler"]:
handler = attrs["handler"]
#Insert code to verify if handler is part of the collection
#we want.
self.handlerList.append(handler)
def extractSignals(uiFile)
import xml.parsers.expat
p = xml.parsers.expat.ParserCreate()
p.StartElementHandler = self.start_element
p.ParseFile(uiFile)
self.handlerList = []
extractSignals(uiFile)
for handler in handlerList:
dic[handler] = eval(''. join(["self.", handler, "_cb"]))
builder.connect_signals
({
"on_window_destroy" : gtk.main_quit,
"on_buttonQuit_clicked" : gtk.main_quit
})
I have a problem trying to unpickle subclasses of this class. When I unpickle it, the stuff isn't there. What gives?
class Account:
def __init__(self, server, port, smtp_server, smtp_port):
self.server = server
self.port = port
self.smtp_server = smtp_server
self.smtp_port = smtp_port
self.save()
def save(self):
#save account for later loading
self.name = tkFileDialog.asksaveasfilename(title = "Save as..")
pickle.dump(self, open(self.name, "wr"))
Does your class inherit object?
Either way, you can specify what you want to pickle by overwriting __getstate__. Otherwise it should normally copy __dict__ if you're inheriting object.
So, Here's how I just figured it out- i moved the ugly pickle stuff (see comment) to the unpickling class, imported the classes I was pickling, and it seems like it works.