I have a problem trying to unpickle subclasses of this class. When I unpickle it, the stuff isn't there. What gives?
class Account:
def __init__(self, server, port, smtp_server, smtp_port):
self.server = server
self.port = port
self.smtp_server = smtp_server
self.smtp_port = smtp_port
self.save()
def save(self):
#save account for later loading
self.name = tkFileDialog.asksaveasfilename(title = "Save as..")
pickle.dump(self, open(self.name, "wr"))
Does your class inherit object?
Either way, you can specify what you want to pickle by overwriting __getstate__. Otherwise it should normally copy __dict__ if you're inheriting object.
So, Here's how I just figured it out- i moved the ugly pickle stuff (see comment) to the unpickling class, imported the classes I was pickling, and it seems like it works.
Related
I have classes that contain unpickleable attributes, e.g. COM interfaces. When I pickle the classes I can remove those attributes to make the whole instance of this class pickleable.
But when I unpickle the dumped data again, of course I want to re-establish those previously removed attributes. The easiest for me would be to provide the new values, e.g. the COM interfaces, as additional parameters to __setstate__() but of course this is not possible.
Is there another nice solution to this without setting all those attributes one by another from externally? Maybe something using __getnewargs__(), which I have not really understood yet?
Here is a minimal example:
import win32com.client as COM
import pickle
class Test:
def __init__(self, app):
self.app = app
def __getstate__(self):
attr = self.__dict__.copy()
# Remove the unpicklable entries.
attr['app'] = None
return attr
def __setstate__(self, attr):
self.__dict__.update(attr)
# FIXME: Re-establish COM interfaces here! Would be nice to provide
# app as parameter an do self.app = app here.
# This must be done outside the class since it is only that easy in this
# minimal example.
app = COM.Dispatch('designer.Application')
test = Test(app)
with open('Test.pickle', 'wb') as fid:
pickle.dump(test, fid)
with open('Test.pickle', 'rb') as fid:
test_new = pickle.load(fid)
# Of course now I could do
# test_new.app = app
# But this is cumbersome with a lot of such attributes to be set...
print(test.app, test_new.app)
How do I re-initialize the base class using self.__init__?
In the following example my goal is to inherit eComm which is a socket derived driver. This handles connect/disconnect. If it disconnects we need to reinitialize it using __init__, however it appears to refer to the comDriver when self.__init__ is called.
How do I properly initialize the superclass to allow for this?
(else: self.__init__ in connect() is referring to comDriver, not eComm like it should)
The following is a simple case to reproduce the error in Python 3.x
class eComm():
def __init__(self):
self.s = "example object"
self.initialized = True
self.connected = False
def connect(self, IP_ADDRESS, PORT):
if self.initialized:
print(IP_ADDRESS, PORT)
else:
print("REINITIALIZING")
self.__init__()
self.connected = True
return(True)
class comDriver(eComm):
def __init__(self, IP_ADDRESS, PORT):
self.IP = IP_ADDRESS
self.PORT = PORT
super().__init__()
pass
def getTemp(self):
print("EXAMPLE FUNCTION")
return(1)
x = comDriver("192", 7)
x.connect("161", 6)
x.initialized = False
x.connect("111", 5)
IMO you're using the special method __init__ wrongly. It's meant to initialize a Python object, not anything outside that scope.
With your intention, I recommend that you create a separate initializer function, and call it from __init__. Here's an example:
class eComm():
def __init__(self):
self.initialize_eComm()
def initialize_eComm(self):
self.s = "example object"
self.initialized = True
self.connected = False
And then you can replace self.__init__() with self.initialize_eComm() to avoid name conflict in subclasses.
self.__init__ in connect() is referring to comDriver, not eComm like it should
This doesn't quite hold -- self refers to the calling object, which is comDriver. If you want to call to the __init__ method on eComm regardless of what classes extend it, you will have to reference in explicitly.
eComm.__init__(self)
But, the other answers and comments are right that this is not a good use of __init__.
Context:
I developed a python script to be run on a remote linux server. Running using Python 3.6.1. The script worked but was very messy, and procedurally written as opposed to OO. So, I re-wrote this script into 2 different classes. One main class and a blueprint class for objects.
My script is a lot more complicated, i just simplified it for this question.
Desired Function:
Read values from CSV file. Create Objects from these values, 1 object per line. Do some calculations on the values on init'ing the object (in the objects class). Have these objects be accessible from the main class (Base class).
Problems:
I need some clarification on:
The main method is not running. Tried variants on the method call, like Base.main(), including the "if name" statement inside the Base class, and it complains about self not being defined
The "self" reference. Are my usages of this correct? For example: Adding the attribute "age" into the Person objects so you can access it with person.age for example. My method call "self.input_file_handling(Base.inputFilePath)" etc.
Script:
import csv
class Person:
def calculate_age(self):
self.age = 2017 - self.birthYear
def __init__(self, name, birthYear):
self.name = self.strip_characters(self, name)
self.birthYear = int(birthYear)
self.calculate_age()
class Base:
inputFilePath = "input.csv"
people = []
def main():
self.input_file_handling(Base.inputFilePath)
#More methods here
#staticmethod
def input_file_handling(input_file_path):
input_file_path = str(input_file_path)
with open(input_file_path, 'r') as csv_file:
csv_reader = csv.DictReader(csv_file)
for line in csv_reader:
name = line['Name']
age = line['age']
person = Person(name, age)
people.append(person)
if __name__ == '__main__':
main()
First the main method of Base class is not static because it use the self variable, so is necessary receive that.
If you want call the main method and use the self variable you need make something like that:
class Base:
def main(self):
pass
if __name__ == '__main__':
instance_of_base = Base()
instance_of_base.main()
You can call the input_file_handling method without using self, because it's static
Base.input_file_handling(Base.inputFilePath)
I think you need learn more about how python resolve static things and the class and object variables.
Python is not C. There is no main function that automagically executes.
The main method that you defined is inside Base class, but it doesn't accept an argument for the instance.
Either modify it so it accept it (ie self by the convention) or make it a static method.
Then in if __name__ == '__main__': either use Base().main() or Base.main(), depending on what approach you decided to take.
But you don't seem to need any of this, and only doing it for the sake of forcing Python to look/work as other languages (looking at you C++/Java). Python doesn't require you to have a class or a 'main' function/method to execute code.
Your code written in a Pythonic way would be: (Python3)
import csv
from time import time, gmtime
INPUT_FILE_PATH = "input.csv"
class Person:
def __init__(self, name, birth_year):
self.name = name.strip()
self.birth_year = birth_year
#property
def birth_year(self):
return self._birth_year
#setter.birth_year
def birth_year(self, value):
self._birth_year = value
self._age = gmtime(time()).tm_year - value
#property
def age(self):
return self._age
#setter.age
def age(self, value):
self._birth_year = gmtime(time()).tm_year - value
self._age = value
def input_file_handling(input_file_path):
people = []
with open(input_file_path, 'r') as csv_file:
csv_reader = csv.DictReader(csv_file)
for line in csv_reader:
people.append(Person(line['Name'], int(line['age'])))
return people
if __name__ == '__main__':
people = input_file_handling(INPUT_FILE_PATH)
You seem to come from a OOP-only language (C# maybe?).
Some tips:
Avoid globals when able for variables, use them for function definition, class definition and constants.
Do not use a new class to store functions that do not require it
Use lower case and '' for variable and function names, Use CamelCase for class names, use caps and '' for constants.
Use duck typing: do not check that a argument is of a given type, try to use it as if it was and handle throw exceptions if it isn't.
Properties ar your friends if you want to force a specific bahaviour at getting or setting a class attributes
If you do not understand somehting ask in the comments.
I would like to extend an object with new attributes and methods, but in run-time. Basically I would prefer to just inherit and extend a class, but new objects of the base class are not usually created using it's constructor but by using fairly complex function.
Instead of...
from win32com import client
excel = client.Dispatch("Excel.Application")
excel.Visible = 1
excel.Workbooks.Add()
print(excel.Range("A1").value)
...I need something like (obviously broken):
from win32com import client
class Excel(client.CDispatch):
def __init__(self):
self = client.Dispatch("Excel.Application")
def get(self, cell):
return self.Range(cell).value
def show(self):
self.Visible = 1
excel = Excel()
excel.show()
excel.Workbooks.Add() # I want this to be still working
print(excel.get("A1"))
I still would like to be able to use original methods and attributes, but also my new ones. I have trouble wrapping my head around the concept, I am even not sure how to call the principle. Any ideas?
Another way to get desired functionality is like this:
from win32com import client
class Excel():
def __init__(self):
self.excel = client.Dispatch("Excel.Application")
self.Workbooks = self.excel.Workbooks
# I do not really want to repeat all base class
# functionality here to bind it to my new class
def get(self, cell):
return self.excel.Range(cell).value
def show(self):
self.excel.Visible = 1
excel = Excel()
excel.show()
excel.Workbooks.Add()
print(excel.get("A1"))
That works, however requires me to do a lot of lines similar to self.Workbooks = self.excel.Workbooks.
Implementation inheritence is mostly a variant of the composition / delegation pattern. The good news is that Python makes delegation quite easy. I've not tried (not working on Windows) but the following snippet might just work:
from win32com import client
class Excel(object):
def __init__(self):
self._app = client.Dispatch("Excel.Application")
def get(self, cell):
return self._app.Range(cell).value
def show(self):
self._app.Visible = 1
def __getattr__(self, name):
try:
return getattr(self._app, name)
except AttributeError:
raise AttributeError(
"'%s' object has no attribute '%s'" % (type(self).__name__, name))
traits_pickle_problem.py
from traits.api import HasTraits, List
import cPickle
class Client(HasTraits):
data = List
class Person(object):
def __init__(self):
self.client = Client()
# dynamic handler
self.client.on_trait_event(self.report,'data_items')
def report(self,obj,name,old,new):
print 'client added-- ' , new.added
if __name__ == '__main__':
p = Person()
p.client.data = [1,2,3]
p.client.data.append(10)
cPickle.dump(p,open('testTraits.pkl','wb'))
The above code reports a dynamic trait. Everything works as expected in this code. However, using a new python process and doing the following:
>>> from traits_pickle_problem import Person, Client
>>> p=cPickle.load(open('testTraits.pkl','rb'))
>>> p.client.data.append(1000)
causes no report of the list append. However, re-establishing the listener separately as follows:
>>> p.client.on_trait_event(p.report,'data_items')
>>> p.client.data.append(1000)
client added-- [1000]
makes it work again.
Am I missing something or does the handler need to be re-established in __setstate__ during the unpickling process.
Any help appreciated. This is for Python 2.7 (32-bit) on windows with traits version 4.30.
Running pickletools.dis(cPickle.dumps(p)), you can see the handler object being referenced:
...
213: c GLOBAL 'traits.trait_handlers TraitListObject'
...
But there's no further information on how it should be wired to the report method. So either the trait_handler doesn't pickle itself out properly, or it's an ephemeral thing like a file handle that can't be pickled in the first place.
In either case, your best option is to overload __setstate__ and re-wire the event handler when the object is re-created. It's not ideal, but at least everything is contained within the object.
class Person(object):
def __init__(self):
self.client = Client()
# dynamic handler
self.client.on_trait_event(self.report, 'data_items')
def __setstate__(self, d):
self.client = d['client']
self.client.on_trait_event(self.report, 'data_items')
def report(self, obj, name, old, new):
print 'client added-- ', new.added
Unpickling the file now correctly registers the event handler:
p=cPickle.load(open('testTraits.pkl','rb'))
p.client.data.append(1000)
>>> client added-- [1000]
You might find this talk Alex Gaynor did at PyCon interesting. It goes into the high points of how pickling work under the hood.
EDIT - initial response used on_trait_change - a typo that appears to work. Changed it back to on_trait_event for clarity.
I had the same problem but came around like this: Imaging I want to pickle only parts of a quiet big class and some of the objects has been set so transient=True so they're not pickled because there is nothing important to save, e.g.
class LineSpectrum(HasTraits):
andor_cam = Instance(ANDORiKonM, transient=True)
In difference to objects which should be saved, e.g.
spectrometer = Instance(SomeNiceSpectrometer)
In my LineSpectrum class, I have a
def __init__(self, f):
super(LineSpectrum, self).__init__()
self.load_spectrum(f)
def __setstate__(self, state): # WORKING!
print("LineSpectrum: __setstate__ with super(...) call")
self.__dict__.update(state)
super(LineSpectrum, self).__init__() # this has to be done, otherwise pickled sliders won't work, also first update __dict__!
self.from_pickle = True # is not needed by traits, need it for myself
self.andor_cam = ANDORiKonM(self.filename)
self.load_spectrum(self.filename)
In my case, this works perfectly - all sliders are working, all values set at the time the object has been pickled are set back.
Hope this works for you or anybody who's having the same problem. Got Anaconda Python 2.7.11, all packages updated.
PS: I know the thread is old, but didn't want to open a new one just for this.