Develop Reference

Initializing an attribute in a child class that is used in the parent class

I am using a 3rd party Python library (wxPython), which has a buggy class in one of its modules.
The problematic code section looks like this:
def OnText(self, event):
value = self.GetValue()
if value != self.__oldvalue:
pass # Here some more code follows ...
self.__oldvalue = value
The problem is the if statement, because at the first call to this method self.__oldvalue has not been initialized yet. So for a workaround until this bug has been fixed by the library devs I thought I could fix this with a little workaround. I simply wanted to derive a child class from that faulty class and initialize self.__oldvalue in this constructor:
class MyIntCtrl(wx.lib.intctrl.IntCtrl):
def __init__(self, *args, **kw):
self.__oldvalue = None
super().__init__(*args, **kw)
However, now when I use this new class MyIntCtrl instead of the original IntCtrl class, I do get exactly the same error as before:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/wx/lib/intctrl.py", line 509, in OnText
if value != self.__oldvalue:
AttributeError: 'MyIntCtrl' object has no attribute '_IntCtrl__oldvalue'
Now I am wondering: What am I doing wrong, how else can I fix this issue in a child class?

Any member of class which starts with __ (double underscore) is private, you can use single underscore _ or not use underscores in naming for access them in derived classes.
class Parent:
def __init__(self):
self.__private_field = "private field"
self._protected_field = "protected field"
self.public_field = "public field"
class Child(Parent):
def __init__(self):
pass
def do(self):
print(self.__private_field) # It will throw exception
print(self._protected_field) # It will not throw exception
print(self.public_field) # It will not throw exception
Or you can bypass private/protected members by calling them like:
print(_Parent__private_field)

Construct object via __init__ and ignore constructor exception

I have a Python class whose __init__ method raises a custom exception called WrongFileSpecified.
However, when I write a unit test, I want to assign the attributes of the instance object from a test fixture. So normally what I would be doing is reading data off a file and then working with the instance object.
But with the test, I cannot use any test files, so I basically need to hard code the data in the instance object in the setUp method of the unit test. Is there any way to get a instance created without __init__ complaining about the exception?
Sample code:
class A(object):
def __init__(self, folderPath):
#check folder path using os.isdir() otherwise raise exception
#...
self.folderPath = folderPath
#Call load record
self._load_records() #uses self.folderPath and raises exceptions as well
#Note i cannot avoid raising these exceptions, its required
class TestA(unittest.TestCase):
.......
obj = None
def setUp(self):
obj = A('fake folder path')
obj.val1 = "testparam1"
obj.param2 = "testparam2"
def test_1(self):
.....

You can create an empty object, bypassing __init__ by using __new__.
obj = obj_type.__new__(obj_type)
Note that obj_type is the appropriate type object. This is a little hacky but it works. You are reponsible for setting the object's members.
Edit: here is an example.
class Foo():
def __init__(self):
self.x = 1
self.y = 2
def say_hello(self):
print('Hello!')
r = Foo.__new__(Foo)
r.say_hello()
print(r.x)
Console output:
Hello!
Traceback (most recent call last):
File "C:\WinPython-64bit-3.3.5.7\python-
3.3.5.amd64\Scripts\projects\luc_utils\dev\test\
unit_test_serialization.py", line 29, in <module>
print(r.x)
AttributeError: 'Foo' object has no attribute 'x'

Here are two options:
Refactor the file loading out to a class method, which is the Pythonic method of providing an alternate constructor (see below); or
Provide an additional parameter to __init__ to suppress the exceptions when necessary (e.g. def __init__(self, folderPath, suppress=False), or validate=True, whichever makes more sense for your usage).
The latter is a bit awkward, in my opinion, but would mean that you don't have to refactor existing code creating A instances. The former would look like:
class A(object):
def __init__(self, ...):
"""Pass whatever is loaded from the file to __init__."""
...
#classmethod
def from_file(cls, folderPath):
"""Load the data from the file, or raise an exception."""
...
and you would replace e.g. a = A(whatever) with a = A.from_file(whatever).

There is a very useful module called mock, you can check it out later, I feel that in this case it will be too much. Instead, you should consider redesigning your class, like this, for example:
class A(object):
def __init__(self, folderPath):
self.folderPath = folderPath
def _load_records(self)
#check folder path using os.isdir() otherwise raise exception
...
#uses self.folderPath and raises exceptions as well
...
#classmethod
def load_records(cls, folderpath):
obj = cls(folderpath)
obj._load_records()
return obj
# Usage
records = A.load_records('/path/to/records')
Then you can do:
class TestA(unittest.TestCase):
.......
obj = None
def setUp(self):
self.obj = A('fake folder path')
self.obj.val1 = "testparam1"
self.obj.param2 = "testparam2"
def test_1(self):
self.assertRaises(self.obj._load_records, HorribleFailureError)
Also i highly recommend to check out pytest, it has wonderful facilities for test fixtures, including fixtures for files and folders.

Why is PySide's exception handling extending this object's lifetime?

tl;dr -- In a PySide application, an object whose method throws an exception will remain alive even when all other references have been deleted. Why? And what, if anything, should one do about this?
In the course of building a simple CRUDish app using a Model-View-Presenter architecture with a PySide GUI, I discovered some curious behavior. In my case:
The interface is divided into multiple Views -- i.e., each tab page displaying a different aspect of data might be its own class of View
Views are instantiated first, and in their initialization, they instantiate their own Presenter, keeping a normal reference to it
A Presenter receives a reference to the View it drives, but stores this as a weak reference (weakref.ref) to avoid circularity
No other strong references to a Presenter exist. (Presenters can communicate indirectly with the pypubsub messaging library, but this also stores only weak references to listeners, and is not a factor in the MCVE below.)
Thus, in normal operation, when a View is deleted (e.g., when a tab is closed), its Presenter is subsequently deleted as its reference count becomes 0
However, a Presenter of which a method has thrown an exception does not get deleted as expected. The application continues to function, because PySide employs some magic to catch exceptions. The Presenter in question continues to receive and respond to any View events bound to it. But when the View is deleted, the exception-throwing Presenter remains alive until the whole application is closed. An MCVE (link for readability):
import logging
import sys
import weakref
from PySide import QtGui
class InnerPresenter:
def __init__(self, view):
self._view = weakref.ref(view)
self.logger = logging.getLogger('InnerPresenter')
self.logger.debug('Initializing InnerPresenter (id:%s)' % id(self))
def __del__(self):
self.logger.debug('Deleting InnerPresenter (id:%s)' % id(self))
#property
def view(self):
return self._view()
def on_alert(self):
self.view.show_alert()
def on_raise_exception(self):
raise Exception('From InnerPresenter (id:%s)' % id(self))
class OuterView(QtGui.QMainWindow):
def __init__(self, *args, **kwargs):
super(OuterView, self).__init__(*args, **kwargs)
self.logger = logging.getLogger('OuterView')
# Menus
menu_bar = self.menuBar()
test_menu = menu_bar.addMenu('&Test')
self.open_action = QtGui.QAction('&Open inner', self, triggered=self.on_open, enabled=True)
test_menu.addAction(self.open_action)
self.close_action = QtGui.QAction('&Close inner', self, triggered=self.on_close, enabled=False)
test_menu.addAction(self.close_action)
def closeEvent(self, event, *args, **kwargs):
self.logger.debug('Exiting application')
event.accept()
def on_open(self):
self.setCentralWidget(InnerView(self))
self.open_action.setEnabled(False)
self.close_action.setEnabled(True)
def on_close(self):
self.setCentralWidget(None)
self.open_action.setEnabled(True)
self.close_action.setEnabled(False)
class InnerView(QtGui.QWidget):
def __init__(self, *args, **kwargs):
super(InnerView, self).__init__(*args, **kwargs)
self.logger = logging.getLogger('InnerView')
self.logger.debug('Initializing InnerView (id:%s)' % id(self))
self.presenter = InnerPresenter(self)
# Layout
layout = QtGui.QHBoxLayout(self)
alert_button = QtGui.QPushButton('Alert!', self, clicked=self.presenter.on_alert)
layout.addWidget(alert_button)
raise_button = QtGui.QPushButton('Raise exception!', self, clicked=self.presenter.on_raise_exception)
layout.addWidget(raise_button)
self.setLayout(layout)
def __del__(self):
super(InnerView, self).__del__()
self.logger.debug('Deleting InnerView (id:%s)' % id(self))
def show_alert(self):
QtGui.QMessageBox(text='Here is an alert').exec_()
if __name__ == '__main__':
logging.basicConfig(level=logging.DEBUG)
app = QtGui.QApplication(sys.argv)
view = OuterView()
view.show()
sys.exit(app.exec_())
Open and close the inner view, and you'll see both view and presenter are deleted as expected. Open the inner view, click the button to trigger an exception on the presenter, then close the inner view. The view will be deleted, but the presenter won't until the application exits.
Why? Presumably whatever it is that catches all exceptions on behalf of PySide is storing a reference to the object that threw it. Why would it need to do that?
How should I proceed (aside from writing code that never causes exceptions, of course)? I have enough sense not to rely on __del__ for resource management. I get that I have no right to expect anything subsequent to a caught-but-not-really-handled exception to go ideally but this just strikes me as unnecessarily ugly. How should I approach this in general?

The problem is sys.last_tracback and sys.last_value.
When a traceback is raised interactively, and this seems to be what is emulated, the last exception and its traceback are stores in sys.last_value and sys.last_traceback respectively.
Doing
del sys.last_value
del sys.last_traceback
# for consistency, see
# https://docs.python.org/3/library/sys.html#sys.last_type
del sys.last_type
will free the memory.
It's worth noting that at most one exception and traceback pair can get cached. This means that, because you're sane and don't rely on del, there isn't a massive amount of damage to be done.
But if you want to reclaim the memory, just delete those values.

Python use destructor in a memory safe way

I have a class that represents a database connection, that has a close method. To make it more comfortable to use, the class itself is a context manager that closes the connection when finished working with it.
The problem is that I got no guarantee that the user of this class will always remember to either use the context manager or close it explicitly, so I would like to implement a method to be called when the object is garbage collected.
The __del__ method looks pretty much like what I need, but I have read in many places that it is risky since it can disturb the garbage collector, especially if there are some circular references.
How true is that? will the following code entail some potential memory leaks?
def __del__(self):
self.close()

Here's an example where you would need the weakref so the child doesn't block the parent from cleaning up:
import weakref
class Spam(object):
def __init__(self, name):
self.name = name
def __del__(self):
print '%r got deleted' % self
def __repr__(self):
return '<%s:%s>' % (self.__class__.__name__, self.name)
class SpamChild(Spam):
def __init__(self, name, parent):
Spam.__init__(self, name)
self.parent = weakref.ref(parent)
def __del__(self):
print '%r, child of %r got deleted' % (self, self.parent)
a = Spam('a')
b = SpamChild('b', a)
del a

pygtk gtk.Builder.connect_signals onto multiple objects?

I am updating some code from using libglade to GtkBuilder, which is supposed to be the way of the future.
With gtk.glade, you could call glade_xml.signal_autoconnect(...) repeatedly to connect signals onto objects of different classes corresponding to different windows in the program. However Builder.connect_signals seems to work only once, and (therefore) to give warnings about any handlers that aren't defined in the first class that's passed in.
I realize I can connect them manually but this seems a bit laborious. (Or for that matter I could use some getattr hackery to let it connect them through a proxy to all the objects...)
Is it a bug there's no function to hook up handlers across multiple objects? Or am I missing something?
Someone else has a similar problem http://www.gtkforums.com/about1514.html which I assume means this can't be done.

Here's what I currently have. Feel free to use it, or to suggest something better:
class HandlerFinder(object):
"""Searches for handler implementations across multiple objects.
"""
# See <http://stackoverflow.com/questions/4637792> for why this is
# necessary.
def __init__(self, backing_objects):
self.backing_objects = backing_objects
def __getattr__(self, name):
for o in self.backing_objects:
if hasattr(o, name):
return getattr(o, name)
else:
raise AttributeError("%r not found on any of %r"
% (name, self.backing_objects))

I have been looking for a solution to this for some time and found that it can be done by passing a dict of all the handlers to connect_signals.
The inspect module can extract methods using
inspect.getmembers(instance, predicate=inspect.ismethod
These can then be concatenated into a dictionary using d.update(d3), watching out for duplicate functions such as on_delete.
Example code:
import inspect
...
handlers = {}
for c in [win2, win3, win4, self]: # self is the main window
methods = inspect.getmembers(c, predicate=inspect.ismethod)
handlers.update(methods)
builder.connect_signals(handlers)
This will not pick up alias method names declared using #alias. For an example of how to do that, see the code for Builder.py, at def dict_from_callback_obj.

I'm only a novice but this is what I do, maybe it can inspire;-)
I instantiate the major components from a 'control' and pass the builder object so that the instantiated object can make use of any of the builder objects (mainwindow in example) or add to the builder (aboutDialog example). I also pass a dictionary (dic) where each component adds "signals" to it.
Then the 'connect_signals(dic)' is executed.
Of course I need to do some manual signal connecting when I need to pass user arguments to the callback method, but those are few.
#modules.control.py
class Control:
def __init__(self):
# Load the builder obj
guibuilder = gtk.Builder()
guibuilder.add_from_file("gui/mainwindow.ui")
# Create a dictionnary to store signal from loaded components
dic = {}
# Instanciate the components...
aboutdialog = modules.aboutdialog.AboutDialog(guibuilder, dic)
mainwin = modules.mainwindow.MainWindow(guibuilder, dic, self)
...
guibuilder.connect_signals(dic)
del dic
#modules/aboutdialog.py
class AboutDialog:
def __init__(self, builder, dic):
dic["on_OpenAboutWindow_activate"] = self.on_OpenAboutWindow_activate
self.builder = builder
def on_OpenAboutWindow_activate(self, menu_item):
self.builder.add_from_file("gui/aboutdialog.ui")
self.aboutdialog = self.builder.get_object("aboutdialog")
self.aboutdialog.run()
self.aboutdialog.destroy()
#modules/mainwindow.py
class MainWindow:
def __init__(self, builder, dic, controller):
self.control = controller
# get gui xml and/or signals
dic["on_file_new_activate"] = self.control.newFile
dic["on_file_open_activate"] = self.control.openFile
dic["on_file_save_activate"] = self.control.saveFile
dic["on_file_close_activate"] = self.control.closeFile
...
# get needed gui objects
self.mainWindow = builder.get_object("mainWindow")
...
Edit: alternative to auto attach signals to callbacks:
Untested code
def start_element(name, attrs):
if name == "signal":
if attrs["handler"]:
handler = attrs["handler"]
#Insert code to verify if handler is part of the collection
#we want.
self.handlerList.append(handler)
def extractSignals(uiFile)
import xml.parsers.expat
p = xml.parsers.expat.ParserCreate()
p.StartElementHandler = self.start_element
p.ParseFile(uiFile)
self.handlerList = []
extractSignals(uiFile)
for handler in handlerList:
dic[handler] = eval(''. join(["self.", handler, "_cb"]))

builder.connect_signals
({
"on_window_destroy" : gtk.main_quit,
"on_buttonQuit_clicked" : gtk.main_quit
})