Related
Im trying to catch any exception that is raised in any servicer so I can make sure that I only propagate known exceptions and not unexpected ones like ValueError, TypeError etc.
I'd like to be able to catch any raised error, and format them or convert them to other errors to better control the info that is exposed.
I don't want to have to enclose every servicer method with try/except.
I've tried with an interceptor, but im not able to catch the errors there.
Is there a way of specifying an error handler for the grpc Server? like what you do with flask or any other http server?
gRPC Python currently don't support server-side global error handler. The interceptor won't execute the server handler inside the intercept_service function, so there is no way to try/except.
Also, I found the gRPC Python server interceptor implementation is different from what they proposed original at L13-Python-Interceptors.md#server-interceptors. If the implementation stick to the original design, we can use interceptor as global error handler easily with handler and request/request_iterator.
# Current Implementation
intercept_service(self, continuation, handler_call_details)
# Original Design
intercept_unary_unary_handler(self, handler, method, request, servicer_context)
intercept_unary_stream_handler(self, handler, method, request, servicer_context)
intercept_stream_unary_handler(self, handler, method, request_iterator, servicer_context)
intercept_stream_stream_handler(self, handler, method, request_iterator, servicer_context)
Please submit a feature request issue to https://github.com/grpc/grpc/issues.
Maybe this will help you :)
def _wrap_rpc_behavior(handler, fn):
if handler is None:
return None
if handler.request_streaming and handler.response_streaming:
behavior_fn = handler.stream_stream
handler_factory = grpc.stream_stream_rpc_method_handler
elif handler.request_streaming and not handler.response_streaming:
behavior_fn = handler.stream_unary
handler_factory = grpc.stream_unary_rpc_method_handler
elif not handler.request_streaming and handler.response_streaming:
behavior_fn = handler.unary_stream
handler_factory = grpc.unary_stream_rpc_method_handler
else:
behavior_fn = handler.unary_unary
handler_factory = grpc.unary_unary_rpc_method_handler
return handler_factory(fn(behavior_fn,
handler.request_streaming,
handler.response_streaming),
request_deserializer=handler.request_deserializer,
response_serializer=handler.response_serializer)
class TracebackLoggerInterceptor(grpc.ServerInterceptor):
def intercept_service(self, continuation, handler_call_details):
def latency_wrapper(behavior, request_streaming, response_streaming):
def new_behavior(request_or_iterator, servicer_context):
try:
return behavior(request_or_iterator, servicer_context)
except Exception as err:
logger.exception(err, exc_info=True)
return new_behavior
return _wrap_rpc_behavior(continuation(handler_call_details), latency_wrapper)
As some of the previous comments suggested, I tried the meta-class approach which works quite well.
Attached is a simple example to demonstrate how to intercept the grpc calls.
You could extend this by providing the metaclass a list of decorators which you could apply on each function.
Also, it would be wise to be more selective regarding the methods you apply the wrapper to. A good option would be to list the methods of the autogenerated base class and only wrap those.
from types import FunctionType
from functools import wraps
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
# do stuff here
return method(*args, **kwargs)
return wrapped
class ServicerMiddlewareClass(type):
def __new__(meta, classname, bases, class_dict):
new_class_dict = {}
for attribute_name, attribute in class_dict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
new_class_dict[attribute_name] = attribute
return type.__new__(meta, classname, bases, new_class_dict)
# In order to use
class MyGrpcService(grpc.MyGrpcServicer, metaclass=ServicerMiddlewareClass):
...
I struggled to think of a good title so I'll just explain it here. I'm using Python in Maya, which has some event callback options, so you can do something like on save: run function. I have a user interface class, which I'd like it to update when certain events are triggered, which I can do, but I'm looking for a cleaner way of doing it.
Here is a basic example similar to what I have:
class test(object):
def __init__(self, x=0):
self.x = x
def run_this(self):
print self.x
def display(self):
print 'load user interface'
#Here's the main stuff that used to be just 'test().display()'
try:
callbacks = [callback1, callback2, ...]
except NameError:
pass
else:
for i in callbacks:
try:
OpenMaya.MEventMessage.removeCallback(i)
except RuntimeError:
pass
ui = test(5)
callback1 = OpenMaya.MEventMessage.addEventCallback('SomeEvent', ui.run_this)
callback2 = OpenMaya.MEventMessage.addEventCallback('SomeOtherEvent', ui.run_this)
callback3 = ......
ui.display()
The callback persists until Maya is restarted, but you can remove it using removeCallback if you pass it the value that is returned from addEventCallback. The way I have currently is just check if the variable is set before you set it, which is a lot more messy than the previous one line of test().display()
Would there be a way that I can neatly do it in the function? Something where it'd delete the old one if I ran the test class again or something similar?
There are two ways you might want to try this.
You can an have a persistent object which represents your callback manager, and allow it to hook and unhook itself.
import maya.api.OpenMaya as om
import maya.cmds as cmds
om.MEventMessage.getEventNames()
class CallbackHandler(object):
def __init__(self, cb, fn):
self.callback = cb
self.function = fn
self.id = None
def install(self):
if self.id:
print "callback is currently installed"
return False
self.id = om.MEventMessage.addEventCallback(self.callback, self.function)
return True
def uninstall(self):
if self.id:
om.MEventMessage.removeCallback(self.id)
self.id = None
return True
else:
print "callback not currently installed"
return False
def __del__(self):
self.uninstall()
def test_fn(arg):
print "callback fired 2", arg
cb = CallbackHandler('NameChanged', test_fn)
cb.install()
# callback is active
cb.uninstall()
# callback not active
cb.install()
# callback on again
del(cb) # or cb = None
# callback gone again
In this version you'd store the CallbackHandlers you create for as long as you want the callback to persist and then manually uninstall them or let them fall out of scope when you don't need them any more.
Another option would be to create your own object to represent the callbacks and then add or remove any functions you want it to trigger in your own code. This keeps the management entirely on your side instead of relying on the api, which could be good or bad depending on your needs. You'd have an Event() class which was callable (using __call__() and it would have a list of functions to fire then its' __call__() was invoked by Maya. There's an example of the kind of event handler object you'd want here
I am finding that I am using plenty of context managers in Python. However, I have been testing a number of things using them, and I am often needing the following:
class MyTestCase(unittest.TestCase):
def testFirstThing(self):
with GetResource() as resource:
u = UnderTest(resource)
u.doStuff()
self.assertEqual(u.getSomething(), 'a value')
def testSecondThing(self):
with GetResource() as resource:
u = UnderTest(resource)
u.doOtherStuff()
self.assertEqual(u.getSomething(), 'a value')
When this gets to many tests, this is clearly going to get boring, so in the spirit of SPOT/DRY (single point of truth/dont repeat yourself), I'd want to refactor those bits into the test setUp() and tearDown() methods.
However, trying to do that has lead to this ugliness:
def setUp(self):
self._resource = GetSlot()
self._resource.__enter__()
def tearDown(self):
self._resource.__exit__(None, None, None)
There must be a better way to do this. Ideally, in the setUp()/tearDown() without repetitive bits for each test method (I can see how repeating a decorator on each method could do it).
Edit: Consider the undertest object to be internal, and the GetResource object to be a third party thing (which we aren't changing).
I've renamed GetSlot to GetResource here—this is more general than specific case—where context managers are the way which the object is intended to go into a locked state and out.
How about overriding unittest.TestCase.run() as illustrated below? This approach doesn't require calling any private methods or doing something to every method, which is what the questioner wanted.
from contextlib import contextmanager
import unittest
#contextmanager
def resource_manager():
yield 'foo'
class MyTest(unittest.TestCase):
def run(self, result=None):
with resource_manager() as resource:
self.resource = resource
super(MyTest, self).run(result)
def test(self):
self.assertEqual('foo', self.resource)
unittest.main()
This approach also allows passing the TestCase instance to the context manager, if you want to modify the TestCase instance there.
Manipulating context managers in situations where you don't want a with statement to clean things up if all your resource acquisitions succeed is one of the use cases that contextlib.ExitStack() is designed to handle.
For example (using addCleanup() rather than a custom tearDown() implementation):
def setUp(self):
with contextlib.ExitStack() as stack:
self._resource = stack.enter_context(GetResource())
self.addCleanup(stack.pop_all().close)
That's the most robust approach, since it correctly handles acquisition of multiple resources:
def setUp(self):
with contextlib.ExitStack() as stack:
self._resource1 = stack.enter_context(GetResource())
self._resource2 = stack.enter_context(GetOtherResource())
self.addCleanup(stack.pop_all().close)
Here, if GetOtherResource() fails, the first resource will be cleaned up immediately by the with statement, while if it succeeds, the pop_all() call will postpone the cleanup until the registered cleanup function runs.
If you know you're only ever going to have one resource to manage, you can skip the with statement:
def setUp(self):
stack = contextlib.ExitStack()
self._resource = stack.enter_context(GetResource())
self.addCleanup(stack.close)
However, that's a bit more error prone, since if you add more resources to the stack without first switching to the with statement based version, successfully allocated resources may not get cleaned up promptly if later resource acquisitions fail.
You can also write something comparable using a custom tearDown() implementation by saving a reference to the resource stack on the test case:
def setUp(self):
with contextlib.ExitStack() as stack:
self._resource1 = stack.enter_context(GetResource())
self._resource2 = stack.enter_context(GetOtherResource())
self._resource_stack = stack.pop_all()
def tearDown(self):
self._resource_stack.close()
Alternatively, you can also define a custom cleanup function that accesses the resource via a closure reference, avoiding the need to store any extra state on the test case purely for cleanup purposes:
def setUp(self):
with contextlib.ExitStack() as stack:
resource = stack.enter_context(GetResource())
def cleanup():
if necessary:
one_last_chance_to_use(resource)
stack.pop_all().close()
self.addCleanup(cleanup)
pytest fixtures are very close to your idea/style, and allow for exactly what you want:
import pytest
from code.to.test import foo
#pytest.fixture(...)
def resource():
with your_context_manager as r:
yield r
def test_foo(resource):
assert foo(resource).bar() == 42
The problem with calling __enter__ and __exit__ as you did, is not that you have done so: they can be called outside of a with statement. The problem is that your code has no provision to call the object's __exit__ method properly if an exception occurs.
So, the way to do it is to have a decorator that will wrap the call to your original method in a withstatement. A short metaclass can apply the decorator transparently to all methods named test* in the class -
# -*- coding: utf-8 -*-
from functools import wraps
import unittest
def setup_context(method):
# the 'wraps' decorator preserves the original function name
# otherwise unittest would not call it, as its name
# would not start with 'test'
#wraps(method)
def test_wrapper(self, *args, **kw):
with GetSlot() as slot:
self._slot = slot
result = method(self, *args, **kw)
delattr(self, "_slot")
return result
return test_wrapper
class MetaContext(type):
def __new__(mcs, name, bases, dct):
for key, value in dct.items():
if key.startswith("test"):
dct[key] = setup_context(value)
return type.__new__(mcs, name, bases, dct)
class GetSlot(object):
def __enter__(self):
return self
def __exit__(self, *args, **kw):
print "exiting object"
def doStuff(self):
print "doing stuff"
def doOtherStuff(self):
raise ValueError
def getSomething(self):
return "a value"
def UnderTest(*args):
return args[0]
class MyTestCase(unittest.TestCase):
__metaclass__ = MetaContext
def testFirstThing(self):
u = UnderTest(self._slot)
u.doStuff()
self.assertEqual(u.getSomething(), 'a value')
def testSecondThing(self):
u = UnderTest(self._slot)
u.doOtherStuff()
self.assertEqual(u.getSomething(), 'a value')
unittest.main()
(I also included mock implementations of "GetSlot" and the methods and functions in your example so that I myself could test the decorator and metaclass I am suggesting on this answer)
I'd argue you should separate your test of the context manager from your test of the Slot class. You could even use a mock object simulating the initialize/finalize interface of slot to test the context manager object, and then test your slot object separately.
from unittest import TestCase, main
class MockSlot(object):
initialized = False
ok_called = False
error_called = False
def initialize(self):
self.initialized = True
def finalize_ok(self):
self.ok_called = True
def finalize_error(self):
self.error_called = True
class GetSlot(object):
def __init__(self, slot_factory=MockSlot):
self.slot_factory = slot_factory
def __enter__(self):
s = self.s = self.slot_factory()
s.initialize()
return s
def __exit__(self, type, value, traceback):
if type is None:
self.s.finalize_ok()
else:
self.s.finalize_error()
class TestContextManager(TestCase):
def test_getslot_calls_initialize(self):
g = GetSlot()
with g as slot:
pass
self.assertTrue(g.s.initialized)
def test_getslot_calls_finalize_ok_if_operation_successful(self):
g = GetSlot()
with g as slot:
pass
self.assertTrue(g.s.ok_called)
def test_getslot_calls_finalize_error_if_operation_unsuccessful(self):
g = GetSlot()
try:
with g as slot:
raise ValueError
except:
pass
self.assertTrue(g.s.error_called)
if __name__ == "__main__":
main()
This makes code simpler, prevents concern mixing and allows you to reuse the context manager without having to code it in many places.
A friend and I have been playing around with pygame some and came across this tutorial for building games using pygame. We really liked how it broke out the game into a model-view-controller system with events as a go-between, but the code makes heavy use of isinstance checks for the event system.
Example:
class CPUSpinnerController:
...
def Notify(self, event):
if isinstance( event, QuitEvent ):
self.keepGoing = 0
This results in some extremely unpythonic code. Does anyone have any suggestions on how this could be improved? Or an alternative methodology for implementing MVC?
This is a bit of code I wrote based on #Mark-Hildreth answer (how do I link users?) Does anyone else have any good suggestions? I'm going to leave this open for another day or so before picking a solution.
class EventManager:
def __init__(self):
from weakref import WeakKeyDictionary
self.listeners = WeakKeyDictionary()
def add(self, listener):
self.listeners[ listener ] = 1
def remove(self, listener):
del self.listeners[ listener ]
def post(self, event):
print "post event %s" % event.name
for listener in self.listeners.keys():
listener.notify(event)
class Listener:
def __init__(self, event_mgr=None):
if event_mgr is not None:
event_mgr.add(self)
def notify(self, event):
event(self)
class Event:
def __init__(self, name="Generic Event"):
self.name = name
def __call__(self, controller):
pass
class QuitEvent(Event):
def __init__(self):
Event.__init__(self, "Quit")
def __call__(self, listener):
listener.exit(self)
class RunController(Listener):
def __init__(self, event_mgr):
Listener.__init__(self, event_mgr)
self.running = True
self.event_mgr = event_mgr
def exit(self, event):
print "exit called"
self.running = False
def run(self):
print "run called"
while self.running:
event = QuitEvent()
self.event_mgr.post(event)
em = EventManager()
run = RunController(em)
run.run()
This is another build using the examples from #Paul - impressively simple!
class WeakBoundMethod:
def __init__(self, meth):
import weakref
self._self = weakref.ref(meth.__self__)
self._func = meth.__func__
def __call__(self, *args, **kwargs):
self._func(self._self(), *args, **kwargs)
class EventManager:
def __init__(self):
# does this actually do anything?
self._listeners = { None : [ None ] }
def add(self, eventClass, listener):
print "add %s" % eventClass.__name__
key = eventClass.__name__
if (hasattr(listener, '__self__') and
hasattr(listener, '__func__')):
listener = WeakBoundMethod(listener)
try:
self._listeners[key].append(listener)
except KeyError:
# why did you not need this in your code?
self._listeners[key] = [listener]
print "add count %s" % len(self._listeners[key])
def remove(self, eventClass, listener):
key = eventClass.__name__
self._listeners[key].remove(listener)
def post(self, event):
eventClass = event.__class__
key = eventClass.__name__
print "post event %s (keys %s)" % (
key, len(self._listeners[key]))
for listener in self._listeners[key]:
listener(event)
class Event:
pass
class QuitEvent(Event):
pass
class RunController:
def __init__(self, event_mgr):
event_mgr.add(QuitEvent, self.exit)
self.running = True
self.event_mgr = event_mgr
def exit(self, event):
print "exit called"
self.running = False
def run(self):
print "run called"
while self.running:
event = QuitEvent()
self.event_mgr.post(event)
em = EventManager()
run = RunController(em)
run.run()
A cleaner way of handling events (and also a lot faster, but possibly consumes a bit more memory) is to have multiple event handler functions in your code. Something along these lines:
The Desired Interface
class KeyboardEvent:
pass
class MouseEvent:
pass
class NotifyThisClass:
def __init__(self, event_dispatcher):
self.ed = event_dispatcher
self.ed.add(KeyboardEvent, self.on_keyboard_event)
self.ed.add(MouseEvent, self.on_mouse_event)
def __del__(self):
self.ed.remove(KeyboardEvent, self.on_keyboard_event)
self.ed.remove(MouseEvent, self.on_mouse_event)
def on_keyboard_event(self, event):
pass
def on_mouse_event(self, event):
pass
Here, the __init__ method receives an EventDispatcher as an argument. The EventDispatcher.add function now takes the type of the event you are interested in, and the listener.
This has benefits for efficiency since the listener only ever gets called for events that it is interested in. It also results in more generic code inside the EventDispatcher itself:
EventDispatcher Implementation
class EventDispatcher:
def __init__(self):
# Dict that maps event types to lists of listeners
self._listeners = dict()
def add(self, eventcls, listener):
self._listeners.setdefault(eventcls, list()).append(listener)
def post(self, event):
try:
for listener in self._listeners[event.__class__]:
listener(event)
except KeyError:
pass # No listener interested in this event
But there is a problem with this implementation. Inside NotifyThisClass you do this:
self.ed.add(KeyboardEvent, self.on_keyboard_event)
The problem is with self.on_keyboard_event: it is a bound method which you passed to the EventDispatcher. Bound methods hold a reference to self; this means that as long as the EventDispatcher has the bound method, self will not be deleted.
WeakBoundMethod
You will need to create a WeakBoundMethod class that holds only a weak reference to self (I see you already know about weak references) so that the EventDispatcher does not prevent the deletion of self.
An alternative would be to have a NotifyThisClass.remove_listeners function that you call before deleting the object, but that's not really the cleanest solution and I find it very error prone (easy to forget to do).
The implementation of WeakBoundMethod would look something like this:
class WeakBoundMethod:
def __init__(self, meth):
self._self = weakref.ref(meth.__self__)
self._func = meth.__func__
def __call__(self, *args, **kwargs):
self._func(self._self(), *args, **kwargs)
Here's a more robust implementation I posted on CodeReview, and here's an example of how you'd use the class:
from weak_bound_method import WeakBoundMethod as Wbm
class NotifyThisClass:
def __init__(self, event_dispatcher):
self.ed = event_dispatcher
self.ed.add(KeyboardEvent, Wbm(self.on_keyboard_event))
self.ed.add(MouseEvent, Wbm(self.on_mouse_event))
Connection Objects (Optional)
When removing listeners from the manager/ dispatcher, instead of making the EventDispatcher needlessly search through the listeners until it finds the right event type, then search through the list until it finds the right listener, you could have something like this:
class NotifyThisClass:
def __init__(self, event_dispatcher):
self.ed = event_dispatcher
self._connections = [
self.ed.add(KeyboardEvent, Wbm(self.on_keyboard_event)),
self.ed.add(MouseEvent, Wbm(self.on_mouse_event))
]
Here EventDispatcher.add returns a Connection object that knows where in the EventDispatcher's dict of lists it resides. When a NotifyThisClass object is deleted, so is self._connections, which will call Connection.__del__, which will remove the listener from the EventDispatcher.
This could make your code both faster and easier to use because you only have to explicitly add the functions, they are removed automatically, but it's up to you to decide if you want to do this. If you do it, note that EventDispatcher.remove shouldn't exist anymore.
I stumbled upon SJ Brown's tutorial on making games in the past. It's a great page, one of the best I've read. However, like you, I didn't like the calls to isinstance, or the fact that all the listeners receive all the events.
First, isinstance is slower than checking that two strings are equals, so I ended up storing a name on my events and test for the name rather than the class. But still, the notify function with its battery of if was itching me because it felt like a waste of time. We can do two optimizations here:
Most listeners are interested in only a few types of events. For performance reasons, when QuitEvent is posted, only the listeners interested in it should be notified. The event manager keeps track of which listener wants to listen to which event.
Then, to avoid going through a tons of if statements in a single notify method, we will have one method per type of event.
Example:
class GameLoopController(...):
...
def onQuitEvent(self, event):
# Directly called by the event manager when a QuitEvent is posted.
# I call this an event handler.
self._running = False
Because I want the developer to type as little as possible, I made the following thing:
When a listener is registered to an event manager, the event manager scans all the methods of the listener. When one method starts with 'on' (or any prefix you like), then it looks at the rest ("QuitEvent") and binds this name to this method. Later, when the event manager pumps its event list, it looks at the event class name: "QuitEvent". It knows that name, and therefore can directly call all the corresponding event handlers directly. The developer has nothing to do but adding onWhateverEvent methods to have them working.
It has some drawbacks:
If I make a typo in the name of the handler ("onRunPhysicsEvent"
instead of "onPhysicsRanEvent" for example") then my handler will
never be called and I'll wonder why. But I know the trick so I
don't wonder why very long.
I cannot add an event handler after the listener has been
registered. I must un-register and re-register. Indeed, the
events handlers are scanned only during the registration. Then
again, I never had to do that anyway so I don't miss it.
Despite these drawbacks I like it much more than having the constructor of the listener explicitly explain the event manager that it wants to stay tuned of this, this, this and this event. And it's the same execution speed anyway.
Second point:
When designing our event manager, we want to be careful. Very often, a listener will respond to an event by creating-registering or unregistering-destroying listeners. This happens all the time. If we don't think about it then our game may break with RuntimeError: dictionary changed size during iteration. The code that you propose iterates over a copy of the dictionary so you're protected against explosions; but it has consequences to be aware of:
- Listeners registered because of an event will not receive that event.
- Listeners unregistered because of an event will still receive that
event.
I never found it to be a problem though.
I implemented that myself for the game I am developing. I can link you to two articles and a half I wrote on the subject:
http://niriel.wordpress.com/2011/08/06/who-controls-the-controllers/
http://niriel.wordpress.com/2011/08/08/the-event-management-is-in-place/
http://niriel.wordpress.com/2011/08/11/the-first-screenshot-of-infiniworld/
The links to my github account will bring you directly to the source code of the relevant parts. If you cannot wait, here's the thing: https://github.com/Niriel/Infiniworld/blob/v0.0.2/src/evtman.py . In there you'll see that the code for my event class is a bit big, but that every inherited event is declared in 2 lines: the base Event class is making your life easy.
So, this all works using python's introspection mechanism, and using the fact that methods are objects like any other that can be put in dictionaries. I think it's quite pythony :).
Give each event a method (possibly even using __call__), and pass in the Controller object as an argument. The "call" method should then call the controller object. For example...
class QuitEvent:
...
def __call__(self, controller):
controller.on_quit(self) # or possibly... controller.on_quit(self.val1, self.val2)
class CPUSpinnerController:
...
def on_quit(self, event):
...
Whatever code you're using to route your events to your controllers will call the __call__ method with the correct controller.
I stumbled upon the same issue (almost a decade later!), and here is an implementation I've been using so that the EventManager would notify only a subset of listeners.
It is based on defaultdict: the _listeners attribute of EventManager is a defaultdict of WeakKeyDictionary().
Event are all inherited from an empty abstract Event class, so listeners can focus only on some classes of events they want to listen to.
Here is a minimalist code to get the idea behind it:
from collections import defaultdict
from weakref import WeakKeyDictionary
class Event:
def __init__(self):
pass
class KeyboardEvent(Event): # for instance, a keyboard event class with the key pressed
def __init__(self, key):
self._key = key
class EventManager:
def __init__(self):
self._listeners = defaultdict(lambda: WeakKeyDictionary())
def register_listener(self, event_types, listener):
for event_type in event_types:
self._listeners[event_type][listener] = 1
def unregister_listener(self, listener):
for event_type in self._listeners:
self._listeners[event_type].pop(listener, None)
def post_event(self, event):
for listener in self._listeners[event.__class__]:
listener.notify(event)
When registering, a listener tells the event manager which event type it wants to be notified of.
When posting events, the event manager will only notify the listeners which registered to be notified for that type of event.
This piece of code has, of course, much less scope than the very general (and very elegant) solution proposed by #Paul Manta, but in my case, it helped remove some repetitive calls to isinstance and other checks while keeping things as simple as I could.
One of the drawbacks of this is that all type of events have to be objects of some class, but in OO python, this is supposed to be the way to go.
The problem is basically this, in python's gobject and gtk bindings. Assume we have a class that binds to a signal when constructed:
class ClipboardMonitor (object):
def __init__(self):
clip = gtk.clipboard_get(gtk.gdk.SELECTION_CLIPBOARD)
clip.connect("owner-change", self._clipboard_changed)
The problem is now that, no instance of ClipboardMonitor will ever die. The gtk clipboard is an application-wide object, and connecting to it keeps a reference to the object, since we use the callback self._clipboard_changed.
I'm debating how to work around this using weak references (weakref module), but I have yet to come up with a plan. Anyone have an idea how to pass a callback to the signal registration, and have it behave like a weak reference (if the signal callback is called when the ClipboardMonitor instance is out of scope, it should be a no-op).
Addition: Phrased independently of GObject or GTK+:
How do you provide a callback method to an opaque object, with weakref semantics? If the connecting object goes out of scope, it should be deleted and the callback should act as a no-op; the connectee should not hold a reference to the connector.
To clarify: I explicitly want to avoid having to call a "destructor/finalizer" method
The standard way is to disconnect the signal. This however needs to have a destructor-like method in your class, called explicitly by code which maintains your object. This is necessary, because otherwise you'll get circular dependency.
class ClipboardMonitor(object):
[...]
def __init__(self):
self.clip = gtk.clipboard_get(gtk.gdk.SELECTION_CLIPBOARD)
self.signal_id = self.clip.connect("owner-change", self._clipboard_changed)
def close(self):
self.clip.disconnect(self.signal_id)
As you pointed out, you need weakrefs if you want to avoid explicite destroying. I would write a weak callback factory, like:
import weakref
class CallbackWrapper(object):
def __init__(self, sender, callback):
self.weak_obj = weakref.ref(callback.im_self)
self.weak_fun = weakref.ref(callback.im_func)
self.sender = sender
self.handle = None
def __call__(self, *things):
obj = self.weak_obj()
fun = self.weak_fun()
if obj is not None and fun is not None:
return fun(obj, *things)
elif self.handle is not None:
self.sender.disconnect(self.handle)
self.handle = None
self.sender = None
def weak_connect(sender, signal, callback):
wrapper = CallbackWrapper(sender, callback)
wrapper.handle = sender.connect(signal, wrapper)
return wrapper
(this is a proof of concept code, works for me -- you should probably adapt this piece to your needs). Few notes:
I am storing callback object and function separatelly. You cannot simply make a weakref of a bound method, because bound methods are very temporary objects. Actually weakref.ref(obj.method) will destroy the bound method object instantly after creating a weakref. I didn't check whether it is needed to store a weakref to the function too... I guess if your code is static, you probably can avoid that.
The object wrapper will remove itself from the signal sender when it notices that the weak reference ceased to exist. This is also necessary to destroy the circular dependence between the CallbackWrapper and the signal sender object.
(This answer tracks my progress)
This second version will disconnect as well; I have a convenience function for gobjects, but I actually need this class for a more general case -- both for D-Bus signal callbacks and GObject callbacks.
Anyway, what can one call the WeakCallback implementation style? It is a very clean encapsulation of the weak callback, but with the gobject/dbus specialization unnoticeably tacked on. Beats writing two subclasses for those two cases.
import weakref
class WeakCallback (object):
"""A Weak Callback object that will keep a reference to
the connecting object with weakref semantics.
This allows to connect to gobject signals without it keeping
the connecting object alive forever.
Will use #gobject_token or #dbus_token if set as follows:
sender.disconnect(gobject_token)
dbus_token.remove()
"""
def __init__(self, obj, attr):
"""Create a new Weak Callback calling the method #obj.#attr"""
self.wref = weakref.ref(obj)
self.callback_attr = attr
self.gobject_token = None
self.dbus_token = None
def __call__(self, *args, **kwargs):
obj = self.wref()
if obj:
attr = getattr(obj, self.callback_attr)
attr(*args, **kwargs)
elif self.gobject_token:
sender = args[0]
sender.disconnect(self.gobject_token)
self.gobject_token = None
elif self.dbus_token:
self.dbus_token.remove()
self.dbus_token = None
def gobject_connect_weakly(sender, signal, connector, attr, *user_args):
"""Connect weakly to GObject #sender's #signal,
with a callback in #connector named #attr.
"""
wc = WeakCallback(connector, attr)
wc.gobject_token = sender.connect(signal, wc, *user_args)
not actually tried it yet, but:
class WeakCallback(object):
"""
Used to wrap bound methods without keeping a ref to the underlying object.
You can also pass in user_data and user_kwargs in the same way as with
rpartial. Note that refs will be kept to everything you pass in other than
the callback, which will have a weakref kept to it.
"""
def __init__(self, callback, *user_data, **user_kwargs):
self.im_self = weakref.proxy(callback.im_self, self._invalidated)
self.im_func = weakref.proxy(callback.im_func)
self.user_data = user_data
self.user_kwargs = user_kwargs
def __call__(self, *args, **kwargs):
kwargs.update(self.user_kwargs)
args += self.user_data
self.im_func(self.im_self, *args, **kwargs)
def _invalidated(self, im_self):
"""Called by the weakref.proxy object."""
cb = getattr(self, 'cancel_callback', None)
if cb is not None:
cb()
def add_cancel_function(self, cancel_callback):
"""
A ref will be kept to cancel_callback. It will be called back without
any args when the underlying object dies.
You can wrap it in WeakCallback if you want, but that's a bit too
self-referrential for me to do by default. Also, that would stop you
being able to use a lambda as the cancel_callback.
"""
self.cancel_callback = cancel_callback
def weak_connect(sender, signal, callback):
"""
API-compatible with the function described in
http://stackoverflow.com/questions/1364923/. Mostly used as an example.
"""
cb = WeakCallback(callback)
handle = sender.connect(signal, cb)
cb.add_cancel_function(WeakCallback(sender.disconnect, handle))