I am trying to make a class that will make api requests, retrying based on configuration options passed in to the retrying.retry decorator, and handle different error codes in the correct way for each job.
Here is my code:
from retrying import retry
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
#retry(**self.retry_kwargs) # Obviously doesn't know what self is
def make_request(self):
pass
How can I pass in parameters to this method decorator? I tried making them a class attribute, which didn't work either.
A couple of notes/questions:
The #retry decorator will be applied to the make_request method at the time the class is created, while retry_kwargs will only become available when an instance of the class is created, and thus the former must precede the latter.
In which case, the former cannot depend on information that becomes available in the latter, ... as long as you use the decorator syntax ...
The decorator syntax
#decorator
def xxx(...):
...
is just syntax sugar for
def xxx(...):
...
xxx = decorate(xxx)
which means that, along with the fact that Python is very dynamic, you could force the issue by doing something like
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
APIRequester.make_request = retry(**kwargs)(APIRequester.make_request)
def make_request(self):
pass
Whether this particular decorator chokes on the self parameter or not, I cannot tell you.
Will you have more than one instance of APIRequester? If so, note that the method will be re-decorated each time a new instance is created: can this work sensibly? (I doubt it.) But see the edit below ...
If you do not have more that one instance, then you probably don't need to rely on information that becomes availale at the singleton's construction time.
The above were some general Python principles. I doubt that you really want to force the issue in this case. It seems to me that you are trying to use the decorator in a way that it was not designed to be used.
Edit: instancemethods
If you replace the line that does the decorating in the constructor with
self.make_request = retry(**kwargs)(self.make_request)
then each instance will get its own decorated version of the function. This should avoid any problems with re-decoration of the same function. There may will still be problems with self getting in the way. In that case, you could remove the self parameter from the definition and wrap it with staticmethod:
self.make_request = retry(**kwargs)(staticmethod(self.make_request))
Or better still, use decorator syntax to apply staticmethod to make_request at the place where you define it, the way Guido inteded it.
Like this, it even stands a chance of working! :-)
Decorator is just a syntax sugar for func=decorator(func). You can do the assignment yourself:
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.make_request = retry(**kwargs)(self.make_request)
def make_request(self):
pass
This will internally replace a method (descriptor) by a function, but it will work as expected.
Of course self is available in the decorator at the call time. See answers to How to decorate a method inside a class? , on which I based my answer here:
def my_retry(fn):
from functools import wraps
#wraps(fn)
def wrapped(self):
print(self.retry_kwargs)
for i in range(self.retry_kwargs["times"]):
# you have total control
fn(self)
# around your method. Can even call it multiple times,
# call with original retry:
retry(**self.retry_kwargs)(fn)(self)
# check exceptions, return some value (here None), etc
#
return wrapped
class APIRequester(object):
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
#my_retry
def make_request(self):
print("method")
a = APIRequester('http://something', times=3)
a.make_request()
That is, original decorator is wrapped with a new, configuration-aware decorator. No need to change the constructor, syntax remains simple.
Retry decorator doesn't support class method, because instance of the class is implicitly passed to function.
Please decorate normal function.
If you want to wrap function into class, please decorate static method.
Related
I have a module, that contains a bunch of functions that generate xml's
In other module I am constructing a class, and I want to assign one of those function to a class variable
Problem is, the fuction acts as a class method and when i call it from another class method, it passes self as a first argument.
Did i choose a good design approach? How to avoid passing self to a function? (#staticmethod decorator before generate = gen.generate doesn't work)
I would like to avoid making a class out of generate function if possible
Thanks!
generators.py
def generate(id, date):
pass
def generate_another():
pass
main.py
import generators as gen
class Client():
generate = gen.generate
def get_result(self, *args, **qwargs):
request = self.generate(id, date)
You must understand, that:
#decorator
def some_func():
pass
Is just syntactic sugar for
def some_func():
pass
some_func = decorator(some_func)
So in this case, you just want:
import generators as gen
class Client:
generate = staticmethod(gen.generate)
def get_result(self, *args, **qwargs):
request = self.generate(id, date)
As to whether or not this is a good design decision, it's probably too much of an opion-based question. Personally, I tend to avoid staticmethod. What advantage does this design offer over simply calling gen.generate inside get_result?
This is nearly a duplicate of #juanpa.arrivillaga's answer, but with a simplification which highlights a potential design advantage, addressing Juanpa's question in the case where we're truly using a static method (no self arg needed).
Code which originally looks like this (class with a decorated static method)...
import generators as gen
class Client:
#staticmethod
def get_result(*args, **qwargs):
request = gen.generate(id, date)
... could be simplified as:
import generators as gen
class Client:
get_result = staticmethod(gen.generate)
In Juanpa's answer, the class has two ways of calling the same underlying gen.generate:
client = Client()
# 1. as a class method
client.get_result()
# 2. as a static method
client.generate()
This all neglects the details of what is done with the request object etc, but if the same external function can be recycled completely as a staticmethod in multiple classes, Juanpa's answer helps keep static methods DRY:
def recycled_function(some_arg):
...
class ClassOne:
static_func = staticfunction(recycled_function)
class ClassTwo:
static_func = staticfunction(recycled_function)
This approach is more concise than copying the full implementation of recycled_function into each class decorated by #staticmethod, or even by making a wrapper method or static function to call the external recycled_function.
I have a set of different classes that share most of their functionality. Their differences can be isolated in a single method and in the decorator to be applied to one of the base methods.
What's the cleanest for the derived classes to set this decorator, which has to be applied to a base class method? I tried something along these lines, but it didn't work, as the method to be decorated was already bound:
class Base(other):
decorator = lambda x:x
def __init__(self, *args, **kwargs):
self.post = self.decorator(self.post)
super(Base, self).__init__(*args, **kwargs)
def post(self):
pass
class Derived(Base):
decorator = some_decorator
The short version is: What you want here is effectively the same thing as a static method, and that's the easiest way to solve it.
The problem isn't that the method, self.post, is bound, but that the decorator, self.decorator, is.
When you store a function as a class attribute, that's basically the same thing as defining a new method. So accessing it as self.decorator is going to get you a bound method. (If you don't understand why, either read the Descriptor HowTo, or take it on faith.) Which means it will be called with that self as its first argument.
You could always add an explicit self parameter to decorator and just ignore it… but if you want a method without a self parameter, that's exactly what a static method is: something that, when used as a method, doesn't take a magic self. So:
class Derived(Base):
#staticmethod
def decorator(func):
whatever(fund)
… or:
class Derived(Base):
decorator = staticmethod(whatever)
If you really want to look up decorator as a data attribute even though it's a function, the easy way is to move it to the instance:
class Derived(Base):
def __init__(self, *args, **kwargs):
self.decorator = whatever
super(Derived, self).__init__(*args, **kwargs)
Or, of course, you can reverse the descriptory methodness:
self.post = self.decorator.im_func(self.post)
… or just avoid it by doing the lookup manually:
decorator = type(self).__dict__['decorator']
self.post = decorator(self.post)
These are all hacky, but then you're trying to do something hacky, so I don't think it's a problem that the hackiness is explicit.
I'm trying to wrap a function when defining the class, use method in this class's instance
Below is a code that works
class A(object):
def __init__(self):
pass
#self_decorator
def to_be_decorated(self):
pass
def self_decorator(fn):
from functools import wraps
#wraps(fn)
def wrapper(*args, **kwargs):
self = args[0]
return self.app.route('/twip')(fn(*args, **kwargs))
return wrapper
What I actually tries to get:
class A(object):
def __init__(self):
self.app = APP()
#self.app.wrapper_function # which apparently doesn't work
def to_be_decorated(self):
pass
So, is it possible for my way of decorating to work?
At the class definition there's no self as the class do not exists yet and cannot be instantiated.
You can use
class A(object):
app = APP()
#app.wrapper_function
def to_be_decorated(self):
pass
But that would be a class variable.
A decorator is just a function that gets called at the time of definition.
When you write this:
#self.app.wrapper_function # which apparently doesn't work
def to_be_decorated(self):
pass
it's roughly the same as writing this:
def to_be_decorated(self):
pass
to_be_decorated = self.app.wrapper_function(to_be_decorated)
Once you see it that way, it's obvious why this doesn't work, and couldn't possibly work.
First, A's class definition doesn't have a self variable, nor does A have a member app that could be accessed even if it did. Instead, each instance of A will have its own unique self.app.
And even if you wish the interpreter could just "do what I mean", if you think about it, that can't mean anything. You want to call "the" self.app.wrapper_function method, but there is no such thing. Unless you read through all of the relevant code everywhere the method could be defined or redefined, there's absolutely no guarantee that the self.app.wrapper_function from different instances of A will even have the same underlying func_code. But, even if they did, self.app.wrapper_function is a bound method—a method bound together with the object it's called on—so they're still all different from each other.
There are plenty of ways around this, but they're all going to be the same trick: Use some kind of indirection to get a function which isn't a member of self.app or self, and references self.app at call time. This is basically what your self_decorator does, and it's unavoidable.
I need to decorate a object's method. It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program (arguments supplied with argv), so a same object could be decorated 3 times, 2 times, or not be decorated at all.
Here is some context, the program is a puzzle solver, the main behavior is to find a solution for the puzzle automatically, by automatically I mean without user intervention. And here is where the decoration gets to play, one of the things I want to is draw a graph of what happened during the execution, but I want to do so only when the flag --draw-graph is used.
Here is what I've tried:
class GraphDecorator(object):
def __init__(self, wrappee):
self.wrappee = wrappee
def method(self):
# do my stuff here
self.wrappee.method()
# do more of stuff here
def __getattr__(self,attr):
return getattr(self.wrappee,attr)
And why it did NOT work:
It did not work because of the way I built the application, when a method that did not exist in my Decorator class was called it felt back to the implementation of the decorated class, the problem is that the application always started invoking the method run that did not need to be decorated, so the undecorated fall back was used and from inside the undecorated form it always called undecorated methods, what I needed was to replace the method from the object, not to proxy it:
# method responsible to replace the undecorated form by the decorated one
def graphDecorator(obj):
old_method = obj.method
def method(self):
# do my stuff here
old_method()
# do more of my stuff
setattr(obj,'method',method) # replace with the decorated form
And here is my problem, the decorated form does not receive self when it is called resulting on a TypeError because of the wrong number of arguments.
The problem was that I couldn't use func(self) as a method. The reason is that setattr() method does not bound the function, and the function acts like it a static method - not a class method -, thanks to the introspective nature of python I've able to come up with this solution:
def decorator(obj):
old_func = obj.func # can't call 'by name' because of recursion
def decorated_func(self):
# do my stuff here
old_func() # does not need pass obj
# do some othere stuff here
# here is the magic, this get the type of a 'normal method' of a class
method = type(obj.func)
# this bounds the method to the object, so self is passed by default
obj.func = method(decorated_func, obj)
I think this is the best way to decorate a object's method at runtime, though it would be nice to find a way to call method() directly, without the line method = type(obj.func)
You might want to use __getattribute__ instead of __getattr__ (the latter being only called if "standard" lookup fails):
class GraphDecorator(object):
def __init__(self, wrappee):
self.__wrappee = wrappee
def method(self):
# do my stuff here
self.wrappe.method()
# do more of stuff here
def __getattribute__(self, name):
try:
wrappee = object.__getattribute__(self, "_GraphDecorator__wrappee")
return getattr(wrappee, name)
except AttributeError:
return object.__getattribute__(self, name)
I need to decorate a object's method. It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program (arguments supplied with argv), so a same object could be decorated 3 times, 2 times, or not be decorated at all.
The above is unfortunately incorrect, and what you are trying to do is unnecessary.
You can do this at runtime like so. Example:
import sys
args = sys.argv[1:]
class MyClass(object):
pass
if args[0]=='--decorateWithFoo':
MyClass = decoratorFoo(MyClass)
if args[1]=='--decorateWithBar'
MyClass = decoratorBar(MyClass)
The syntax:
#deco
define something
Is the same thing as:
define something
something = deco(something)
You could also make a decorator factory #makeDecorator(command_line_arguments)
"It needs to be at runtime because the decorators applied on the object depends on the arguments that the user gave when calling the program"
The don't use decorators. Decorators are only syntactical support for wrappers, you can just as well use normal function/method calls instead.
I have a class that has an output() method which returns a matplotlib Figure instance. I have a decorator I wrote that takes that fig instance and turns it into a Django response object.
My decorator looks like this:
class plot_svg(object):
def __init__(self, view):
self.view = view
def __call__(self, *args, **kwargs):
print args, kwargs
fig = self.view(*args, **kwargs)
canvas=FigureCanvas(fig)
response=HttpResponse(content_type='image/svg+xml')
canvas.print_svg(response)
return response
and this is how it was being used:
def as_avg(self):
return plot_svg(self.output)()
The only reason I has it that way instead of using the "#" syntax is because when I do it with the "#":
#plot_svg
def as_svg(self):
return self.output()
I get this error:
as_svg() takes exactly 1 argument (0 given)
I'm trying to 'fix' this by putting it in the "#" syntax but I can't figure out how to get it working. I'm thinking it has something to do with self not getting passed where it's supposed to...
Right: when you decorate with a class, instead of with a function, you have to make it a descriptor (give it a __get__ method, at least) to get the "automatic self". Simplest is to decorate with a function instead:
def plot_svg(view):
def wrapper(*args, **kwargs):
print args, kwargs
fig = view(*args, **kwargs)
canvas = FigureCanvas(fig)
response = HttpResponse(content_type='image/svg+xml')
canvas.print_svg(response)
return response
return wrapper
Background: the reason functions "become methods" (when defined in a class and accessed on an instance thereof by attribute-get notation), in other words, the reason such functions can get their automatic self is that they're descriptors -- the function type has a __get__.
A class doesn't have a __get__ method -- unless you explicitly add one. So why not just decorate with a function instead, as in the above example? This way you get the nice __get__ of functions automatically -- and as you see the nested function "lexical closure" property doesn't pose any problem at all (indeed, it simplifies things -- the nested function calls view, not self.view which might be rather confusing if self could mean either an instance of your decorator class OR an instance of the class whose method you're decorating...!-).
OK, there seem to be a couple of issues.
First, some syntax. You are getting the "takes exactly one argument" error because
your decorator, #plot_svg, is expecting one parameter (a view). There are other syntax errors as well.
Second, and more importantly, you want to write a function instead of a decorator. A decorator destroys your access to one concept and only gives you access to the combined
concept. You just want a new function like this:
def as_svg(an_object):
return django_response(an_object.output())
Alternately, I misunderstand your question from having a little too little code example.