I recently started learning about Decorators in Python and found the following piece of code which got me confused.
HANDLERS = {}
def handler_for(template):
def decorator(f):
HANDLERS[template] = f
return f
return decorator
def get_template(template, **kwargs):
#This function got me all confused. How does HANDLERS.get(template, None) works and how does it return demo_func() function?
return HANDLERS.get(template, None)
def email(template, **kwargs):
handler = get_template(template, **kwargs)
contents = handler(**kwargs)
print contents
#handler_for('demo')
def demo_func(**kwargs):
#Do something and return the String
if __name__ == "__main__":
email('demo')
I tried debugging with Python Debugger but I'm still getting confused as how the get_template() function calls the decorator which returns the function name.
I tried debugging with Python Debugger but I'm still getting confused as how the get_template() function calls the decorator which returns the function name.
Well… it doesn't. The decorator gets called by you using #handler_for('demo').
Decorators aren't as complicated as they seem, once you keep in mind that the following are equivalent:
#beans
def spam(eggs):
print(eggs)
def spam(eggs):
print(eggs)
spam = beans(spam)
That beans doesn't have to be just an identifier, it can be (almost) any expression, even a function call. So, your code is doing this:
demo_func = handler_for('demo')(demo_func)
handler_for takes 'demo' as the value of its template parameter. It then defines and returns a local function, decorator. So, when it's called, it has access to its own argument, f, and also to the closed-over template value from when it was constructed. That function is immediately called with demo_func as its f value, and it just stashes the function in the HANDLERS dictionary and returns it unchanged. So, inlining everything:
def demo_func(**kwargs):
#Do something and return the String
template = 'demo'
HANDLERS[template] = demo_func
demo_func = demo_func
So, because get_template just looks things up in HANDLERS, when you later call get_template('demo') it will find demo_func.
Related
I would like to write a decorator to redirect stdout for the main function to a specific log file. The argument that the main function takes is - item, and I want the decorator to allocate a different log path with each item for the main function. How do I achieve this?
Currently I have:
def redirect_stdout(func):
def wrapper():
with open(f"{log_path}{item}.log", "w") as log, contextlib.redirect_stdout(log), contextlib.redirect_stderr(log):
func(item)
return wrapper()
#redirect_stdout
def main(item):
But I am not sure how the item argument goes into the decorator. Thanks!
What you are looking for is something like below
def redirect_stdout(func):
def wrapper(item):
with open(f"{log_path}{item}.log", "w") as log, contextlib.redirect_stdout(log), contextlib.redirect_stderr(log):
func(item)
return wrapper
To understand how this works you need to properly understand how the decorator works. Check below I have tried to explain how the decorator works. ==> I have used to indicate this is equivalent to.
#redirect_stdout
def main(item):
pass
== >
def main(item):
pass
main = redirect_stdout(main) = wrapper
---------
main(item) => wrapper(item)
The current code I have, allows the function to call the wrapper decorator, and uses the function name in its code. However, I'm looking for a way to give the function a 'alias' in a way as an argument. Here's the current code:
import os, sys
# Take user input
message = input('type command: ')
# Command wrapper
ALLCOMMANDS = {}
def command(function):
ALLCOMMANDS[function.__name__] = function
return function
# Commands
#command
def foo():
print("bar")
#command
def goo():
print('ber')
# Run appropriate command
if message in ALLCOMMANDS:
ALLCOMMANDS[message]()
For example I would want to be able to call the function by a name such as '!foo' from the user input, so maybe the argument would look like #command(name='!foo'), I just don't know where to go from there to use that argument in the decorator since it already has an argument.
I attempted
# Command wrapper
ALLCOMMANDS = {}
def command(name):
ALLCOMMANDS[name] = name
return name
but keep getting errors and I assume I am missing something
You should read up a bit more on python decorators. You're getting an error with:
def command(name):
ALLCOMMANDS[name] = name
return name
Because of the return name.
Decorators are just syntactic sugar. This:
#command
def foo():
print('bar')
Is equivalent to:
def foo():
print('bar')
foo = command(foo)
From this you can see why your original decorator works. At the end you return function.
Things get a little tricker when you have a decorator that takes arguments. Desugared the following:
#command('nickname')
def foo():
print('bar')
Looks like this:
def foo():
print('bar')
foo = command('nickname')(foo)
So, to write a decorator that takes arguments, the decorator needs to return a function that takes the function to decorate as an argument:
def command(nickname):
def wrapped(f):
ALLCOMMANDS[nickname] = f
return f
return wrapped
Also consider making ALLCOMMANDS an attribute on your command instead of a global (UPPER_SNAKE is usually reserved for constants):
def command(nickname):
def wrapped(f):
command._functions[nickname] = f
return f
return wrapped
command._functions = {}
Let's consider this piece of code where I would like to create bar dynamically with a decorator
def foo():
def bar():
print "I am bar from foo"
print bar()
def baz():
def bar():
print "I am bar from baz"
print bar()
I thought I could create bar from the outside with a decorator:
def bar2():
print "I am super bar from foo"
setattr(foo, 'bar', bar2)
But the result is not what I was expecting (I would like to get I am super bar from foo:
>>> foo()
I am bar from foo
Is it possible to override a sub-function on an existing function with a decorator?
The actual use case
I am writing a wrapper for a library and to avoid boilerplate code I would like to simplify my work.
Each library function has a prefix lib_ and returns an error code. I would like to add the prefix to the current function and treat the error code. This could be as simple as this:
def call():
fname = __libprefix__ + inspect.stack()[1][3]
return_code = getattr(__lib__, fname)(*args)
if return_code < 0: raise LibError(fname, return_code)
def foo():
call()
The problem is that call might act differently in certain cases. Some library functions do not return an error_code so it would be easier to write it like
this:
def foo():
call(check_status=True)
Or much better in my opinion (this is the point where I started thinking about decorators):
#LibFunc(check_status=True)
def foo():
call()
In this last example I should declare call inside foo as a sub-function created dynamically by the decorator itself.
The idea was to use something like this:
class LibFunc(object):
def __init__(self,**kwargs):
self.kwargs = kwargs
def __call__(self, original_func):
decorator_self = self
def wrappee( *args, **kwargs):
def call(*args):
fname = __libprefix__ + original_func.__name__
return_code = getattr(__lib__, fname)(*args)
if return_code < 0: raise LibError(fname, return_code)
print original_func
print call
# <<<< The part that does not work
setattr(original_func, 'call', call)
# <<<<
original_func(*args,**kwargs)
return wrappee
Initially I was tempted to call the call inside the decorator itself to minimize the writing:
#LibFunc():
foo(): pass
Unfortunately, this is not an option since other things should sometime be done before and after the call:
#LibFunc():
foo(a,b):
value = c_float()
call(a, pointer(value), b)
return value.value
Another option that I thought about was to use SWIG, but again this is not an option because I will need to rebuild the existing library with the SWIG wrapping functions.
And last but not least, I may get inspiration from SWIG typemaps and declare my wrapper as this:
#LibFunc(check_exit = true, map = ('<a', '>c_float', '<c_int(b)')):
foo(a,b): pass
This looks like the best solution to me, but this is another topic and another question...
Are you married to the idea of a decorator? Because if your goal is bunch of module-level functions each of which wraps somelib.lib_somefunctionname, I don't see why you need one.
Those module-level names don't have to be functions, they just have to be callable. They could be a bunch of class instances, as long as they have a __call__ method.
I used two different subclasses to determine how to treat the return value:
#!/usr/bin/env python3
import libtowrap # Replace with the real library name.
class Wrapper(object):
'''
Parent class for all wrapped functions in libtowrap.
'''
def __init__(self, name):
self.__name__ = str(name)
self.wrapped_name = 'lib_' + self.__name__
self.wrapped_func = getattr(libtowrap, self.wrapped_name)
self.__doc__ = self.wrapped_func.__doc__
return
class CheckedWrapper(Wrapper):
'''
Wraps functions in libtowrap that return an error code that must
be checked. Negative return values indicate an error, and will
raise a LibError. Successful calls return None.
'''
def __call__(self, *args, **kwargs):
error_code = self.wrapped_func(*args, **kwargs)
if error_code < 0:
raise LibError(self.__name__, error_code)
return
class UncheckedWrapper(Wrapper):
'''
Wraps functions in libtowrap that return a useful value, as
opposed to an error code.
'''
def __call__(self, *args, **kwargs):
return self.wrapped_func(*args, **kwargs)
strict = CheckedWrapper('strict')
negative_means_failure = CheckedWrapper('negative_means_failure')
whatever = UncheckedWrapper('whatever')
negative_is_ok = UncheckedWrapper('negative_is_ok')
Note that the wrapper "functions" are assigned while the module is being imported. They are in the top-level module namespace, and not hidden by any if __name__ == '__main__' test.
They will behave like functions for most purposes, but there will be minor differences. For example, I gave each instance a __name__ that matches the name they're assigned to, not the lib_-prefixed name used in libtowrap... but I copied the original __doc__, which might refer to a prefixed name like lib_some_other_function. Also, testing them with isinstance will probably surprise people.
For more about decorators, and for many more annoying little discrepancies like the ones I mentioned above, see Graham Dumpleton's half-hour lecture "Advanced Methods for Creating Decorators" (PyCon US 2014; slides). He is the author of the wrapt module (Python Package Index; Git Hub; Read the Docs), which corrects all(?) of the usual decorator inconsistencies. It might solve your problem entirely (except for the old lib_-style names showing up in __doc__).
I am fairly new to Python and have been learning about decorators. After messing around with Flask, I am trying to write some code that simulates their route handler/decorators, just to understand how decorators (with arguments) work.
In the code below, the route decorator seems to call itself once the script runs. My question is, how is it possible that app.route() gets called when i run this script, and what is really happening here? Notice i don't call my index() function anywhere directly.
# test.py
class Flask(object):
def __init__(self, name):
self.scriptname = name
def route(self, *rargs, **kargs):
args = list(rargs)
if kargs:
print(kargs['methods'])
def decorator(f):
f(args[0])
return decorator
app = Flask(__name__)
#app.route("/", methods = ["GET","PUT"])
def index(rt):
print('route: ' + rt)
the above prints this in my terminal:
$ python test.py
['GET', 'PUT']
route: /
Any insight would be appreciated.
#app.route("/", methods = ["GET","PUT"]) is an executable statement: it calls the route() method of the app object. Since it's at module level, it will be executed when the script is imported.
Now, the result of calling app.route(...) is a function, and because you've used the # to mark it as a decorator, that function will wrap index. Note that the syntax is just a shortcut for this:
index = app.route(...)(index)
in other words, Python will call the function returned by app.route() with index as a parameter, and store the result as the new index function.
However, you're missing a level here. A normal decorator, without params, is written like this:
#foo
def bar()
pass
and when the module is imported, foo() is run and returns a function that wraps bar. But you're calling your route() function within the decorator call! So actually your function needs to return a decorator function that itself returns a function that wraps the original function... headscratching, to be sure.
Your route method should look more like this:
def route(self, *rargs, **kargs):
args = list(rargs)
if kargs:
print(kargs['methods'])
def decorator(f):
def wrapped(index_args):
f(args[0])
return wrapped
return decorator
Basically... app.route(index, "/", ["GET", "PUT"]) is a function. And this is the function which is going to be called instead of index.
In your code, when you call index(), it calls app.route(index, "/", ["GET", "PUT"]). This starts by printing kargs['methods'], then creates the decorator function:
def decorator(f):
f(args[0])
This decorator will call the decorated function (f) with one argument, args[0], which here is "/". This prints route: /.
The best explanation of decorators I've found is here: How to make a chain of function decorators?
If you dont want the self-firing, you can define your decorator this way:
def route(*rargs, **kargs):
args = list(rargs)
if kargs:
print(kargs['methods'])
def decorator(f):
f(args[0])
return decorator
#app.route("/", methods = ["GET","PUT"])
def index(rt):
print('route: ' + rt)
However, the rt argument of index will never be used, because route always calls index with args[0] which is always \...
I've got some code in a decorator that I only want run once. Many other functions (utility and otherwise) will be called later down the line, and I want to ensure that other functions that may have this decorator aren't accidentally used way down in the nest of function calls.
I also want to be able to check, at any point, whether or not the current code has been wrapped in the decorator or not.
I've written this, but I just wanted to see if anyone else can think of a better/more elegant solution than checking for the (hopefully!) unique function name in the stack.
import inspect
def my_special_wrapper(fn):
def my_special_wrapper(*args, **kwargs):
""" Do some magic, only once! """
# Check we've not done this before
for frame in inspect.stack()[1:]: # get stack, ignoring current!
if frame[3] == 'my_special_wrapper':
raise StandardError('Special wrapper cannot be nested')
# Do magic then call fn
# ...
fn(*args, **kwargs)
return my_special_wrapper
def within_special_wrapper():
""" Helper to check that the function has been specially wrapped """
for frame in inspect.stack():
if frame[3] == 'my_special_wrapper':
return True
return False
#my_special_wrapper
def foo():
print within_special_wrapper()
bar()
print 'Success!'
#my_special_wrapper
def bar():
pass
foo()
Here is an example of using a global for this task - in what I believe is a relatively safe way:
from contextlib import contextmanager
from functools import wraps
_within_special_context = False
#contextmanager
def flag():
global _within_special_context
_within_special_context = True
try:
yield
finally:
_within_special_context = False
#I'd argue this would be best replaced by just checking the variable, but
#included for completeness.
def within_special_wrapper():
return _within_special_context
def my_special_wrapper(f):
#wraps(f)
def internal(*args, **kwargs):
if not _within_special_context:
with flag():
...
f(*args, **kwargs)
else:
raise Exception("No nested calls!")
return internal
#my_special_wrapper
def foo():
print(within_special_wrapper())
bar()
print('Success!')
#my_special_wrapper
def bar():
pass
foo()
Which results in:
True
Traceback (most recent call last):
File "/Users/gareth/Development/so/test.py", line 39, in <module>
foo()
File "/Users/gareth/Development/so/test.py", line 24, in internal
f(*args, **kwargs)
File "/Users/gareth/Development/so/test.py", line 32, in foo
bar()
File "/Users/gareth/Development/so/test.py", line 26, in internal
raise Exception("No nested calls!")
Exception: No nested calls!
Using a context manager ensures that the variable is unset. You could just use try/finally, but if you want to modify the behaviour for different situations, the context manager can be made to be flexible and reusable.
The obvious solution is to have special_wrapper set a global flag, and just skip its magic if the flag is set.
This is about the only good use of a global variable - to allow a single piece of code to store information that is only used within that code, but which needs to survive the life of execution in that code.
It doesn't need to be set in global scope. The function could set the flag on itself, for example, or on any object or class, as long as nothing else will touch it.
As noted by Lattyware in comments, you'll want to use either a try/except, or perhaps even better, a context manager to ensure the variable is unset.
Update: If you need the wrapped code to be able to check if it is wrapped, then provide a function which returns the value of the flag. You might want to wrap it all up with a class for neatness.
Update 2: I see you're doing this for transaction management. There are probably already libraries which do this. I strongly recommend that you at least look at their code.
While my solution technically works, it requires a manual reset of the decorator, but you could very well modify things such that the outermost function is instead a class (with the instances being the wrappers of the decorated functions passed to it in __init__), and have reset() being called in __exit__(), which would then allow you to use the with statement to create the decorator to be usable only once within the context. Also note that it requires Python 3 due to the nonlocal keyword, but that can easily be adapted to 2.7 with a dict in place of the flag variable.
def once_usable(decorator):
"Apply this decorator function to the decorator you want to be usable only once until it is reset."
def outer_wrapper():
flag = False
def inner_wrapper(*args, **kwargs):
nonlocal flag
if not flag:
flag = True
return decorator(*args, **kwargs)
else:
print("Decorator currently unusable.") # raising an Error also works
def decorator_reset():
nonlocal flag
flag = False
return (inner_wrapper, decorator_reset)
return outer_wrapper()
Testing:
>>> def a(aa):
return aa*2
>>> def b(bb):
def wrapper(*args, **kwargs):
print("Decorated.")
return bb(*args, **kwargs)
return wrapper
>>> dec, reset = once_usable(b)
>>> aa = dec(a)
>>> aa(22)
Decorated.
44
>>> aaa = dec(a)
Decorator currently unusable.
>>> reset()
>>> aaa = dec(a)
>>> aaa(11)
Decorated.
22