I am learning to write the python decorators. Following are two examples.
Example 1:
def dec():
def wrapper(func):
print(func)
return func
return wrapper
#dec
def hello():
print('hello')
Example 2:
def dec(name):
def wrapper(fn):
print(fn)
return fn
return wrapper
#dec('a')
def hello(x):
print(x)
So my question is why the example 1 throws an exception that dec() takes 0 positional arguments but 1 was given while the example 2 works well.
Bot examples are incorrect. the outer function of a decorator (dec in this case) must take an argument which is the function it decorates.
So example 1 should look like this:
def dec(func):
def wrapper():
print(func)
return func
return wrapper
#dec
def hello():
print('hello')
hello()
Example two while doesn't crash immediately, would if you tried to call hello (the decorated function).
For a decorator to take an argument you need another level of nesting.
So example 2 should look like this:
def dec_args(name):
def dec(func):
def wrapper():
print(func)
print(name)
return func
return wrapper
return dec
#dec_args('a')
def hello():
print('hello')
hello()
The inner decorator function (wrapper) should take the argument that are passed to the decorated function (hello). In these cases you don't have any
Also note that the wrapper function doesn't really need to return func. wrapper is essentially a replacement of func. That's what decorators do. They replace the function you decorate with the inner function of the decorator (by calling the outer function with the decorated function as an argument).
Related
If I create a decorator like following:
def my_decorator(some_fun):
def wrapper():
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
return wrapper
#my_decorator
def just_some_function():
print("Wheee!")
Another decorator can be defined as:
def my_decorator(some_fun):
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
#my_decorator
def just_some_fun():
print("some fun")
Both decorators will work the same. What is the benefit of using "wrapper" function inside decorator. I didn't get the purpose.
The purpose of having a wrapper function is that a function decorator receives a function object to decorate, and it must return the decorated function.
Your 2nd version of my_decorator doesn't have an explicit return statement, so it returns None. When my_decorator is called via the # decorator syntax
before some_function() is called.
some fun
after some_function() is called.
gets printed, and then None gets assigned to the name just_some_fun. So if you add print(just_some_fun) to the end of that code it will print None.
It may be easier to understand what's going on if we get rid of the # syntactic sugar and re-write your code using normal function calling syntax:
def my_decorator(some_fun):
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
def just_some_fun():
print("some fun")
just_some_fun = my_decorator(just_some_fun)
The decorators in Python are callable objects which in the simplest case is function taking one parameter which is some function or class. The decorator should return again same type which it takes (so if it takes function it should return function). The point is in the time when the decorator is called.
When you import a Python file or just run it directly the Python interpreter goes through the content and gathering information about what classes and function are defined and if it encounter on some code (not declaration) it will execute it.
If the interpreter encounter on decorator it takes the decorated function, call the decorator and replace the decorated function with returned value from the decorator.
Let's say you have this code:
#my_decorator
def my_function()
print("My_function")
It's equivalent to this call:
def my_function()
print("My_function")
my_function = my_decorator(my_function)
If the decorator would be like this one
def my_decorator(func):
print("decorated)
return 42
then the my_function is not even a function it would be an integer (you can try print(my_function))
So when you define decorator as
def my_decorator2(some_fun):
print("before")
some_fun()
print("after")
then this decorator returns nothing (in python it means it returns None).
#my_decorator2
def decorated():
print("inside")
prints
before
inside
after
but calling decorated() will raise exception 'NoneType' object is not callable because the decorated was replaced with None.
You should always create decorator which return something useful like function or class (which is usually the "wrap" function inside). Sometimes can be useful to return from decorator something else then function/class but it usually obfuscate your code and convert it into something totally non-maintainable.
Its already explained why to use wrapper function, out of curiosity I am just giving
examples what we can do if we don't need a wrapper function.
Type 1
Return a small function which returns None or pass
def decorator_func(to_be_decorated_function):
print("Logging IN: Currrently in function")
to_be_decorated_function()
print("Logging OUT: Currrently in function")
def a(): None # or def a(): pass
return (a)
#decorator_func
def to_be_decorated_function():
print('October 16, 2000')
to_be_decorated_function()
# equivalent to
#to_be_decorated_function = decorator_func(to_be_decorated_function)
Type 2
This merely tears out the usage of decorators, and just a slight tweak.What if we don't return , as well not use callable object at all.
def decorator_func(to_be_decorated_function):
print("Logging IN: Currrently in function")
to_be_decorated_function()
print("Logging OUT: Currrently in function")
#decorator_func
def to_be_decorated_function():
print('October 16, 2000')
to_be_decorated_function # notice I'm just using the object and not callable function
# equivalent to
#decorator_func(to_be_decorated_function)
Output from both the types
Logging IN: Currrently in function
October 16, 2000
Logging OUT: Currrently in function
I am absolutly new in Python (I came from Java and C#)
I am stufying the decorator topic. So I have the following example:
# DECORATOR:
def my_decorator(func):
def wrap_func():
print('**************')
func()
print('**************')
return wrap_func
#my_decorator
def hello():
print('Hello World')
hello()
It seems to me that this is the logic (but I am not sure of this:
I am defining a my_decorator function taking another function as parameter. This my_decorator function wrap the function passed as parameter into a wrap_func() function that execute the function passed as parameter. In addition this wrap_func() function can perform some extra logic before and after the execution of the function passed as parameter (it is decorating the original function.
To say to the interpreter that a specific function have to use the decorator I have to use the syntax #decorator_name before the function definition.
So in my case: when I perform hello() call: Python know that the hello() function have to be decorated by my_decorator decorator so it is not directly executing the hello() function but it is performing the my_decorator() function passing hello() reference as parameter. So it can add extra logic before and after the hello() invoaction.
Is this reasoning correct or am I missing something??
I have another doubt: why the decorator function return the function that wrap our decoration logic?
A decorator is just syntactic sugar for a function call.
The decoration
#my_decorator
def hello():
print('Hello world')
is equivalent to
def hello():
print('Hello world')
hello = my_decorator(hello)
So in this case, using the decorator syntax is the same as defining the following function directly:
def hello():
print('**************')
print('Hello world')
print('**************')
The decorator creates a function that calls print('**************'), then calls the original function, then calls print('**************') again.
If you had only planned on decorating once function, this is obviously just a lot of unnecessary boiler plate. But if you were decorating multiple functions, the abstraction pays off:
# DECORATOR:
def my_decorator(func):
def wrap_func():
print('**************')
func()
print('**************')
return wrap_func
#my_decorator
def hello():
print('Hello World')
#my_decorator
def how_are_you():
print("How are you doing?")
#my_decorator
def goodbye():
print('Goodbye, world')
vs
def hello():
print('**************')
print('Hello world')
print('**************')
def how_are_your():
print('**************')
print('How are you doing')
print('**************')
def good_bye():
print('**************')
print('Goodbye world')
print('**************')
especially if you later decide to wrap each output with ============== instead of **************.
Decorators are just a syntactic sugar. This:
#decorator
def func():
pass
is equivalent to this:
def func():
pass
func = decorator(func)
I have a decorator that records the functions present in my script:
registry=[]
def register(func):
print('running register(%s)' % func)
registry.append(func)
return func
I then have a series of decorated functions:
#register
def func1():
print('running f1')
#register
def func2():
print('running f2')
This works, after running the script, print(registry) returns:
[<function func1 at 0x0000000008433950>, <function func2 at 0x0000000008F06AE8>]
However calling the functions individually, for example:
func1()
Returns only 'running f1': just the function, without the decoration.
I was expecting it to return something like 'running register( func1) \n running func1'.
So my question is, when you have a decorated function and call it; when will it call the function in isolation and when will it call the decorated function?
Thanks very much.
Your register (decorator) function is only run once when the code is interpreted.
If you want to alter the function behaviour, try something like:
def register(func):
registry.append(func)
print('adding register(%s)' % func)
def wrap(*args, **kwargs):
print('running register(%s)' % func)
return func(*args, **kwargs)
return wrap
The first print is done once, the second one before each call.
Adding the arguments will make your decorator more transparent.
What we call a "decorator" is just a higher order function, and the #decorator syntax nothing more than syntactic sugar so this:
#decorate
def func():
pass
is strictly equivalent to
def func():
pass
func = decorate(func)
As mentionned by Guillaume Deslandes, if this code is at your module or script's top-level, the decorator is only invoked when the module or script is first loaded by the runtime.
In your case, the decorator function register returns it's argument (the function it's applied to) unchanged, so calling the "decorated" function will work exactly as if it never had been decorated.
If you want to modify the decorated function in any way (by executing code before and or after the original function or whatever), you have to return a new function that will "replace" the original (but - usually - keeping a reference to the original function so this new "wrapper" function can still call the original), which is usually done using the fact that Python functions are closure:
def decorate(fun):
def wrapper(*args, **kw):
res = fun(*args, **kw)
print("function {} called with *{}, *{} returned {}".format(fun, args, kw, res)
return res
return wrapper
#decorate
def fun(a):
return a * 2
If I create a decorator like following:
def my_decorator(some_fun):
def wrapper():
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
return wrapper
#my_decorator
def just_some_function():
print("Wheee!")
Another decorator can be defined as:
def my_decorator(some_fun):
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
#my_decorator
def just_some_fun():
print("some fun")
Both decorators will work the same. What is the benefit of using "wrapper" function inside decorator. I didn't get the purpose.
The purpose of having a wrapper function is that a function decorator receives a function object to decorate, and it must return the decorated function.
Your 2nd version of my_decorator doesn't have an explicit return statement, so it returns None. When my_decorator is called via the # decorator syntax
before some_function() is called.
some fun
after some_function() is called.
gets printed, and then None gets assigned to the name just_some_fun. So if you add print(just_some_fun) to the end of that code it will print None.
It may be easier to understand what's going on if we get rid of the # syntactic sugar and re-write your code using normal function calling syntax:
def my_decorator(some_fun):
print("before some_function() is called.")
some_fun()
print("after some_function() is called.")
def just_some_fun():
print("some fun")
just_some_fun = my_decorator(just_some_fun)
The decorators in Python are callable objects which in the simplest case is function taking one parameter which is some function or class. The decorator should return again same type which it takes (so if it takes function it should return function). The point is in the time when the decorator is called.
When you import a Python file or just run it directly the Python interpreter goes through the content and gathering information about what classes and function are defined and if it encounter on some code (not declaration) it will execute it.
If the interpreter encounter on decorator it takes the decorated function, call the decorator and replace the decorated function with returned value from the decorator.
Let's say you have this code:
#my_decorator
def my_function()
print("My_function")
It's equivalent to this call:
def my_function()
print("My_function")
my_function = my_decorator(my_function)
If the decorator would be like this one
def my_decorator(func):
print("decorated)
return 42
then the my_function is not even a function it would be an integer (you can try print(my_function))
So when you define decorator as
def my_decorator2(some_fun):
print("before")
some_fun()
print("after")
then this decorator returns nothing (in python it means it returns None).
#my_decorator2
def decorated():
print("inside")
prints
before
inside
after
but calling decorated() will raise exception 'NoneType' object is not callable because the decorated was replaced with None.
You should always create decorator which return something useful like function or class (which is usually the "wrap" function inside). Sometimes can be useful to return from decorator something else then function/class but it usually obfuscate your code and convert it into something totally non-maintainable.
Its already explained why to use wrapper function, out of curiosity I am just giving
examples what we can do if we don't need a wrapper function.
Type 1
Return a small function which returns None or pass
def decorator_func(to_be_decorated_function):
print("Logging IN: Currrently in function")
to_be_decorated_function()
print("Logging OUT: Currrently in function")
def a(): None # or def a(): pass
return (a)
#decorator_func
def to_be_decorated_function():
print('October 16, 2000')
to_be_decorated_function()
# equivalent to
#to_be_decorated_function = decorator_func(to_be_decorated_function)
Type 2
This merely tears out the usage of decorators, and just a slight tweak.What if we don't return , as well not use callable object at all.
def decorator_func(to_be_decorated_function):
print("Logging IN: Currrently in function")
to_be_decorated_function()
print("Logging OUT: Currrently in function")
#decorator_func
def to_be_decorated_function():
print('October 16, 2000')
to_be_decorated_function # notice I'm just using the object and not callable function
# equivalent to
#decorator_func(to_be_decorated_function)
Output from both the types
Logging IN: Currrently in function
October 16, 2000
Logging OUT: Currrently in function
I am using decorator: deferred_set_context to set the function module, name and time as the context string(basically this context is needed to trace in-case there are multiple calls to register_request)
#deferred_set_context()
def register_request(self, machine_model):
def deferred_set_context(name=None):
def __outer__(func):
context_name = name
if not context_name:
context_name = func.__name__
#defer.inlineCallbacks
def __inner__(*args, **kwargs):
r = None
with base.this_context_if_no_context(base.Context('%s.%s.%s' % (func.__module__, context_name,
datetime.utcnow().strftime(
'%Y-%m-%dT%H:%M:%S.%f')))):
r = yield func(*args, **kwargs)
defer.returnValue(r)
return __inner__
return __outer__
Now I need to set pass the machine_model name(which is argument of register_request) to this decorator: register_request. Something like this:
#deferred_set_context(name=machine_model)
def register_request(self, machine_model):
How shall I do it?
Note that the following two code snippets are equivalent
# first
#decorator
def func():
pass
# second
def func():
pass
func = decorator(func)
You need to make your decorator take a function as an argument. If you want to pass name in as a variable, then it should be a parameter in the function returned by the decorator.
Assuming you don't want to change the behavior of deferred_set_context or register_request, you could make a second shim decorator that spies on the correct function argument and passes it along to your original decorator.
Something like this:
def new_shim_decorator(register_request):
def wrapper(self, machine_model):
return deferred_set_context(machine_model)(self, machine_model)
return wrapper
#new_shim_decorator
def register_request(self, machine_model):
print("my_function called with {}".format(machine_model))
This is kind of a hacky solution in my opinion since this decorator that proxies to the original decorator would need to know the arguments passed to register_request and mirror them.
I would prefer to see a keyword-only argument that get be optionally get-ed from the keyword argument dictionary in either a shim decorator or the original:
def deferred_set_context_by_kwarg(kwarg_name):
def outer(func):
def wrapper(*args, **kwargs):
return deferred_set_context(name=kwargs.get(kwarg_name))(*args, **kwargs)
return wrapper
return outer
#deferred_set_context_by_kwarg('machine_model')
def register_request(self, *, machine_model):
print("my_function called with {}".format(machine_model))
Notice the *, machine_model. That * is a Python 3 feature that requires machine_model to be specified as a keyword argument. This allows our deferred_set_context_by_kwarg decorator to accept a string representing the name of the keyword argument we want and use that name to retrieve the keyword argument value and pass to deferred_set_context upon each function call.
If you are sure your function register_request takes always the same 2 arguments then pick them inside the decorator code.