I'm using unittest and nose-parametarized, and want to apply different decorators to a test based on a condition.
I have a test and I want to skip unittest.skip the test or execute it #parameterized.expand(args)based on the arguments passed to args.
I think I need to have another decorator which applies proper decorator to the test , but now sure how.
pseudo code could be something like this :
#validate_data(args)
def test(args):
...
where #validate_data(args) is a decorator which applies unittest.skip if args ==None or #parameterized.expand(args)otherwise
Any comments/suggestions is appreciated.
A decorator can also be called as a function. #decorator is equivalent to decorator(func) and #decorator(args) to decorator(args)(func). So you could return the value of those function returns conditionally in your decorator. Here is an example below:
def parameterized_or_skip(args=None):
if args:
return parameterized.expand(args)
return unittest.skip(reason='No args')
...
#parameterized_or_skip(args)
def my_testcase(self, a, b):
pass
Related
I'm trying to understand the behavior of decorator.
I understand that a decorator has to return an object so I can understand the syntax below:
def my_deco(fonction):
print("Deco is called with parameter the function {0}".format(fonction))
return fonction
#my_deco
def hello():
print("hello !")
Deco is called with parameter the function <function salut at 0x00BA5198>
Here the decorator does not do much, but in the case I need to modify the function, I'd define a decorator like this
def my_deco(fonction):
def modified_func():
print("Warning ! calling {0}".format(fonction))
return fonction()
return modified_func
#my_deco
def hello():
print("Salut !")
The initial function behavior is modified through modified_func.This is fine
It includes the call to the initial function. This is fine
Now what I don't understand is: why do we return the result of the function? in my case the function is a simple 'print' so I don't get why I should return something
Thanks for your explanation
As it is in the comments: usually when you write a decorator, you make it so that it can be used with any possible function. And the way to do that is to return either whatever the original function returned, or transform that return value (which can also be done in the wrapper function).
In Python, all functions actually do return something. Functions without an explicit return statement return the value None. So, if your wrpper function, inside the decorator, always returns whatever the decorated function returned, it will be on the safe side: even if the decorated function had no explicit return, it will return a None that is just forwarded by your wrapper.
Now, that is not "mandatory". If you know beforehand that your decorator will only be applied to functions with no return value, you are free not to put a return statement in the wrapper function as well - it is not an incorrect syntax (but it is likely a trap for your future self).
I am learning a bit about decorators from a great tutorial on thecodeship but have found myself rather confused by one example.
First a simple example followed by an explanation is given for what a decorator is.
def p_decorate(func):
def func_wrapper(name):
return "<p>{0}</p>".format(func(name))
return func_wrapper
def get_text(name):
return "lorem ipsum, {0} dolor sit amet".format(name)
my_get_text = p_decorate(get_text)
print my_get_text("John")
Now this makes sense to me. A decorator is simply a wrapper to a function. And in this guys explanation he says a decorator is a function that takes another function as an argument, generates a new function, and returns the generated function to be used anywhere.
And now the equivalent to the above code is:
def p_decorate(func):
def func_wrapper(name):
return "<p>{0}</p>".format(func(name))
return func_wrapper
#p_decorate
def get_text(name):
return "lorem ipsum, {0} dolor sit amet".format(name)
print get_text("John")
I believe I understand the way a decorator is initialized when given no arguments. Correct me if I am wrong.
The decorator by default passes in the function get_text and because p_decorate returns a function func_wrapper, we end up with the true statement get_text = func_wrapper.
What is important to me is the first code block equivalent, because I see and understand how the decorator is behaving.
What very much confuses me is the following code:
def tags(tag_name):
def tags_decorator(func):
def func_wrapper(name):
return "<{0}>{1}</{0}>".format(tag_name, func(name))
return func_wrapper
return tags_decorator
#tags("p")
def get_text(name):
return "Hello "+name
print get_text("John")
Again, correct me if I'm wrong but this is my understanding.
The decorator accepts the tag string "p" instead of the
default function name. And in turn the function tags_decorator
assumes that the parameter that will be passed in is the function
being decorated, get_text.
It might be helpful for me to see the equivalent block of code in "non-decorator" form but I can't seem to wrap my head around what that would begin to look like. I also don't comprehend why tags_decorator and func_wrapper are both returned. What is the purpose of returning two different functions if a decorator only needs to return 1 function to wrap get_text.
As a side note, it really comes down to the following.
Can this block be simplified to something less than a set of 3 functions?
Can decorators accept more than 1 argument to simplify code?
Within limits, everything after the # is executed to produce a decorator. In your first example, what follows after the # is just a name:
#p_decorate
so Python looks up p_decorate and calls it with the decorated function as an argument:
get_text = p_decorate(get_text)
(oversimplified a bit, get_text is not initially bound to the original function, but you got the gist already).
In your second example, the decorator expression is a little more involved, it includes a call:
#tags("p")
so Python uses tags("p") to find the decorator. In the end this is what then is executed:
get_text = tags("p")(get_text)
The output of tags("p") is the decorator here! I call the tags function itself a decorator factory, it produces a decorator when called. When you call tags(), it returns tags_decorator(). That's the real decorator here.
You could instead remove the decorator function and hardcode the "p" value and use that directly:
def tags_decorator_p(func):
def func_wrapper(name):
return "<{0}>{1}</{0}>".format("p", func(name))
return func_wrapper
#tags_decorator_p
def get_text(name):
# ...
but then you'd have to create separate decorators for each possible value of the argument to tags(). That's the value of a decorator factory, you get to add parameters to the decorator and alter how your function is decorated.
A decorator factory can take any number of arguments; it is just a function you call to produce a decorator. The decorator itself can only accept one argument, the function-to-decorate.
I said, within limits at the start of my answer for a reason; the syntax for the expression following the # only allows a dotted name (foo.bar.baz, so attribute access) and a call ((arg1, arg2, keyword=arg3)). See the reference documentation. The original PEP 318 states:
The decorator statement is limited in what it can accept -- arbitrary expressions will not work. Guido preferred this because of a gut feeling [17] .
I have a decorator function my_fun(I,k) and it is applied to a function add(x,y) as such
#my_fun(4,5)
def add(x,y): return x+y
I am new to Python would like to know if I am writing the my_fun function
How can I access x,y in the add method in my_fun?
How can I access the return value of add in the decorator function?
I am a little confused on syntax and concepts any explanation would be help.
A decorator consists of the decorator function and a function wrapper (and if you want additional arguments for the decorator another outer layer of function around it):
# Takes the arguments for the decorator and makes them accessible inside
def my_fun(decorator_argument1, decorator_argument2):
# Takes the function so that it can be wrapped.
def wrapfunc(func):
# Here we are actually going to wrap the function ... finally
def wrapper(*args, **kwargs):
# Call the function with the args and kwargs
res = func(*args, **kwargs)
# return this result
return res
# Replace the decorated function with the wrapper
return wrapper
# Return the wrapper for the function wrapper :-)
return wrapfunc
In your case if you only want to use the decorator with your function you don't need to bother with the *args, **kwargs and replace it by:
def wrapper(x, y):
# Here you can do stuff with x and y, i.e. print(x)
# Call the function with x and y
res = func(x, y)
# Here you can do stuff with the result, i.e. res = res * decorator_argument1
return res
I indicated the places where you can access x and y and the result.
If you want to predefine values for x and y a custom decorator is not the best way. You could use defaults:
def add(x=4,y=5): return x+y
add() # returns 9
add(2) # returns 7
add(5, 10) # returns 15
or if you want to fix an argument you should use functools.partial
If you're passing arguments to the decorator with #my_fun(4, 5), you need three levels of nested functions to implement the decorator in the simplest way. The outer level is the "decorator factory". It returns the middle level function, the decorator. The decorator gets called with the function it's decorating as an argument and needs to return the inner most nested function, the wrapper. The wrapper function is the one that gets called by the user.
def decorator_factory(deco_arg, deco_arg2): # name this whatever you want to use with #syntax
def decorator(func):
def wrapper(func_arg, func_arg2):
# This is a closure!
# In here you can write code using the arguments from the enclosing scpoes. e.g.:
return func(func_arg*deco_arg, func_arg2*deco_arg2) # uses args from all levels
return wrapper
return decorator
The inner functions here are closures. They can see the variables in the scope surrounding the place they were defined in, even after the functions those scope belonged to have finished running.
(Note, if you want your decorator to be able to decorate many different functions, you may want the wrapper function to accept *args and **kwargs and pass them along to func. The example above only works for functions that accept exactly two arguments. A limitation like that may be perfectly reasonable for some uses, but not always.)
I'd like to modify the arguments passed to a method in a module, as opposed to replacing its return value.
I've found a way around this, but it seems like something useful and has turned into a lesson in mocking.
module.py
from third_party import ThirdPartyClass
ThirdPartyClass.do_something('foo', 'bar')
ThirdPartyClass.do_something('foo', 'baz')
tests.py
#mock.patch('module.ThirdPartyClass.do_something')
def test(do_something):
# Instead of directly overriding its return value
# I'd like to modify the arguments passed to this function.
# change return value, no matter inputs
do_something.return_value = 'foo'
# change return value, based on inputs, but have no access to the original function
do_something.side_effect = lambda x, y: y, x
# how can I wrap do_something, so that I can modify its inputs and pass it back to the original function?
# much like a decorator?
I've tried something like the following, but not only is it repetitive and ugly, it doesn't work. After some PDB introspection.. I'm wondering if it's simply due to however this third party library works, as I do see the original functions being called successfully when I drop a pdb inside the side_effect.
Either that, or some auto mocking magic I'm just not following that I'd love to learn about.
def test():
from third_party import ThirdPartyClass
original_do_something = ThirdPartyClass.do_something
with mock.patch('module.ThirdPartyClass.do_something' as mocked_do_something:
def side_effect(arg1, arg2):
return original_do_something(arg1, 'overridden')
mocked_do_something.side_effect = side_effect
# execute module.py
Any guidance is appreciated!
You may want to use parameter wraps for the mock call. (Docs for reference.) This way the original function will be called, but it will have everything from Mock interface.
So for changing parameters called to original function you may want to try it like that:
org.py:
def func(x):
print(x)
main.py:
from unittest import mock
import org
of = org.func
def wrapped(a):
of('--{}--'.format(a))
with mock.patch('org.func', wraps=wrapped):
org.func('x')
org.func.assert_called_with('x')
result:
--x--
The trick is to pass the original underlying function that you still want to access as a parameter to the function.
Eg, for race condition testing, have tempfile.mktemp return an existing pathname:
def mock_mktemp(*, orig_mktemp=tempfile.mktemp, **kwargs):
"""Ensure mktemp returns an existing pathname."""
temp = orig_mktemp(**kwargs)
open(temp, 'w').close()
return temp
Above, orig_mktemp is evaluated when the function is declared, not when it is called, so all invocations will have access to the original method of tempfile.mktemp via orig_mktemp.
I used it as follows:
#unittest.mock.patch('tempfile.mktemp', side_effect=mock_mktemp)
def test_retry_on_existing_temp_path(self, mock_mktemp):
# Simulate race condition: creation of temp path after tempfile.mktemp
...
Let's assume we have a decorator:
def decor(function):
def result():
printf('decorated')
return function()
return result
What is the difference between following code:
#decor
def my_foo():
print('my_foo')
and:
def my_foo():
print('my_foo')
my_foo = decor(my_foo)
Your last code snippet is almost the definition of a decorator. The only difference is that in the first case, the name decor is evaluated before the function definition, while in the second case it is evaluated after the function definition. This only makes a difference if executing the function definition changes what the name refers to.
Nonsensical example:
def decor(function):
def result():
printf('decorated')
return function()
return result
def plonk():
global decor
decor = lambda x: x
return None
Now
#decor
def my_foo(foo=plonk()):
print('my_foo')
is different from
def my_foo(foo=plonk()):
print('my_foo')
my_foo = decor(my_foo)
There isn't a difference. The #decorator syntax simply makes it easier to understand that a decorator is being applied. (This is an example of syntactic sugar.)
If there is a difference, is that python versions prior to Python 2.4 does not support the #decorator syntax while the explicit decorator call is supported since the stone age. Also, the #decorator syntax had to be applied at function definition and had to use the same function name, while the explicit decorator call can be applied later and can rename the decorated function.
Use the #decorator syntax unless you had a really, really, really good reason not to; which is almost never.