I need to patch current datetime in tests. I am using this solution:
def _utcnow():
return datetime.datetime.utcnow()
def utcnow():
"""A proxy which can be patched in tests.
"""
# another level of indirection, because some modules import utcnow
return _utcnow()
Then in my tests I do something like:
with mock.patch('***.utils._utcnow', return_value=***):
...
But today an idea came to me, that I could make the implementation simpler by patching __call__ of function utcnow instead of having an additional _utcnow.
This does not work for me:
from ***.utils import utcnow
with mock.patch.object(utcnow, '__call__', return_value=***):
...
How to do this elegantly?
When you patch __call__ of a function, you are setting the __call__ attribute of that instance. Python actually calls the __call__ method defined on the class.
For example:
>>> class A(object):
... def __call__(self):
... print 'a'
...
>>> a = A()
>>> a()
a
>>> def b(): print 'b'
...
>>> b()
b
>>> a.__call__ = b
>>> a()
a
>>> a.__call__ = b.__call__
>>> a()
a
Assigning anything to a.__call__ is pointless.
However:
>>> A.__call__ = b.__call__
>>> a()
b
TLDR;
a() does not call a.__call__. It calls type(a).__call__(a).
Links
There is a good explanation of why that happens in answer to "Why type(x).__enter__(x) instead of x.__enter__() in Python standard contextlib?".
This behaviour is documented in Python documentation on Special method lookup.
[EDIT]
Maybe the most interesting part of this question is Why I cannot patch somefunction.__call__?
Because the function don't use __call__'s code but __call__ (a method-wrapper object) use function's code.
I don't find any well sourced documentation about that, but I can prove it (Python2.7):
>>> def f():
... return "f"
...
>>> def g():
... return "g"
...
>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>
>>> g
<function g at 0x7f15763817d0>
>>> g.__call__
<method-wrapper '__call__' of function object at 0x7f15763817d0>
Replace f's code by g's code:
>>> f.func_code = g.func_code
>>> f()
'g'
>>> f.__call__()
'g'
Of course f and f.__call__ references are not changed:
>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>
Recover original implementation and copy __call__ references instead:
>>> def f():
... return "f"
...
>>> f()
'f'
>>> f.__call__ = g.__call__
>>> f()
'f'
>>> f.__call__()
'g'
This don't have any effect on f function. Note: In Python 3 you should use __code__ instead of func_code.
I Hope that somebody can point me to the documentation that explain this behavior.
You have a way to work around that: in utils you can define
class Utcnow(object):
def __call__(self):
return datetime.datetime.utcnow()
utcnow = Utcnow()
And now your patch can work like a charm.
Follow the original answer that I consider even the best way to implement your tests.
I've my own gold rule: never patch protected methods. In this case the things are little bit smoother because protected method was introduced just for testing but I cannot see why.
The real problem here is that you cannot to patch datetime.datetime.utcnow directly (is C extension as you wrote in the comment above). What you can do is to patch datetime by wrap the standard behavior and override utcnow function:
>>> with mock.patch("datetime.datetime", mock.Mock(wraps=datetime.datetime, utcnow=mock.Mock(return_value=3))):
... print(datetime.datetime.utcnow())
...
3
Ok that is not really clear and neat but you can introduce your own function like
def mock_utcnow(return_value):
return mock.Mock(wraps=datetime.datetime,
utcnow=mock.Mock(return_value=return_value)):
and now
mock.patch("datetime.datetime", mock_utcnow(***))
do exactly what you need without any other layer and for every kind of import.
Another solution can be import datetime in utils and to patch ***.utils.datetime; that can give you some freedom to change datetime reference implementation without change your tests (in this case take care to change mock_utcnow() wraps argument too).
As commented on the question, since datetime.datetime is written in C, Mock can't replace attributes on the class (see Mocking datetime.today by Ned Batchelder). Instead you can use freezegun.
$ pip install freezegun
Here's an example:
import datetime
from freezegun import freeze_time
def my_now():
return datetime.datetime.utcnow()
#freeze_time('2000-01-01 12:00:01')
def test_freezegun():
assert my_now() == datetime.datetime(2000, 1, 1, 12, 00, 1)
As you mention, an alternative is to track each module importing datetime and patch them. This is in essence what freezegun does. It takes an object mocking datetime, iterates through sys.modules to find where datetime has been imported and replaces every instance. I guess it's arguable whether you can do this elegantly in one function.
Related
I have a unit test where the setup mocks a client like so:
def setUp(self):
self.mock_client = mock.patch.object(module_name, 'ClientClassName', autospec=True).start()
Then in my test I have a faked return value:
def myTest(self):
self.mock_client.my_method.return_value = ...
Now I want to get the arguments that my_method was called with, however I've been tearing my hair out trying to access them. It seems that I can't just do:
mock_args, mock_kwargs = self.mock_client.my_method.call_args
This gives me back First off why doesn't this work? I did make a little headway and found that:
self.mock_client.method_calls[0]
will give me back a call object that looks like call().my_method(...the arguments), but I have been trying for hours to get access to the individual arguments and cant seem to do it. Where am I going wrong?
Call args are just accessed with subscription on the mock.call object, i.e. __getitem__.
>>> from unittest.mock import MagicMock
>>> m = MagicMock()
>>> m(123, xyz="hello")
<MagicMock name='mock()' id='140736989479888'>
>>> m("another call")
<MagicMock name='mock()' id='140736989479888'>
>>> m.call_args_list
[call(123, xyz='hello'), call('another call')]
>>> m.call_args_list[0][0]
(123,)
>>> m.call_args_list[0][1]
{'xyz': 'hello'}
Item 0 will be a tuple of args, and item 1 will be a dict of kwargs. Attribute access also works, like a namedtuple (item 0 is attribute "args", and item 1 is attribute "kwargs"). If you only need to access the most recent call, you can use call_args instead of call_args_list.
Note that accessing the call args items directly is usually not required, you can use an assertion against another call instance in the tests:
>>> from unittest.mock import call
>>> m(k=123)
<MagicMock name='mock()' id='140736989479888'>
>>> assert m.call_args == call(k=123) # pass
>>> assert m.call_args == call(k=124) # fail
AssertionError
Or an even higher level, you can use m.assert_has_calls on the mock directly.
When mocking methods, whether the mock calls have self or not can be influenced by autospec:
>>> from unittest import mock
>>> class A(object):
... def f(self, *args, **kwargs):
... pass
...
>>> with mock.patch("__main__.A.f") as m:
... a = A()
... a.f('without autospec', n=1)
...
>>> m.call_args
call('without autospec', n=1)
>>> with mock.patch("__main__.A.f", autospec=True) as m:
... a = A()
... a.f('with autospec', n=2)
...
>>> m.call_args
call(<__main__.A object at 0x7fffe3d4e6a0>, 'with autospec', n=2)
This is discussed in more detail in the docs here.
In addition to wim's answer, you can actually dig down fairly deeply into this arguments, although sometimes you seem to find a string instead of a real object.
The main point to understand is that when you iterate through my_mock.call_args_list you get objects of type unittest.mock._Call. These can indeed be compared to call(...) objects which you have created yourself. But that's not all.
unittest.mock._Call is itself iterable, and consists of 2 elements: one is a tuple, the other is a dict. These are none other than the *args and **kwargs passed to the mock method.
Given that this is, as far as I can make out, completely undocumented in the docs, I suppose it is not beyond the bounds of possibility that this could break one day. It does often prove handy to know though, in my experience.
Is it possible to do something like the following:
class foo():
def bar(): # a method that doesn't take any args
# slow calculation
return somefloat
b = bar # bar is a function but b just gives you the float attribute
f = foo()
f.b # returns somefloat but doesn't require the empty parentheses
I hope the example is clear since I'm not super clear on what the terminology is for what I want to do. My basic goal is to remove a bunch of parentheses for methods that don't have arguments to make the code cleaner to read.
The function is slow and rarely used so it would be easiest to calculate it real time rather than calculate it once ahead of time and store the variable.
Is this possible? Is it good practice? Is there a better way?
The standard way to achieve this is to use property, which is a decorator:
class Foo():
#property
def bar(self):
# slow calculation
return somefloat
f = Foo()
f.bar # returns somefloat but doesn't require the empty parentheses
A couple of things to notice:
You still need self in the method signature as usual, because sometimes you're going to need to refer to e.g. self.some_attribute inside the method. As you can see, that doesn't affect the use of the property at all.
There's no need to clutter your API with both a f.bar() method and a f.b property - it's better to decide what makes most sense for your class than offer a heap of different ways to do the same thing.
b = bar obviously wouldn't work. However a property would for the simplest "doesn't require the empty parentheses" ask of yours:
b = property(bar)
Now every access to f.b will call f.bar() "behind the curtains".
However this means that if you access f.b twice, f.bar() gets called twice, repeating the computation. If the repetition is irrelevant (i.e if the result doesn't change for repeated computations on the same object) you can do better ("caching" the result in f.b forever once it's first been computed) -- something like:
class foo(object):
def bar(self): # a method that doesn't take any args
# slow calculation
return somefloat
def _cache_bar(self):
result = self.bar()
setattr(self, 'b', result)
return result
b = property(_cache_bar)
By static method, but need to call by parentheses.
class foo(object):
#staticmethod
def bar(): # a method that doesn't take any args
# slow calculation
return "abc"
b = bar # bar is a function but b just gives you the float attribute
f = foo()
print f.b()
output:
$ python test.py
abc
Can I somehow call a function without the ()? Maybe abusing the magic methods such as __call__() somehow?
I'd like to be able to something similar to
from IPython import embed as qq
but call embed() only via qq rather than qq()
This is more out of curiosity, and as a learning exercise for python, rather than practical purposes.
If you are using the REPL (the Python shell), then you can hack your way around this, because the REPL will call repr() on objects for you (which in turn invokes their __repr__ method):
from IPython import embed
class WrappedFunctionCall(object):
def __init__(self, fn):
self.fn = fn
def __repr__(self):
self.fn()
return "" # `__repr__` must return a string
qq = WrappedFunctionCall(embed)
# Typing "qq" will invoke embed now and load iPython.
But really, you should not be doing this!
And of course, it won't work outside of the REPL, because there won't be anything to call __repr__ in that case. Obviously, passing arguments isn't "supported" either.
__call__ will be invoked only if the function is invoked with (). If the function is in a class, then you can use #property decorator, to do something like this
import math
class Circle(object):
def __init__(self, radius):
self.radius = radius
#property
def area(self):
return math.pi * (self.radius ** 2)
print(Circle(5).area)
# 78.53981633974483
Read more about getter and setter here
If you want to learn, play around with Python.
In [1]: def foo():
...: pass
...:
In [2]: foo
Out[2]: <function __main__.foo>
In [3]: foo()
In [4]: bar = foo
In [5]: bar
Out[5]: <function __main__.foo>
In [6]: bar()
As you see, foo will not call the function, it will return it. And that is a good thing becaus you can pass it as an argument and assign it, for example bar = foo.
In pure Python, the only way I can think of is to use an object and a property:
>>> class Wtf(object):
... #property
... def yadda(self):
... print "Yadda"
...
>>> w = Wtf()
>>> w.yadda
Yadda
>>>
Else you might want to check IPython's doc on how to define your own custom "magic" commands: http://ipython.org/ipython-doc/dev/config/custommagics.html
You can call the function foo without using () (on that function):
def call_function(fun_name,*args):
return fun_name(*args)
def foo(a,b):
return a+b
print call_function(foo,1,2)
# Prints 3
Note that this answer isn't entirely serious, but it does contain a snippet of interesting Python code.
I am learning a tutorial on python.It is explaining how functions are first class objects in Python.
def foo():
pass
print(foo.__class__)
print(issubclass(foo.__class__,object))
The output that I get for the above code is
<type 'function'>
True
This program is supposed to demonstrate that functions are first class objects in python? My questions are as follows.
How does the above code prove that functions are fist class objects?
What are the attributes of a first class object?
what does function.__class__ signify? It returns a tuple <type,function> which doesn't mean much?
Here's what Guido says about first class objects in his blog:
One of my goals for Python was to make it so that all objects were "first class." By this, I meant that I wanted all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. That is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth.
The whole blog post is worth reading.
In the example you posted, the tutorial may be making the point that first class objects are generally descendents of the "object" class.
First-class simply means that functions can be treated as a value -- that is you can assign them to variables, return them from functions, as well as pass them in as a parameter. That is you can do code like:
>>> def say_hi():
print "hi"
>>> def say_bye():
print "bye"
>>> f = say_hi
>>> f()
hi
>>> f = say_bye
>>> f()
bye
This is useful as you can now assign functions to variables like any ordinary variable:
>>> for f in (say_hi, say_bye):
f()
hi
bye
Or write higher order functions (that take functions as parameters):
>>> def call_func_n_times(f, n):
for i in range(n):
f()
>>> call_func_n_times(say_hi, 3)
hi
hi
hi
>>> call_func_n_times(say_bye, 2)
bye
bye
About __class__ in python tells what type of object you have. E.g., if you define an list object in python: a = [1,2,3], then a.__class__ will be <type 'list'>. If you have a datetime (from datetime import datetime and then d = datetime.now(), then the type of d instance will be <type 'datetime.datetime'>. They were just showing that in python a function is not a brand new concept. It's just an ordinary object of <type 'function'>.
You proved that functions are first class objects because you were allowed to pass foo as an argument to a method.
The attributes of first class objects was nicely summarised in this post: https://stackoverflow.com/a/245208/3248346
Depending on the language, this can
imply:
being expressible as an anonymous literal value
being storable in variables
being storable in data structures
having an intrinsic identity (independent of any given name)
being comparable for equality with other entities
being passable as a parameter to a procedure/function
being returnable as the result of a procedure/function
being constructible at runtime
being printable
being readable
being transmissible among distributed processes
being storable outside running processes
Regarding your third question, <type 'function'> isn't a tuple. Python's tuple notation is (a,b), not angle brackets.
foo.__class__ returns a class object, that is, an object which represents the class to which foo belongs; class objects happen to produce descriptive strings in the interpreter, in this case telling you that the class of foo is the type called 'function'. (Classes and types are basically the same in modern Python.)
It doesn't mean a whole lot other than that, like any other object, functions have a type:
>>> x = 1
>>> x.__class__
<type 'int'>
>>> y = "bar"
>>> y.__class__
<type 'str'>
>>> def foo(): pass
...
>>> foo.__class__
<type 'function'>
Regarding your comment to #I.K.s answer, f_at_2() in the following would be the method.
def f_at_2(f):
return f(2)
def foo(n):
return n ** n
def bar(n):
return n * n
def baz(n):
return n / 2
funcs = [foo, bar, baz]
for f in funcs:
print f.func_name, f_at_2(f)
...
>>>
foo 4
bar 4
baz 1
>>>
A method is a function of/in a class, but the concept also applies to a function (outside of a class). The functions (as objects) are contained in a data structure and passed to another object.
I have a utility method that behaves like this
def my_patch_method(self):
pass
def patch_my_lib():
from mylib import MyClass
MyClass.target_method = my_patch_method
return MyClass()
This test fails:
self.assertEqual(my_patch_method, patch_my_lib().target_method)
Whereas this one works:
self.assertEqual(my_patch_method.__name__, patch_my_lib().target_method.__name__)
As the patch method does not have the same name, this is still acceptable proof that patch_my_lib() is doing what it's payed for but why doesn't the first work as I would expect ? And, is there a way to "fix" it ?
The reason your first test fails is that once you monkey-patch the function into your class, it isn't really the same object any more.
>>> def foo(self): pass
...
>>> class Foo: pass
...
>>> Foo.bar = foo
>>> type(Foo.bar)
<type 'instancemethod'>
>>> type(foo)
<type 'function'>
>>>
>>> Foo.bar is foo
False
>>> Foo.bar == foo
False
In fact, the original function and the new method have different types. Instead, have your first test check this condition:
>>> Foo.bar.im_func is foo
True
So maybe this: self.assertIs(my_patch_method, patch_my_lib().target_method.im_func)
Try:
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
You are returning an instance from patch_my_lib, so comparing the function to the bound method
Something like this should pass
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
But it's probably better to check that the behaviour you are patching is working
MyClass.target_method = my_patch_method sets your function as a class function for MyClass, but you return a instance of that class with return MyClass().