I have a utility method that behaves like this
def my_patch_method(self):
pass
def patch_my_lib():
from mylib import MyClass
MyClass.target_method = my_patch_method
return MyClass()
This test fails:
self.assertEqual(my_patch_method, patch_my_lib().target_method)
Whereas this one works:
self.assertEqual(my_patch_method.__name__, patch_my_lib().target_method.__name__)
As the patch method does not have the same name, this is still acceptable proof that patch_my_lib() is doing what it's payed for but why doesn't the first work as I would expect ? And, is there a way to "fix" it ?
The reason your first test fails is that once you monkey-patch the function into your class, it isn't really the same object any more.
>>> def foo(self): pass
...
>>> class Foo: pass
...
>>> Foo.bar = foo
>>> type(Foo.bar)
<type 'instancemethod'>
>>> type(foo)
<type 'function'>
>>>
>>> Foo.bar is foo
False
>>> Foo.bar == foo
False
In fact, the original function and the new method have different types. Instead, have your first test check this condition:
>>> Foo.bar.im_func is foo
True
So maybe this: self.assertIs(my_patch_method, patch_my_lib().target_method.im_func)
Try:
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
You are returning an instance from patch_my_lib, so comparing the function to the bound method
Something like this should pass
self.assertEqual(my_patch_method, patch_my_lib().target_method.im_func)
But it's probably better to check that the behaviour you are patching is working
MyClass.target_method = my_patch_method sets your function as a class function for MyClass, but you return a instance of that class with return MyClass().
Related
I tried to write simple class factory using the "type" method.
I called myclass_factory twice and It returned identical namespaces
(case1 and case2). But the values of myclass attribute in that namespaces were different (!).
Actualy it just what I need but I can not undestnad why I got that result.
For my udestanding, since case1 and case2 are not objects but just same namespace (<class 'main.MyClass'>) they should refer to the same memory and should be case1.myclass = case2.myclass
Please explain how It could be so?
>>> def myclass_factory(myclass):
... return type('MyClass',(object,),{'myclass': myclass})
...
>>> class class1:
... pass
...
>>> class class2:
... pass
...
>>> case1 = myclass_factory(class1)
>>> case2 = myclass_factory(class2)
>>>
>>> case1
<class '__main__.MyClass'>
>>> case2
<class '__main__.MyClass'>
>>>
>>> case1.myclass
<class '__main__.class1'>
>>> case2.myclass
<class '__main__.class2'>
>>>
>>>
Actually case 1 and case 2 are objects since you instantiate them. They are type objects and thus ordinary python objects. So you create two different type objects.
also see this answer: Link to answer
Also see This description
This is essentially equivalent to the following "normal" code for defining classes:
def MyClass(object):
myclass = class1
case1 = MyClass
def MyClass(object):
myclass = class2
case2 = MyClass
print(case1.myclass)
print(case2.myclass)
Types/classes are first-class objects, and creating a new type with the same name has no effect on the previous class one with that name that was saved in the variable case1.
I have a unit test where the setup mocks a client like so:
def setUp(self):
self.mock_client = mock.patch.object(module_name, 'ClientClassName', autospec=True).start()
Then in my test I have a faked return value:
def myTest(self):
self.mock_client.my_method.return_value = ...
Now I want to get the arguments that my_method was called with, however I've been tearing my hair out trying to access them. It seems that I can't just do:
mock_args, mock_kwargs = self.mock_client.my_method.call_args
This gives me back First off why doesn't this work? I did make a little headway and found that:
self.mock_client.method_calls[0]
will give me back a call object that looks like call().my_method(...the arguments), but I have been trying for hours to get access to the individual arguments and cant seem to do it. Where am I going wrong?
Call args are just accessed with subscription on the mock.call object, i.e. __getitem__.
>>> from unittest.mock import MagicMock
>>> m = MagicMock()
>>> m(123, xyz="hello")
<MagicMock name='mock()' id='140736989479888'>
>>> m("another call")
<MagicMock name='mock()' id='140736989479888'>
>>> m.call_args_list
[call(123, xyz='hello'), call('another call')]
>>> m.call_args_list[0][0]
(123,)
>>> m.call_args_list[0][1]
{'xyz': 'hello'}
Item 0 will be a tuple of args, and item 1 will be a dict of kwargs. Attribute access also works, like a namedtuple (item 0 is attribute "args", and item 1 is attribute "kwargs"). If you only need to access the most recent call, you can use call_args instead of call_args_list.
Note that accessing the call args items directly is usually not required, you can use an assertion against another call instance in the tests:
>>> from unittest.mock import call
>>> m(k=123)
<MagicMock name='mock()' id='140736989479888'>
>>> assert m.call_args == call(k=123) # pass
>>> assert m.call_args == call(k=124) # fail
AssertionError
Or an even higher level, you can use m.assert_has_calls on the mock directly.
When mocking methods, whether the mock calls have self or not can be influenced by autospec:
>>> from unittest import mock
>>> class A(object):
... def f(self, *args, **kwargs):
... pass
...
>>> with mock.patch("__main__.A.f") as m:
... a = A()
... a.f('without autospec', n=1)
...
>>> m.call_args
call('without autospec', n=1)
>>> with mock.patch("__main__.A.f", autospec=True) as m:
... a = A()
... a.f('with autospec', n=2)
...
>>> m.call_args
call(<__main__.A object at 0x7fffe3d4e6a0>, 'with autospec', n=2)
This is discussed in more detail in the docs here.
In addition to wim's answer, you can actually dig down fairly deeply into this arguments, although sometimes you seem to find a string instead of a real object.
The main point to understand is that when you iterate through my_mock.call_args_list you get objects of type unittest.mock._Call. These can indeed be compared to call(...) objects which you have created yourself. But that's not all.
unittest.mock._Call is itself iterable, and consists of 2 elements: one is a tuple, the other is a dict. These are none other than the *args and **kwargs passed to the mock method.
Given that this is, as far as I can make out, completely undocumented in the docs, I suppose it is not beyond the bounds of possibility that this could break one day. It does often prove handy to know though, in my experience.
So disclaimer: this question has piqued my curiosity a bit, and I'm asking this for purely educational purposes. More of a challenge for the Python gurus here I suppose!
Is it possible to make the output of type(foo) return a different value than the actual instance class? i.e. can it pose as an imposter and pass a check such as type(Foo()) is Bar?
#juanpa.arrivillaga made a suggestion of manually re-assigning __class__ on the instance, but that has the effect of changing how all other methods would be called. e.g.
class Foo:
def test(self):
return 1
class Bar:
def test(self):
return 2
foo = Foo()
foo.__class__ = Bar
print(type(foo) is Bar)
print(foo.test())
>>> True
>>> 2
The desired outputs would be True, 1. i.e The class returned in type is different than the instance, and the instance methods defined in the real class still get invoked.
No - the __class__ attribute is a fundamental information on the layout of all Python objects as "seen" on the C API level itself. And that is what is checked by the call to type.
That means: every Python object have a slot in its in-memory layout with space for a single pointer, to the Python object that is that object's class.
Even if you use ctypes or other means to override protection to that slot and change it from Python code (since modifying obj.__class__ with = is guarded at the C level), changing it effectively changes the object type: the value in the __class__ slot IS the object's class, and the test method would be picked from the class in there (Bar) in your example.
However there is more information here: in all documentation, type(obj) is regarded as equivalent as obj.__class__ - however, if the objects'class defines a descriptor with the name __class__, it is used when one uses the form obj.__class__. type(obj) however will check the instance's __class__ slot directly and return the true class.
So, this can "lie" to code using obj.__class__, but not type(obj):
class Bar:
def test(self):
return 2
class Foo:
def test(self):
return 1
#property
def __class__(self):
return Bar
Property on the metaclass
Trying to mess with creating a __class__ descriptor on the metaclass of Foo itself will be messy -- both type(Foo()) and repr(Foo()) will report an instance of Bar, but the "real" object class will be Foo. In a sense, yes, it makes type(Foo()) lie, but not in the way you were thinking about - type(Foo()) will output the repr of Bar(), but it is Foo's repr that is messed up, due to implementation details inside type.__call__:
In [73]: class M(type):
...: #property
...: def __class__(cls):
...: return Bar
...:
In [74]: class Foo(metaclass=M):
...: def test(self):
...: return 1
...:
In [75]: type(Foo())
Out[75]: <__main__.Bar at 0x55665b000578>
In [76]: type(Foo()) is Bar
Out[76]: False
In [77]: type(Foo()) is Foo
Out[77]: True
In [78]: Foo
Out[78]: <__main__.Bar at 0x55665b000578>
In [79]: Foo().test()
Out[79]: 1
In [80]: Bar().test()
Out[80]: 2
In [81]: type(Foo())().test()
Out[81]: 1
Modifying type itself
Since no one "imports" type from anywhere, and just use
the built-in type itself, it is possible to monkeypatch the builtin
type callable to report a false class - and it will work for all
Python code in the same process relying on the call to type:
original_type = __builtins__["type"] if isinstance("__builtins__", dict) else __builtins__.type
def type(obj_or_name, bases=None, attrs=None, **kwargs):
if bases is not None:
return original_type(obj_or_name, bases, attrs, **kwargs)
if hasattr(obj_or_name, "__fakeclass__"):
return getattr(obj_or_name, "__fakeclass__")
return original_type(obj_or_name)
if isinstance(__builtins__, dict):
__builtins__["type"] = type
else:
__builtins__.type = type
del type
There is one trick here I had not find in the docs: when acessing __builtins__ in a program, it works as a dictionary. However, in an interactive environment such as Python's Repl or Ipython, it is a
module - retrieving the original type and writting the modified
version to __builtins__ have to take that into account - the code above
works both ways.
And testing this (I imported the snippet above from a .py file on disk):
>>> class Bar:
... def test(self):
... return 2
...
>>> class Foo:
... def test(self):
... return 1
... __fakeclass__ = Bar
...
>>> type(Foo())
<class '__main__.Bar'>
>>>
>>> Foo().__class__
<class '__main__.Foo'>
>>> Foo().test()
1
Although this works for demonstration purposes, replacing the built-in type caused "dissonances" that proved fatal in a more complex environment such as IPython: Ipython will crash and terminate immediately if the snippet above is run.
I need to patch current datetime in tests. I am using this solution:
def _utcnow():
return datetime.datetime.utcnow()
def utcnow():
"""A proxy which can be patched in tests.
"""
# another level of indirection, because some modules import utcnow
return _utcnow()
Then in my tests I do something like:
with mock.patch('***.utils._utcnow', return_value=***):
...
But today an idea came to me, that I could make the implementation simpler by patching __call__ of function utcnow instead of having an additional _utcnow.
This does not work for me:
from ***.utils import utcnow
with mock.patch.object(utcnow, '__call__', return_value=***):
...
How to do this elegantly?
When you patch __call__ of a function, you are setting the __call__ attribute of that instance. Python actually calls the __call__ method defined on the class.
For example:
>>> class A(object):
... def __call__(self):
... print 'a'
...
>>> a = A()
>>> a()
a
>>> def b(): print 'b'
...
>>> b()
b
>>> a.__call__ = b
>>> a()
a
>>> a.__call__ = b.__call__
>>> a()
a
Assigning anything to a.__call__ is pointless.
However:
>>> A.__call__ = b.__call__
>>> a()
b
TLDR;
a() does not call a.__call__. It calls type(a).__call__(a).
Links
There is a good explanation of why that happens in answer to "Why type(x).__enter__(x) instead of x.__enter__() in Python standard contextlib?".
This behaviour is documented in Python documentation on Special method lookup.
[EDIT]
Maybe the most interesting part of this question is Why I cannot patch somefunction.__call__?
Because the function don't use __call__'s code but __call__ (a method-wrapper object) use function's code.
I don't find any well sourced documentation about that, but I can prove it (Python2.7):
>>> def f():
... return "f"
...
>>> def g():
... return "g"
...
>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>
>>> g
<function g at 0x7f15763817d0>
>>> g.__call__
<method-wrapper '__call__' of function object at 0x7f15763817d0>
Replace f's code by g's code:
>>> f.func_code = g.func_code
>>> f()
'g'
>>> f.__call__()
'g'
Of course f and f.__call__ references are not changed:
>>> f
<function f at 0x7f1576381848>
>>> f.__call__
<method-wrapper '__call__' of function object at 0x7f1576381848>
Recover original implementation and copy __call__ references instead:
>>> def f():
... return "f"
...
>>> f()
'f'
>>> f.__call__ = g.__call__
>>> f()
'f'
>>> f.__call__()
'g'
This don't have any effect on f function. Note: In Python 3 you should use __code__ instead of func_code.
I Hope that somebody can point me to the documentation that explain this behavior.
You have a way to work around that: in utils you can define
class Utcnow(object):
def __call__(self):
return datetime.datetime.utcnow()
utcnow = Utcnow()
And now your patch can work like a charm.
Follow the original answer that I consider even the best way to implement your tests.
I've my own gold rule: never patch protected methods. In this case the things are little bit smoother because protected method was introduced just for testing but I cannot see why.
The real problem here is that you cannot to patch datetime.datetime.utcnow directly (is C extension as you wrote in the comment above). What you can do is to patch datetime by wrap the standard behavior and override utcnow function:
>>> with mock.patch("datetime.datetime", mock.Mock(wraps=datetime.datetime, utcnow=mock.Mock(return_value=3))):
... print(datetime.datetime.utcnow())
...
3
Ok that is not really clear and neat but you can introduce your own function like
def mock_utcnow(return_value):
return mock.Mock(wraps=datetime.datetime,
utcnow=mock.Mock(return_value=return_value)):
and now
mock.patch("datetime.datetime", mock_utcnow(***))
do exactly what you need without any other layer and for every kind of import.
Another solution can be import datetime in utils and to patch ***.utils.datetime; that can give you some freedom to change datetime reference implementation without change your tests (in this case take care to change mock_utcnow() wraps argument too).
As commented on the question, since datetime.datetime is written in C, Mock can't replace attributes on the class (see Mocking datetime.today by Ned Batchelder). Instead you can use freezegun.
$ pip install freezegun
Here's an example:
import datetime
from freezegun import freeze_time
def my_now():
return datetime.datetime.utcnow()
#freeze_time('2000-01-01 12:00:01')
def test_freezegun():
assert my_now() == datetime.datetime(2000, 1, 1, 12, 00, 1)
As you mention, an alternative is to track each module importing datetime and patch them. This is in essence what freezegun does. It takes an object mocking datetime, iterates through sys.modules to find where datetime has been imported and replaces every instance. I guess it's arguable whether you can do this elegantly in one function.
Is it possible in python to check if an object is a class object. IE if you have
class Foo(object):
pass
How could you check if o is Foo (or some other class) or an instance of Foo (or any other class instance)? In Java this would be a simple matter. Just check if the object is an instance of Class. Is there something similar in python or are you supposed to just not care?
Slight clarification: I'm trying to make a function that prints information about the parameter its given. So if you pass in o, where o = Foo() it prints out information about Foo. If you pass in Foo it should print out the exact same information. Not information about Type.
Use the isinstance builtin function.
>>> o = Foo()
>>> isinstance(o, Foo)
True
>>> isinstance(13, Foo)
False
This also works for subclasses:
>>> class Bar(Foo): pass
>>> b = Bar()
>>> isinstance(b, Foo)
True
>>> isinstance(b, Bar)
True
Yes, normally, you are supposed to not particularly care what type the object is. Instead, you just call the method you want on o so that people can plug in arbitrary objects that conform to your interface. This wouldn't be possible if you were to aggressively check the types of objects that you're using. This principle is called duck typing, and allows you a bit more freedom in how you choose to write your code.
Python is pragmatic though, so feel free to use isinstance if it makes sense for your particular program.
Edit:
To check if some variable is a class vs an instance, you can do this:
>>> isinstance(Foo, type) # returns true if the variable is a type.
True
>>> isinstance(o, type)
False
My end goal is to make a function that prints out information about an object if its an instance and print something different if its a class. So this time I do care.
First, understand that classes are instances — they're instances of type:
>>> class Foo(object):
... pass
...
>>> isinstance(Foo, type)
True
So, you can pick out classes that way, but keep in mind that classes are instances too. (And thus, you can pass classes to functions, return them from functions store them in lists, create the on the fly…)
the isinstance() function
isinstance(o, Foo)
and you can also use it to compare o to object
In [18]: class Foo(object): pass
In [20]: o_instance = Foo()
In [21]: o_class = Foo
In [22]: isinstance(o_instance, Foo)
Out[22]: True
In [23]: isinstance(o_class, Foo)
Out[23]: False
In [24]: isinstance(o_instance, object)
Out[24]: True
In [25]: isinstance(o_class, object)
Out[25]: True
I had to do like Thanatos said and check
isinstance(Foo, type)
But in the case of old class types you have to also do
isinstance(Foo, types.ClassType)