I want to mock out methods on any instance of some class in the production code in order to facilitate testing. Is there any library in Python which could facilitate this?
Basically, I want to do the following, but in Python (the following code is Ruby, using the Mocha library):
def test_stubbing_an_instance_method_on_all_instances_of_a_class
Product.any_instance.stubs(:name).returns('stubbed_name')
assert_equal 'stubbed_name', SomeClassThatUsesProduct.get_new_product_name
end
The important thing to note from above is that I need to mock it out on the class level, since I'm actually need to mock out methods on an instance created by the thing I'm testing.
Use Case:
I have a class QueryMaker which calls a method on an instance of RemoteAPI. I want to mock out the RemoteAPI.get_data_from_remote_server method to return some constant. How do I do this inside a test without having to put a special case within the RemoteAPI code to check for what environment it's running in.
Example of what I wanted in action:
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
def new_foo(self):
return "New foo"
A.foo = new_foo
y = B()
if y.bar() == "New foo":
print "Success!"
Needing to mock out methods when testing is very common and there are lots of tools to help you with it in Python. The danger with "monkey patching" classes like this is that if you don't undo it afterwards then the class has been modified for all other uses throughout your tests.
My library mock, which is one of the most popular Python mocking libraries, includes a helper called "patch" that helps you to safely patch methods or attributes on objects and classes during your tests.
The mock module is available from:
http://pypi.python.org/pypi/mock
The patch decorator can be used as a context manager or as a test decorator. You can either use it to patch out with functions yourself, or use it to automatically patch with Mock objects that are very configurable.
from a import A
from b import B
from mock import patch
def new_foo(self):
return "New foo"
with patch.object(A, 'foo', new_foo):
y = B()
if y.bar() == "New foo":
print "Success!"
This handles the unpatching for you automatically. You could get away without defining the mock function yourself:
from mock import patch
with patch.object(A, 'foo') as mock_foo:
mock_foo.return_value = "New Foo"
y = B()
if y.bar() == "New foo":
print "Success!"
Mock is the way to do it, alright.
It can be a bit tricky to make sure you're patching the instance method on any instances created from the class.
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test.py
from a import A
from b import B
import mock
mocked_a_class = mock.Mock()
mocked_a_instance = mocked_a_class.return_value
mocked_a_instance.foo.return_value = 'New foo'
with mock.patch('b.A', mocked_a_class): # Note b.A not a.A
y = B()
if y.bar() == "New foo":
print "Success!"
Referenced in the docs, at the para starting "To configure return values on methods of instances on the patched class..."
Easiest way is probably to use a class method. You really should use an instance method, but it's a pain to create those, whereas there's a built-in function that creates a class method. With a class method, your stub will get a reference to the class (rather than the instance) as the first argument, but since it's a stub this probably doesn't matter. So:
Product.name = classmethod(lambda cls: "stubbed_name")
Note that the signature of the lambda must match the signature of the method you're replacing. Also, of course, since Python (like Ruby) is a dynamic language, there is no guarantee that someone won't switch out your stubbed method for something else before you get your hands on the instance, though I expect you will know pretty quickly if that happens.
Edit: On further investigation, you can leave out the classmethod():
Product.name = lambda self: "stubbed_name"
I was trying to preserve the original method's behavior as closely as possible, but it looks like it's not actually necessary (and doesn't preserve the behavior as I'd hoped, anyhow).
I don't know Ruby quite well enough to tell exactly what you're trying to do, but check out the __getattr__ method. If you define it in your class, Python will call it when code tries to access any attribute of your class that isn't otherwise defined. Since you want it to be a method, it will need to create a method on the fly that it returns.
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
... def __getattr__(self, attr_name):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
'stubbed_name'
>>> p.get_number()
My number is 172
>>> p.other_method()
'stubbed_other_method'
Also note that __getattr__ needs to not use any other undefined attributes of your class, or else it will be infinitely recursive, calling __getattr__ for the attribute that doesn't exist.
... def __getattr__(self, attr_name):
... return self.x
>>> p.y
Traceback (most recent call last):
#clipped
RuntimeError: maximum recursion depth exceeded while calling a Python object
If this is something you only want to do from your test code, not the production code, then put your normal class definition in the production code file, then in the test code define the __getattr__ method (unbound), and then bind it to the class you want:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... print "My number is %d" % self.number
...
#test code
>>> def __getattr__(self, attr):
... return lambda:"stubbed_"+attr_name
...
>>> p = Product(172)
>>> p.number
172
>>> p.name()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: Product instance has no attribute 'name'
>>> Product.__getattr__ = __getattr__
>>> p.name()
'stubbed_name'
I'm not sure how this would react with a class that was already using __getattribute__ (as opposed to __getattr__, __getattribute__ is called for all attributes whether or not they exist).
If you only want to do this for specific methods that already exist, then you could do something like:
#production code
>>> class Product:
... def __init__(self, number):
... self.number = number
... def get_number(self):
... return self.number
...
>>> p = Product(172)
>>> p.get_number()
172
#test code
>>> def get_number(self):
... return "stub_get_number"
...
>>> Product.get_number = get_number
>>> p.get_number()
'stub_get_number'
Or if you really wanted to be elegant, you could create a wrapper function to make doing multiple methods easy:
#test code
>>> import functools
>>> def stubber(fn):
... return functools.wraps(fn)(lambda self:"stub_"+fn.__name__)
...
>>> Product.get_number = stubber(Product.get_number)
>>> p.get_number()
'stub_get_number'
#Orignal Class definition - path "module.Product"
class Product:
def method_A(self):
# do something
pass
def method_B(self):
self.random_attr = 1
#Test case
from module import Product
class MockedProduct(Product):
def method_B(self):
self.random_attr = 2
with mock.patch('module.Product', new=MockedProduct):
#Write test case logic here
#Now method_B function call on product class instance should return 2
#instead of 1
minimal reproducible example using pytest and monkeypatch
# a.py
class A(object):
def foo(self):
return "A's foo"
# b.py
from a import A
class B(object):
def bar(self):
x = A()
return x.foo()
# test_ab.py
import pytest
from a import A
from b import B
def new_foo(self):
return "New foo"
def test_mock_instance_method(monkeypatch):
y = B()
print(y.bar())
monkeypatch.setattr(A, 'foo', new_foo)
print(y.bar())
gives you
$ pytest -rP .
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/user/220225so
plugins: cov-3.0.0
collected 1 item
test_ab.py . [100%]
==================================== PASSES ====================================
__________________________ test_mock_instance_method ___________________________
----------------------------- Captured stdout call -----------------------------
A's foo
New foo
============================== 1 passed in 0.01s ===============================
$
Related
Please check the simple code below:
def extract_class_from_func(func_var):
# should return class of the func_var
return ...
class A:
def m(self):
print(self, "m")
func_var = A.m
# This check should succeed
assert A == extract_class_from_func(func_var)
Please help me in implementing method extract_class_from_func
You can't. In Python 2, this reference was available at A.m.im_class. But, to quote from PEP 3155 - Qualified name for classes and functions:
This possibility is gone in Python 3.
There is no longer any such thing as an "unbound method" and the function A.m is no different from a regular function - it does not hold any reference to the class object. In fact, you can even delete the class and see the "method" still works:
>>> class Ameta(type):
... def __del__(self):
... print("goodbye A")
...
>>> class A(metaclass=Ameta):
... def m(self):
... print(self, "m")
...
>>> f = A.m
>>> import gc
>>> del A
>>> gc.collect()
goodbye A
6
>>> f("👻")
👻 m
However, the PEP did provide some limited support for what you wanted: if you look in A.m.__qualname__ you will find a string from which you may be able to introspect the class A.
The module can be found using func_var.__module__ and class name can be found using __qualname__. Then you just perform string import as below:
import importlib
def extract_class_from_func(func_var):
className = func_var.__qualname__.split('.')[0]
return getattr(importlib.import_module(func_var.__module__), className)
class Greet:
def __init__(self):
pass
def sayHi(self):
print("Hi!")
>>> Greet == extract_class_from_func(Greet.sayHi)
>>> True
You can do like this.
def extract_class_from_func(func_var):
return func_var.__qualname__.split('.')[0]
I have a file copy_x_to_y.py which goes like this:
from abcd import F
def function_to_be_tested():
F()
in abcd.py file, i have something like this:
from xyz import XY
class F():
def __init__(self, arg1):
self.xy = XY(arg1)
I want to mock init of XY in my test case.
I have tried mocking F's init with:
def mock_func(*args, **kwargs):
pass
#patch('path/to/copy_x_to_y.F.__init__', mock_func)
def test():
assert function_to_be_tested() is None
but it always happens to call XY's init, resulting in error as its initialization calls
to connect with S3 with arg1. How to test this kind of structure?
What is the reason for wanting to mock __init__ of XY? Do you want it to return a specific object of XY, do you want to check if XY.__init__ was called with specific arguments or something else?
A possible solution to your problem would be to mock the entire class, but have it return a "normal" object. Here's an example:
>>> from unittest.mock import patch
>>> class MyClass:
... def __init__(self, val):
... self._val = val
... def foo(self):
... print(self._val)
...
>>> a = MyClass(1)
>>> a.foo()
1
>>> patcher = patch('__main__.MyClass', return_value=a)
>>> mock_class = patcher.start()
>>> b = MyClass(2) # This will return a.
>>> b.foo()
1
>>> mock_class.call_args_list
[call(2)]
>>> patcher.stop()
Which in your case would be:
from xyz import XY
from path/to/copy_x_to_y import function_to_be_tested
def test():
arg1 = ...
a = XY(arg1) # Has to be called before the patch to get a "normal" object.
with patch('xyz.XY', return_value=a) as mock_xy:
# Run funcion to be tested here and check results.
function_to_be_tested()
assert ...
Some side notes:
It is possible to mock __init__ directly though, if that's really what you need to do.
>>> def my_init(self, *args, **kwargs):
... self._val = 1
>>> patcher = patch.object(MyClass, '__init__', my_init)
>>> mock_init = patcher.start()
>>> a = MyClass(2)
>>> a.foo()
1
If you use the patch decorator, you have to supply the decorated function with one extra argument that is the mock of the class or object.
https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch
#patch('path/to/SomeClass', ...)
def test(mock_class):
...
Also patch is typically (exclusively?) used to patch a class while patch.object is used to patch a member inside a class or module. https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch.object
I know how fierce the SO community is so I'll try my best to keep the question minimal, complete and verifiable.
What I simply want to know is can monkey patching be used to replace the definition of an existing function?
for example:
class A():
def foo():
print '2'
def foo():
print '5'
A.foo = foo
This way doesn't seem to work also as to why I don't just add a new function instead of replacing an existing one, I call these functions in other classes and it is my understanding that monkey patching adds those functions at run-time and I need my python code to run on an Apache spark server which throws an error deeming the calls to that function unreferenced.
So please be nice and help me out or suggest a work around.
Thanks.
Edit: The goal of the code is to print 5 when A.foo is called.
Your only problem is that you aren't defining foo correctly in the first place. It needs to take an explicit argument for the instance calling it.
class A(object):
def __init__(self)
self.x = 2
def foo(self):
print(self.x)
def foo(this):
print(this.x + 3)
A.foo = foo
a = A()
a.foo() # outputs 5 in Python 2 and Python 3
In a very real sense, monkey patching is how classes are created in the first place. A class statement is almost just syntactic sugar for the following code:
def foo(self):
print(self.x)
A = type('A', (object,), {'foo': foo})
del foo
It's not too much of a simplification to image the definition of type being something like
def type(name, bases, d):
new_class = magic_function_to_make_a_class()
new_class.name = name
new_class.bases = bases
for k, v in d.items():
setattr(new_class, k, v)
return new_class
I hope I understand what you are trying to do here. This would work in Python 3:
class A():
def foo():
print('2')
def foo():
A.foo = lambda: print('5')
A.foo() # Print '2'
foo() # Assign the new method
A.foo() # Prints '5'
In Python 2 however there are several caveats.
print is not a method as it is in Python 3 (see here: Why doesn't print work in a lambda?)
You can't just call unbound messages (see here: In Python, can you call an instance method of class A, but pass in an instance of class B?)
So you have to do it like this:
from __future__ import print_function
class A():
def foo():
print('2')
def foo():
A.foo = lambda: print('5')
A.foo.__func__() # Print '2'
foo() # Assign the new method
A.foo.__func__() # Prints '5'
Edit:
After seeing your question in the comment I think you actually want something different. Which is this:
class A():
def foo(self):
print '2'
def foo(self):
print '5'
a = A()
a.foo() # Print '2'
A.foo = foo # Assign the new method
a.foo() # Prints '5'
This works just fine in Python 2.
The self is a reference to the current instance the method is bound to. It is not used when you just call something like print which access any properties or methods attached to that instance. But for a different case please have a look at the following example:
class A():
msg = "Some message"
def foo(self):
print self.msg
def bar(self):
self.msg = "Some other message"
a = A()
a.foo() # Print old msg
A.bar = bar # Assign the new method
a.bar() # Assigns new message
a.foo() # Prints new message
Also as chepner points out in a comment under his post:
The name self isn't special; it's just a convention. You could use
this if you really wanted to, and it doesn't need to be the same name
in both functions defined here. What is important is that the first
argument to a function used as an instance method will be a reference
to the calling object. a.foo() is almost exactly the same as A.foo(a)
How can I get the class that defined a method in Python?
I'd want the following example to print "__main__.FooClass":
class FooClass:
def foo_method(self):
print "foo"
class BarClass(FooClass):
pass
bar = BarClass()
print get_class_that_defined_method(bar.foo_method)
import inspect
def get_class_that_defined_method(meth):
for cls in inspect.getmro(meth.im_class):
if meth.__name__ in cls.__dict__:
return cls
return None
I don't know why no one has ever brought this up or why the top answer has 50 upvotes when it is slow as hell, but you can also do the following:
def get_class_that_defined_method(meth):
return meth.im_class.__name__
For python 3 I believe this changed and you'll need to look into .__qualname__.
In Python 3, if you need the actual class object you can do:
import sys
f = Foo.my_function
vars(sys.modules[f.__module__])[f.__qualname__.split('.')[0]] # Gets Foo object
If the function could belong to a nested class you would need to iterate as follows:
f = Foo.Bar.my_function
vals = vars(sys.modules[f.__module__])
for attr in f.__qualname__.split('.')[:-1]:
vals = vals[attr]
# vals is now the class Foo.Bar
Thanks Sr2222 for pointing out I was missing the point...
Here's the corrected approach which is just like Alex's but does not require to import anything. I don't think it's an improvement though, unless there's a huge hierarchy of inherited classes as this approach stops as soon as the defining class is found, instead of returning the whole inheritance as getmro does. As said, this is a very unlikely scenario.
def get_class_that_defined_method(method):
method_name = method.__name__
if method.__self__:
classes = [method.__self__.__class__]
else:
#unbound method
classes = [method.im_class]
while classes:
c = classes.pop()
if method_name in c.__dict__:
return c
else:
classes = list(c.__bases__) + classes
return None
And the Example:
>>> class A(object):
... def test(self): pass
>>> class B(A): pass
>>> class C(B): pass
>>> class D(A):
... def test(self): print 1
>>> class E(D,C): pass
>>> get_class_that_defined_method(A().test)
<class '__main__.A'>
>>> get_class_that_defined_method(A.test)
<class '__main__.A'>
>>> get_class_that_defined_method(B.test)
<class '__main__.A'>
>>> get_class_that_defined_method(C.test)
<class '__main__.A'>
>>> get_class_that_defined_method(D.test)
<class '__main__.D'>
>>> get_class_that_defined_method(E().test)
<class '__main__.D'>
>>> get_class_that_defined_method(E.test)
<class '__main__.D'>
>>> E().test()
1
Alex solution returns the same results. As long as Alex approach can be used, I would use it instead of this one.
Python 3
Solved it in a very simple way:
str(bar.foo_method).split(" ", 3)[-2]
This gives
'FooClass.foo_method'
Split on the dot to get the class and the function name separately
I found __qualname__ is useful in Python3.
I test it like that:
class Cls(object):
def func(self):
print('1')
c = Cls()
print(c.func.__qualname__)
# output is: 'Cls.func'
def single_func():
print(2)
print(single_func.__module__)
# output: '__main__'
print(single_func.__qualname__)
# output: 'single_func'
After my test, I found another answer here.
I started doing something somewhat similar, basically the idea was checking whenever a method in a base class had been implemented or not in a sub class. Turned out the way I originally did it I could not detect when an intermediate class was actually implementing the method.
My workaround for it was quite simple actually; setting a method attribute and testing its presence later. Here's an simplification of the whole thing:
class A():
def method(self):
pass
method._orig = None # This attribute will be gone once the method is implemented
def run_method(self, *args, **kwargs):
if hasattr(self.method, '_orig'):
raise Exception('method not implemented')
self.method(*args, **kwargs)
class B(A):
pass
class C(B):
def method(self):
pass
class D(C):
pass
B().run_method() # ==> Raises Exception: method not implemented
C().run_method() # OK
D().run_method() # OK
UPDATE: Actually call method() from run_method() (isn't that the spirit?) and have it pass all arguments unmodified to the method.
P.S.: This answer does not directly answer the question. IMHO there are two reasons one would want to know which class defined a method; first is to point fingers at a class in debug code (such as in exception handling), and the second is to determine if the method has been re-implemented (where method is a stub meant to be implemented by the programmer). This answer solves that second case in a different way.
if you get this error:
'function' object has no attribute 'im_class'
try this:
import inspect
def get_class_that_defined_method(meth):
class_func_defided = meth.__globals__[meth.__qualname__.split('.')[0]]
#full_func_name = "%s.%s.%s"%(class_func_defided.__module__,class_func_defided.__name__,meth.__name__)
if inspect.isfunction(class_func_defided):
print("%s is not part of a class."%meth.__name__)
return None
return class_func_defided
sample test:
class ExampleClass:
#staticmethod
def ex_static_method():
print("hello from static method")
def ex_instance_method(self):
print("hello from instance method")
def ex_funct(self):
print("hello from simple function")
if __name__ == "__main__":
static_method_class = get_class_that_defined_method(ExampleClass.ex_static_method)
static_method_class.ex_static_method()
instance_method_class = get_class_that_defined_method(ExampleClass.ex_instance_method)
instance_method_class().ex_instance_method()
function_class = get_class_that_defined_method(ex_funct)
Let's say, I have a function named my_function and I'm passing an object as its parameter in the following way:
my_obj = MyClass()
my_function(my_obj)
Is there a way using which I can print the name of the function inside a method of MyClass?
Let's say that MyClass has a method called display_info inside it and it prints the name of the function in where the object of MyClass is passed.
An example would be:
class MyClass:
def print_log(self):
# some code
def random_method_1():
pass
def random_method_2():
def my_function(param1, some_name, some_number):
# some code
# Instantiating MyClass
my_object = MyClass()
# Calling my_function
my_function(my_object, "John Doe", 38478347)
my_object.print_log()
# Above line of code should be able to print "my_function"
Design your program better
If you need this systematically you might want a better designed program, e.g. one that injects the name of the parent or calling entity into e.g. my_obj.display_info(caller='something')
class MyClass:
def display_info(self, caller):
print caller
def my_function(my_obj):
my_obj.display_info(caller='my_function')
The above allows that param to be injected by each caller.
Or keep a reference to the caller in MyClass.caller at init time if possible, or set it on the spot as needed e.g.
# pseudo code
class MyClass:
def __init__(self, caller):
self.caller = caller
def display_info(self):
print self.caller
my_obj = MyClass(caller='caller')
my_obj.display_info()
my_obj.caller = 'new_caller'
my_obj.display_info()
The inspect hack to avoid
The inspect module provides several useful functions to help get
information about live objects such as modules, classes, methods,
functions, tracebacks, frame objects, and code objects.
Yes usng inspect. When you call inspect.stack() you get a record of frame objects. So we use stack()[1] to get the previous frame, which is a tuple where the third element is the name of the caller (my_function). Hence stack()[1][3].
>>> import inspect
>>> class MyClass():
... def display_info(self):
... print inspect.stack()[1][3]
...
>>> def my_function(my_obj):
... my_obj.display_info()
...
>>> my_obj = MyClass()
>>> my_function(my_obj)
my_function # <--- The result you wanted
>>>
>>> def another_fn(my_obj):
... my_obj.display_info()
...
>>> another_fn(my_obj)
another_fn
This provides the result you want (prints the name of the calling function)