I'm trying to mock a property on only a specific instance of a class. To simplify the example, I have a Thing which gets initialised with a name, and based on this name the class will load a configuration file in a pre-defined location /conf/{name}_conf.yaml.
When testing, a couple of instances of Thing are created and I just want to override the configuration for one of these.
I commented below the initial code that I had to change to make it work:
class Thing():
def __init__(self, name):
self.name = name
# I wasn't able to mock this:
# self.configuration_name = f'/configuration/{self.name}_configuration.yaml'
# #property <- nor this
def configuration_filename(self):
return f'/configuration/{self.name}_configuration.yaml'
And in my tests, the mock should be able to take as parameter a different configuration file (specific to the test), but only be applied to the instance of Thing named test_thing.
I got it working with the above implementation like this:
configuration_filename_orig = Thing.configuration_filename
def my_patched_configuration_filename(self, configuration_filename, *args, **kwargs):
if self.slug == 'cmstest':
return configuration_filename
else:
return configuration_filename_orig(self, *args, **kwargs)
Then I can "inject" a custom test configuration file for each test class like this:
from functools import partial
from test.utils import my_patched_configuration_filename
...
#patch.object(Thing, 'configuration_filename', autospec=True, side_effect=partial(my_patched_configuration_filename, configuration_filename='testdata/test_1.yaml'))
class ConfigurationTests(TestCase):
def test_1(self, mocked_conf):
# test something
def test_2(self, mocked_conf):
# test something else
#patch.object(Thing, 'configuration_filename', autospec=True, side_effect=partial(my_patched_configuration_filename, configuration_filename='testdata/test_2.yaml'))
class ConfigurationTestsAdvanced(TestCase):
def test_1(self, mocked_conf):
# test something
def test_2(self, mocked_conf):
# test something else
Now... this works but I wonder if there's a way to do something similar but with a real property on the Thing class (either with the #property decorator or with the property initialised in the the __init__ method).
I've spent a couple of hours trying different things... but the main issue seems that using return_value doesn't pass the self argument to the mock, so I can't use it.
Any idea ?
ok there might be a better way but I got this working as follow:
I can use the #property decorator on my class, that's what I want to mock:
class Thing():
def __init__(self, name):
self.name = name
#property
def configuration_filename(self):
return f'/configuration/{self.name}_configuration.yaml'
I create a new mock class, based on Mock:
# first copy original
configuration_filename_orig = Thing.configuration_filename.__get__
class ConfigurationPropertyMock(Mock):
# and here, add the `self` to the args
def __get__(self, obj, obj_type=None, *args, **kwargs):
return self(obj, obj_type, *args, **kwargs)
def patched_filename(self, *args, **kwargs):
configuration_filename = kwargs.pop('configuration_filename')
if self.slug == 'cmstest' and configuration_filename:
return configuration_filename
else:
return configuration_filename_orig(self, *args, **kwargs)
And I patch the test class where I can pass a custom configuration_filename:
from unittest.mock import patch
from tests.utils import ConfigurationPropertyMock, patched_filename
...
#patch('somewhere.Thing.configuration_filename',
new_callable=ConfigurationPropertyMock,
side_effect=partial(patched_filename, configuration_filename='test_conf.yaml'))
)
class YAMLApiConfigurationTests(TestCase):
def test_api_configuration_document(self, mocked_conf):
# test here, the test configuration is loaded
pass
Voilà :)
Related
I have a class MyClass with a complex __init__ function.
This class had a method my_method(self) which I would like to test.
my_method only needs attribute my_attribute from the class instance.
Is there a way I can mock class instances without calling __init__ and by setting the attributes of each class instance instead?
What I have:
# my_class.py
from utils import do_something
class MyClass(object):
def __init__(self, *args, **kwargs):
# complicated function which I would like to bypass when initiating a mocked instance class
pass
def my_method(self):
return do_something(self.my_attribute)
What I tried
#mock.patch("my_class.MyClass")
def test_my_method(class_mock, attribute):
instance = class_mock.return_value
instance.my_attribute = attribute
example_instance = my_class.MyClass()
out_my_method = example_instance.my_method()
# then perform some assertions on `out_my_method`
however this still makes usage of __init__ which I hope we can by-pass or mock.
As I mentioned in the comments, one way to test a single method without having to create an instance is:
MyClass.my_method(any_object_with_my_attribute)
The problem with this, as with both options in quamrana's answer, is that we have now expanded the scope of any future change just because of the tests. If a change to my_method requires access to an additional attribute, we now have to change both the implementation and something else (the SuperClass, the MockMyClass, or in this case any_object_with_my_attribute_and_another_one).
Let's have a more concrete example:
import json
class MyClass:
def __init__(self, filename):
with open(filename) as f:
data = json.load(f)
self.foo = data.foo
self.bar = data.bar
self.baz = data.baz
def my_method(self):
return self.foo ** 2
Here any test that requires an instance of MyClass. is painful because of the file access in __init__. A more testable implementation would split apart the detail of how the data is accessed and the initialisation of a valid instance:
class MyClass:
def __init__(self, foo, bar, baz):
self.foo = foo
self.bar = bar
self.baz = baz
def my_method(self):
return self.foo ** 2
#classmethod
def from_json(cls, filename):
with open(filename) as f:
data = json.load(f)
return cls(data.foo, data.bar, data.baz)
You have to refactor MyClass("path/to/file") to MyClass.from_json("path/to/file"), but wherever you already have the data (e.g. in your tests) you can use e.g. MyClass(1, 2, 3) to create the instance without requiring a file (you only need to consider the file in the tests of from_json itself). This makes it clearer what the instance actually needs, and allows the introduction of other ways to construct an instance without changing the interface.
There are at least two options I can see:
Extract a super class:
class SuperClass:
def __init__(self, attribute):
self.my_attribute = attribute
def my_method(self):
return do_something(self.my_attribute)
class MyClass(SuperClass):
def __init__(self, *args, **kwargs):
super().__init__(attribute) # I don't know where attribute comes from
# complicated function which I would like to bypass when initiating a mocked instance class
Your tests can instantiate SuperClass and call my_method().
Inherit from MyClass as is and make your own simple __init__():
class MockMyClass(MyClass):
def __init__(self, attribute):
self.my_attribute = attribute
Now your test code can instantiate MockMyClass with the required attribute and call my_method()
For instance, you could write the test as follows
def test_my_method(attribute):
class MockMyClass(MyClass):
def __init__(self, attribute):
self.my_attribute = attribute
out_my_method = MockMyClass(attribute).my_method()
# perform assertions on out_my_method
I am trying to add a decorator to required class methods and I have come up with the following code for it. I need this to work with all the similar classes.
import allure
def class_method_dec(cls):
"""
Decorator function to decorate required class methods.
"""
if cls.method1:
cls.method1= allure.step(cls.method1)
if cls.method2:
cls.method2= allure.step(cls.method2)
if cls.method3:
cls.method3= allure.step(cls.method3)
return cls
#class_method_dec
class TestClass:
def __init__(self, a, b):
self.a = a
self.b = b
def method1(self):
"""
method docstring
"""
pass
def method2(self):
"""
method docstring
"""
pass
def method3(self):
"""
method docstring
"""
pass
Is this the right way to do it? I am looking for the best way to do this.
Also, I understand that we can use functools.wraps to preserve the docstring when decorating functions. Is there a need of something like it when we are decorating classes?
From Satwik Kansal’s brilliant Metaprogramming in Python IBM tutorial , I discovered this gem:
Satwik first defined a decorator:
from functools import wraps
import random
import time
def wait_random(min_wait=1, max_wait=30):
def inner_function(func):
#wraps(func)
def wrapper(args, **kwargs):
time.sleep(random.randint(min_wait, max_wait))
return func(args, **kwargs)
return wrapper
return inner_function
And then he created a class wrapper that will apply this decorator to a class:
def classwrapper(cls):
for name, val in vars(cls).items():
#callable return True if the argument is callable
#i.e. implements the __call
if callable(val):
#instead of val, wrap it with our decorator.
setattr(cls, name, wait_random()(val))
return cls
Application:
# decorate a function
#wait_random(10, 15)
def function_to_scrape():
#some scraping stuff
# decorate a class
#classwrapper
class Scraper:
# some scraping stuff
To make use of it in your case, substitute wait_random decorator with your own. Turn your function to a decorator.
E.g
from functools import wraps
import allure
def apply_allure():
def inner_function(func):
#wraps(func)
def wrapper(args, **kwargs):
func = allure.step(func)
return func(args, **kwargs)
return wrapper
return inner_function
In the classwrapper replace wait_random with apply_allure:
Do read the tutorial for more information and explanations
I’m trying to create a decorator that is called within a class, which would pull attributes from that class, and use those class attributes to edit the function’s docstring.
My problem is that I have found examples of decorators that edit the docstring of the function (setting the function's __doc__ attribute equal to a new string), and I have also found examples of decorators that pull attributes from the parent class (by passing self into the decorator), but I haven’t been able to find an example of a decorator that is able to do both.
I have tried to combine these two examples, but it isn't working:
def my_decorator(func):
def wrapper(self, *args, **kwargs):
name = func.__name__ # pull function name
cls = self.__class__.__name__ # pull class name
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), name, cls) # set them to docstring
return func(self, *args, **kwargs)
return wrapper
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
With this, I would hope that the following would return "Docstring is now new for the function: example":
Test().example.__doc__
Instead it returns None.
Edit: Note that I am not interested in how to access the name of the class specifically, so much as how to access the class attributes in general (where here self.__class__.__name__ is used as an example).
example is replaced with wrapper; the decoration is equivalent to
def example(self, examplearg=1):
"""Docstring"""
pass
example = my_decorator(example)
so you need to set wrapper.__doc__, not func.__doc__.
def my_decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for the function {}".format(
str(func.__doc__),
func.__name__)
return wrapper
Note that at the time you call my_decorator, you don't have any information about what class the decorated function/method belongs to. You would have to pass its name explicitly:
def my_decorator(cls_name):
def _decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for function {} in class {}".format(
func.__doc__,
func.__name__,
cls_name)
return wrapper
return _decorator
class Test():
#my_decorator("Test")
def example(self, examplearg=1):
"""Docstring"""
# or
# def example(self, examplearg=1):
# """Docstring"""
#
# example = my_decorator("Test")(example)
You can simply modify the __doc__ attribute when the decorator is called instead, and use the first token of the dot-delimited __qualname__ attribute of the function to obtain the class name:
def my_decorator(func):
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), func.__name__, func.__qualname__.split('.')[0])
return func
so that:
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
print(Test().example.__doc__)
would output:
Docstring is new for the function example in class Test
Turns out that accessing class attributes from within a class is impossible, as the class has yet to be executed when the decorator is called. So the original goal - using a decorator within a class to access class attributes - does not seem to be possible.
However, thanks to jdehesa for pointing me to a workaround that allows access to the class attributes using a class decorator, here: Can a Python decorator of an instance method access the class?.
I was able to use the class decorator to alter the specific method's docstring using class attributes like so:
def class_decorator(cls):
for name, method in cls.__dict__.items():
if name == 'example':
# do something with the method
method.__doc__ = "{} is new for function {} in class {}".format(method.__doc__, name, cls.__name__)
# Note that other class attributes such as cls.__base__
# can also be accessed in this way
return cls
#class_decorator
class Test():
def example(self, examplearg=1):
"""Docstring"""
print(Test().example.__doc__)
# Returns "Docstring is new for function example in class Test"
I am implementing the classes in Python 2.7 as below:
class MyClass(object):
def do_something(self):
"""
do something here
"""
class MyClassManager(object):
def get_my_class_obj(self):
"""read my_class_obj from db"""
return instance # instance has type MyClass
class MyClassUser(object):
my_class_obj = new MyClass() # this is a class variable
In main:
MyClassUser.my_class_obj = MyClassManager().get_my_class_obj()
"""
do a lot of different things else in main
"""
From somewhere else:
"""only when some condition happens"""
MyClassUser.my_class_obj.do_something()
Is there a way I can defer the read obj (read from db inside get_my_class_obj) process in MyClassManager until obj.do_something method is actually invoked? Provided that I have to call MyClassManager.get_my_class_obj for some setup at the beginning. Suppose the situation is in the context of a web server and do_something will only be invoked when there is some request but I need to set up it first
A very quick&dirty, dumbed down possible solution (and certainly not how I'd design this but anyway). This example assume your object only has an id (db primary key) and a name (stored in db).
import functools
class MyClassManager(object):
def get(self, object_id):
return MyClass(object_id)
def load(self, object):
obj.name = self.get_from_db(object.id)
def autoload(func):
#functools.wraps(func)
def wrapper(self, *args, **kw):
self._load()
return func(self, *args, **kw)
return wrapper
class MyClass(object):
def __init__(self, id, name=None):
self.id = id
self._name = name
self._loaded = name is not None
def _load(self):
if self._loaded:
return
MyManager.load(self)
#property
#autoload
def name(self):
return self._name
#autoload
def do_something(self):
# your code here
But really, don't do this if you're a beginner (and you wouldn't ask if you were not), use an existing working ORM instead. Unless it's for educational purpose of course, in which case, well, you might either learn a lot or give up in despair - or both ;-)
I'm attempting to use sphinx-doc :automodule: in conjunction with Mock-ed out modules as per this answer. Specifically I'm using Mock for PyQt5 module imports which are not available on ReadTheDocs.
Strangely, I'm finding that any class that inherits from a Mock-ed module's class is not included in the resulting documentation. It appears as though sphinx-doc can't see them for some reason.
My slightly-custom Mock is as follows:
from mock import Mock as MagicMock
class Mock(MagicMock):
__all__ = ['QApplication','pyqtSignal','pyqtSlot','QObject','QAbstractItemModel','QModelIndex','QTabWidget',
'QWebPage','QTableView','QWebView','QAbstractTableModel','Qt','QWidget','QPushButton','QDoubleSpinBox',
'QListWidget','QDialog','QSize','QTableWidget','QMainWindow','QTreeWidget',
'QAbstractItemDelegate','QColor','QGraphicsItemGroup','QGraphicsItem','QGraphicsPathItem',
'QGraphicsTextItem','QGraphicsRectItem','QGraphicsScene','QGraphicsView',]
def __init__(self, *args, **kwargs):
super(Mock, self).__init__()
#classmethod
def __getattr__(cls, name):
if name in ('__file__', '__path__'):
return os.devnull
else:
return Mock
#classmethod
def __setattr__(*args, **kwargs):
pass
def __setitem__(self, *args, **kwargs):
return
def __getitem__(self, *args, **kwargs):
return Mock
The __all__ is required to allow from x import * style imports for the PyQt5 classes.
I can confirm that changing the superclass to object results in the classes being correctly documented, as does remove the Mock (generating locally). Forcing the documentation by using :autoclass: results in a single line saying that the class inherits from Mock.
I resolved this in the end by not using Mock for Qt objects. In my application there is a qt.py wrapper file that handles differences between PyQt4 and PyQt5 and allows them to be subsequently imported for use (while ignoring the Qt namespace rearrangement).
In this file I wrapped the actual import code in a test for ReadTheDocs and then if detected returned a series of dummy classes inheriting directly from object. Additions were required where objects have attributes, but this is only used once in the code base. It'll need to be kept up to date, but it solves the problem.
# ReadTheDocs
ON_RTD = os.environ.get('READTHEDOCS', None) == 'True'
if not ON_RTD:
#... do the normal import here ...
else:
class QMockObject(object):
def __init__(self, *args, **kwargs):
super(QMockObject, self).__init__()
def __call__(self, *args, **kwargs):
return None
class QApplication(QMockObject):
pass
class pyqtSignal(QMockObject):
pass
class pyqtSlot(QMockObject):
pass
class QObject(QMockObject):
pass
class QAbstractItemModel(QMockObject):
pass
class QModelIndex(QMockObject):
pass
class QTabWidget(QMockObject):
pass
class QWebPage(QMockObject):
pass
class QTableView(QMockObject):
pass
class QWebView(QMockObject):
pass
class QAbstractTableModel(QMockObject):
pass
class Qt(QMockObject):
DisplayRole = None
class QWidget(QMockObject):
pass
class QPushButton(QMockObject):
pass
class QDoubleSpinBox(QMockObject):
pass
class QListWidget(QMockObject):
pass
class QDialog(QMockObject):
pass
class QSize(QMockObject):
pass
class QTableWidget(QMockObject):
pass
class QMainWindow(QMockObject):
pass
class QTreeWidget(QMockObject):
pass
class QAbstractItemDelegate(QMockObject):
pass
class QColor(QMockObject):
pass
class QGraphicsItemGroup(QMockObject):
pass
class QGraphicsItem(QMockObject):
pass
class QGraphicsPathItem(QMockObject):
pass
class QGraphicsTextItem(QMockObject):
pass
class QGraphicsRectItem(QMockObject):
pass
class QGraphicsScene(QMockObject):
pass
class QGraphicsView(QMockObject):
pass
app = None