Develop Reference

How to mock a #property method with "self" as parameter?

I'm trying to mock a property on only a specific instance of a class. To simplify the example, I have a Thing which gets initialised with a name, and based on this name the class will load a configuration file in a pre-defined location /conf/{name}_conf.yaml.
When testing, a couple of instances of Thing are created and I just want to override the configuration for one of these.
I commented below the initial code that I had to change to make it work:
class Thing():
def __init__(self, name):
self.name = name
# I wasn't able to mock this:
# self.configuration_name = f'/configuration/{self.name}_configuration.yaml'
# #property <- nor this
def configuration_filename(self):
return f'/configuration/{self.name}_configuration.yaml'
And in my tests, the mock should be able to take as parameter a different configuration file (specific to the test), but only be applied to the instance of Thing named test_thing.
I got it working with the above implementation like this:
configuration_filename_orig = Thing.configuration_filename
def my_patched_configuration_filename(self, configuration_filename, *args, **kwargs):
if self.slug == 'cmstest':
return configuration_filename
else:
return configuration_filename_orig(self, *args, **kwargs)
Then I can "inject" a custom test configuration file for each test class like this:
from functools import partial
from test.utils import my_patched_configuration_filename
...
#patch.object(Thing, 'configuration_filename', autospec=True, side_effect=partial(my_patched_configuration_filename, configuration_filename='testdata/test_1.yaml'))
class ConfigurationTests(TestCase):
def test_1(self, mocked_conf):
# test something
def test_2(self, mocked_conf):
# test something else
#patch.object(Thing, 'configuration_filename', autospec=True, side_effect=partial(my_patched_configuration_filename, configuration_filename='testdata/test_2.yaml'))
class ConfigurationTestsAdvanced(TestCase):
def test_1(self, mocked_conf):
# test something
def test_2(self, mocked_conf):
# test something else
Now... this works but I wonder if there's a way to do something similar but with a real property on the Thing class (either with the #property decorator or with the property initialised in the the __init__ method).
I've spent a couple of hours trying different things... but the main issue seems that using return_value doesn't pass the self argument to the mock, so I can't use it.
Any idea ?

ok there might be a better way but I got this working as follow:
I can use the #property decorator on my class, that's what I want to mock:
class Thing():
def __init__(self, name):
self.name = name
#property
def configuration_filename(self):
return f'/configuration/{self.name}_configuration.yaml'
I create a new mock class, based on Mock:
# first copy original
configuration_filename_orig = Thing.configuration_filename.__get__
class ConfigurationPropertyMock(Mock):
# and here, add the `self` to the args
def __get__(self, obj, obj_type=None, *args, **kwargs):
return self(obj, obj_type, *args, **kwargs)
def patched_filename(self, *args, **kwargs):
configuration_filename = kwargs.pop('configuration_filename')
if self.slug == 'cmstest' and configuration_filename:
return configuration_filename
else:
return configuration_filename_orig(self, *args, **kwargs)
And I patch the test class where I can pass a custom configuration_filename:
from unittest.mock import patch
from tests.utils import ConfigurationPropertyMock, patched_filename
...
#patch('somewhere.Thing.configuration_filename',
new_callable=ConfigurationPropertyMock,
side_effect=partial(patched_filename, configuration_filename='test_conf.yaml'))
)
class YAMLApiConfigurationTests(TestCase):
def test_api_configuration_document(self, mocked_conf):
# test here, the test configuration is loaded
pass
Voilà :)

Using class attributes to modify a docstring with a decorator in Python

I’m trying to create a decorator that is called within a class, which would pull attributes from that class, and use those class attributes to edit the function’s docstring.
My problem is that I have found examples of decorators that edit the docstring of the function (setting the function's __doc__ attribute equal to a new string), and I have also found examples of decorators that pull attributes from the parent class (by passing self into the decorator), but I haven’t been able to find an example of a decorator that is able to do both.
I have tried to combine these two examples, but it isn't working:
def my_decorator(func):
def wrapper(self, *args, **kwargs):
name = func.__name__ # pull function name
cls = self.__class__.__name__ # pull class name
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), name, cls) # set them to docstring
return func(self, *args, **kwargs)
return wrapper
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
With this, I would hope that the following would return "Docstring is now new for the function: example":
Test().example.__doc__
Instead it returns None.
Edit: Note that I am not interested in how to access the name of the class specifically, so much as how to access the class attributes in general (where here self.__class__.__name__ is used as an example).

example is replaced with wrapper; the decoration is equivalent to
def example(self, examplearg=1):
"""Docstring"""
pass
example = my_decorator(example)
so you need to set wrapper.__doc__, not func.__doc__.
def my_decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for the function {}".format(
str(func.__doc__),
func.__name__)
return wrapper
Note that at the time you call my_decorator, you don't have any information about what class the decorated function/method belongs to. You would have to pass its name explicitly:
def my_decorator(cls_name):
def _decorator(func):
def wrapper(self, *args, **kwargs):
return func(self, *args, **kwargs)
wrapper.__doc__ = "{} is new for function {} in class {}".format(
func.__doc__,
func.__name__,
cls_name)
return wrapper
return _decorator
class Test():
#my_decorator("Test")
def example(self, examplearg=1):
"""Docstring"""
# or
# def example(self, examplearg=1):
# """Docstring"""
#
# example = my_decorator("Test")(example)

You can simply modify the __doc__ attribute when the decorator is called instead, and use the first token of the dot-delimited __qualname__ attribute of the function to obtain the class name:
def my_decorator(func):
func.__doc__ = "{} is new for the function {} in class {}".format(
str(func.__doc__), func.__name__, func.__qualname__.split('.')[0])
return func
so that:
class Test():
#my_decorator
def example(self, examplearg=1):
"""Docstring"""
pass
print(Test().example.__doc__)
would output:
Docstring is new for the function example in class Test

Turns out that accessing class attributes from within a class is impossible, as the class has yet to be executed when the decorator is called. So the original goal - using a decorator within a class to access class attributes - does not seem to be possible.
However, thanks to jdehesa for pointing me to a workaround that allows access to the class attributes using a class decorator, here: Can a Python decorator of an instance method access the class?.
I was able to use the class decorator to alter the specific method's docstring using class attributes like so:
def class_decorator(cls):
for name, method in cls.__dict__.items():
if name == 'example':
# do something with the method
method.__doc__ = "{} is new for function {} in class {}".format(method.__doc__, name, cls.__name__)
# Note that other class attributes such as cls.__base__
# can also be accessed in this way
return cls
#class_decorator
class Test():
def example(self, examplearg=1):
"""Docstring"""
print(Test().example.__doc__)
# Returns "Docstring is new for function example in class Test"

Lazy class property decorator

I have one django model which needs to do some processing referring the custom user model.
I can't work with the class of this model at class loading time because the loading order of the classes is unknown.
So I need to add some class attributes at runtime, at the moment I'm adding them in the __init__ or __new__ like:
def __new__(cls, *args, **kwargs):
# hack to avoid INSTALLED_APPS initialization conflicts.
# get_user_model() can't be called from this module at class loading time,
# so some class attributes must be added later.
# Metaclasses could me more appropiate but I don't want to override
# dango's metaclasses.
if not hasattr(cls, '_reverse_field_name_to_user'):
cls._find_reverse_field_name_to_user()
return Group.__new__(cls, *args, **kwargs)
It works but looks horrible so I've thought about using something like #lazyclassproperty for these attributes.
I've found several #classproperty and #lazyproperty decorators but not one for both and I have no idea how to write one myself.
Question: How could I code such decorator? or suggest another cleaner alternative to my current silly implementation.

Pyramid framework has a very nice decorator called reify, but it only works at instance level, and you want class level, so let's modify it a bit
class class_reify(object):
def __init__(self, wrapped):
self.wrapped = wrapped
try:
self.__doc__ = wrapped.__doc__
except: # pragma: no cover
pass
# original sets the attributes on the instance
# def __get__(self, inst, objtype=None):
# if inst is None:
# return self
# val = self.wrapped(inst)
# setattr(inst, self.wrapped.__name__, val)
# return val
# ignore the instance, and just set them on the class
# if called on a class, inst is None and objtype is the class
# if called on an instance, inst is the instance, and objtype
# the class
def __get__(self, inst, objtype=None):
# ask the value from the wrapped object, giving it
# our class
val = self.wrapped(objtype)
# and set the attribute directly to the class, thereby
# avoiding the descriptor to be called multiple times
setattr(objtype, self.wrapped.__name__, val)
# and return the calculated value
return val
class Test(object):
#class_reify
def foo(cls):
print("foo called for class", cls)
return 42
print(Test.foo)
print(Test.foo)
Run the program and it prints
foo called for class <class '__main__.Test'>
42
42

Trying to program a "to_class" general decorator

Suppose I have defined:
def to_class(cls):
""" returns a decorator
aimed to force the result to be of class cls. """
def decorating_func(func):
def wrapper(*args, **kwargs):
return cls(func(*args, **kwargs))
return wrapper
return decorator(decorating_func)
I wish to use it to create decorators which turn function results to objects of a given class. However, this will not work:
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#staticmethod
#to_test_class
def test_func(value):
return value
to_test_class = to_class(TestClass)
as test_func will look for to_test_class and will not find it. On the other hand, putting the assignment to to_test_class before the class definition will fail as well, as TestClass will not be defined yet.
Trying to put #to_class(TestClass) above the definition of test_func will also fail, as the method is constructed before the class (if I am not wrong).
The only workaround I have found is to define to_test_class manually as a decorator, and not as one returned from the general "to_class" def.
It might be important to mention that this is only a basic example, but I wish to use to_class for many applications, such as modifying the returned value before 'plugging' it into the class' constructor; and I wish to use it as a decorator for other class' methods as well.
I am sure some think a "to_class" decorator is pointless; manipulations can be done within the decorated method, instead. Though, I find it convenient, and it helps me with readability.
Finally I wish to add that this interests me 20% for practical reasons and 80% for studying reasons, as I find this is something I do not fully understand about decorators in Python in general.

Indeed, at class construction time, the class object itself has not yet been constructed, thus you cannot use it as the basis of a decorator.
One work-around I can think of, is to not use the staticmethod decorator. Instead, internally in your own decorator, re-use the classmethod decorator. That way you ensure that Python at the very least passes in the associated class for you:
def to_class(func):
""" returns a decorator
aimed to force the result to be of class cls. """
def wrapper(cls, *args, **kwargs):
return cls(func(*args, **kwargs))
return classmethod(wrapper)
Then use it like this:
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#to_class
def test_func(value):
return value
Demonstration:
>>> def to_class(func):
... """ returns a decorator
... aimed to force the result to be of class cls. """
... def wrapper(cls, *args, **kwargs):
... return cls(func(*args, **kwargs))
... return classmethod(wrapper)
...
>>> class TestClass(object):
... def __init__(self, value):
... self._value = (value, value)
... def __str__(self):
... return str(self._value)
... #to_class
... def test_func(value):
... return value
...
>>> TestClass.test_func('foo')
<__main__.TestClass object at 0x102a77210>
>>> print TestClass.test_func('foo')
('foo', 'foo')
A generic version of your decorator is not easy; the only other workaround to your conundrum is to use a metaclass hack; see another answer of mine where I describe the method in more detail.
You basically need to reach into the class-under-construction namespace, set a temporary metaclass, and then rely on there being at least one instance of the class before your decorator will work; the temporary metaclass approach hooks into the class creation mechanisms to retrieve the constructed class at a later time.
Seeing as you are using this decorator as an alternative class factory however, that is probably not going to be ideal; if someone used your decorated functions to create class instances exclusively the metaclass would be called too late.

Well, you forgot that class is the first parameter passed to method decorated with classmethod, so you can write it like this:
def to_this_class(func):
def wrapped(cls, value):
res = func(cls, value)
return cls(res)
return wrapped
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#classmethod
#to_this_class
def test_func(cls, value):
return value
x = TestClass('a')
print x.test_func('b')

The problem is that a decorator gets evaluated upon defining the thing it decorates, so when defining the method test_func(), the decorator to_test_class gets called, and even if it already exists, the thing it shall work on (the class TestClass) does not exist yet (as this is created after all methods are created).
Maybe you can use a placeholder at the point where the class is used and later (after the class is created) fill in that value (the class) at the point of the placeholder.
Example:
lazyClasses = {}
def to_lazy_class(className):
""" returns a decorator
aimed to force the result to be of class cls. """
def decorating_func(func):
def wrapper(*args, **kwargs):
return lazyClasses[className](func(*args, **kwargs))
return wrapper
return decorating_func
class TestClass(object):
def __init__(self, value):
self._value = (value, value)
def __str__(self):
return str(self._value)
#staticmethod
#to_lazy_class('TestClass')
def test_func(value):
return value
lazyClasses['TestClass'] = TestClass
>>> TestClass.test_func('hallo')
<__main__.TestClass object at 0x7f76d8cba190>

Customizable decorators in Python

I would like to provide a decorator that allows for an optional configuration when applied to a function.
A simple implementation follows:
import functools
class Deco(object):
config = {'message': 'hello'}
def __init__(self, func):
self.func = func
functools.wraps(func)(self)
def __call__(self, *args, **kwargs):
print self.config['message']
return self.func(*args, **kwargs)
#classmethod
def customize(cls, **kwargs):
"""Return a customized instance of this class. """
return type(cls.__name__, (Deco, ), {'config': kwargs})
#Deco
def add(a, b):
return a + b
#Deco.customize(message='bye')
def sub(a, b):
return a - b
>>> add(1, 2)
'hello'
>>> sub(2, 1)
'bye'
I would like to use it to provide user-friendly decorators for Django views.
This approach works without errors, but is there something bad about allowing a class to have a static factory method instantiating customized instances of it self, as a decorator?

You could work without creating an extra sub-class for each time the decorator is used there, but your code is fine. The way without extra subclass could be something along:
class Deco(object):
config = {'message': 'hello'}
def __init__(self, func=None, **kwargs):
if kwargs:
self.config = kwargs
if func is not None:
self._decorate(func)
def _decorate(self, func):
self.func = func
functools.wraps(func)(self)
def __call__(self, *args, **kwargs):
if not hasattr(self, "func"):
self._decorate(func)
return self
print self.config['message']
return self.func(*args, **kwargs)
So, while performance wise there would be no difference to your code (unless you would be decorating at least hundreds of thousands of functions - your code create an extra object - a class - for each time the decorator is used, besides the instance of that class) - there is an impact on people would review your code (either to use your modules, or to maintain it after you are done). I mean "a decorator that dynamically generates subclasses of itself" may sound too advanced and scare people away. Although it is as simple as my suggestion above once one understands the mechanisms of class generation in Python as you had.