How can I delay the __init__ call until an attribute is accessed?

How can I delay the __init__ call until an attribute is accessed? - python

I have a test framework that requires test cases to be defined using the following class patterns:
class TestBase:
def __init__(self, params):
self.name = str(self.__class__)
print('initializing test: {} with params: {}'.format(self.name, params))
class TestCase1(TestBase):
def run(self):
print('running test: ' + self.name)
When I create and run a test, I get the following:
>>> test1 = TestCase1('test 1 params')
initializing test: <class '__main__.TestCase1'> with params: test 1 params
>>> test1.run()
running test: <class '__main__.TestCase1'>
The test framework searches for and loads all TestCase classes it can find, instantiates each one, then calls the run method for each test.
load_test(TestCase1(test_params1))
load_test(TestCase2(test_params2))
...
load_test(TestCaseN(test_params3))
...
for test in loaded_tests:
test.run()
However, I now have some test cases for which I don't want the __init__ method called until the time that the run method is called, but I have little control over the framework structure or methods. How can I delay the call to __init__ without redefining the __init__ or run methods?
Update
The speculations that this originated as an XY problem are correct. A coworker asked me this question a while back when I was maintaining said test framework. I inquired further about what he was really trying to achieve and we figured out a simpler workaround that didn't involve changing the framework or introducing metaclasses, etc.
However, I still think this is a question worth investigating: if I wanted to create new objects with "lazy" initialization ("lazy" as in lazy evaluation generators such as range, etc.) what would be the best way of accomplishing it? My best attempt so far is listed below, I'm interested in knowing if there's anything simpler or less verbose.

First Solution：use property.the elegant way of setter/getter in python.
class Bars(object):
def __init__(self):
self._foo = None
#property
def foo(self):
if not self._foo:
print("lazy initialization")
self._foo = [1,2,3]
return self._foo
if __name__ == "__main__":
f = Bars()
print(f.foo)
print(f.foo)
Second Solution:the proxy solution,and always implement by decorator.
In short, Proxy is a wrapper that wraps the object you need. Proxy could provide additional functionality to the object that it wraps and doesn't change the object's code. It's a surrogate which provide the abitity of control access to a object.there is the code come form user Cyclone.
class LazyProperty:
def __init__(self, method):
self.method = method
self.method_name = method.__name__
def __get__(self, obj, cls):
if not obj:
return None
value = self.method(obj)
print('value {}'.format(value))
setattr(obj, self.method_name, value)
return value
class test:
def __init__(self):
self._resource = None
#LazyProperty
def resource(self):
print("lazy")
self._resource = tuple(range(5))
return self._resource
if __name__ == '__main__':
t = test()
print(t.resource)
print(t.resource)
print(t.resource)
To be used for true one-time calculated lazy properties. I like it because it avoids sticking extra attributes on objects, and once activated does not waste time checking for attribute presence

Metaclass option
You can intercept the call to __init__ using a metaclass. Create the object with __new__ and overwrite the __getattribute__ method to check if __init__ has been called or not and call it if it hasn't.
class DelayInit(type):
def __call__(cls, *args, **kwargs):
def init_before_get(obj, attr):
if not object.__getattribute__(obj, '_initialized'):
obj.__init__(*args, **kwargs)
obj._initialized = True
return object.__getattribute__(obj, attr)
cls.__getattribute__ = init_before_get
new_obj = cls.__new__(cls, *args, **kwargs)
new_obj._initialized = False
return new_obj
class TestDelayed(TestCase1, metaclass=DelayInit):
pass
In the example below, you'll see that the init print won't occur until the run method is executed.
>>> new_test = TestDelayed('delayed test params')
>>> new_test.run()
initializing test: <class '__main__.TestDelayed'> with params: delayed test params
running test: <class '__main__.TestDelayed'>
Decorator option
You could also use a decorator that has a similar pattern to the metaclass above:
def delayinit(cls):
def init_before_get(obj, attr):
if not object.__getattribute__(obj, '_initialized'):
obj.__init__(*obj._init_args, **obj._init_kwargs)
obj._initialized = True
return object.__getattribute__(obj, attr)
cls.__getattribute__ = init_before_get
def construct(*args, **kwargs):
obj = cls.__new__(cls, *args, **kwargs)
obj._init_args = args
obj._init_kwargs = kwargs
obj._initialized = False
return obj
return construct
#delayinit
class TestDelayed(TestCase1):
pass
This will behave identically to the example above.

In Python, there is no way that you can avoid calling __init__ when you instantiate a class cls. If calling cls(args) returns an instance of cls, then the language guarantees that cls.__init__ will have been called.
So the only way to achieve something similar to what you are asking is to introduce another class that will postpone the calling of __init__ in the original class until an attribute of the instantiated class is being accessed.
Here is one way:
def delay_init(cls):
class Delay(cls):
def __init__(self, *arg, **kwarg):
self._arg = arg
self._kwarg = kwarg
def __getattribute__(self, name):
self.__class__ = cls
arg = self._arg
kwarg = self._kwarg
del self._arg
del self._kwarg
self.__init__(*arg, **kwarg)
return getattr(self, name)
return Delay
This wrapper function works by catching any attempt to access an attribute of the instantiated class. When such an attempt is made, it changes the instance's __class__ to the original class, calls the original __init__ method with the arguments that were used when the instance was created, and then returns the proper attribute. This function can be used as decorator for your TestCase1 class:
class TestBase:
def __init__(self, params):
self.name = str(self.__class__)
print('initializing test: {} with params: {}'.format(self.name, params))
class TestCase1(TestBase):
def run(self):
print('running test: ' + self.name)
>>> t1 = TestCase1("No delay")
initializing test: <class '__main__.TestCase1'> with params: No delay
>>> t2 = delay_init(TestCase1)("Delayed init")
>>> t1.run()
running test: <class '__main__.TestCase1'>
>>> t2.run()
initializing test: <class '__main__.TestCase1'> with params: Delayed init
running test: <class '__main__.TestCase1'>
>>>
Be careful where you apply this function though. If you decorate TestBase with delay_init, it will not work, because it will turn the TestCase1 instances into TestBase instances.

In my answer I'd like to focus on cases when one wants to instantiate a class whose initialiser (dunder init) has side effects. For instance, pysftp.Connection, creates an SSH connection, which may be undesired until it's actually used.
In a great blog series about conceiving of wrapt package (nit-picky decorator implementaion), the author describes Transparent object proxy. This code can be customised for the subject in question.
class LazyObject:
_factory = None
'''Callable responsible for creation of target object'''
_object = None
'''Target object created lazily'''
def __init__(self, factory):
self._factory = factory
def __getattr__(self, name):
if not self._object:
self._object = self._factory()
return getattr(self._object, name)
Then it can be used as:
obj = LazyObject(lambda: dict(foo = 'bar'))
obj.keys() # dict_keys(['foo'])
But len(obj), obj['foo'] and other language constructs which invoke Python object protocols (dunder methods, like __len__ and __getitem__) will not work. However, for many cases, which are limited to regular methods, this is a solution.
To proxy object protocol implementations, it's possible to use neither __getattr__, nor __getattribute__ (to do it in a generic way). The latter's documentation notes:
This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions. See Special method lookup.
As a complete solution is demanded, there are examples of manual implementations like werkzeug's LocalProxy and django's SimpleLazyObject. However a clever workaround is possible.
Luckily there's a dedicated package (based on wrapt) for the exact use case, lazy-object-proxy which is described in this blog post.
from lazy_object_proxy import Proxy
obj = Proxy(labmda: dict(foo = 'bar'))
obj.keys() # dict_keys(['foo'])
len(len(obj)) # 1
obj['foo'] # 'bar'

One alternative would be to write a wrapper that takes a class as input and returns a class with delayed initialization until any member is accessed. This could for example be done as this:
def lazy_init(cls):
class LazyInit(cls):
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
self._initialized = False
def __getattr__(self, attr):
if not self.__dict__['_initialized']:
cls.__init__(self,
*self.__dict__['args'], **self.__dict__['kwargs'])
self._initialized = True
return self.__dict__[attr]
return LazyInit
This could then be used as such
load_test(lazy_init(TestCase1)(test_params1))
load_test(lazy_init(TestCase2)(test_params2))
...
load_test(lazy_init(TestCaseN)(test_params3))
...
for test in loaded_tests:
test.run()

Answering your original question (and the problem I think you are actually trying to solve), "How can I delay the init call until an attribute is accessed?": don't call init until you access the attribute.
Said another way: you can make the class initialization simultaneous with the attribute call. What you seem to actually want is 1) create a collection of TestCase# classes along with their associated parameters; 2) run each test case.
Probably your original problem came from thinking you had to initialize all your TestCase classes in order to create a list of them that you could iterate over. But in fact you can store class objects in lists, dicts etc. That means you can do whatever method you have for finding all TestCase classes and store those class objects in a dict with their relevant parameters. Then just iterate that dict and call each class with its run() method.
It might look like:
tests = {TestCase1: 'test 1 params', TestCase2: 'test 2 params', TestCase3: 'test 3 params'}
for test_case, param in tests.items():
test_case(param).run()

Overridding __new__
You could do this by overriding __new__ method and replacing __init__ method with a custom function.
def init(cls, real_init):
def wrapped(self, *args, **kwargs):
# This will run during the first call to `__init__`
# made after `__new__`. Here we re-assign the original
# __init__ back to class and assign a custom function
# to `instances.__init__`.
cls.__init__ = real_init
def new_init():
if new_init.called is False:
real_init(self, *args, **kwargs)
new_init.called = True
new_init.called = False
self.__init__ = new_init
return wrapped
class DelayInitMixin(object):
def __new__(cls, *args, **kwargs):
cls.__init__ = init(cls, cls.__init__)
return object.__new__(cls)
class A(DelayInitMixin):
def __init__(self, a, b):
print('inside __init__')
self.a = sum(a)
self.b = sum(b)
def __getattribute__(self, attr):
init = object.__getattribute__(self, '__init__')
if not init.called:
init()
return object.__getattribute__(self, attr)
def run(self):
pass
def fun(self):
pass
Demo:
>>> a = A(range(1000), range(10000))
>>> a.run()
inside __init__
>>> a.a, a.b
(499500, 49995000)
>>> a.run(), a.__init__()
(None, None)
>>> b = A(range(100), range(10000))
>>> b.a, b.b
inside __init__
(4950, 49995000)
>>> b.run(), b.__init__()
(None, None)
Using cached properties
The idea is to do the heavy calculation only once by caching results. This approach will lead to much more readable code if the whole point of delaying initialization is improving performance.
Django comes with a nice decorator called #cached_property. I tend to use it a lot in both code and unit-tests for caching results of heavy properties.
A cached_property is a non-data descriptor. Hence once the key is set in instance's dictionary, the access to property would always get the value from there.
class cached_property(object):
"""
Decorator that converts a method with a single self argument into a
property cached on the instance.
Optional ``name`` argument allows you to make cached properties of other
methods. (e.g. url = cached_property(get_absolute_url, name='url') )
"""
def __init__(self, func, name=None):
self.func = func
self.__doc__ = getattr(func, '__doc__')
self.name = name or func.__name__
def __get__(self, instance, cls=None):
if instance is None:
return self
res = instance.__dict__[self.name] = self.func(instance)
return res
Usage:
class A:
#cached_property
def a(self):
print('calculating a')
return sum(range(1000))
#cached_property
def b(self):
print('calculating b')
return sum(range(10000))
Demo:
>>> a = A()
>>> a.a
calculating a
499500
>>> a.b
calculating b
49995000
>>> a.a, a.b
(499500, 49995000)

I think you can use a wrapper class to hold the real class you want to instance, and use call __init__ yourself in your code, like(Python 3 code):
class Wrapper:
def __init__(self, cls):
self.cls = cls
self.instance = None
def your_method(self, *args, **kwargs):
if not self.instance:
self.instnace = cls()
return self.instance(*args, **kwargs)
class YourClass:
def __init__(self):
print("calling __init__")
but it's a dump way, but without any trick.

Related

Dynamic Wrapper in Python

I'm looking to create a dynamic wrapper class that exposes the API calls from a provided object using data in the object.
Statically it looks like this:
class Concrete:
def __init__(self, data):
self.data = data
def print_data(self):
print(self.data)
class Wrapper:
'''
One day this will wrap a variety of objects. But today
it can only handle Concrete objects.
'''
def wrap_it(self, concrete):
self.cco = concrete # concreteobject=cco
def print_data(self):
self.cco.print_data()
cco = Concrete(5)
wcco = Wrapper()
wcco.wrap_it(cco)
wcco.print_data()
Produces
5
I'd like to figure out how to do the same thing but make
wrap_it dynamic. It should search the concrete object
find the functions, and create functions of the same name
that call the same function in the concrete object.
I imagine that the solution involves inspect.signature or
at least some use of *args and **kwargs, but I've not seen
an example on how to put all this together.

You can use the __getattr__ magic method to hook getting undefined attributes, and forward them to the concrete object:
class DynamicWrapper():
def wrap_it(self, concrete):
self.cco = concrete
def __getattr__(self, k):
def wrapper(*args, **kwargs):
print(f'DynamicWrapper calling {k} with args {args} {kwargs}')
return getattr(self.cco, k)(*args, **kwargs)
if hasattr(self.cco, k):
return wrapper
else:
raise AttributeError(f'No such field/method: {k}')
cco = Concrete(5)
dwcco = DynamicWrapper()
dwcco.wrap_it(cco)
dwcco.print_data()

Use the dir() function to get the attributes of the given object, check if they are callable and assign them to your wrapper, like this:
class Wrapper:
def wrap_it(self, objToWrap):
for attr in dir(objToWrap):
if not attr.startswith('__') and callable(getattr(objToWrap, attr)):
exec('self.%s = objToWrap.%s' % (attr, attr))
And now, for testing.
>>> cco = Concrete(5)
>>> wcco = Wrapper()
>>> wcco.wrap_it(cco)
>>> wcco.print_data()
5

Using classes as method decorators [duplicate]

This question already has answers here:
How can I decorate an instance method with a decorator class?
(2 answers)
Closed 4 years ago.
While there are plenty of resources about using classes as decorators, I haven't been able to find any that deal with the problem of decorating methods. The goal of this question is to fix that. I will post my own solution, but of course everyone else is invited to post theirs as well.
Why the "standard" implementation doesn't work
The problem with the standard decorator class implementation is that python will not create a bound method of the decorated function:
class Deco:
def __init__(self, func):
self.func= func
def __call__(self, *args):
self.func(*args)
class Class:
#Deco
def hello(self):
print('hello world')
Class().hello() # throws TypeError: hello() missing 1 required positional argument: 'self'
A method decorator needs to overcome this hurdle.
Requirements
Taking the classes from the previous example, the following things are expected to work:
>>> i= Class()
>>> i.hello()
hello world
>>> i.hello
<__main__.Deco object at 0x7f4ae8b518d0>
>>> Class.hello is Class().hello
False
>>> Class().hello is Class().hello
False
>>> i.hello is i.hello
True
Ideally, the function's __doc__ and signature and similar attributes are preserved as well.

Usually when a method is accessed as some_instance.some_method(), python's descriptor protocol kicks in and calls some_method.__get__(), which returns a bound method. However, because the method has been replaced with an instance of the Deco class, that does not happen - because Deco is not a descriptor. In order to make Deco work as expected, it must implement a __get__ method that returns a bound copy of itself.
Implementation
Here's basic "do nothing" decorator class:
import inspect
import functools
from copy import copy
class Deco(object):
def __init__(self, func):
self.__self__ = None # "__self__" is also used by bound methods
self.__wrapped__ = func
functools.update_wrapper(self, func)
def __call__(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy
bound = copy(self)
bound.__self__ = instance
# update __doc__ and similar attributes
functools.update_wrapper(bound, self.__wrapped__)
# add the bound instance to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
To make the decorator do something, add your code in the __call__ method.
Here's one that takes parameters:
class DecoWithArgs(object):
#== change the constructor's parameters to fit your needs ==
def __init__(self, *args):
self.args = args
self.__wrapped__ = None
self.__self__ = None
def __call__(self, *args, **kwargs):
if self.__wrapped__ is None:
return self.__wrap(*args, **kwargs)
else:
return self.__call_wrapped_function(*args, **kwargs)
def __wrap(self, func):
# update __doc__ and similar attributes
functools.update_wrapper(self, func)
return self
def __call_wrapped_function(self, *args, **kwargs):
# if bound to an object, pass it as the first argument
if self.__self__ is not None:
args = (self.__self__,) + args
#== change the following line to make the decorator do something ==
return self.__wrapped__(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
# create a bound copy of this object
bound = copy(self)
bound.__self__ = instance
bound.__wrap(self.__wrapped__)
# add the bound decorator to the object's dict so that
# __get__ won't be called a 2nd time
setattr(instance, self.__wrapped__.__name__, bound)
return bound
An implementation like this lets us use the decorator on methods as well as functions, so I think it should be considered good practice.

How do I directly mock a superclass with python mock?

I am using the python mock framework for testing (http://www.voidspace.org.uk/python/mock/) and I want to mock out a superclass and focus on testing the subclasses' added behavior.
(For those interested I have extended pymongo.collection.Collection and I want to only test my added behavior. I do not want to have to run mongodb as another process for testing purposes.)
For this discussion, A is the superclass and B is the subclass. Furthermore, I define direct and indirect superclass calls as shown below:
class A(object):
def method(self):
...
def another_method(self):
...
class B(A):
def direct_superclass_call(self):
...
A.method(self)
def indirect_superclass_call(self):
...
super(A, self).another_method()
Approach #1
Define a mock class for A called MockA and use mock.patch to substitute it for the test at runtime. This handles direct superclass calls. Then manipulate B.__bases__ to handle indirect superclass calls. (see below)
The issue that arises is that I have to write MockA and in some cases (as in the case for pymongo.collection.Collection) this can involve a lot of work to unravel all of the internal calls to mock out.
Approach #2
The desired approach is to somehow use a mock.Mock() class to handle calls on the the mock just in time, as well as defined return_value or side_effect in place in the test. In this manner, I have to do less work by avoiding the definition of MockA.
The issue that I am having is that I cannot figure out how to alter B.__bases__ so that an instance of mock.Mock() can be put in place as a superclass (I must need to somehow do some direct binding here). Thus far I have determined, that super() examines the MRO and then calls the first class that defines the method in question. I cannot figure out how to get a superclass to handle the check to it and succeed if it comes across a mock class. __getattr__ does not seem to be used in this case. I want super to to think that the method is defined at this point and then use the mock.Mock() functionality as usual.
How does super() discover what attributes are defined within the class in the MRO sequence? And is there a way for me to interject here and to somehow get it to utilize a mock.Mock() on the fly?
import mock
class A(object):
def __init__(self, value):
self.value = value
def get_value_direct(self):
return self.value
def get_value_indirect(self):
return self.value
class B(A):
def __init__(self, value):
A.__init__(self, value)
def get_value_direct(self):
return A.get_value_direct(self)
def get_value_indirect(self):
return super(B, self).get_value_indirect()
# approach 1 - use a defined MockA
class MockA(object):
def __init__(self, value):
pass
def get_value_direct(self):
return 0
def get_value_indirect(self):
return 0
B.__bases__ = (MockA, ) # - mock superclass
with mock.patch('__main__.A', MockA):
b2 = B(7)
print '\nApproach 1'
print 'expected result = 0'
print 'direct =', b2.get_value_direct()
print 'indirect =', b2.get_value_indirect()
B.__bases__ = (A, ) # - original superclass
# approach 2 - use mock module to mock out superclass
# what does XXX need to be below to use mock.Mock()?
#B.__bases__ = (XXX, )
with mock.patch('__main__.A') as mymock:
b3 = B(7)
mymock.get_value_direct.return_value = 0
mymock.get_value_indirect.return_value = 0
print '\nApproach 2'
print 'expected result = 0'
print 'direct =', b3.get_value_direct()
print 'indirect =', b3.get_value_indirect() # FAILS HERE as the old superclass is called
#B.__bases__ = (A, ) # - original superclass

is there a way for me to interject here and to somehow get it to utilize a mock.Mock() on the fly?
There may be better approaches, but you can always write your own super() and inject it into the module that contains the class you're mocking. Have it return whatever it should based on what's calling it.
You can either just define super() in the current namespace (in which case the redefinition only applies to the current module after the definition), or you can import __builtin__ and apply the redefinition to __builtin__.super, in which case it will apply globally in the Python session.
You can capture the original super function (if you need to call it from your implementation) using a default argument:
def super(type, obj=None, super=super):
# inside the function, super refers to the built-in

I played around with mocking out super() as suggested by kindall. Unfortunately, after a great deal of effort it became quite complicated to handle complex inheritance cases.
After some work I realized that super() accesses the __dict__ of classes directly when resolving attributes through the MRO (it does not do a getattr type of call). The solution is to extend a mock.MagicMock() object and wrap it with a class to accomplish this. The wrapped class can then be placed in the __bases__ variable of a subclass.
The wrapped object reflects all defined attributes of the target class to the __dict__ of the wrapping class so that super() calls resolve to the properly patched in attributes within the internal MagicMock().
The following code is the solution that I have found to work thus far. Note that I actually implement this within a context handler. Also, care has to be taken to patch in the proper namespaces if importing from other modules.
This is a simple example illustrating the approach:
from mock import MagicMock
import inspect
class _WrappedMagicMock(MagicMock):
def __init__(self, *args, **kwds):
object.__setattr__(self, '_mockclass_wrapper', None)
super(_WrappedMagicMock, self).__init__(*args, **kwds)
def wrap(self, cls):
# get defined attribtues of spec class that need to be preset
base_attrs = dir(type('Dummy', (object,), {}))
attrs = inspect.getmembers(self._spec_class)
new_attrs = [a[0] for a in attrs if a[0] not in base_attrs]
# pre set mocks for attributes in the target mock class
for name in new_attrs:
setattr(cls, name, getattr(self, name))
# eat up any attempts to initialize the target mock class
setattr(cls, '__init__', lambda *args, **kwds: None)
object.__setattr__(self, '_mockclass_wrapper', cls)
def unwrap(self):
object.__setattr__(self, '_mockclass_wrapper', None)
def __setattr__(self, name, value):
super(_WrappedMagicMock, self).__setattr__(name, value)
# be sure to reflect to changes wrapper class if activated
if self._mockclass_wrapper is not None:
setattr(self._mockclass_wrapper, name, value)
def _get_child_mock(self, **kwds):
# when created children mocks need only be MagicMocks
return MagicMock(**kwds)
class A(object):
x = 1
def __init__(self, value):
self.value = value
def get_value_direct(self):
return self.value
def get_value_indirect(self):
return self.value
class B(A):
def __init__(self, value):
super(B, self).__init__(value)
def f(self):
return 2
def get_value_direct(self):
return A.get_value_direct(self)
def get_value_indirect(self):
return super(B, self).get_value_indirect()
# nominal behavior
b = B(3)
assert b.get_value_direct() == 3
assert b.get_value_indirect() == 3
assert b.f() == 2
assert b.x == 1
# using mock class
MockClass = type('MockClassWrapper', (), {})
mock = _WrappedMagicMock(A)
mock.wrap(MockClass)
# patch the mock in
B.__bases__ = (MockClass, )
A = MockClass
# set values within the mock
mock.x = 0
mock.get_value_direct.return_value = 0
mock.get_value_indirect.return_value = 0
# mocked behavior
b = B(7)
assert b.get_value_direct() == 0
assert b.get_value_indirect() == 0
assert b.f() == 2
assert b.x == 0

Namespaces inside class in Python3

I am new to Python and I wonder if there is any way to aggregate methods into 'subspaces'. I mean something similar to this syntax:
smth = Something()
smth.subspace.do_smth()
smth.another_subspace.do_smth_else()
I am writing an API wrapper and I'm going to have a lot of very similar methods (only different URI) so I though it would be good to place them in a few subspaces that refer to the API requests categories. In other words, I want to create namespaces inside a class. I don't know if this is even possible in Python and have know idea what to look for in Google.
I will appreciate any help.

One way to do this is by defining subspace and another_subspace as properties that return objects that provide do_smth and do_smth_else respectively:
class Something:
#property
def subspace(self):
class SubSpaceClass:
def do_smth(other_self):
print('do_smth')
return SubSpaceClass()
#property
def another_subspace(self):
class AnotherSubSpaceClass:
def do_smth_else(other_self):
print('do_smth_else')
return AnotherSubSpaceClass()
Which does what you want:
>>> smth = Something()
>>> smth.subspace.do_smth()
do_smth
>>> smth.another_subspace.do_smth_else()
do_smth_else
Depending on what you intend to use the methods for, you may want to make SubSpaceClass a singleton, but i doubt the performance gain is worth it.

I had this need a couple years ago and came up with this:
class Registry:
"""Namespace within a class."""
def __get__(self, obj, cls=None):
if obj is None:
return self
else:
return InstanceRegistry(self, obj)
def __call__(self, name=None):
def decorator(f):
use_name = name or f.__name__
if hasattr(self, use_name):
raise ValueError("%s is already registered" % use_name)
setattr(self, name or f.__name__, f)
return f
return decorator
class InstanceRegistry:
"""
Helper for accessing a namespace from an instance of the class.
Used internally by :class:`Registry`. Returns a partial that will pass
the instance as the first parameter.
"""
def __init__(self, registry, obj):
self.__registry = registry
self.__obj = obj
def __getattr__(self, attr):
return partial(getattr(self.__registry, attr), self.__obj)
# Usage:
class Something:
subspace = Registry()
another_subspace = Registry()
#MyClass.subspace()
def do_smth(self):
# `self` will be an instance of Something
pass
#MyClass.another_subspace('do_smth_else')
def this_can_be_called_anything_and_take_any_parameter_name(obj, other):
# Call it `obj` or whatever else if `self` outside a class is unsettling
pass
At runtime:
>>> smth = Something()
>>> smth.subspace.do_smth()
>>> smth.another_subspace.do_smth_else('other')
This is compatible with Py2 and Py3. Some performance optimizations are possible in Py3 because __set_name__ tells us what the namespace is called and allows caching the instance registry.

Python and the Singleton Pattern [duplicate]

This question already has answers here:
Creating a singleton in Python
(38 answers)
Closed 4 years ago.
There seem to be many ways to define singletons in Python. Is there a consensus opinion on Stack Overflow?

I don't really see the need, as a module with functions (and not a class) would serve well as a singleton. All its variables would be bound to the module, which could not be instantiated repeatedly anyway.
If you do wish to use a class, there is no way of creating private classes or private constructors in Python, so you can't protect against multiple instantiations, other than just via convention in use of your API. I would still just put methods in a module, and consider the module as the singleton.

Here's my own implementation of singletons. All you have to do is decorate the class; to get the singleton, you then have to use the Instance method. Here's an example:
#Singleton
class Foo:
def __init__(self):
print 'Foo created'
f = Foo() # Error, this isn't how you get the instance of a singleton
f = Foo.instance() # Good. Being explicit is in line with the Python Zen
g = Foo.instance() # Returns already created instance
print f is g # True
And here's the code:
class Singleton:
"""
A non-thread-safe helper class to ease implementing singletons.
This should be used as a decorator -- not a metaclass -- to the
class that should be a singleton.
The decorated class can define one `__init__` function that
takes only the `self` argument. Also, the decorated class cannot be
inherited from. Other than that, there are no restrictions that apply
to the decorated class.
To get the singleton instance, use the `instance` method. Trying
to use `__call__` will result in a `TypeError` being raised.
"""
def __init__(self, decorated):
self._decorated = decorated
def instance(self):
"""
Returns the singleton instance. Upon its first call, it creates a
new instance of the decorated class and calls its `__init__` method.
On all subsequent calls, the already created instance is returned.
"""
try:
return self._instance
except AttributeError:
self._instance = self._decorated()
return self._instance
def __call__(self):
raise TypeError('Singletons must be accessed through `instance()`.')
def __instancecheck__(self, inst):
return isinstance(inst, self._decorated)

You can override the __new__ method like this:
class Singleton(object):
_instance = None
def __new__(cls, *args, **kwargs):
if not cls._instance:
cls._instance = super(Singleton, cls).__new__(
cls, *args, **kwargs)
return cls._instance
if __name__ == '__main__':
s1 = Singleton()
s2 = Singleton()
if (id(s1) == id(s2)):
print "Same"
else:
print "Different"

A slightly different approach to implement the singleton in Python is the borg pattern by Alex Martelli (Google employee and Python genius).
class Borg:
__shared_state = {}
def __init__(self):
self.__dict__ = self.__shared_state
So instead of forcing all instances to have the same identity, they share state.

The module approach works well. If I absolutely need a singleton I prefer the Metaclass approach.
class Singleton(type):
def __init__(cls, name, bases, dict):
super(Singleton, cls).__init__(name, bases, dict)
cls.instance = None
def __call__(cls,*args,**kw):
if cls.instance is None:
cls.instance = super(Singleton, cls).__call__(*args, **kw)
return cls.instance
class MyClass(object):
__metaclass__ = Singleton

See this implementation from PEP318, implementing the singleton pattern with a decorator:
def singleton(cls):
instances = {}
def getinstance():
if cls not in instances:
instances[cls] = cls()
return instances[cls]
return getinstance
#singleton
class MyClass:
...

The Python documentation does cover this:
class Singleton(object):
def __new__(cls, *args, **kwds):
it = cls.__dict__.get("__it__")
if it is not None:
return it
cls.__it__ = it = object.__new__(cls)
it.init(*args, **kwds)
return it
def init(self, *args, **kwds):
pass
I would probably rewrite it to look more like this:
class Singleton(object):
"""Use to create a singleton"""
def __new__(cls, *args, **kwds):
"""
>>> s = Singleton()
>>> p = Singleton()
>>> id(s) == id(p)
True
"""
it_id = "__it__"
# getattr will dip into base classes, so __dict__ must be used
it = cls.__dict__.get(it_id, None)
if it is not None:
return it
it = object.__new__(cls)
setattr(cls, it_id, it)
it.init(*args, **kwds)
return it
def init(self, *args, **kwds):
pass
class A(Singleton):
pass
class B(Singleton):
pass
class C(A):
pass
assert A() is A()
assert B() is B()
assert C() is C()
assert A() is not B()
assert C() is not B()
assert C() is not A()
It should be relatively clean to extend this:
class Bus(Singleton):
def init(self, label=None, *args, **kwds):
self.label = label
self.channels = [Channel("system"), Channel("app")]
...

As the accepted answer says, the most idiomatic way is to just use a module.
With that in mind, here's a proof of concept:
def singleton(cls):
obj = cls()
# Always return the same object
cls.__new__ = staticmethod(lambda cls: obj)
# Disable __init__
try:
del cls.__init__
except AttributeError:
pass
return cls
See the Python data model for more details on __new__.
Example:
#singleton
class Duck(object):
pass
if Duck() is Duck():
print "It works!"
else:
print "It doesn't work!"
Notes:
You have to use new-style classes (derive from object) for this.
The singleton is initialized when it is defined, rather than the first time it's used.
This is just a toy example. I've never actually used this in production code, and don't plan to.

I'm very unsure about this, but my project uses 'convention singletons' (not enforced singletons), that is, if I have a class called DataController, I define this in the same module:
_data_controller = None
def GetDataController():
global _data_controller
if _data_controller is None:
_data_controller = DataController()
return _data_controller
It is not elegant, since it's a full six lines. But all my singletons use this pattern, and it's at least very explicit (which is pythonic).

The one time I wrote a singleton in Python I used a class where all the member functions had the classmethod decorator.
class Foo:
x = 1
#classmethod
def increment(cls, y=1):
cls.x += y

Creating a singleton decorator (aka an annotation) is an elegant way if you want to decorate (annotate) classes going forward. Then you just put #singleton before your class definition.
def singleton(cls):
instances = {}
def getinstance():
if cls not in instances:
instances[cls] = cls()
return instances[cls]
return getinstance
#singleton
class MyClass:
...

There are also some interesting articles on the Google Testing blog, discussing why singleton are/may be bad and are an anti-pattern:
Singletons are Pathological Liars
Where Have All the Singletons Gone?
Root Cause of Singletons

I think that forcing a class or an instance to be a singleton is overkill. Personally, I like to define a normal instantiable class, a semi-private reference, and a simple factory function.
class NothingSpecial:
pass
_the_one_and_only = None
def TheOneAndOnly():
global _the_one_and_only
if not _the_one_and_only:
_the_one_and_only = NothingSpecial()
return _the_one_and_only
Or if there is no issue with instantiating when the module is first imported:
class NothingSpecial:
pass
THE_ONE_AND_ONLY = NothingSpecial()
That way you can write tests against fresh instances without side effects, and there is no need for sprinkling the module with global statements, and if needed you can derive variants in the future.

The Singleton Pattern implemented with Python courtesy of ActiveState.
It looks like the trick is to put the class that's supposed to only have one instance inside of another class.

class Singleton(object[,...]):
staticVar1 = None
staticVar2 = None
def __init__(self):
if self.__class__.staticVar1==None :
# create class instance variable for instantiation of class
# assign class instance variable values to class static variables
else:
# assign class static variable values to class instance variables

class Singeltone(type):
instances = dict()
def __call__(cls, *args, **kwargs):
if cls.__name__ not in Singeltone.instances:
Singeltone.instances[cls.__name__] = type.__call__(cls, *args, **kwargs)
return Singeltone.instances[cls.__name__]
class Test(object):
__metaclass__ = Singeltone
inst0 = Test()
inst1 = Test()
print(id(inst1) == id(inst0))

OK, singleton could be good or evil, I know. This is my implementation, and I simply extend a classic approach to introduce a cache inside and produce many instances of a different type or, many instances of same type, but with different arguments.
I called it Singleton_group, because it groups similar instances together and prevent that an object of the same class, with same arguments, could be created:
# Peppelinux's cached singleton
class Singleton_group(object):
__instances_args_dict = {}
def __new__(cls, *args, **kwargs):
if not cls.__instances_args_dict.get((cls.__name__, args, str(kwargs))):
cls.__instances_args_dict[(cls.__name__, args, str(kwargs))] = super(Singleton_group, cls).__new__(cls, *args, **kwargs)
return cls.__instances_args_dict.get((cls.__name__, args, str(kwargs)))
# It's a dummy real world use example:
class test(Singleton_group):
def __init__(self, salute):
self.salute = salute
a = test('bye')
b = test('hi')
c = test('bye')
d = test('hi')
e = test('goodbye')
f = test('goodbye')
id(a)
3070148780L
id(b)
3070148908L
id(c)
3070148780L
b == d
True
b._Singleton_group__instances_args_dict
{('test', ('bye',), '{}'): <__main__.test object at 0xb6fec0ac>,
('test', ('goodbye',), '{}'): <__main__.test object at 0xb6fec32c>,
('test', ('hi',), '{}'): <__main__.test object at 0xb6fec12c>}
Every object carries the singleton cache... This could be evil, but it works great for some :)

My simple solution which is based on the default value of function parameters.
def getSystemContext(contextObjList=[]):
if len( contextObjList ) == 0:
contextObjList.append( Context() )
pass
return contextObjList[0]
class Context(object):
# Anything you want here

Being relatively new to Python I'm not sure what the most common idiom is, but the simplest thing I can think of is just using a module instead of a class. What would have been instance methods on your class become just functions in the module and any data just becomes variables in the module instead of members of the class. I suspect this is the pythonic approach to solving the type of problem that people use singletons for.
If you really want a singleton class, there's a reasonable implementation described on the first hit on Google for "Python singleton", specifically:
class Singleton:
__single = None
def __init__( self ):
if Singleton.__single:
raise Singleton.__single
Singleton.__single = self
That seems to do the trick.

Singleton's half brother
I completely agree with staale and I leave here a sample of creating a singleton half brother:
class void:pass
a = void();
a.__class__ = Singleton
a will report now as being of the same class as singleton even if it does not look like it. So singletons using complicated classes end up depending on we don't mess much with them.
Being so, we can have the same effect and use simpler things like a variable or a module. Still, if we want use classes for clarity and because in Python a class is an object, so we already have the object (not and instance, but it will do just like).
class Singleton:
def __new__(cls): raise AssertionError # Singletons can't have instances
There we have a nice assertion error if we try to create an instance, and we can store on derivations static members and make changes to them at runtime (I love Python). This object is as good as other about half brothers (you still can create them if you wish), however it will tend to run faster due to simplicity.

In cases where you don't want the metaclass-based solution above, and you don't like the simple function decorator-based approach (e.g. because in that case static methods on the singleton class won't work), this compromise works:
class singleton(object):
"""Singleton decorator."""
def __init__(self, cls):
self.__dict__['cls'] = cls
instances = {}
def __call__(self):
if self.cls not in self.instances:
self.instances[self.cls] = self.cls()
return self.instances[self.cls]
def __getattr__(self, attr):
return getattr(self.__dict__['cls'], attr)
def __setattr__(self, attr, value):
return setattr(self.__dict__['cls'], attr, value)

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

How can I delay the init call until an attribute is accessed? - python

Related

Dynamic Wrapper in Python

Using classes as method decorators [duplicate]

How do I directly mock a superclass with python mock?

Namespaces inside class in Python3

Python and the Singleton Pattern [duplicate]

Categories

Resources