I am trying to make a python decorator that adds attributes to methods of a class so that I can access and modify those attributes from within the method itself. The decorator code is
from types import MethodType
class attribute(object):
def __init__(self, **attributes):
self.attributes = attributes
def __call__(self, function):
class override(object):
def __init__(self, function, attributes):
self.__function = function
for att in attributes:
setattr(self, att, attributes[att])
def __call__(self, *args, **kwargs):
return self.__function(*args, **kwargs)
def __get__(self, instance, owner):
return MethodType(self, instance, owner)
retval = override(function, self.attributes)
return retval
I tried this decorator on the toy example that follows.
class bar(object):
#attribute(a=2)
def foo(self):
print self.foo.a
self.foo.a = 1
Though I am able to access the value of attribute 'a' from within foo(), I can't set it to another value. Indeed, when I call bar().foo(), I get the following AttributeError.
AttributeError: 'instancemethod' object has no attribute 'a'
Why is this? More importantly how can I achieve my goal?
Edit
Just to be more specific, I am trying to find a simple way to implement static variable that are located within class methods. Continuing from the example above, I would like instantiate b = bar(), call both foo() and doo() methods and then access b.foo.a and b.doo.a later on.
class bar(object):
#attribute(a=2)
def foo(self):
self.foo.a = 1
#attribute(a=4)
def doo(self):
self.foo.a = 3
The best way to do this is to not do it at all.
First of all, there is no need for an attribute decorator; you can just assign it yourself:
class bar(object):
def foo(self):
print self.foo.a
self.foo.a = 1
foo.a = 2
However, this still encounters the same errors. You need to do:
self.foo.__dict__['a'] = 1
You can instead use a metaclass...but that gets messy quickly.
On the other hand, there are cleaner alternatives.
You can use defaults:
def foo(self, a):
print a[0]
a[0] = 2
foo.func_defaults = foo.func_defaults[:-1] + ([2],)
Of course, my preferred way is to avoid this altogether and use a callable class ("functor" in C++ words):
class bar(object):
def __init__(self):
self.foo = self.foo_method(self)
class foo_method(object):
def __init__(self, bar):
self.bar = bar
self.a = 2
def __call__(self):
print self.a
self.a = 1
Or just use classic class attributes:
class bar(object):
def __init__(self):
self.a = 1
def foo(self):
print self.a
self.a = 2
If it's that you want to hide a from derived classes, use whatever private attributes are called in Python terminology:
class bar(object):
def __init__(self):
self.__a = 1 # this will be implicitly mangled as __bar__a or similar
def foo(self):
print self.__a
self.__a = 2
EDIT: You want static attributes?
class bar(object):
a = 1
def foo(self):
print self.a
self.a = 2
EDIT 2: If you want static attributes visible to only the current function, you can use PyExt's modify_function:
import pyext
def wrap_mod(*args, **kw):
def inner(f):
return pyext.modify_function(f, *args, **kw)
return inner
class bar(object):
#wrap_mod(globals={'a': [1]})
def foo(self):
print a[0]
a[0] = 2
It's slightly ugly and hackish. But it works.
My recommendation would be just to use double underscores:
class bar(object):
__a = 1
def foo(self):
print self.__a
self.__a = 2
Although this is visible to the other functions, it's invisible to anything else (actually, it's there, but it's mangled).
FINAL EDIT: Use this:
import pyext
def wrap_mod(*args, **kw):
def inner(f):
return pyext.modify_function(f, *args, **kw)
return inner
class bar(object):
#wrap_mod(globals={'a': [1]})
def foo(self):
print a[0]
a[0] = 2
foo.a = foo.func_globals['a']
b = bar()
b.foo() # prints 1
b.foo() # prints 2
# external access
b.foo.a[0] = 77
b.foo() # prints 77
While You can accomplish Your goal by replacing self.foo.a = 1 with self.foo.__dict__['a'] = 1 it is generally not recommended.
If you are using Python2 - (and not Python3) - whenever you retrieve a method from an instance, a new instance method object is created which is a wrapper to the original function defined in the class body.
The instance method is a rather transparent proxy to the function - you can retrieve the function's attributes through it, but not set them - that is why setting an item in self.foo.__dict__ works.
Alternatively you can reach the function object itself using: self.foo.im_func - the im_func attribute of instance methods point the underlying function.
Based on other contributors's answers, I came up with the following workaround. First, wrap a dictionnary in a class resolving non-existant attributes to the wrapped dictionnary such as the following code.
class DictWrapper(object):
def __init__(self, d):
self.d = d
def __getattr__(self, key):
return self.d[key]
Credits to Lucas Jones for this code.
Then implement a addstatic decorator with a statics attribute that will store the static attributes.
class addstatic(object):
def __init__(self, **statics):
self.statics = statics
def __call__(self, function):
class override(object):
def __init__(self, function, statics):
self.__function = function
self.statics = DictWrapper(statics)
def __call__(self, *args, **kwargs):
return self.__function(*args, **kwargs)
def __get__(self, instance, objtype):
from types import MethodType
return MethodType(self, instance)
retval = override(function, self.statics)
return retval
The following code is an example of how the addstatic decorator can be used on methods.
class bar(object):
#attribute(a=2, b=3)
def foo(self):
self.foo.statics.a = 1
self.foo.statics.b = 2
Then, playing with an instance of the bar class yields :
>>> b = bar()
>>> b.foo.statics.a
2
>>> b.foo.statics.b
3
>>> b.foo()
>>> b.foo.statics.a
3
>>> b.foo.statics.b
5
The reason for using this statics dictionnary follows jsbueno's answer which suggest that what I want would require overloading the dot operator of and instance method wrapping the foo function, which I am not sure is possible. Of course, the method's attribute could be set in self.foo.__dict__, but since it not recommended (as suggested by brainovergrow), I came up with this workaround. I am not certain this would be recommended either and I guess it is up for comments.
Related
I am trying to find a good way for returning a (new) class object in class method that can be extended as well.
I have a class (classA) which has among other methods, a method that returns a new classA object after some processing
class classA:
def __init__(): ...
def methodX(self, **kwargs):
process data
return classA(new params)
Now, I am extending this class to another classB. I need methodX to do the same, but return classB this time, instead of classA
class classB(classA):
def __init__(self, params):
super().__init__(params)
self.newParams = XYZ
def methodX(self, **kwargs):
???
This may be something trivial but I simply cannot figure it out. In the end I dont want to rewrite the methodX each time the class gets extended.
Thank you for your time.
Use the __class__ attribute like this:
class A:
def __init__(self, **kwargs):
self.kwargs = kwargs
def methodX(self, **kwargs):
#do stuff with kwargs
return self.__class__(**kwargs)
def __repr__(self):
return f'{self.__class__}({self.kwargs})'
class B(A):
pass
a = A(foo='bar')
ax = a.methodX(gee='whiz')
b = B(yee='haw')
bx = b.methodX(cool='beans')
print(a)
print(ax)
print(b)
print(bx)
class classA:
def __init__(self, x):
self.x = x
def createNew(self, y):
t = type(self)
return t(y)
class classB(classA):
def __init__(self, params):
super().__init__(params)
a = classA(1)
newA = a.createNew(2)
b = classB(1)
newB = b.createNew(2)
print(type(newB))
# <class '__main__.classB'>
I want to propose what I think is the cleanest approach, albeit similar to existing answers. The problem feels like a good fit for a class method:
class A:
#classmethod
def method_x(cls, **kwargs):
return cls(<init params>)
Using the #classmethod decorator ensures that the first input (traditionally named cls) will refer to the Class to which the method belongs, rather than the instance.
(usually we call the first method input self and this refers to the instance to which the method belongs)
Because cls refers to A, rather than an instance of A, we can call cls() as we would call A().
However, in a class that inherits from A, cls will instead refer to the child class, as required:
class A:
def __init__(self, x):
self.x = x
#classmethod
def make_new(cls, **kwargs):
y = kwargs["y"]
return cls(y) # returns A(y) here
class B(A):
def __init__(self, x):
super().__init__(x)
self.z = 3 * x
inst = B(1).make_new(y=7)
print(inst.x, inst.z)
And now you can expect that print statement to produce 7 21.
That inst.z exists should confirm for you that the make_new call (which was only defined on A and inherited unaltered by B) has indeed made an instance of B.
However, there's something I must point out. Inheriting the unaltered make_new method only works because the __init__ method on B has the same call signature as the method on A. If this weren't the case then the call to cls might have had to be altered.
This can be circumvented by allowing **kwargs on the __init__ method and passing generic **kwargs into cls() in the parent class:
class A:
def __init__(self, **kwargs):
self.x = kwargs["x"]
#classmethod
def make_new(cls, **kwargs):
return cls(**kwargs)
class B(A):
def __init__(self, x, w):
super().__init__(x=x)
self.w = w
inst = B(1,2).make_new(x="spam", w="spam")
print(inst.x, inst.w)
Here we were able to give B a different (more restrictive!) signature.
This illustrates a general principle, which is that parent classes will typically be more abstract/less specific than their children.
It follows that, if you want two classes that substantially share behaviour but which do quite specific different things, it will be better to create three classes: one rather abstract one that defines the behaviour-in-common, and two children that give you the specific behaviours you want.
I am writing a metaclass that amongst other things wants to add a method to the classes it creates. Let's forget about the metaclass for now though and just look at simple adding of methods.
In order to dynamically add an instance method I can do:
class Foo:
def bar(self, x):
print(f"In bar() with {x} and {self}")
def func(self, x):
print(f"In func() with {x} and {self}")
Foo.func = func
After that I can do:
>>> f = Foo()
>>> f.bar(7)
In bar() with 7 and <__main__.Foo object at 0x7f912a7e57f0>
>>> f.func(7)
In func() with 7 and <__main__.Foo object at 0x7f912a7e57f0>
So the methods bar and func seem to function identically, but there are some discernible differences, e.g.:
>>> f.bar.__qualname__
'Foo.bar'
>>> f.func.__qualname__
'func'
f.func.__module__ could also potentially be different from f.bar.__module__, depending on where everything is defined.
What do I have to change in Construction 2 (below) in order for both constructions to behave exactly the same (no code that uses the Foo class could change its behaviour depending on which construction is used)?
# Construction 1
class Foo:
def func(self):
pass
# Construction 2
class Foo:
pass
def func(self):
pass
Foo.func = func
I have created a decorator that hopefully implements a sensible version of Construction 2, but what could I still be missing/breaking by monkey patching like that?
class instance_method_of:
def __init__(self, cls, name=None):
self.cls = cls
self.name = name
def __call__(self, func):
if self.name is not None:
func.__name__ = self.name
func.__qualname__ = f'{self.cls.__qualname__}.{func.__name__}'
func.__module__ = self.cls.__module__
setattr(self.cls, func.__name__, func)
return func
class Foo:
pass
#instance_method_of(Foo)
def func(self):
pass
It seems to cover everything! Although I would go with a function decorator like :
from pathlib import Path
def patch(f):
cls = next(iter(f.__annotations__.values()))
name = f.__defaults__[0]
f.__qualname__ = f"{cls.__name__}.{f.__name__}"
f.__module__ = cls.__module__
if name is None:
setattr(cls,f.__name__,f)
else:
f.__qualname__ = f"{cls.__name__}.{name}"
setattr(cls,name,f)
return f
#patch
def new(self:Path, name:str=None):
"new"
return list(self.iterdir())
path = Path()
path.new()
mostly adapted from fastai version
I've imported a package that provides me with a class and a wrapper function that creates an instance of that class.
For example:
class Foo:
def __init__(self, a, b):
self.a = a
self.b = b
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
def makeFoo(x, y):
a = x + y
b = x - y
return Foo(a, b)
I want to have a similar class NamedFoo, that has the same properties/methods, also has a name property, and with a constructor that calls makeFoo. I figure that this should be solved using inheritance, with NamedFoo being a subclass of Foo. However, I don't know how to make the NamedFoo constructor utilize makeFoo correctly:
class NamedFoo(Foo):
def __init__(self, x, y, name):
# ???
# Foo = makeFoo(x, y) ??
# self.Foo = makeFoo(x, y) ??
self.name = name
def printName(self):
print(self.name)
Example data:
myNamedFoo = NamedFoo(2,5,"first")
myNamedFoo.print_a() # (From makeFoo: a = x + y) ==> 2 + 5 = 7
myNamedFoo.print_b() # (From makeFoo: a = x - y) ==> 2 - 5 = -3
I'm not too familiar with object-oriented programming, so I might just be using the wrong search terms, but I haven't found anything similar to what I need. Is this possible, and if so how can I do it?
I'm also not sure if this is an X/Y problem, but here are the alternatives I've considered and why I don't think they're ideal:
Composition of Foo and the property name: It's ugly and doesn't seem right.
Manually adding the name property to each Foo object, and perhaps wrapping it in a function: Doesn't quite have the elegance of a one liner constructor.
Rewriting the constructor for the Foo class, to have the same code as what's in makeFoo: makeFoo is rather complex and needs to do a lot of setup, and this would in any case lead to code duplication
In the NamedFoo constructor, create an instance of the Foo class from the makeFoo wrapper function. Pass this instance's attributes to the super().__init__.
class NamedFoo(Foo):
def __init__(self, x, y, name):
_foo = makeFoo(x,y) # use the wrapper to handle complex logic from input params
super().__init__(_foo.a,_foo.b) # pass the properly derived Foo attributes to the superclass constructor
self.name = name
This way, we're instantiating NamedFoo from whatever magic happens within the makeFoo function. Pass your x and y to that, which creates a throwaway Foo instance (so we can have it properly constructed with whatever complex logic resides in the helper function). The final NamedFoo class is then instantiated from the Foo constructor.
i think this should work..
class Foo:
def __init__(self,a,b):
self.a = a + b
self.b = a - b
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
class NamedFoo(Foo):
def __init__(self,a,b,name):
super().__init__(a,b)
self.name = name
def main():
example = NamedFoo(2,5,"first")
example.print_a()
example.print_b()
main()
this prints out
7
-3
or if you really want to use a function to create self.a and self.b use this:
class Foo:
def __init__(self, a, b):
self.a, self.b = make_foo(a,b)
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
class NamedFoo(Foo):
def __init__(self, a,b,name):
super().__init__(a,b)
self.name = name
def make_foo(x,y):
return x+y, x-y
def main():
example = NamedFoo(2,5,"first")
example.print_a()
example.print_b()
main()
I need to refactor existing code by collapsing a method that's copy-and-pasted between various classed that inherit from one another into a single method.
So I produced the following code:
class A(object):
def rec(self):
return 1
class B(A):
def rec(self):
return self.rec_gen(B)
def rec_gen(self, rec_class):
return super(rec_class, self).rec() + 1
class C(B):
def rec(self):
return self.rec_gen(C)
if __name__=='__main__':
b = B(); c = C()
print c.rec()
print b.rec()
And the output:
3
2
What still bothers me is that in the 'rec' method I need to tell 'rec_gen' the context of the class in which it's running. Is there a way for 'rec_gen' to figure it out by itself in runtime?
This capability has been added to Python 3 - see PEP 3135. In a nutshell:
class B(A):
def rec(self):
return super().rec() + 1
I think you've created the convoluted rec()/rec_gen() setup because you couldn't automatically find the class, but in case you want that anyway the following should work:
class A(object):
def rec(self):
return 1
class B(A):
def rec(self):
# __class__ is a cell that is only created if super() is in the method
super()
return self.rec_gen(__class__)
def rec_gen(self, rec_class):
return super(rec_class, self).rec() + 1
class C(B):
def rec(self):
# __class__ is a cell that is only created if super() is in the method
super()
return self.rec_gen(__class__)
The simplest solution in Python 2 is to use a private member to hold the super object:
class B(A):
def __init__(self):
self.__super = super(B)
def rec(self):
return self.__super.rec() + 1
But that still suffers from the need to specify the actual class in one place, and if you happen to have two identically-named classes in the class hierarchy (e.g. from different modules) this method will break.
There were a couple of us who made recipes for automatic resolution for Python 2 prior to the existence of PEP 3135 - my method is at self.super on ActiveState. Basically, it allows the following:
class B(A, autosuper):
def rec(self):
return self.super().rec() + 1
or in the case that you're calling a parent method with the same name (the most common case):
class B(A, autosuper):
def rec(self):
return self.super() + 1
Caveats to this method:
It's quite slow. I have a version sitting around somewhere that does bytecode manipulation to improve the speed a lot.
It's not consistent with PEP 3135 (although it was a proposal for the Python 3 super at one stage).
It's quite complex.
It's a mix-in base class.
I don't know if the above would enable you to meet your requirements. With a small change to the recipe though you could find out what class you're in and pass that to rec_gen() - basically extract the class-finding code out of _getSuper() into its own method.
An alternative solution for python 2.x would be to use a metaclass to automatically define the rec method in all your subclasses:
class RecGen(type):
def __new__(cls, name, bases, dct):
new_cls = super(RecGen, cls).__new__(cls, name, bases, dct)
if bases != (object,):
def rec(self):
return super(new_cls, self).rec() + 1
new_cls.rec = rec
return new_cls
class A(object):
__metaclass__ = RecGen
def rec(self):
return 1
class B(A):
pass
class C(B):
pass
Note that if you're just trying to get something like the number of parent classes, it would be easier to use self.__class__.__mro__ directly:
class A(object):
def rec(self):
return len(self.__class__.__mro__)-1
class B(A):
pass
class C(B):
pass
I'm not sure exactly what you're trying to achieve, but if it is just to have a method that returns a different constant value for each class then use class attributes to store the value. It isn't clear at all from your example that you need to go anywhere near super().
class A(object):
REC = 1
def rec(self):
return self.REC
class B(A):
REC = 2
class C(B):
REC = 3
if __name__=='__main__':
b = B(); c = C()
print c.rec()
print b.rec()
I'm trying to decorate a class with another class. I also want to inherit from the decorated class, but I get some errors. Here's my code:
class Decorator:
def __init__(self, decorated):
pass
#Decorator
class Foo:
pass
class Goo(Foo):
pass
The error I get when I try to subclass from Foo is this:
Traceback (most recent call last):
File "test.py", line 9, in
class Goo(Foo):
TypeError: __init__() takes exactly 2 positional arguments (4 given)
By adding another init function to Decorator...
def __init__(self, *args):
for arg in args:
print(arg)
... I get the following output:
<class '__main__.Foo'>
Goo
(<__main__.Decorator object at 0x010073B0>,)
{'__module__': '__main__'}
What are those parameters and how should I be using them inside Decorator?
I'll try to answer the "what are those parameters" question. This code:
#Decorator
class Foo:
pass
is equivalent to:
class Foo:
pass
Foo = Decorator(Foo)
This means that Foo ends up being an instance of the Decorator class instead of being a class.
When you try to use this instance as a base of a class (Goo), Python will have to determine a metaclass that will be used to create the new class. In this case it will use Foo.__class__ which equals to Decorator. Then it will call the metaclass with (name, bases, dict) arguments and expect it to return a new class.
This is how you end up with these arguments in Decorator.__init__.
More about this can be found here:
http://www.python.org/download/releases/2.2.3/descrintro/#metaclasses
(particularly the "When a class statement is executed..." part)
Are you trying to add a MixIn to a class after the class has been defined?
If so, you could inject the MixIn this way:
def inject_class(mixin):
def _inject_class(cls):
return type(cls.__name__,(mixin,)+cls.__bases__,dict(cls.__dict__))
return _inject_class
class MixIn(object):
def mix(self):
print('mix')
#inject_class(MixIn)
class Foo(object):
def foo(self):
print('foo')
class Goo(Foo):
def goo(self):
print('goo')
goo=Goo()
goo.mix()
goo.foo()
goo.goo()
prints
mix
foo
goo
If you don't want the generality of inject_class, you could make a specialized class decorator which mixes in Decorator only:
def decorate(cls):
class Decorator(object):
def deco(self):
print('deco')
return type(cls.__name__,(Decorator,)+cls.__bases__,dict(cls.__dict__))
#decorate
class Foo(object):
def foo(self):
print('foo')
the result is the same.
I had the same problem and the following solution works for me:
from functools import update_wrapper
class decoratorBase():
def __new__(cls, logic):
self = object.__new__(cls)
self.__init__(logic)
def new (cls):
#cls is the decorated class type, not the decorator class type itself
self._createInstance(cls)
self._postInstanceCreation()
return self
self._logic.__new__ = new
#return the wrapped class and not a wrapper
return self._logic
def __init__(self, logic):
#logic is the decorated class
self._logic = logic
def _createInstance(self, cls):
self._logicInstance = object.__new__(cls)
self._logicInstance.__init__()
def _postInstanceCreation(self):
pass
class factory(decoratorBase):
def __init__(self, *largs, **kwargs):
super().__init__(*largs, **kwargs)
self.__instance = None
def _createInstance(self, cls):
self._logicInstance = None
self._cls = cls
def _postInstanceCreation(self):
update_wrapper(self, self._cls)
def __call__(self, userData, *largs, **kwargs):
logicInstance = object.__new__(self._cls)
logicInstance.__init__(*largs, **kwargs)
logicInstance._update(userData)
return logicInstance
class singelton(decoratorBase):
def _postInstanceCreation(self):
update_wrapper(self, self._logicInstance)
def __call__(self, userData):
self._logicInstance._update(userData)
return self._logicInstance
class base():
def __init__(self):
self.var = 0
print ("Create new object")
def __call__(self):
self.var += self._updateValue()
def _update(self, userData):
print ("Update object static value with {0}".format(userData))
self.var = userData
#factory
class factoryTestBase(base):
def __call__(self):
super().__call__()
print("I'm a factory, here is the proof: {0}".format(self.var))
def _updateValue(self):
return 1
class factoryTestDerived(factoryTestBase):
def _updateValue(self):
return 5
#singelton
class singeltonTestBase(base):
def __call__(self):
super().__call__()
print("I'm a singelton, here is the proof: {0}".format(self.var))
def _updateValue(self):
return 1
class singeltonTestDerived(singeltonTestBase):
def _updateValue(self):
return 5
The magic in this approach is the overloading of the __new__() method, as well for the decorator itself as for the "wrapper" which is returned by the decorator. I set the word wrapper in quotes, because actually there is no wrapper. Instead the decorated class is alternated by the decorator and returned. Using this scheme, you are able to inherit from a decorated class. The most important thing is the change of the __new__() method of the decorated class, which is made by the following lines:
def new (cls):
self._createInstance(cls)
self._postInstanceCreation()
return self
self._logic.__new__ = new
Using this, you have access to the decorator methods like self._createInstance() during creation of an object from a decorated class. You even have the opportunity to inherit from your decorators (as it is shown in the example).
Now lets run a simple example:
>>> factoryObjCreater = factoryTestBase()
>>> factoryObj1 = factoryObjCreater(userData = 1)
Create new object
Update object static value with 1
>>> factoryObj2 = factoryObjCreater(userData = 1)
Create new object
Update object static value with 1
>>> factoryObj1()
I'm a factory, here is the proof: 2
>>> factoryObj2()
I'm a factory, here is the proof: 2
>>> factoryObjDerivedCreater = factoryTestDerived()
>>> factoryObjDerived1 = factoryObjDerivedCreater(userData = 2)
Create new object
Update object static value with 2
>>> factoryObjDerived2 = factoryObjDerivedCreater(userData = 2)
Create new object
Update object static value with 2
>>> factoryObjDerived1()
I'm a factory, here is the proof: 7
>>> factoryObjDerived2()
I'm a factory, here is the proof: 7
>>> singeltonObjCreater = singeltonTestBase()
Create new object
>>> singeltonObj1 = singeltonObjCreater(userData = 1)
Update object static value with 1
>>> singeltonObj2 = singeltonObjCreater(userData = 1)
Update object static value with 1
>>> singeltonObj1()
I'm a singelton, here is the proof: 2
>>> singeltonObj2()
I'm a singelton, here is the proof: 3
>>> singeltonObjDerivedCreater = singeltonTestDerived()
Create new object
>>> singeltonObjDerived1 = singeltonObjDerivedCreater(userData = 2)
Update object static value with 2
>>> singeltonObjDerived2 = singeltonObjDerivedCreater(userData = 2)
Update object static value with 2
>>> singeltonObjDerived1()
I'm a singelton, here is the proof: 7
>>> singeltonObjDerived2()
I'm a singelton, here is the proof: 12
>>>