I am building a child class inheriting from the xarray Dataset class but I wish to use the xarray open_dataset function to return the instance. The reason for this is that xarray's open_dataset can link to large datasets without loading them to memory as I would have to do when manually initialising a Dataset instance. I do not wish to modify this function and so to get around this I modify the 'new' method for Dataset to return my own class as follows.
def open_dataset(nc_file, data_type):
data_objs = {
"my_class": MyClass,
}
obj = data_objs[data_type]
def __new__(cls, *args, **kwargs):
if cls == xr.Dataset:
return object.__new__(obj)
return object.__new__(cls)
xr.Dataset.__new__ = staticmethod(__new__)
obj = xr.open_dataset(nc_file)
return obj
I can then define my class as follows and am successfully able to obtain MyClass instances from the open_dataset function which have the my_method method accessible and useable.
class MyClass(xr.Dataset)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.my_attribute = 1
print("in the init")
def my_method(self):
print("hello!")
However, the instances do not have my_attribute initialised. I have checked by adding a print statement that 'init' is being executed and I do get a print out. I therefore don't understand why the attributes in this init are not being initialised.
Related
When I create a mixin class that extends the logic of __init__, the regular thing to do is:
class ExtraValuemixin:
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
# some extra initialization
self._extra_value = 1
def retrieve_extra_value(self):
return self._extra_value
However this looks wrong to mypy, as it says:
Too many arguments for "__init__" of "object"
I get it, there's no *args or **kwargs in the object's constructor signature; but this is a mixin, and it relies on its childen's constructors. Ho do I make mypy understand this?
Full example:
class ExtraValuemixin:
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
# some extra initialization
self._extra_value = 1
def retrieve_extra_value(self):
return self._extra_value
class ParentObj:
def __init__(self, value):
self.value = value
class ChildObj(ExtraValuemixin, ParentObj):
pass
obj = ChildObj(value=5)
print(obj.retrieve_extra_value())
super().__init__(...)
Calls the __init__ function of it's parent class.
Defining a class with
class XXX:
(Omitting the hierarchal parent), leaves the parent as the default: our friend and beloved object class.
And we know the source of that class looks like:
class object:
def __init__():
Meaning, that it accepts no arguments. You essentially called this __init__ with two arguments. Hence, your exact error of:
Too many arguments for "__init__" of "object"
There is an answered question about classmethod and property combined together: Using property() on classmethods
I still don't understand the cause of the problem, please help.
My understanding of classmethod was that it simply replaces self with cls. With this in mind I wrote several classmethods during the past few years and now I see I was wrong all that time.
So what is the difference between #classmethod and #cm from the code below?
def cm(func):
def decorated(self, *args, **kwargs):
return func(self.__class__, *args, **kwargs)
return decorated
class C:
V = 0
#property
#classmethod
def inc1(cls):
cls.V += 1
print("V1 =", cls.V)
#property
#cm
def inc3(cls):
cls.V += 3
print("V3 =", cls.V)
c = C()
#c.inc1 # fails with: TypeError: 'classmethod' object is not callable
c.inc3 # works
inc3 with cm works, but inc1 with classmethod does not.
what is the difference between #classmethod and #cm from the code below?
decorator is calling during class creation time before an instance is created.
In your case, since #cm returns func(self.__class__, *args, **kwargs), which is relied on self, it should be used as a instance method.
On the other hand, #classmethod is able to use before an instance is created.
def cm(func):
def decorated(self, *args, **kwargs):
return func(self.__class__, *args, **kwargs)
return decorated
class C:
#classmethod
def inc1(cls):
(blablabla)
#cm
def inc3(cls):
(blablabla)
C().inc1() # works as a instance method
C.inc1() # works as a classmethod
C().inc3() # works as a instance method
C.inc3() # TypeError: unbound method decorated() must be called with C instance as first argument (got nothing instead)
For a combination of classmethod and property, it could be done by return an customized object. Reference
class ClassPropertyDescriptor(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.f.__get__(obj, klass)()
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class C:
#classproperty
def inc1(cls):
(blablabla)
C.inc1 # works as a classmethod property
[Edit]
Q. What does the classmethod() call do with the method it decorates to achieve that?
The implementation can be done by using descriptor
class ClassMethodDescriptor(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
def newfunc(*args):
return self.f(klass, *args)
return newfunc
def myclassmethod(func):
return ClassMethodDescriptor(func)
class C:
#myclassmethod
def inc1(cls):
(blablabla)
C.inc1() # works as a classmethod
Q. Why is the result not callable?
Because the implementation of ClassMethodDescriptor does not define __call__ function. Once using #property, it will return ClassMethodDescriptor which is not callable.
The difference is that classmethod is not callable, and cm method is callable. This means that when the property(class) makes a call to the inputed func(which it is supposed to do), it works as you'll except for cm, but will not work for classmethod since classmethod does not have a call implemented.
class method does not know anything about instance and does not require it.
instance method knows about it's instance and it's class.
class Foo:
some = 'some'
class Bar(Foo):
def __init__(self):
self.some = 'not some'
#classmethod
def cls_some(cls):
print(cls.some)
def instance_some(self):
print(self.some)
Bar.cls_some()
>>>some
Bar().instance_some()
>>>not some
Also as you can see you don't need an instance to call classmethod.
I am trying to pass some attributes from a method in a parent class to a method in a child class. And after that I wish to use a decorator that uses that attribute.
Suppose I have two classes and a decorator like these:
def my_decorator(fun):
inner(*args, **kwargs):
if fun.some_attr == 'value':
out = fun(*args, **kwargs)
else:
out = None
return out
return inner
class A:
def some_method_in_A(self):
return 1
some_method_in_A.some_attr = 'value'
class B(A):
#my_decorator
def some_method_in_B(self):
super(B, self).some_method_in_A()
What I want is to have some_attr set to 'value' in some_method_in_B. This attribute has to be available for my_decorator.
Is it possible to do it?
This is something that I would have to do for lots of methods in class B so it would be nice to do it in a compact way.
I have the following classes.
Validator is a decorator that receives a class which defines validation criteria for a decorated function. ValidateKeys is the validation criteria for this example. Node2D is a class using validation criteria.
class Validator(object):
def __init__(self, TheValidator, *args, **kwargs):
self.validator = TheValidator(*args,**kwargs)
def __call__(self,f):
def wrapped_f(instance, *args,**kwargs):
self.TheValidator(instance, *args, **kwargs)
return f(instance,*args,**kwargs)
return wrapped_f
class ValidateKeys(object):
def __init__(self,*keysIterable):
self.validkeys = keysIterable
def __call__(self, instance, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance.__dict__.update(kwargs)
class Node2D(object):
#property
def coords(self):
return self.__dict__
#coords.setter
def coords(self,Coords):
self.set_coords(**Coords)
#Validator(ValidateKeys, 'x','y')
def set_coords(self,**Coords):
pass
From what I understand, as things are written here, every instance of Node2D will produce a duplicate Validator (as will any other class decorated with Validator) and ValidateKeys.
EDIT: THIS IS WRONG! See answer below.
Note that this is primarily a learning exercise for me and although I would be interested in hearing criticisms/suggestions for improving my over all approach, my primary goal is to learn more about how to use decorators effectively.
Also note that I normally would not use capitalization for a decorator class but am using it here since it makes it easier to read on SO.
My assumption was incorrect.
As things are written, only one instance of Validator and ValidateKeys is created per class. I did not realize that the line #Validator(ValidateKeys, 'x','y') only runs once (at the time of class definition) and not at instance creation.
I should have realized this, since decorator expressions appear at the same level of hierarchy as class attributes, e.g.:
class MyClass():
class_attribute = None #only one class_attribute is created
#decorator #only one decorator (i.e., decorated method) is created
def method():
pass
Test:
class Validator(object):
def __init__(self, TheValidator, *args, **kwargs):
print("New Validator Object")
self.TheValidator = TheValidator(*args,**kwargs)
def __call__(self,f):
def wrapped_f(instance, *args,**kwargs):
self.TheValidator(instance, *args, **kwargs)
return f(instance,*args,**kwargs)
return wrapped_f
class ValidateKeys(object):
def __init__(self,*keysIterable):
print("New ValidateKeys Object")
self.validkeys = keysIterable
def __call__(self, instance, **kwargs):
for a in kwargs:
if not a in self.validkeys:
raise Exception()
instance.__dict__.update(kwargs)
class Node2D(object):
#property
def coords(self):
return self.__dict__
#coords.setter
def coords(self,Coords):
self.set_coords(**Coords)
#Validator(ValidateKeys, 'x','y')
def set_coords(self,**Coords):
pass
n1 = Node2D()
n2 = Node2D()
n1.setcoords(x=1,y=2)
n1.coords
Output:
'New Validator Object' #<-- Seen only once when module is loaded (class defined)
'New ValidateKeys Object' #<-- Seen only once when module is loaded (class defined)
'{'x': 1, 'y': 2}'
I do not have the problem I thought I had. Thanks to all for the help.
I have come across this singleton implementation here: http://blog.amir.rachum.com/post/21850841339/implementing-the-singleton-pattern-in-python in the first reply.
def singleton(cls):
return cls()
#singleton
class Foo(object):
def bar(self):
pass
if __name__ == '__main__':
print id(Foo)
print id(Foo)
But I don't understand the inner workings, the decorator returns a class instance, but why the same instance every time ?
You can rewrite that code to
class Foo(object):
pass
Foo = singleton(Foo)
# which is
Foo = Foo()
So here the name of the class is replaced by an instantiation of it. A bit cheesy in my opinion, especially since you can still create new objects of the same class by using Foo.__class__ and you are messing with the naming schema.
The singleton does that by holding internal state. This state here would probably be an instance of the class. The decorator can be something arbitrary.
Have a look at this:
http://hairysun.com/downloads/DecoratorHandout.pdf
class Decorator(object):
# in __init__ set up state
def __call__(self, function):
#functools.wraps(function)
def wrapper(*args, **kw): # 1.
print "before func"
result = function(*args, **kw) # 2.
print "after func"
return result
return wrapper # 3.
>>> decorator2 = Decorator()
>>> #decorator2
... def nothing(): pass
The decorator is essentially a function that
Defines a function
That calls the function that you passed in
Returns the newly 'wrapped' function to be called later
The surrounding class (here: the decorator) could do something like this:
class Singleton(object):
def __init__(self):
self.instance = None
def __call__(self, function):
#functools.wraps(function)
def wrapper(*args, **kw):
if self.instance is None:
self.instance = function(*args, **kw)
return self.instance
return wrapper
I did not run the code, but I assume this is in general how it works. If there is no instance available create one. If one is available, don't create a new one - return the single old one instead. One might probably want to check for other properties of the callable before using this in production.