I am trying to learn more about Python - I am using 3.6.3 currently.
What is the best practice for argument validation/parsing when using kwargs?
Or, what is the best practice for argument validation/parsing without kwargs?
class foo:
def __init__(self, **kwargs):
if 'a' in kwargs:
self.a = kwargs['a']
else:
self.a = 0
class bar(foo):
def __init__(self, **kwargs):
super().__init__()
if 'x' in kwargs:
self.x = kwargs['x']
else:
self.x = 23
# b and y are undefined, but my classes use kwargs - what is the best practice for validating / parsing dynamic arguments?
test = bar(b=1,y=24)
You can pass a default value to get() for keys that are not in the kwargs dictionary:
def __init__(self, **kwargs):
self.a = kwargs.get("a", 0)
self.x = kwargs.get("x", 23)
# etc.
Alternatively if you want any value in kwargs to be set as an instance variable of your class, you could do:
def __init__(self, **kwargs):
for k, v in kwargs.items():
self.__setattr__(k, v)
class Foo:
def __init__(self,a=0):
self.a = a
class Bar(Foo):
def __init__(self,b=21,**kwargs):
self.b = b # accept whatever you want for this child
Foo.__init__(self,**kwargs) # pass remaining kwargs down the line
does exactly what your Foo class does and is much more clear
most problems with using kwargs come from the fact that its not at all self documenting ... I have no idea what arguments can and should be supplied for neither Foo or Bar where as explicitly declared arguments with default values make it very clear what options are available to pass into the functions
Related
I am trying to find a good way for returning a (new) class object in class method that can be extended as well.
I have a class (classA) which has among other methods, a method that returns a new classA object after some processing
class classA:
def __init__(): ...
def methodX(self, **kwargs):
process data
return classA(new params)
Now, I am extending this class to another classB. I need methodX to do the same, but return classB this time, instead of classA
class classB(classA):
def __init__(self, params):
super().__init__(params)
self.newParams = XYZ
def methodX(self, **kwargs):
???
This may be something trivial but I simply cannot figure it out. In the end I dont want to rewrite the methodX each time the class gets extended.
Thank you for your time.
Use the __class__ attribute like this:
class A:
def __init__(self, **kwargs):
self.kwargs = kwargs
def methodX(self, **kwargs):
#do stuff with kwargs
return self.__class__(**kwargs)
def __repr__(self):
return f'{self.__class__}({self.kwargs})'
class B(A):
pass
a = A(foo='bar')
ax = a.methodX(gee='whiz')
b = B(yee='haw')
bx = b.methodX(cool='beans')
print(a)
print(ax)
print(b)
print(bx)
class classA:
def __init__(self, x):
self.x = x
def createNew(self, y):
t = type(self)
return t(y)
class classB(classA):
def __init__(self, params):
super().__init__(params)
a = classA(1)
newA = a.createNew(2)
b = classB(1)
newB = b.createNew(2)
print(type(newB))
# <class '__main__.classB'>
I want to propose what I think is the cleanest approach, albeit similar to existing answers. The problem feels like a good fit for a class method:
class A:
#classmethod
def method_x(cls, **kwargs):
return cls(<init params>)
Using the #classmethod decorator ensures that the first input (traditionally named cls) will refer to the Class to which the method belongs, rather than the instance.
(usually we call the first method input self and this refers to the instance to which the method belongs)
Because cls refers to A, rather than an instance of A, we can call cls() as we would call A().
However, in a class that inherits from A, cls will instead refer to the child class, as required:
class A:
def __init__(self, x):
self.x = x
#classmethod
def make_new(cls, **kwargs):
y = kwargs["y"]
return cls(y) # returns A(y) here
class B(A):
def __init__(self, x):
super().__init__(x)
self.z = 3 * x
inst = B(1).make_new(y=7)
print(inst.x, inst.z)
And now you can expect that print statement to produce 7 21.
That inst.z exists should confirm for you that the make_new call (which was only defined on A and inherited unaltered by B) has indeed made an instance of B.
However, there's something I must point out. Inheriting the unaltered make_new method only works because the __init__ method on B has the same call signature as the method on A. If this weren't the case then the call to cls might have had to be altered.
This can be circumvented by allowing **kwargs on the __init__ method and passing generic **kwargs into cls() in the parent class:
class A:
def __init__(self, **kwargs):
self.x = kwargs["x"]
#classmethod
def make_new(cls, **kwargs):
return cls(**kwargs)
class B(A):
def __init__(self, x, w):
super().__init__(x=x)
self.w = w
inst = B(1,2).make_new(x="spam", w="spam")
print(inst.x, inst.w)
Here we were able to give B a different (more restrictive!) signature.
This illustrates a general principle, which is that parent classes will typically be more abstract/less specific than their children.
It follows that, if you want two classes that substantially share behaviour but which do quite specific different things, it will be better to create three classes: one rather abstract one that defines the behaviour-in-common, and two children that give you the specific behaviours you want.
I am trying to design a class structure that allows the user to define their own class that overloads predefined methods in other classes. In this case the user would create the C class to overload the "function" method in D. The user created C class has common logic for other user created classes A and B so they inherit from C to overload "function" but also inherit from D to use D's other methods. The issue I am having is how to pass "value" from A and B to D and ignore passing it to C. What I currently have written will produce an error as C does not have "value" as an argument.
I know that I can add "value" (or *args) to C's init method and the super call but I don't want to have to know what inputs other classes need in order to add new classes to A and B. Also, if I swap the order of C and D I won't get an error but then I don't use C's overloaded "function". Is there an obvious way around this?
class D(SomethingElse):
def __init__(self, value, **kwargs):
super(D, self).__init__(**kwargs)
self.value = value
def function(self):
return self.value
def other_method(self):
pass
class C(object):
def __init__(self):
super(C, self).__init__()
def function(self):
return self.value*2
class B(C, D):
def __init__(self, value, **kwargs):
super(B, self).__init__(value, **kwargs)
class A(C, D):
def __init__(self, value, **kwargs):
super(A, self).__init__(value, **kwargs)
a = A(3)
print(a.function())
>>> 6
Essentially, there are two things you need to do to make your __init__ methods play nice with multiple inheritance in Python:
Always take a **kwargs parameter, and always call super().__init__(**kwargs), even if you think you are the base class. Just because your superclass is object doesn't mean you are last (before object) in the method resolution order.
Don't pass your parent class's __init__ arguments explicitly; only pass them via **kwargs. Your parent class isn't necessarily the next one after you in the method resolution order, so positional arguments might be passed to the wrong other __init__ method.
This is called "co-operative subclassing". Let's try with your example code:
class D:
def __init__(self, value, **kwargs):
self.value = value
super().__init__(**kwargs)
def function(self):
return self.value
class C:
# add **kwargs parameter
def __init__(self, **kwargs):
# pass kwargs to super().__init__
super().__init__(**kwargs)
def function(self):
return self.value * 2
class B(C, D):
# don't take parent class's value arg explicitly
def __init__(self, **kwargs):
# pass value arg via kwargs
super().__init__(**kwargs)
class A(C, D):
# don't take parent class's value arg explicitly
def __init__(self, **kwargs):
# pass value arg via kwargs
super().__init__(**kwargs)
Demo:
>>> a = A(value=3)
>>> a.value
3
>>> a.function()
6
Note that value must be passed to the A constructor as a keyword argument, not as a positional argument. It's also recommended to set self.value = value before calling super().__init__.
I've also simplified class C(object): to class C:, and super(C, self) to just super() since these are equivalent in Python 3.
So I'm trying to understand the point of A AND B. I'm guessing that maybe you want to mix in the superclass behavior and sometimes have local behavior. So suppose A is just mixing together behaviors, and B has some local behavior and state.
If you don't need your own state, you probably don't need an __init__. So for A and C just omit __init__.
class SomethingElse(object):
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
class D(SomethingElse):
def __init__(self, value, *args, **kwargs):
super(D, self).__init__(*args, **kwargs)
self.value = value
def function(self):
return self.value
def other_method(self):
return self.__dict__
class C(object):
#def __init__(self):
# super(C, self).__init__()
def function(self):
return self.value*2
class B(C, D):
def __init__(self, value, bstate, *args, **kwargs):
super(B, self).__init__(value, *args, **kwargs)
self.bstate = bstate
def __repr__(self):
return (self.__class__.__name__ + ' ' +
self.bstate + ' ' + str(self.other_method()))
class A(C, D):
pass
a = A(3)
b = B(21, 'extra')
a.function()
6
b.function()
42
repr(a)
'<xx.A object at 0x107cf5e10>'
repr(b)
"B extra {'args': (), 'bstate': 'extra', 'value': 21, 'kwargs': {}}"
I've kept python2 syntax assuming you might still be using it, but as another answer points out, python3 simplifies super() syntax, and you really should be using python3 now.
If you swap C and D you are changing the python method resolution order, and that will indeed change the method to which a call to A.function resolves.
I've imported a package that provides me with a class and a wrapper function that creates an instance of that class.
For example:
class Foo:
def __init__(self, a, b):
self.a = a
self.b = b
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
def makeFoo(x, y):
a = x + y
b = x - y
return Foo(a, b)
I want to have a similar class NamedFoo, that has the same properties/methods, also has a name property, and with a constructor that calls makeFoo. I figure that this should be solved using inheritance, with NamedFoo being a subclass of Foo. However, I don't know how to make the NamedFoo constructor utilize makeFoo correctly:
class NamedFoo(Foo):
def __init__(self, x, y, name):
# ???
# Foo = makeFoo(x, y) ??
# self.Foo = makeFoo(x, y) ??
self.name = name
def printName(self):
print(self.name)
Example data:
myNamedFoo = NamedFoo(2,5,"first")
myNamedFoo.print_a() # (From makeFoo: a = x + y) ==> 2 + 5 = 7
myNamedFoo.print_b() # (From makeFoo: a = x - y) ==> 2 - 5 = -3
I'm not too familiar with object-oriented programming, so I might just be using the wrong search terms, but I haven't found anything similar to what I need. Is this possible, and if so how can I do it?
I'm also not sure if this is an X/Y problem, but here are the alternatives I've considered and why I don't think they're ideal:
Composition of Foo and the property name: It's ugly and doesn't seem right.
Manually adding the name property to each Foo object, and perhaps wrapping it in a function: Doesn't quite have the elegance of a one liner constructor.
Rewriting the constructor for the Foo class, to have the same code as what's in makeFoo: makeFoo is rather complex and needs to do a lot of setup, and this would in any case lead to code duplication
In the NamedFoo constructor, create an instance of the Foo class from the makeFoo wrapper function. Pass this instance's attributes to the super().__init__.
class NamedFoo(Foo):
def __init__(self, x, y, name):
_foo = makeFoo(x,y) # use the wrapper to handle complex logic from input params
super().__init__(_foo.a,_foo.b) # pass the properly derived Foo attributes to the superclass constructor
self.name = name
This way, we're instantiating NamedFoo from whatever magic happens within the makeFoo function. Pass your x and y to that, which creates a throwaway Foo instance (so we can have it properly constructed with whatever complex logic resides in the helper function). The final NamedFoo class is then instantiated from the Foo constructor.
i think this should work..
class Foo:
def __init__(self,a,b):
self.a = a + b
self.b = a - b
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
class NamedFoo(Foo):
def __init__(self,a,b,name):
super().__init__(a,b)
self.name = name
def main():
example = NamedFoo(2,5,"first")
example.print_a()
example.print_b()
main()
this prints out
7
-3
or if you really want to use a function to create self.a and self.b use this:
class Foo:
def __init__(self, a, b):
self.a, self.b = make_foo(a,b)
def print_a(self):
print(self.a)
def print_b(self):
print(self.b)
class NamedFoo(Foo):
def __init__(self, a,b,name):
super().__init__(a,b)
self.name = name
def make_foo(x,y):
return x+y, x-y
def main():
example = NamedFoo(2,5,"first")
example.print_a()
example.print_b()
main()
I am trying to make a python decorator that adds attributes to methods of a class so that I can access and modify those attributes from within the method itself. The decorator code is
from types import MethodType
class attribute(object):
def __init__(self, **attributes):
self.attributes = attributes
def __call__(self, function):
class override(object):
def __init__(self, function, attributes):
self.__function = function
for att in attributes:
setattr(self, att, attributes[att])
def __call__(self, *args, **kwargs):
return self.__function(*args, **kwargs)
def __get__(self, instance, owner):
return MethodType(self, instance, owner)
retval = override(function, self.attributes)
return retval
I tried this decorator on the toy example that follows.
class bar(object):
#attribute(a=2)
def foo(self):
print self.foo.a
self.foo.a = 1
Though I am able to access the value of attribute 'a' from within foo(), I can't set it to another value. Indeed, when I call bar().foo(), I get the following AttributeError.
AttributeError: 'instancemethod' object has no attribute 'a'
Why is this? More importantly how can I achieve my goal?
Edit
Just to be more specific, I am trying to find a simple way to implement static variable that are located within class methods. Continuing from the example above, I would like instantiate b = bar(), call both foo() and doo() methods and then access b.foo.a and b.doo.a later on.
class bar(object):
#attribute(a=2)
def foo(self):
self.foo.a = 1
#attribute(a=4)
def doo(self):
self.foo.a = 3
The best way to do this is to not do it at all.
First of all, there is no need for an attribute decorator; you can just assign it yourself:
class bar(object):
def foo(self):
print self.foo.a
self.foo.a = 1
foo.a = 2
However, this still encounters the same errors. You need to do:
self.foo.__dict__['a'] = 1
You can instead use a metaclass...but that gets messy quickly.
On the other hand, there are cleaner alternatives.
You can use defaults:
def foo(self, a):
print a[0]
a[0] = 2
foo.func_defaults = foo.func_defaults[:-1] + ([2],)
Of course, my preferred way is to avoid this altogether and use a callable class ("functor" in C++ words):
class bar(object):
def __init__(self):
self.foo = self.foo_method(self)
class foo_method(object):
def __init__(self, bar):
self.bar = bar
self.a = 2
def __call__(self):
print self.a
self.a = 1
Or just use classic class attributes:
class bar(object):
def __init__(self):
self.a = 1
def foo(self):
print self.a
self.a = 2
If it's that you want to hide a from derived classes, use whatever private attributes are called in Python terminology:
class bar(object):
def __init__(self):
self.__a = 1 # this will be implicitly mangled as __bar__a or similar
def foo(self):
print self.__a
self.__a = 2
EDIT: You want static attributes?
class bar(object):
a = 1
def foo(self):
print self.a
self.a = 2
EDIT 2: If you want static attributes visible to only the current function, you can use PyExt's modify_function:
import pyext
def wrap_mod(*args, **kw):
def inner(f):
return pyext.modify_function(f, *args, **kw)
return inner
class bar(object):
#wrap_mod(globals={'a': [1]})
def foo(self):
print a[0]
a[0] = 2
It's slightly ugly and hackish. But it works.
My recommendation would be just to use double underscores:
class bar(object):
__a = 1
def foo(self):
print self.__a
self.__a = 2
Although this is visible to the other functions, it's invisible to anything else (actually, it's there, but it's mangled).
FINAL EDIT: Use this:
import pyext
def wrap_mod(*args, **kw):
def inner(f):
return pyext.modify_function(f, *args, **kw)
return inner
class bar(object):
#wrap_mod(globals={'a': [1]})
def foo(self):
print a[0]
a[0] = 2
foo.a = foo.func_globals['a']
b = bar()
b.foo() # prints 1
b.foo() # prints 2
# external access
b.foo.a[0] = 77
b.foo() # prints 77
While You can accomplish Your goal by replacing self.foo.a = 1 with self.foo.__dict__['a'] = 1 it is generally not recommended.
If you are using Python2 - (and not Python3) - whenever you retrieve a method from an instance, a new instance method object is created which is a wrapper to the original function defined in the class body.
The instance method is a rather transparent proxy to the function - you can retrieve the function's attributes through it, but not set them - that is why setting an item in self.foo.__dict__ works.
Alternatively you can reach the function object itself using: self.foo.im_func - the im_func attribute of instance methods point the underlying function.
Based on other contributors's answers, I came up with the following workaround. First, wrap a dictionnary in a class resolving non-existant attributes to the wrapped dictionnary such as the following code.
class DictWrapper(object):
def __init__(self, d):
self.d = d
def __getattr__(self, key):
return self.d[key]
Credits to Lucas Jones for this code.
Then implement a addstatic decorator with a statics attribute that will store the static attributes.
class addstatic(object):
def __init__(self, **statics):
self.statics = statics
def __call__(self, function):
class override(object):
def __init__(self, function, statics):
self.__function = function
self.statics = DictWrapper(statics)
def __call__(self, *args, **kwargs):
return self.__function(*args, **kwargs)
def __get__(self, instance, objtype):
from types import MethodType
return MethodType(self, instance)
retval = override(function, self.statics)
return retval
The following code is an example of how the addstatic decorator can be used on methods.
class bar(object):
#attribute(a=2, b=3)
def foo(self):
self.foo.statics.a = 1
self.foo.statics.b = 2
Then, playing with an instance of the bar class yields :
>>> b = bar()
>>> b.foo.statics.a
2
>>> b.foo.statics.b
3
>>> b.foo()
>>> b.foo.statics.a
3
>>> b.foo.statics.b
5
The reason for using this statics dictionnary follows jsbueno's answer which suggest that what I want would require overloading the dot operator of and instance method wrapping the foo function, which I am not sure is possible. Of course, the method's attribute could be set in self.foo.__dict__, but since it not recommended (as suggested by brainovergrow), I came up with this workaround. I am not certain this would be recommended either and I guess it is up for comments.
Is is possible to access the arguments which were passed to __init__, without explicitly having to store them?
e.g.
class thing(object):
def __init__(self, name, data):
pass # do something useful here
t = thing('test', [1,2,3,])
print t.__args__ # doesn't exist
>> ('test', [1,2,3])
The use-case for this is creating a super-class which can automatically store the arguments used to create an instance of a class derived from it, without having to pass all the arguments explicitly to the super's __init__. Maybe there's an easier way to do it!
No, you have to store them. Otherwise they are gone after __init__() returns, as all local variables.
If you don't want to pass all arguments on explicitly, you can use **kwargs:
class Base(object):
def __init__(self, name, data):
# store name and data
class Derived(Base):
def __init__(self, **kwargs):
Base.__init__(self, **kwargs)
Derived(name="Peter", data=42)
This is not entirely recommended, but here is a wrapper that automatically stores parameter variables:
from functools import wraps
def init_wrapper(f):
#wraps(f)
def wrapper(self, *args, **kwargs):
func_parameters = f.func_code.co_varnames[1:f.func_code.co_argcount]
#deal with default args
diff = len(func_parameters) - len(args)
if diff > 0:
args += f.func_defaults[-diff:]
#set instance variables
for pos, arg in enumerate(func_parameters):
print pos, arg
setattr(self, arg, args[pos])
f(self, *args, **kwargs) #not necessary to use return on __init__()
return wrapper
Usage:
class A(object):
#init_wrapper
def __init__(self, a, b, c):
print a + b + c
Example:
>>> a = A(1, 2, 3)
6
>>> a.a
1
>>> a.b
2
>>> a.c
3
In a word: No.
What you could do is:
def __init__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
If you find yourself needing to do this a lot, you could also use a decorator to abstract the task.
I think that you are looking for arbitrary argument lists and keyword arguments combined with super.__init__.
Give "Python's Super is nifty, but you can't use it" a read before you start down this path though.