In the following code:
class A(object):
VALUE = 1
def __init__(self, value=VALUE):
self.value = value
class B(A):
VALUE = 2
i'd expect that B().value should be equal to 2, however:
B().value = 1
Is there an elegant way to define a class hierarchy where child classes can just declare class variables they want to override and have them be defaults for the instance variables? I still want to allow for these to be changed on a per-instance level, eg.
b = B(value=3)
This is another default arguments question. The point is that when you write
def foo(value=VALUE):
the code inside the function is compiled and made into a function object. It is at this time -- not at call time! -- that the default arguments are stored. So by the time you have defined B it is too late: the default value of foo is already set and changing VALUE won't have any effect.
If this seems a strange thing to do, suppose foo was a global function:
default = 3
def foo(x=default): pass
Then any other code, anywhere, could screw up foo by doing
global default
default = 4
This is arguably just as confusing.
To force the lookups to be done at runtime not compile time, you need to put them inside the function:
def foo(value=None):
self.value = self.VALUE if value is None else value
or (not quite the same but prettier)
self.value = value or self.VALUE
(This is different because it will treat any 'falsy' value as a sentinel -- that is, 0, [], {} etc will all be overwritten by VALUE.)
EDIT: #mgilson pointed out another way of doing this:
def foo(**kwargs):
self.value = kwargs.get("value", self.VALUE)
This is neater in that it doesn't require you to make a sentinel value (like None or object(), but it does change the argument specification of foo quite fundamentally since now it will accept arbitrary keyword arguments. Your call.
The default value should not be define in the func declaration line, otherwise, when the python reads this line, the default value will be focused, and whatever you change the VALUE later, the default value will not be changed.
You could write it as follows:
class A:
VALUE = 1
def __init__(self, value=None):
if value is None:
value = self.VALUE
self.value = value
class B(A):
VALUE = 2
print(A().value)
print(B().value)
Related
I am trying to modify an already defined class by changing an attribute's value. Importantly, I want this change to propagate internally.
For example, consider this class:
class Base:
x = 1
y = 2 * x
# Other attributes and methods might follow
assert Base.x == 1
assert Base.y == 2
I would like to change x to 2, making it equivalent to this.
class Base:
x = 2
y = 2 * x
assert Base.x == 2
assert Base.y == 4
But I would like to make it in the following way:
Base = injector(Base, x=2)
Is there a way to achieve this WITHOUT recompile the original class source code?
The effect you want to achieve belongs to the realm of "reactive programing" - a programing paradigm (from were the now ubiquitous Javascript library got its name as an inspiration).
While Python has a lot of mechanisms to allow that, one needs to write his code to actually make use of these mechanisms.
By default, plain Python code as the one you put in your example, uses the Imperative paradigm, which is eager: whenever an expression is encoutered, it is executed, and the result of that expression is used (in this case, the result is stored in the class attribute).
Python's advantages also can make it so that once you write a codebase that will allow some reactive code to take place, users of your codebase don't have to be aware of that, and things work more or less "magically".
But, as stated above, that is not free. For the case of being able to redefine y when x changes in
class Base:
x = 1
y = 2 * x
There are a couple paths that can be followed - the most important is that, at the time the "*" operator is executed (and that happens when Python is parsing the class body), at least one side of the operation is not a plain number anymore, but a special object which implements a custom __mul__ method (or __rmul__) in this case. Then, instead of storing a resulting number in y, the expression is stored somewhere, and when y is retrieved either as a class attribute, other mechanisms force the expression to resolve.
If you want this at instance level, rather than at class level, it would be easier to implement. But keep in mind that you'd have to define each operator on your special "source" class for primitive values.
Also, both this and the easier, instance descriptor approach using property are "lazily evaluated": that means, the value for y is calcualted when it is to be used (it can be cached if it will be used more than once). If you want to evaluate it whenever x is assigned (and not when y is consumed), that will require other mechanisms. Although caching the lazy approach can mitigate the need for eager evaluation to the point it should not be needed.
1 - Before digging there
Python's easiest way to do code like this is simply to write the expressions to be calculated as functions - and use the property built-in as a descriptor to retrieve these values. The drawback is small:
you just have to wrap your expressions in a function (and then, that function
in something that will add the descriptor properties to it, such as property). The gain is huge: you are free to use any Python code inside your expression, including function calls, object instantiation, I/O, and the like. (Note that the other approach requires wiring up each desired operator, just to get started).
The plain "101" approach to have what you want working for instances of Base is:
class Base:
x = 1
#property
def y(self):
return self.x * 2
b = Base()
b.y
-> 2
Base.x = 3
b.y
-> 6
The work of property can be rewritten so that retrieving y from the class, instead of an instance, achieves the effect as well (this is still easier than the other approach).
If this will work for you somehow, I'd recommend doing it. If you need to cache y's value until x actually changes, that can be done with normal coding
2 - Exactly what you asked for, with a metaclass
as stated above, Python'd need to know about the special status of your y attribute when calculcating its expression 2 * x. At assignment time, it would be already too late.
Fortunately Python 3 allow class bodies to run in a custom namespace for the attribute assignment by implementing the __prepare__ method in a metaclass, and then recording all that takes place, and replacing primitive attributes of interest by special crafted objects implementing __mul__ and other special methods.
Going this way could even allow values to be eagerly calculated, so they can work as plain Python objects, but register information so that a special injector function could recreate the class redoing all the attributes that depend on expressions. It could also implement lazy evaluation, somewhat as described above.
from collections import UserDict
import operator
class Reactive:
def __init__(self, value):
self._initial_value = value
self.values = {}
def __set_name__(self, owner, name):
self.name = name
self.values[owner] = self._initial_value
def __get__(self, instance, owner):
return self.values[owner]
def __set__(self, instance, value):
raise AttributeError("value can't be set directly - call 'injector' to change this value")
def value(self, cls=None):
return self.values.get(cls, self._initial_value)
op1 = value
#property
def result(self):
return self.value
# dynamically populate magic methods for operation overloading:
for name in "mul add sub truediv pow contains".split():
op = getattr(operator, name)
locals()[f"__{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(self, other, operator)))(op)
locals()[f"__r{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(other, self, operator)))(op)
class ReactiveExpr(Reactive):
def __init__(self, value, op2, operator):
self.op2 = op2
self.operator = operator
super().__init__(value)
def result(self, cls):
op1, op2 = self.op1(cls), self.op2
if isinstance(op1, Reactive):
op1 = op1.result(cls)
if isinstance(op2, Reactive):
op2 = op2.result(cls)
return self.operator(op1, op2)
def __get__(self, instance, owner):
return self.result(owner)
class AuxDict(UserDict):
def __init__(self, *args, _parent, **kwargs):
self.parent = _parent
super().__init__(*args, **kwargs)
def __setitem__(self, item, value):
if isinstance(value, self.parent.reacttypes) and not item.startswith("_"):
value = Reactive(value)
super().__setitem__(item, value)
class MetaReact(type):
reacttypes = (int, float, str, bytes, list, tuple, dict)
def __prepare__(*args, **kwargs):
return AuxDict(_parent=__class__)
def __new__(mcls, name, bases, ns, **kwargs):
pre_registry = {}
cls = super().__new__(mcls, name, bases, ns.data, **kwargs)
#for name, obj in ns.items():
#if isinstance(obj, ReactiveExpr):
#pre_registry[name] = obj
#setattr(cls, name, obj.result()
for name, reactive in pre_registry.items():
_registry[cls, name] = reactive
return cls
def injector(cls, inplace=False, **kwargs):
original = cls
if not inplace:
cls = type(cls.__name__, (cls.__bases__), dict(cls.__dict__))
for name, attr in cls.__dict__.items():
if isinstance(attr, Reactive):
if isinstance(attr, ReactiveExpr) and name in kwargs:
raise AttributeError("Expression attributes can't be modified by injector")
attr.values[cls] = kwargs.get(name, attr.values[original])
return cls
class Base(metaclass=MetaReact):
x = 1
y = 2 * x
And, after pasting the snippet above in a REPL, here is the
result of using injector:
In [97]: Base2 = injector(Base, x=5)
In [98]: Base2.y
Out[98]: 10
The idea is complicated with that aspect that Base class is declared with dependent dynamically evaluated attributes. While we can inspect class's static attributes, I think there's no other way of getting dynamic expression except for parsing the class's sourcecode, find and replace the "injected" attribute name with its value and exec/eval the definition again. But that's the way you wanted to avoid. (moreover: if you expected injector to be unified for all classes).
If you want to proceed to rely on dynamically evaluated attributes define the dependent attribute as a lambda function.
class Base:
x = 1
y = lambda: 2 * Base.x
Base.x = 2
print(Base.y()) # 4
Is it possible to define an instance variable in a class as a function of another? I haven't gotten it to work unless you redefine the "function instance variable" all the time.
Basically you could have a scenario where you have one instance variable that is a list of integers, and want to have the sum of these as an instance variable, that automatically redefines every time the list is updated.
Is this possible?
class Example:
list_variable = []
sum_variable = sum(list_variable)
def __init__(self, list_variable):
self.list_variable = list_variable
return
This will result in sum_variable = 0 unless you change it.
I understand that this is far from a major issue, you could either define sum_variable as a method or redefine it every time you change list_variable, I'm just wondering if it's possible to skip those things/steps.
Python offers the property decorator for a syntatically identical use of your example:
class Example:
list_variable = []
def __init__(self, list_variable):
self.list_variable = list_variable
return
#property
def sum_variable(self):
return sum(self.list_variable)
e = Example(list_variable=[10, 20, 30])
e.sum_variable # returns 60
I'm creating a class sequence, which inherits from the builtin list and will hold an ordered collection of a second class: d0 which inherits from int. d0, in addition to its int value must contain a secondary value, i which denotes where it exists in the class and a reference to the class itself.
My understanding is because int is an immutable type, I have to use the __new__ method, and because it will have other attributes, I need to use __init__.
I've been trying for a while to get this to work and I've explored a few options.
Attempt 1:
class sequence(list):
def __init__(self, data):
for i, elem in enumerate(data): self.append( d0(elem, i, self) )
class d0(int):
def __new__(self, val, i, parent):
self.i = i
self.parent = parent
return int.__new__(d0, val)
x = sequence([1,2,3])
print([val.i for val in x])
This was the most intuitive to me, but every time self.i is assigned, it overwrites the i attribute for all other instances of d0 in sequence. Though I'm not entirely clear why this happens, I understand that __new__ is not the place instantiate an object.
Attempt 2:
class sequence(list):
def __init__(self, data):
for i, val in enumerate(data): self.append( d0(val, i, self) )
class d0(int):
def __new__(cls, *args):
return super().__new__(cls, *args)
def __init__(self, *args):
self = args[0]
self.i = args[1]
self.parent = args[2]
x = sequence([1,2,3])
print([val.i for val in x])
This raises TypeError: int() takes at most 2 arguments (3 given), though I'm not sure why.
Attempt 3:
class sequence(list):
def __init__(self, data):
for i, val in enumerate(data):
temp = d0.__new__(d0, val)
temp.__init__(i, self)
self.append(temp)
class d0(int):
def __new__(cls, val):
return int.__new__(d0, val)
def __init__(self, i, parent):
self.i = i
self.parent = parent
x = sequence([1,2,3])
print([val.i for val in x])
This accomplishes the task, but is cumbersome and otherwise just feels strange to have to explicitly call __new__ and __init__ to instantiate an object.
What is the proper way to accomplish this? I would also appreciate any explanation for the undesired behavior in attempts 1 and 2.
First, your sequence isn’t much of a type so far: calling append on it won’t preserve its indexed nature (let alone sort or slice assignment!). If you just want to make lists that look like this, just write a function that returns a list. Note that list itself behaves like such a function (it was one back in the Python 1 days!), so you can often still use it like a type.
So let’s talk just about d0. Leaving aside the question of whether deriving from int is a good idea (it’s at least less work than deriving from list properly!), you have the basic idea correct: you need __new__ for an immutable (base) type, because at __init__ time it’s too late to choose its value. So do so:
class d0(int):
def __new__(cls,val,i,parent):
return super().__new__(cls,val)
Note that this is a class method: there’s no instance yet, but we do need to know what class we’re instantiating (what if someone inherits from d0?). This is what attempt #1 got wrong: it thought the first argument was an instance to which to assign attributes.
Note also that we pass only one (other) argument up: int can’t use our ancillary data. (Nor can it ignore it: consider int('f',16).) Thus failed #2: it sent all the arguments up.
We can install our other attributes now, but the right thing to do is use __init__ to separate manufacturing an object from initializing it:
# d0 continued
def __init__(self,val,i,parent):
# super().__init__(val)
self.i=i; self.parent=parent
Note that all the arguments appear again, even val which we ignore. This is because calling a class involves only one argument list (cf. d0(elem,i,self)), so __new__ and __init__ have to share it. (It would therefore be formally correct to pass val to int.__init__, but what would it do with it? There’s no use in calling it at all since we know int is already completely set up.) Using #3 was painful because it didn’t follow this rule.
So, I have defined the following class which should resemble a probability mass function. However, its logic seems broken and it will raise SUM_ERROR every time I try to initialize a new object.
class ProbabilityMass(dict):
class InvalidEntries(Exception):
pass
SUM_ERROR = InvalidEntries("all values must add upto '1'")
VAL_ERROR = InvalidEntries("negative values are not allowed")
def __init__(self, pm):
dict.__init__(pm)
# Input requirements
if not self.sumsUptoOne():
raise ProbabilityMass.SUM_ERROR
if not self.isNonnegative():
raise ProbabilityMass.VAL_ERROR
def isNonnegative(self):
return all(d < 0 for d in self.values())
def sumsUptoOne(self):
return sum(self.values()) == 1
How can I fix this?
Calling dict.__init__() does not initialize the class. The correct call to super should look like this:
def __init__(self, pm):
super(ProbabilityMass, self).__init__(pm)
# Input requirements
...
As a side note, your isNonnegative() method is also incorrect. Change it to:
def isNonnegative(self):
return all(d >= 0 for d in self.values())
Usually, when dict.__init__() is called, it is because you used dict(). When a class is called like a function, an instance is created, and the instance's .__init__() method is called with the arguments given to the class. Well, calling an instance method is the same thing as calling the class method with the instance as a first argument. Therefore, x = dict() is short for:
x = new dict instance
dict.__init__(x)
If you already have an instance of dict (or a subclass) that was not initialized, you can call __init__() yourself. You must, however, remember to pass the instance as the first argument:
dict.__init__(self, pm)
The more common way is to use the built-in super():
super(ProbabilityMass, self).__init__(pm)
i'm python newbie, and member variable of class works weird in my python code.
some works like normal variable, but some works like static variable!
class Chaos:
list_value = []
value = "default"
def set_value(self, word):
self.list_value.append(word)
self.value = word
def show(self, num):
print(str(num) + "====")
print("value : " + self.value)
for st in self.list_value:
sys.stdout.write(st)
print("\n=====\n")
a = Chaos()
a.show(0)
a.set_value("A")
a.show(1)
b = Chaos()
a.show(2)
b.show(3)
result
0====
value : default
=====
1====
value : A
A
=====
2====
value : A
A
=====
3====
value : default
A
=====
but the last result of the test is different from what i expected in last test.
There should be no "A" in the 'list_value' of the instance of 'b'.
It was just created, and never have been added 'A' before.
I added 'A' to the instance of 'a', not 'b'.
But the result show me that there are also 'A' in 'b'
More over, the 'list_value' and the 'value' in the class works differently.
It looks like the both have same syntax. why do they work differently?
Those are, in fact, class variables. To create instance variables, initialize them in the __init__ function:
class Chaos:
def __init__(self):
self.list_value = []
self.value = "default"
The reason value is behaving like instance variables is because you're setting it using self.value. When Python sees self.X it looks if there's a property X in your object, and if there is none, it looks at its class. Since you never set self.list_value, it's accessing the class variable, that is shared among all instances, so any modifiations will reflect in every other object.
The key difference is that you are appending to list_value, and you are assigning to value. They are called "class variables". Each instance has its own reference to the class variable, which is why the list is shared. However, when you assign, you are changing that instance's reference to point to a different variable, which explains the behavior of value.
If you are looking for instance variable behavior from list_value, initialize it in the constructor (a class method called __init__) instead.